A Simple , ☁️ Lightweight , πŸ’ͺ Efficent JSON based database for 🐍 Python.

Overview

data

A Simple, Lightweight, Efficent JSON based DataBase for Python

PyPI version Downloads .github/workflows/Build.yml CodeFactor Discord GitHub Repo stars GitHub commit activity Downloads


The current stable version is v1.6.1

pip install pysondb==1.6.1

Support the project here

Hacktoberfest

While we are accepting pull requests for Hacktoberfest, we will reject any low-quality PR's.

We are accepting PRs for:

  • Packages - updating package versions, adding new packages Documentation updates
  • More features and fixes (Refer here)
  • CLI/API improvements - Redoing the CLI to reduce the usage of CLI packages.
  • We are planning to reduce the project dependencies and remove other unwanted API's.

Checkout the active issues here

Features

  • Lightweight JSON based database.
  • Supports CRUD commands.
  • No Database drivers required.
  • Unique ID assigned for each JSON document added.
  • Strict about Schema of data added.
  • Inbuilt CLI to delete,display,create JSON database.
> a.addMany([{"name":"pysondb","type":"DB"},{"name":"pysondb-cli","type":"CLI"}]) >> a.getAll() >> [{"name":"pysondb","type":"DB"},{"name":"pysondb-cli","type":"CLI"}]">
>> from pysondb import db
>> a=db.getDb("path/to/json.json")
>> a.addMany([{"name":"pysondb","type":"DB"},{"name":"pysondb-cli","type":"CLI"}])
>> a.getAll()
>> [{"name":"pysondb","type":"DB"},{"name":"pysondb-cli","type":"CLI"}]
  • See its simple..

Quick Links

Install

pip install pysondb

Create a database

  • You can create a database using CLI.
pysondb create database_name.json
  • Or in the python file.
from pysondb import db

a=db.getDb("db.json")
  • The above piece of code will create a database with {data:[]} in it.
  • Even if the json file exists there is no problem.

See more Documentation here

What makes pysonDB different

  • CLI support to create,delete and display database.
  • Unique Id automatically assigned for each JSON data added.
  • Schema regularity is checked.

What pysonDB can't do.

  • Cannot store images,videos etc.

TODO

  • CLI to convert CSV to pysonDB required json.
  • Use CSV file in the database.
  • Make the database available to use local servers (Flask,Django)

Contributors.

Projects using PysonDB

Contributing

Whether reporting bugs, discussing improvements and new ideas or writing extensions: Contributions to pysonDB are welcome! Here's how to get started:

  • Join Discord channel Here
  • Check for issues or open a issue or open a feature requests.
  • Fork the repository on Github
  • Create a new branch off the master branch.
  • Write a test which shows that the bug was fixed or that the feature works as expected.
  • Send a pull request to us and wait till it get merged.
Comments
  • Move errors to a separate file

    Move errors to a separate file

    Hacktoberfest - task

    issue - Currently all the errors raised by pysondb are in the same file where JsonStorage class is. This is not considered to be a good practice. So moving the errors to a different file (errors.py) will make the code base look more concise and beautiful.

    enhancement good first issue hacktoberfest 
    opened by Adwaith-Rajesh 13
  • Only one UniqueID will be find

    Only one UniqueID will be find

    Info

    pyVersion: Python 3.10 OS: Windows 10

    Describe the bug

    Only the latest added entry can be found else this error accrued pysondb.errors.db_errors.IdNotFoundError: Id '184766099262345250' does not exist in the JSON db How to reproduce

    GET Output

    [
      {
        "id": 130978302791852290, 
        "name": "lol", 
        "state": true
      }, 
      {
        "id": 184766099262345253, 
        "name": "lol", 
        "state": true
      }, 
      {
        "id": 135312220569528861, 
        "name": "lol", 
        "state": true
      }
    ]
    
    

    Used Code e.g. Delete Data by ID

            id = data["id"]
            database=db.getDb("data.json")
            #id =database.getBy({"id":id})
            database.deleteById(id)
            return id + " deleted"
    

    If I delete the last one ID in this Out here 135312220569528861 this works fine but another IDs couln't be found. Expected behavior

    Find the correct ID

    opened by DoM1niC 11
  • AttributeError: module 're' has no attribute 'Pattern'

    AttributeError: module 're' has no attribute 'Pattern'

    Hi , im having this error when i try to import the db. so sorry im new here.

    i have intalled this version: pysondb 1.5.6

    (test_project_env) [[email protected] test_testing-json.py]# python3 Python 3.6.8 (default, Nov 16 2020, 16:55:22) [GCC 4.8.5 20150623 (Red Hat 4.8.5-44)] on linux Type "help", "copyright", "credits" or "license" for more information.

    from pysondb import db Traceback (most recent call last): File "", line 1, in File "/data/ftp_project_env/lib64/python3.6/site-packages/pysondb/db.py", line 74, in class JsonDatabase: File "/data/ftp_project_env/lib64/python3.6/site-packages/pysondb/db.py", line 234, in JsonDatabase self, key: str, _re: Union[str, re.Pattern] AttributeError: module 're' has no attribute 'Pattern'

    opened by TCP-88 11
  • Can't stop library from logging in Azure Databricks , It writes to much logs

    Can't stop library from logging in Azure Databricks , It writes to much logs

    Thanks for the awesome library but I am facing a small issue here I am using the library in Azure Databricks to write some logs but the library for some reason changing the log level for the cluster machines to be DEBUG so it makes it write to many logs like the image below so if you can provide a way to stop or change the log level it will be great

    Edit: I believe the issue is here https://github.com/fredysomy/pysonDB/blob/29c1d363aac33c8c6e332d27e3043f52a7f54f60/pysondb/db.py#L12

    image

    opened by yehiaelsayed 11
  • Help us and other developers by writing a blog post about pysonDB

    Help us and other developers by writing a blog post about pysonDB

    Hey guys, if you like how pysondb works, and want other people to know about us, then write a blog post about using pysondb in you projects and your user experience and spread the word. One you have done it. Share the link here.

    Let's take pysondb to the moon πŸš€πŸš€πŸš€πŸš€

    I got the idea for this issue from this post

    hacktoberfest 
    opened by Adwaith-Rajesh 9
  • Database Export to CSV

    Database Export to CSV

    How would this feature be useful? I'd like to contribute to this feature is it's valid. This command can be used to export an existing DB into CSV format.

    Describe the solution you'd like A CLI command which takes in the name of the DB and the CSV file name to be exported into.

    Describe alternatives you've considered Before proceeding, I wanted to know if this is a valid idea for implementation. Any other suggestions that can be used to implement this will be helpful.

    opened by VaishnaviNandakumar 7
  • .find() function does not work after a single use

    .find() function does not work after a single use

    Describe the bug When writing data to the JSON db, after the first write to the file, the .find() method will return the record based on the ID. After adding a 2nd record and trying again, the IdNotFoundError error is thrown despite the record being in the DB!

    How to reproduce python 3.9.6

    from pysondb import db
    
    if __name__ == "__main__":
        db = db.getDb("test.json")
        while True:
            id = db.add({"dev": "test"})
            print(db.find(id))
    

    Expected behavior The record should be returning as expected.


    This is a neat library, great work!

    bug good first issue hacktoberfest 
    opened by avi-perl 7
  • suggestion: using ids as key for objects, instead of using them inside object

    suggestion: using ids as key for objects, instead of using them inside object

    How would this feature be useful? retrieving object by its id (primary key) in O(1) times instead of O(n) time.

    Describe the solution you'd like

    currently json file looks like --- { "data": [ { "id": 1, "k1": "v1" }, { "id": 2, "k2": "v2" }] }

    it should looks like --- { "data": { "1": {"k1": "v1"}, "2": {"k2": "v2" } } }

    now we can fetch single items by its key using db["1"] in O(1) time.

    opened by moinologics 5
  • Types for values

    Types for values

    How would this feature be useful?

    This feature can be useful in preventing the entry wrong type of values.

    Describe the solution you'd like

    Maybe, we can define a schema with the keys and the expected types .

    db.set_schema({"name":str, "age":int, "email":str, "bio":Optional[str]})
    

    Describe alternatives you've considered Β―_(ツ)_/Β―

    opened by pranavbaburaj 4
  • Data Versioning JSON files with pysonDB

    Data Versioning JSON files with pysonDB

    How would this feature be useful?

    I feel versioning json files in Database is going a useful feature. I can see it useful for version data artifacts in form of JSON objects.

    Describe the solution you'd like No ideas at the moment

    Describe alternatives you've considered

    • https://github.com/iterative/dvc
    opened by kurianbenoy 4
  • Problem with installation/distribution 1.6.1. Errors folder does not get installed.

    Problem with installation/distribution 1.6.1. Errors folder does not get installed.

    Describe the bug

    $ pysondb convert a.csv a.json
    Traceback (most recent call last):
      File "/usr/local/miniconda3/bin/pysondb", line 5, in <module>
        from pysondb.cli import main
      File "/usr/local/miniconda3/lib/python3.9/site-packages/pysondb/__init__.py", line 1, in <module>
        from .db import getDb
      File "/usr/local/miniconda3/lib/python3.9/site-packages/pysondb/db.py", line 11, in <module>
        from pysondb.errors.db_errors import IdNotFoundError
    ModuleNotFoundError: No module named 'pysondb.errors'
    

    The pysondb/errors folder does not get installed.

    $ pip install pyjsondb=1.6.1
    pip install pyjsondb==1.6.1
    ERROR: Could not find a version that satisfies the requirement pyjsondb==1.6.1 (from versions: 0.0.1)
    ERROR: No matching distribution found for pyjsondb==1.6.1
    

    I noticed setup.cfg still refers to version = 1.6.0.

    How to reproduce

    Expected behavior

    opened by devleaks 3
  • Mistakes in Documentation

    Mistakes in Documentation

    https://pysondb.github.io/pysonDB/update According to updateArray method example on this page is need to change name=Pyson-CLI for name= ArrayEntry >> # In the file.json the data is updated for all data where name=Pyson-CLI like:

    >> from pysondb import db
    >> a=db.getDb("file.json")
    >> a.getBy({"name":"ArrayEntry"})
    >> [{"name":"ArrayEntry", "epic_array":[0,1,2]}]
    >> a.updateArray({"name":"ArrayEntry"},"epic_array",3)
    >> a.getBy({"name":"ArrayEntry"})
    >> # In the file.json the data is updated for all data where name=ArrayEntry
    >> # We can verify it below.
    >> [{"name": "ArrayEntry", "epic_array":[0,1,2,3]}]
    >> a.updateArray({"name":"ArrayEntry"}, "epic_array", 4, 4)
    >> a.getBy({"name":"ArrayEntry"})
    >> [{"name": "ArrayEntry", "epic_array":[1,2,3,4]}
    

    to the JSON file:file.json for this example, should add {"name":"ArrayEntry", "epic_array":[0,1,2]} or another fields for this db?

    opened by zv09 1
  • Indent on deleteById save

    Indent on deleteById save

    https://github.com/pysonDB/pysonDB/blob/4fad660bfb2465b8524fbe314d03b84b30abcb7c/pysondb/db.py#L274

    Hello,

    The param indent is missing on deleteById method. It should be:

    self._get_dump_function()(db_data, db_file, indent=3, ensure_ascii=False)
    

    in order to be consistent with other methods

    Regards and thanks for the lib again :)

    opened by aalvarezwindey 0
  • SchemaError message in updateById

    SchemaError message in updateById

    https://github.com/pysonDB/pysonDB/blob/630af1e030815a70985a2d07fcc58386df12abbb/pysondb/db.py#L251

    Hello,

    I think that the intention of this line was to be

    "db_keys: " + ",".join(sorted(db_data["data"][0].keys())),
    

    Thanks for the lib :)

    opened by aalvarezwindey 0
  • When updating in loop with 2seconds brokes the file

    When updating in loop with 2seconds brokes the file

    Describe the bug I'm trying to update my json every 2 seconds but after couple updateing it will be broken.

    This is the normal ending of my json file. Ekran Resmi 2022-03-28 23 05 49

    This is the broken json Ekran Resmi 2022-03-28 23 06 27

    This is the error code Ekran Resmi 2022-03-28 23 08 16

    bug 
    opened by MrMirhan 4
  • Generated ids exceed the precision used in many javascript json parsers

    Generated ids exceed the precision used in many javascript json parsers

    How would this feature be useful?

    I am using pysondb to develop an API that will drive a frontend that uses javascript. The javascript number type, which is the default datatype for parsing ints out of json, starts to lose precision around 2 digits shorter than the ids provided by pysondb (18 digits, I believe). Even though we know the issue, a quick fix on the frontend is evading us. Changing the length of the ids on the backend would immediately fix the problem on all frontends that use javascript.

    Describe the solution you'd like

    • Some configuration for a database object which allows the target length of the id to be set. I believe this would be a very minor change

    Here is the relevant code in In db.py

       def _get_id(self) -> int:
            return int(str(uuid.uuid4().int)[:18])
    
    • Alternatively, some configuration to return ids as strings instead of ints would also solve the problem. This wouldn't require the database to actually change its internal logic to use strings instead of ints, it would just require a (optional) casting function to be applied to any incoming or outgoing ids.

    Describe alternatives you've considered

    • We could handle casting from int to str ourselves. The problem with this is that id is used everywhere in the api code, and transitioning every data model class to use strings instead of ints would take a long time and introduce bugs.
    • We briefly considered chopping off the last 2-3 digits of the id ourselves at creation time, but we can't figure out how to actually modify the id in the database entry without some insane hackery.
    opened by aekoch 1
Releases(v1.7.0)
  • v1.7.0(May 22, 2022)

  • v1.6.9(May 20, 2022)

    What's Changed

    • Delete print statement in add method by @ZWerduex in https://github.com/pysonDB/pysonDB/pull/96

    New Contributors

    • @ZWerduex made their first contribution in https://github.com/pysonDB/pysonDB/pull/96

    Full Changelog: https://github.com/pysonDB/pysonDB/compare/v1.6.8...v1.6.9

    Source code(tar.gz)
    Source code(zip)
  • v1.6.8(Apr 29, 2022)

  • v1.6.4(Feb 16, 2022)

    What's Changed

    • Resolve "unicode string" by @lonnstyle in https://github.com/pysonDB/pysonDB/pull/88
    • Added updated links to the documentation by @fredysomy in https://github.com/pysonDB/pysonDB/pull/91

    New Contributors

    • @lonnstyle made their first contribution in https://github.com/pysonDB/pysonDB/pull/88

    Full Changelog: https://github.com/pysonDB/pysonDB/compare/v1.6.2...v1.6.4

    Source code(tar.gz)
    Source code(zip)
  • v1.6.2(Jan 18, 2022)

    What's Changed

    • added missing init.py file. by @devleaks in https://github.com/pysonDB/pysonDB/pull/84

    New Contributors

    • @devleaks made their first contribution in https://github.com/pysonDB/pysonDB/pull/84

    Full Changelog: https://github.com/pysonDB/pysonDB/compare/v1.6.1...v1.6.2

    Source code(tar.gz)
    Source code(zip)
  • v1.6.1(Nov 28, 2021)

    What's Changed

    • changed the operator of test_cli due to flake errors by @fredysomy in https://github.com/pysonDB/pysonDB/pull/80
    • Errors by @fredysomy in https://github.com/pysonDB/pysonDB/pull/79
    • Test by @jakbin in https://github.com/pysonDB/pysonDB/pull/78

    Full Changelog: https://github.com/pysonDB/pysonDB/compare/v1.5.7...v1.6.1

    Source code(tar.gz)
    Source code(zip)
  • v1.5.7(Nov 11, 2021)

  • v1.5.5(Oct 16, 2021)

    • Declarative meta data and typo fix
    • error handling when displaying database and database is null
    • error handling when converting to csv database and database is null
    • Rename CLI command display to show. For ease of use and consistency.
    • Define database file extension in README.md and example uses.
    • Correct file location of CHANGELOG.md in pull request template.
    • Fixed bug in CLI, grammar / typo fix
    • Fixed broken links & fixed typos in docs / code
    Source code(tar.gz)
    Source code(zip)
  • v1.5.2(Oct 3, 2021)

  • v1.5.1(Oct 2, 2021)

  • v1.5.0(Oct 1, 2021)

    • Fixes
      • Removed Imageutils (@Architrixs )
    • Bug Fixes
      • Removed the bug present in find function (now known as getById) referenced in #34 , fixed in #47 (@LyndonFan )
    • Enhancements
      • Used argaparse instead of Fire for CLI , thereby reducing package size, fixed in #38 , (@Flodur871 )
    Source code(tar.gz)
    Source code(zip)
  • v1.4.0(Sep 29, 2021)

    Fixes (@Adwaith-Rajesh )

    • find -> getById
    • update -> updateByQuery
    • getBy -> getByQuery
    • Added Backward compatibility

    removed updateArray method

    Source code(tar.gz)
    Source code(zip)
  • v1.2.1(Jul 13, 2021)

  • V1.1.6(Jul 13, 2021)

  • v0.9.0(Jun 1, 2021)

    • Added Merge command to merge two schema identical Json files from CLI (by @Adwaith-Rajesh )
    • Added Regex search to the commands which let you search data using regex. ( By @Adwaith-Rajesh )
    Source code(tar.gz)
    Source code(zip)
Owner
PysonDB
A Simple , Lightweight , Efficent JSON based database for Python. A place for pysonDB related plugins.
PysonDB
This project is related to a No-SQL database, whose data are referred to autoctone botanic species

This project is related to a No-SQL database, whose data are referred to autoctone botanic species. The final goal is creating a function that performs the estimation of the ornamental value, given t

Amatofrancesco99 2 Mar 08, 2022
Shelf DB is a tiny document database for Python to stores documents or JSON-like data

Shelf DB Introduction Shelf DB is a tiny document database for Python to stores documents or JSON-like data. Get it $ pip install shelfdb shelfquery S

Um Nontasuwan 35 Nov 03, 2022
securedb is a fast and lightweight Python framework to easily interact with JSON-based encrypted databases.

securedb securedb is a Python framework that lets you work with encrypted JSON databases. Features: newkey() to generate an encryption key write(key,

Filippo Romani 2 Nov 23, 2022
PathfinderMonsterDatabase - A database of all monsters in Pathfinder 1e, created by parsing aonprd.com

PathfinderMonsterDatabase A database of all monsters in Pathfinder 1e, created by parsing aonprd.com Setup Run the following line to install all requi

Yoni Lerner 11 Jun 12, 2022
LaikaDB, banco de dados para projetos simples.

LaikaDB LaikaDB Γ© um banco de dados noSQL para uso local e simples, onde vocΓͺ pode realizar gravaçáes e leituras de forma eficiente e simples. Todos o

Jaedson Silva 0 Jun 24, 2022
This repo contains the backend of the KMK project

KMK Backend This repository contains the backend part of the KMK project Demo Watch it on Youtube Getting started Pre-commit hooks After you cloned th

21 Nov 26, 2022
Connect Django Project to PostgreSQL

An application for learning things with creating quizzes and flashcards.Django, PostgresSQL are used for this project.

Cena Ashoori 1 Jan 25, 2022
ChaozzDBPy - A python implementation based on the original ChaozzDB from Chaozznl with some new features

ChaozzDBPy About ChaozzDBPy is a python implementation based on the original Cha

Igor Iglesias 1 May 25, 2022
Metrics-advisor - Analyze reshaped metrics from TiDB cluster Prometheus and give some advice about anomalies and correlation.

metrics-advisor Analyze reshaped metrics from TiDB cluster Prometheus and give some advice about anomalies and correlation. Team freedeaths mashenjun

3 Jan 07, 2022
TinyDB is a lightweight document oriented database optimized for your happiness :)

Quick Links Example Code Supported Python Versions Documentation Changelog Extensions Contributing Introduction TinyDB is a lightweight document orien

Markus Siemens 5.6k Dec 30, 2022
Turn SELECT queries returned by a query into links to execute them

datasette-query-links Turn SELECT queries returned by a query into links to execute them Installation Install this plugin in the same environment as D

Simon Willison 5 Apr 27, 2022
LightDB is a lightweight JSON Database for Python

LightDB What is this? LightDB is a lightweight JSON Database for Python that allows you to quickly and easily write data to a file Installing pip3 ins

Stanislaw 14 Oct 01, 2022
Migrate data from SQL to NoSQL easily

Migrate data from SQL to NoSQL easily Installation πŸ’― pip install sql2nosql --upgrade Dependencies πŸ“’ For the package to work, it first needs "clients

Facundo Padilla 43 Mar 26, 2022
AWS Tags As A Database is a Python library using AWS Tags as a Key-Value database.

AWS Tags As A Database is a Python library using AWS Tags as a Key-Value database. This database is completely free* πŸ’Έ

Oren Leung 42 Nov 25, 2022
Enfilade: Tool to Detect Infections in MongoDB Instances

Enfilade: Tool to Detect Infections in MongoDB Instances

Aditya K Sood 7 Feb 21, 2022
A Persistent Embedded Graph Database for Python

Cog - Embedded Graph Database for Python cogdb.io New release: 2.0.5! Installing Cog pip install cogdb Cog is a persistent embedded graph database im

Arun Mahendra 214 Dec 30, 2022
Python function to extract all the rows from a SQLite database file while iterating over its bytes, such as while downloading it

Python function to extract all the rows from a SQLite database file while iterating over its bytes, such as while downloading it

Department for International Trade 16 Nov 09, 2022
pickleDB is an open source key-value store using Python's json module.

pickleDB pickleDB is lightweight, fast, and simple database based on the json module. And it's BSD licensed! pickleDB is Fun import pickledb

Harrison Erd 738 Jan 04, 2023
Python function to query SQLite files stored on S3

sqlite-s3-query Python function to query a SQLite file stored on S3. It uses multiple HTTP range requests per query to avoid downloading the entire fi

Michal Charemza 189 Dec 27, 2022
A NoSQL database made in python.

CookieDB A NoSQL database made in python.

cookie 1 Nov 30, 2022