Tiny local JSON database for Python.

Overview

Pylowdb

Downloads Downloads

Simple to use local JSON database 🦉

# This is pure python, not specific to pylowdb ;)
db.data['posts'] = ({ 'id': 1, 'title': 'pylowdb is awesome' })

# Save to file
db.write()
# db.json
{
  "posts": [
    { "id": 1, "title": "pylowdb is awesome" }
  ]
}

Support me

Features

  • Lightweight
  • Minimalist and easy to learn API
  • Query and modify data using plain Python
  • Atomic write
  • Hackable:
    • Change storage, file format (JSON, YAML, ...) or add encryption via adapters

Install

pip install pylowdb

Usage

import os
from os import path
from pylowdb import (
    Low,
    JSONFile,
)

# Use JSON file for storage
file = path.join(os.getcwd(), 'db.json')
adapter = JSONFile(file)
db = Low(adapter)

# Read data from JSON file, this will set db.data content
db.read()

# If file.json doesn't exist, db.data will be None
# Set default data
# db.data = db.data or { 'posts': [] }
db.data = db.data or { 'posts': [] }

# Create and query items using plain Python

db.data['posts'].append('hello world')
db.data['posts'][0]

# You can also use this syntax if you prefer
posts = db.data['posts']
posts.append('hello world'

# Write db.data content to db.json
db.write()
// db.json
{
  "posts": [ "hello world" ]
}

More examples

For more example, see examples/ directory.

API

Classes

Pylowdb has classes (for synchronous adapters).

Low(adapter)

from pylowdb import (
    Low,
    JSONFile,
)
db = Low(JSONFile('file.json'))
db.read()
db.write()

Methods

db.read()

Calls adapter.read() and sets db.data.

Note: JSONFile adapter will set db.data to None if file doesn't exist.

db.data  # is None
db.read()
db.data # is not None

db.write()

Calls adapter.write(db.data).

db.data = { 'posts': [] }
db.write() # file.json will be { posts: [] }
db.data = {}
db.write() # file.json will be {}

Properties

db.data

Holds your db content. If you're using the adapters coming with pylowdb, it can be any type supported by json.dumbs.

For example:

db.data = 'string'
db.data = [1, 2, 3]
db.data = { 'key': 'value' }

Adapters

Pylowdb adapters

JSONFile

Adapter for reading and writing JSON files.

Low(JSONFile(filename))

Memory

In-memory adapter. Useful for speeding up unit tests.

Low(Memory())

YAMLFile

Adapter for reading and writing YAML files.

Low(YAMLFile(filename))

TextFile

Adapters for reading and writing text. Useful for creating custom adapters.

Third-party adapters

If you've published an adapter for pylowdb, feel free to create a PR to add it here.

Writing your own adapter

You may want to create an adapter to write db.data to YAML, XML, encrypt data, a remote storage, ...

An adapter is a simple class that just needs to expose two methods:

class CustomAdapter:
    read(self):
        # should return deserialized data
        pass
    write(self, data):
        # should return nothing
        pass

For example, let's say you have some async storage and want to create an adapter for it:

api = YourAPI()

class CustomAdapter:
    # Optional: your adapter can take arguments

    def __init__(self, *args, **kwargs):
        pass

    def read(self):
        data = api.read()
    return data

    def write(self, data):
        api.write(data)

adapter = CustomAdapter()
db = Low(adapter)

See pylowdb/adapters for more examples.

Custom serialization

To create an adapter for another format than JSON, you can use TextFile.

For example:

from pylowdb import (
    Adapter,
    Low,
    TextFile,
)
import yaml

class YAMLFile(Adapter):
    def __init__(self, filename: str):
        self.adapter = TextFile(filename)

    def read(self):
        data = self.adapter.read()
        if data is None:  
            return null
        else:
            return YAML.deserialize(data)

    def write(self, obj):
        return self.adapter.write(YAML.serialize(obj))

adapter = YAMLFile('file.yaml')
db = Low(adapter)

Limits

If you have large Python objects (~10-100MB) you may hit some performance issues. This is because whenever you call db.write, the whole db.data is serialized and written to storage.

Depending on your use case, this can be fine or not. It can be mitigated by doing batch operations and calling db.write only when you need it.

If you plan to scale, it's highly recommended to use databases like PostgreSQL, MySql, Oracle ...

Owner
Hussein Sarea
I love every thing creative,I adore anime and disney movies. Technology is my life, and I aspire to learn every single part about it.
Hussein Sarea
Лабораторные работы по Postgresql за 5 семестр

Практикум по Postgresql ERD для заданий 2.x: ERD для заданий 3.x: Их делал вот тут Ниже есть 2 инструкции — по установке postgresql на manjaro и по пе

Danila 10 Oct 31, 2022
A Modular MWDB Utility to Collect Fresh Malware Samples

MWDB Feeds A Modular MWDB Utility to Collect Fresh Malware Samples This project is FREE as in FREE 🍺 , use it commercially, privately or however you

c3rb3ru5 32 Jul 07, 2022
Codeqlcompile - 自动反编译闭源应用,创建codeql数据库

codeql_compile 自动反编译闭源应用,创建codeql数据库 准备 首先下载ecj.jar和idea提供反编译的java-decompiler.ja

236 Jan 05, 2023
Python function to extract all the rows from a SQLite database file while iterating over its bytes, such as while downloading it

Python function to extract all the rows from a SQLite database file while iterating over its bytes, such as while downloading it

Department for International Trade 16 Nov 09, 2022
A Simple , ☁️ Lightweight , 💪 Efficent JSON based database for 🐍 Python.

A Simple, Lightweight, Efficent JSON based DataBase for Python The current stable version is v1.6.1 pip install pysondb==1.6.1 Support the project her

PysonDB 282 Jan 07, 2023
Oh-My-PickleDB is an open source key-value store using Python's json module.

OH-MY-PICKLEDB oh-my-pickleDB is a lightweight, fast, and intuitive data manager written in python 📝 Table of Contents About Getting Started Deployme

Adrián Toral 6 Feb 20, 2022
This is a simple graph database in SQLite, inspired by

This is a simple graph database in SQLite, inspired by "SQLite as a document database".

Denis Papathanasiou 1.2k Jan 03, 2023
Python object-oriented database

ZODB, a Python object-oriented database ZODB provides an object-oriented database for Python that provides a high-degree of transparency. ZODB runs on

Zope 574 Dec 31, 2022
Python function to query SQLite files stored on S3

sqlite-s3-query Python function to query a SQLite file stored on S3. It uses multiple HTTP range requests per query to avoid downloading the entire fi

Michal Charemza 189 Dec 27, 2022
HTTP graph database built in Python 3

KiwiDB HTTP graph database built in Python 3. Reference Format References are strings in the format: { JanCraft 1 Dec 17, 2021

pickleDB is an open source key-value store using Python's json module.

pickleDB pickleDB is lightweight, fast, and simple database based on the json module. And it's BSD licensed! pickleDB is Fun import pickledb

Harrison Erd 738 Jan 04, 2023
Mongita is to MongoDB as SQLite is to SQL

Mongita is a lightweight embedded document database that implements a commonly-used subset of the MongoDB/PyMongo interface. Mongita differs from MongoDB in that instead of being a server, Mongita is

Scott Rogowski 809 Jan 07, 2023
Migrate data from SQL to NoSQL easily

Migrate data from SQL to NoSQL easily Installation 💯 pip install sql2nosql --upgrade Dependencies 📢 For the package to work, it first needs "clients

Facundo Padilla 43 Mar 26, 2022
Postgres full text search options (tsearch, trigram) examples

postgres-full-text-search Postgres full text search options (tsearch, trigram) examples. Create DB CREATE DATABASE ftdb; To feed db with an example

Jarosław Orzeł 97 Dec 30, 2022
Simple embedded in memory json database

dbj dbj is a simple embedded in memory json database. It is easy to use, fast and has a simple query language. The code is fully documented, tested an

Pedro Gonring 25 Aug 12, 2022
Tiny local JSON database for Python.

Pylowdb Simple to use local JSON database 🦉 # This is pure python, not specific to pylowdb ;) db.data['posts'] = ({ 'id': 1, 'title': 'pylowdb is awe

Hussein Sarea 3 Jan 26, 2022
Shelf DB is a tiny document database for Python to stores documents or JSON-like data

Shelf DB Introduction Shelf DB is a tiny document database for Python to stores documents or JSON-like data. Get it $ pip install shelfdb shelfquery S

Um Nontasuwan 35 Nov 03, 2022
LaikaDB, banco de dados para projetos simples.

LaikaDB LaikaDB é um banco de dados noSQL para uso local e simples, onde você pode realizar gravações e leituras de forma eficiente e simples. Todos o

Jaedson Silva 0 Jun 24, 2022
Decentralised graph database management system

Decentralised graph database management system To get started clone the repo, and run the command below. python3 database.py Now, create a new termina

Omkar Patil 2 Apr 18, 2022
A Persistent Embedded Graph Database for Python

Cog - Embedded Graph Database for Python cogdb.io New release: 2.0.5! Installing Cog pip install cogdb Cog is a persistent embedded graph database im

Arun Mahendra 214 Dec 30, 2022