This is a simple graph database in SQLite, inspired by

Related tags

Databasesimple-graph
Overview

About

This is a simple graph database in SQLite, inspired by "SQLite as a document database".

Structure

The schema consists of just two structures:

  • Nodes - these are any json objects, with the only constraint being that they each contain a unique id value
  • Edges - these are pairs of node id values, specifying the direction, with an optional json object as connection properties

Applications

Usage

Installation

Basic Functions

The python database script provides convenience functions for atomic transactions to add, delete, connect, and search for nodes.

Any single node or path of nodes can also be depicted graphically by using the visualize function within the database script to generate dot files, which in turn can be converted to images with Graphviz.

Testing

There will be more robust and dedicated unit tests with pytest soon, but in the meantime, running the example locally will do in a pinch.

This bit of shell magic will pull out the commands from this document:

grep ">>> " README.md | grep -v "grep" | sed -e 's/>>> //'

Use a final | clip (Windows), | pbcopy (macOS), or | xclip -selection clipboard (most linuxes) to copy all the commands into your clipboard.

If you have the correct version of SQLite installed, everything should just work without errors.

Example

Dropping into a python shell, we can create, upsert, and connect people from the early days of Apple Computer. The resulting database will be saved to a SQLite file named apple.sqlite:

>>> apple = "apple.sqlite"
>>> import database as db
>>> db.initialize(apple)
>>> db.atomic(apple, db.add_node({'name': 'Apple Computer Company', 'type':['company', 'start-up'], 'founded': 'April 1, 1976'}, 1))
>>> db.atomic(apple, db.add_node({'name': 'Steve Wozniak', 'type':['person','engineer','founder']}, 2))
>>> db.atomic(apple, db.add_node({'name': 'Steve Jobs', 'type':['person','designer','founder']}, 3))
>>> db.atomic(apple, db.add_node({'name': 'Ronald Wayne', 'type':['person','administrator','founder']}, 4))
>>> db.atomic(apple, db.add_node({'name': 'Mike Markkula', 'type':['person','investor']}, 5))
>>> db.atomic(apple, db.connect_nodes(2, 1, {'action': 'founded'}))
>>> db.atomic(apple, db.connect_nodes(3, 1, {'action': 'founded'}))
>>> db.atomic(apple, db.connect_nodes(4, 1, {'action': 'founded'}))
>>> db.atomic(apple, db.connect_nodes(5, 1, {'action': 'invested', 'equity': 80000, 'debt': 170000}))
>>> db.atomic(apple, db.connect_nodes(1, 4, {'action': 'divested', 'amount': 800, 'date': 'April 12, 1976'}))
>>> db.atomic(apple, db.connect_nodes(2, 3))
>>> db.atomic(apple, db.upsert_node(2, {'nickname': 'Woz'}))

The nodes can be searched by their ids or any other combination of attributes (either as strict equality, or using _search_like in combination with _search_starts_with or _search_contains):

>>> db.atomic(apple, db.find_node(1))
{'name': 'Apple Computer Company', 'type': ['company', 'start-up'], 'founded': 'April 1, 1976', 'id': 1}
>>> db.atomic(apple, db.find_nodes({'name': 'Steve'}, db._search_like, db._search_starts_with))
[{'name': 'Steve Wozniak', 'type': ['person', 'engineer', 'founder'], 'id': 2, 'nickname': 'Woz'}, {'name': 'Steve Jobs', 'type': ['person', 'designer', 'founder'], 'id': 3}]

Paths through the graph can be discovered with a starting node id, and an optional ending id; the default neighbor expansion is nodes connected nodes in either direction, but that can changed by specifying either find_outbound_neighbors or find_inbound_neighbors instead:

>>> db.traverse(apple, 2, 3)
[2, 3]
>>> db.traverse(apple, 4, 5)
[4, 1, 5]
>>> db.traverse(apple, 5, neighbors_fn=db.find_inbound_neighbors)
[5]
>>> db.traverse(apple, 5, neighbors_fn=db.find_outbound_neighbors)
[5, 1, 4]
>>> db.traverse(apple, 5, neighbors_fn=db.find_neighbors)
[5, 1, 4, 3, 2]

Any path or list of nodes can rendered graphically by using the visualize function. This command produces dot files, which are also rendered as images with Graphviz:

>>> db.visualize(apple, 'apple.dot', [4, 1, 5])

The resulting text file also comes with an associated image (the default is png, but that can be changed by supplying a different value to the format parameter)

The default options include every key/value pair (excluding the id) in the node and edge objects:

Basic visualization

There are display options to help refine what is produced:

>>> db.visualize(apple, 'apple.dot', [4, 1, 5], exclude_node_keys=['type'], hide_edge_key=True)

More refined visualization

The resulting dot file can be edited further as needed; the dot guide has more options and examples.

Comments
  • Find Node

    Find Node

    Why do you use fetchall() and not fetchone()?

    https://github.com/dpapathanasiou/simple-graph/blob/be41db28995f745a4f85677bcade7f0471984848/python/database.py#L112

    opened by chhenning 5
  • Ambiguous nature of ID

    Ambiguous nature of ID

    Id's are numerical in the tests which JSONDecoder is happy with despite the unnecessary need to decode the IDs as JSON.

    If you create a key as a string for instance this will work but will not be possible to decode as the json library in python expects strings to be explicitly quoted. Thus you cannot traverse or visualise if edges aren't double quoted strings.

    Should I be double quoting my strings for IDs?

    I'll post a coded example later on if this isn't clear

    opened by marksbrown 4
  •  support a uuid as node.id

    support a uuid as node.id

    • support a string (e.g. uuid '5b86cc504bda4c7e94d47d00a7696bde') as node.id

    • int64 as node.id are not supported. I tried to us a long int

    uuid.UUID('5b86cc504bda4c7e94d47d00a7696bde').int

    P.S. Thank you for your work. This library has made my day.

    opened by lepy 4
  • Error initializing database

    Error initializing database

    Context: MacOS Mojave, SQLite3, Python 3.8.5

    The default sqlite3 binary in MacOS Mojave does not have the JSON extensions enabled.

    Fix: Install sqlite via brew Add to shell initialization file: export PATH="/usr/local/opt/sqlite/bin:$PATH"

    opened by mrdcbrush 4
  • Searching for nodes where query must match a value in an array in the body

    Searching for nodes where query must match a value in an array in the body

    I'm looking to do something like this

    from simple_graph_sqlite import database as db 
    dbfile = "db.sqlite"
    
    db.initialize(dbfile)
    
    db.atomic(dbfile, db.add_node({'name': 'foo', 'type':['company', 'start-up']}, 1))
    db.atomic(dbfile, db.add_node({'name': 'bar', 'type':['cat', 'apple']}, 1))
    
    db.atomic(apple, db.find_nodes({'type': 'apple'}, db._search_like, db._search_starts_with))
    

    But I don't think the last statement works

    opened by rajarshi 3
  • Multiple Node Connections in visualize

    Multiple Node Connections in visualize

    If I am having a graph with multiple nodes attached to a single node (e.g. node 7), the command db.visualize(graph, 'example.dot', db.traverse(graph, 7, neighbors_fn=db.find_outbound_neighbors)) only visualizes the first connection found in db.find_outbound_neighbors. Is there a way to automatically visualize all connections?

    opened by defaltus 3
  • Thank You

    Thank You

    Hey,

    This isn't an issue, just wanted to say thanks for creating simple-graph!

    I used it as the basis for a Julia package (https://github.com/joshday/SQLiteGraph.jl), since as far as I know, there's no graph database that has a Julia interface. I made some minor changes tailored to my specific use case, but I wouldn't have been able to put it together so easily without the help of your work, so thanks again!

    opened by joshday 1
  • No Traverse with TEXT ID

    No Traverse with TEXT ID

    When I use IDs like meta6 the traverse is not working.

    print(db.traverse(db_file_name, 'meta6', neighbors_fn=db.find_neighbors))
    

    This only prints ['"meta6"'].

    In the atomic(), I enabled tracing via:

    connection.set_trace_callback(print)
    

    and the query is incorrect:

    PRAGMA foreign_keys = TRUE;
    WITH RECURSIVE traverse(id) AS (
      SELECT '"meta6"'
      UNION
      SELECT source FROM edges JOIN traverse ON target = id
      UNION
      SELECT target FROM edges JOIN traverse ON source = id
    ) SELECT id FROM traverse;
    

    Changing it to

    PRAGMA foreign_keys = TRUE;
    WITH RECURSIVE traverse(id) AS (
      SELECT 'meta6'
      UNION
      SELECT source FROM edges JOIN traverse ON target = id
      UNION
      SELECT target FROM edges JOIN traverse ON source = id
    ) SELECT id FROM traverse;
    

    makes it work.

    opened by chhenning 1
  • SQL errors result in database being locked

    SQL errors result in database being locked

    Hi - very handy package!

    I've been playing with it and Ive noticed that if I try to insert a duplicate node, it throws the expected IntegrityError. But after that, subsequent operations fail due to the db being locked.

    Looking at atomic I think this is because the db is not being closed. I modified it to wrap it in try/finally and it seems to solve the problem.

    def atomic(db_file, cursor_exec_fn):
        connection = None
        try:
            connection = sqlite3.connect(db_file)
            cursor = connection.cursor()
            cursor.execute("PRAGMA foreign_keys = TRUE;")
            results = cursor_exec_fn(cursor)
            connection.commit()        
        finally:
            if connection: 
                connection.close()
        return results
    
    opened by rajarshi 0
Releases(v2.1.0)
Owner
Denis Papathanasiou
Denis Papathanasiou
EmployeeDB - Advanced Redis search functionalities on Python applied on an Employee management backend app

EmployeeDB - Advanced Redis search functionalities on Python applied on an Employee management backend app

Ahmad Bazzi 58 Oct 10, 2022
A very simple document database

DockieDb A simple in-memory document database. Installation Build the Wheel Fork or clone this repository and run python setup.py bdist_wheel in the r

1 Jan 16, 2022
Connect Django Project to PostgreSQL

An application for learning things with creating quizzes and flashcards.Django, PostgresSQL are used for this project.

Cena Ashoori 1 Jan 25, 2022
A Painless Simple Way To Create Schema and Do Database Operations Quickly In Python

PainlessDB - Taking Your Pain away to the moon 🚀 Contribute · Community · Documentation 🎫 Introduction : PainlessDB is a Python-based free and open-

Aiden Ellis 3 Jul 15, 2022
ClutterDB - Extremely simple JSON database made for infrequent changes which behaves like a dict

extremely simple JSON database made for infrequent changes which behaves like a dict this was made for ClutterBot

Clutter Development 1 Jan 12, 2022
Turn SELECT queries returned by a query into links to execute them

datasette-query-links Turn SELECT queries returned by a query into links to execute them Installation Install this plugin in the same environment as D

Simon Willison 5 Apr 27, 2022
LightDB is a lightweight JSON Database for Python

LightDB What is this? LightDB is a lightweight JSON Database for Python that allows you to quickly and easily write data to a file Installing pip3 ins

Stanislaw 14 Oct 01, 2022
This is a simple graph database in SQLite, inspired by

This is a simple graph database in SQLite, inspired by "SQLite as a document database".

Denis Papathanasiou 1.2k Jan 03, 2023
Youtube Kanalinda tanittigim ve Programladigim SQLite3 ile calisan Kütüphane Programi

SQLite3 Kütüphane Uygulamasi SQLite3 ile calisan Kütüphane Arayüzü Yükleme Yerel veritabani olusacaktir. Yaptiginiz islemler kaybolmaz! Temel Gereksin

Mikael Pikulski 6 Aug 13, 2022
A Persistent Embedded Graph Database for Python

Cog - Embedded Graph Database for Python cogdb.io New release: 2.0.5! Installing Cog pip install cogdb Cog is a persistent embedded graph database im

Arun Mahendra 214 Dec 30, 2022
Decentralised graph database management system

Decentralised graph database management system To get started clone the repo, and run the command below. python3 database.py Now, create a new termina

Omkar Patil 2 Apr 18, 2022
PathfinderMonsterDatabase - A database of all monsters in Pathfinder 1e, created by parsing aonprd.com

PathfinderMonsterDatabase A database of all monsters in Pathfinder 1e, created by parsing aonprd.com Setup Run the following line to install all requi

Yoni Lerner 11 Jun 12, 2022
Manage your sqlite database very easy (like django) ...

Manage your sqlite database very easy (like django) ...

aWolver 1 Feb 09, 2022
This project is related to a No-SQL database, whose data are referred to autoctone botanic species

This project is related to a No-SQL database, whose data are referred to autoctone botanic species. The final goal is creating a function that performs the estimation of the ornamental value, given t

Amatofrancesco99 2 Mar 08, 2022
A super easy, but really really bad DBMS

Dumb DB Are you looking for a reliable database management system? Then you've come to the wrong place. This is a very small database management syste

Elias Amha 5 Dec 28, 2022
A Python wrapper API for operating and working with the Neo4j Graph Data Science (GDS) library

gdsclient This repo hosts the sources for gdsclient, a Python wrapper API for operating and working with the Neo4j Graph Data Science (GDS) library. g

Neo Technology 101 Jan 05, 2023
Postgres full text search options (tsearch, trigram) examples

postgres-full-text-search Postgres full text search options (tsearch, trigram) examples. Create DB CREATE DATABASE ftdb; To feed db with an example

Jarosław Orzeł 97 Dec 30, 2022
LaikaDB, banco de dados para projetos simples.

LaikaDB LaikaDB é um banco de dados noSQL para uso local e simples, onde você pode realizar gravações e leituras de forma eficiente e simples. Todos o

Jaedson Silva 0 Jun 24, 2022
TelegramDB - A library which uses your telegram account as a database for your projects

TelegramDB A library which uses your telegram account as a database for your projects. Basic Usage from pyrogram import Client from telegram import Te

Kaizoku 79 Nov 22, 2022
Tiny local JSON database for Python.

Pylowdb Simple to use local JSON database 🦉 # This is pure python, not specific to pylowdb ;) db.data['posts'] = ({ 'id': 1, 'title': 'pylowdb is awe

Hussein Sarea 3 Jan 26, 2022