pgvector support for Python

Overview

pgvector-python

pgvector support for Python

Great for online recommendations 🎉

Supports Django, SQLAlchemy, Psycopg 2, Psycopg 3, and asyncpg

Build Status

Installation

Run:

pip install pgvector

And follow the instructions for your database library:

Or check out some examples:

Django

Create the extension

from pgvector.django import VectorExtension

class Migration(migrations.Migration):
    operations = [
        VectorExtension()
    ]

Add a vector field

from pgvector.django import VectorField

class Item(models.Model):
    factors = VectorField(dimensions=3)

Insert a vector

item = Item(factors=[1, 2, 3])
item.save()

Get the nearest neighbors to a vector

from pgvector.django import L2Distance

Item.objects.order_by(L2Distance('factors', [3, 1, 2]))[:5]

Also supports MaxInnerProduct and CosineDistance

Add an approximate index

from pgvector.django import IvfflatIndex

class Item(models.Model):
    class Meta:
        indexes = [
            IvfflatIndex(
                name='my_index',
                fields=['factors'],
                lists=100,
                opclasses=['vector_l2_ops']
            )
        ]

Use vector_ip_ops for inner product and vector_cosine_ops for cosine distance

SQLAlchemy

Add a vector column

from pgvector.sqlalchemy import Vector

class Item(Base):
    factors = Column(Vector(3))

Insert a vector

item = Item(factors=[1, 2, 3])
session.add(item)
session.commit()

Get the nearest neighbors to a vector

session.query(Item).order_by(Item.factors.l2_distance([3, 1, 2])).limit(5).all()

Also supports max_inner_product and cosine_distance

Add an approximate index

index = Index('my_index', Item.factors,
    postgresql_using='ivfflat',
    postgresql_with={'lists': 100},
    postgresql_ops={'factors': 'vector_l2_ops'}
)
index.create(engine)

Use vector_ip_ops for inner product and vector_cosine_ops for cosine distance

Psycopg 2

Register the vector type with your connection or cursor

from pgvector.psycopg2 import register_vector

register_vector(conn)

Insert a vector

factors = np.array([1, 2, 3])
cur.execute('INSERT INTO item (factors) VALUES (%s)', (factors,))

Get the nearest neighbors to a vector

cur.execute('SELECT * FROM item ORDER BY factors <-> %s LIMIT 5', (factors,))
cur.fetchall()

Psycopg 3

Register the vector type with your connection or cursor

from pgvector.psycopg3 import register_vector

register_vector(conn)

Insert a vector

factors = np.array([1, 2, 3])
cur.execute('INSERT INTO item (factors) VALUES (%s)', (factors,))

Get the nearest neighbors to a vector

cur.execute('SELECT * FROM item ORDER BY factors <-> %s LIMIT 5', (factors,)).fetchall()

asyncpg

Register the vector type with your connection

from pgvector.asyncpg import register_vector

await register_vector(conn)

Insert a vector

factors = np.array([1, 2, 3])
await conn.execute('INSERT INTO item (factors) VALUES ($1)', factors)

Get the nearest neighbors to a vector

await conn.fetch('SELECT * FROM item ORDER BY factors <-> $1 LIMIT 5', factors)

History

View the changelog

Contributing

Everyone is encouraged to help improve this project. Here are a few ways you can help:

To get started with development:

git clone https://github.com/ankane/pgvector-python.git
cd pgvector-python
pip install -r requirements.txt
pytest
Owner
Andrew Kane
Andrew Kane
Web UI for your scripts with execution management

Script-server is a Web UI for scripts. As an administrator, you add your existing scripts into Script server and other users would be ab

Iaroslav Shepilov 1.1k Jan 09, 2023
Python script for converting obsidian md-file to html (recursively adds all link/images)

ObsidianToHtmlConverter I made a small python script for converting obsidian md-file to static (local) html (recursively adds all link/images) I made

47 Jan 03, 2023
Automatic and platform-independent unpacker for Windows binaries based on emulation

_ _ __ _ __ _ | | | | / / (_) \ \ | | | | | |_ __ | | _ | | _ __ __ _ ___| | _____ _ __

514 Dec 21, 2022
An example file showing a simple endpoints like a login/logout function and maybe some others.

Flask API Example An example project showing a simple endpoints like a login/logout function and maybe some others. How to use: Open up your IDE (or u

Kevin 1 Oct 27, 2021
On this repo, you'll find every codes I made during my NSI classes (informatical courses)

👨‍💻 👩‍💻 school-codes On this repo, you'll find every codes I made during my NSI classes (informatical courses) French for now since this repo is d

EDM 1.15 3 Dec 17, 2022
Improving the Transferability of Adversarial Examples with Resized-Diverse-Inputs, Diversity-Ensemble and Region Fitting

Improving the Transferability of Adversarial Examples with Resized-Diverse-Inputs, Diversity-Ensemble and Region Fitting

Junhua Zou 7 Oct 20, 2022
ROS Foxy + Raspi + Adafruit BNO055

ROS Foxy + Raspi + Adafruit BNO055

Ar-Ray 3 Nov 04, 2022
A lightweight and unlocked launcher for Lunar Client made in Python.

LCLPy LCL's Python Port of Lunar Client Lite. Releases: https://github.com/Aetopia/LCLPy/releases Build Install PyInstaller. pip install PyInstaller

21 Aug 03, 2022
Socorro is the Mozilla crash ingestion pipeline. It accepts and processes Breakpad-style crash reports. It provides analysis tools.

Socorro Socorro is a Mozilla-centric ingestion pipeline and analysis tools for crash reports using the Breakpad libraries. Support This is a Mozilla-s

Mozilla Services 552 Dec 19, 2022
Hacking and Learning consistently for 100 days straight af.

#100DaysOfHacking Hacking and Learning consistently for 100 days straight af. [yes, no breaks except mental-break ones, Obviously.] This Repo is one s

FENIL SHAH 17 Sep 09, 2022
Script to automate the scanning of "old printed photos"

photoscanner Script to automate the scanning of "old printed photos" Just run: ./scan_photos.py The script is prepared to be run by fades. Otherw

Facundo Batista 2 Jan 21, 2022
All you need to understand CRUD and MVP in DRF

Book-Store-API This an API which has been put in place just to make you order for books, upload books with price, image and all, pay and automtically

Oladipo Adesiyan 6 Jul 03, 2022
Coded in Python 3 - I make for education, easily clone simple website.

Simple Website Cloner - Single Page Coded in Python 3 - I make for education, easily clone simple website. How to use ? Install Python 3 first. Instal

Phạm Đức Thanh 2 Jan 13, 2022
Project of the MSEC_LDD . group

HackathonJuntionXHN Project of team MSEC_LQĐ What did we do? Building application to generate whitelist regex for Web application firewall How to setu

Nguyễn Mạnh Cường 0 Dec 19, 2021
Process GPX files (adding sensor metrics, uploading to InfluxDB, etc.) exported from imxingzhe.com

Xingzhe GPX Processor 行者轨迹处理工具 Xingzhe sells cheap GPS bike meters with sensor support including cadence, heart rate and power. But the GPX files expo

Shengqi Chen 8 Sep 23, 2022
Introduction to Databases Coursework 2 (SQL) - dataset generator

Introduction to Databases Coursework 2 (SQL) - dataset generator This is python script generates a text file with insert queries for the schema.sql fi

Javier Bosch 1 Nov 08, 2021
Strong Typing in Python with Decorators

typy Strong Typing in Python with Decorators Description This light-weight library provides decorators that can be used to implement strongly-typed be

Ekin 0 Feb 06, 2022
A rough GSL work DynSAGE of my graduation project

DynSAGE Codes w.r.t DynSAGE-Diffuse can be found in function apply_dyn_model_v2 of src/utils.py. The training entrance is Line 144 - 155 of src/main.p

Yuhan Wang 3 Mar 22, 2022
LSO, also known as Linux Swap Operator, is a software with both GUI and terminal versions that you can manage the Swap area for Linux operating systems.

LSO - Linux Swap Operator Türkçe - LSO Nedir? LSO, diğer adıyla Linux Swap Operator Linux işletim sistemleri için Swap alanını yönetebileceğiniz hem G

Eren İnce 4 Feb 09, 2022
A partial-transpiler that converts a subset of Python to the Folders esoteric programming language

Py2Folders A partial-transpiler that converts a subset of Python to the Folders esoteric programming language Folders Folders is an esoteric programmi

Daniel Johnson 1 Dec 23, 2021