A fast PostgreSQL Database Client Library for Python/asyncio.

Overview

asyncpg -- A fast PostgreSQL Database Client Library for Python/asyncio

GitHub Actions status

asyncpg is a database interface library designed specifically for PostgreSQL and Python/asyncio. asyncpg is an efficient, clean implementation of PostgreSQL server binary protocol for use with Python's asyncio framework. You can read more about asyncpg in an introductory blog post.

asyncpg requires Python 3.5 or later and is supported for PostgreSQL versions 9.5 to 13. Older PostgreSQL versions or other databases implementing the PostgreSQL protocol may work, but are not being actively tested.

Documentation

The project documentation can be found here.

Performance

In our testing asyncpg is, on average, 3x faster than psycopg2 (and its asyncio variant -- aiopg).

https://raw.githubusercontent.com/MagicStack/asyncpg/master/performance.png

The above results are a geometric mean of benchmarks obtained with PostgreSQL client driver benchmarking toolbench in November 2020 (click on the chart to see full details).

Features

asyncpg implements PostgreSQL server protocol natively and exposes its features directly, as opposed to hiding them behind a generic facade like DB-API.

This enables asyncpg to have easy-to-use support for:

  • prepared statements
  • scrollable cursors
  • partial iteration on query results
  • automatic encoding and decoding of composite types, arrays, and any combination of those
  • straightforward support for custom data types

Installation

asyncpg is available on PyPI and has no dependencies. Use pip to install:

$ pip install asyncpg

Basic Usage

import asyncio
import asyncpg

async def run():
    conn = await asyncpg.connect(user='user', password='password',
                                 database='database', host='127.0.0.1')
    values = await conn.fetch(
        'SELECT * FROM mytable WHERE id = $1',
        10,
    )
    await conn.close()

loop = asyncio.get_event_loop()
loop.run_until_complete(run())

License

asyncpg is developed and distributed under the Apache 2.0 license.

Comments
  • Support pyformat and keyword args

    Support pyformat and keyword args

    It looks like for prepared statements, statements have to be in the form 'select $1 from $2 where baz=$3 with the arguments ('foo', 'bar', 'baz')

    It is common in python database api's to support the pyformat int PEP 249. http://legacy.python.org/dev/peps/pep-0249/#paramstyle

    Can this support at least one of the param styles in the pep? Preferably pyformat.

    Ideally, the parameters could then be passed in as a dictionary instead of a ordered list.

    conn.fetch(myquery, {bar='bar', foo='foo'})

    enhancement feedback 
    opened by nhumrich 49
  • SSL Certificate Verify Failed

    SSL Certificate Verify Failed

    • 13.0:
    • 9.6.1:
    • Do you use a PostgreSQL SaaS? If so, which? Can you reproduce the issue with a local PostgreSQL install? Yes, Heroku.:
    • 3.6.3:
    • Windows 10:
    • Do you use pgbouncer? No:
    • Did you install asyncpg with pip? Yes:

    I was getting the same issue as #119 . I enabled the ssl=True in the await asyncpg.create_pool(self.dsn, ssl=True) - and the same occurs with connect.

    So then I used the SSL keyword param, and got ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:777).

    opened by modelmat 38
  • Invalidate type cache on schema changes affecting statement result.

    Invalidate type cache on schema changes affecting statement result.

    Almost all explanation is written in the commit message.

    The rest I want to mention is at least one feature is left untouched. It is the BaseCursor where the bind_execute and bind calls should be wrapped by the same way the bind_execute in the PreparedStatement.__bind_execute was, but I'd leave it for the other commit (may be by the other author).

    opened by vitaly-burovoy 35
  • asyncpg.exceptions.InvalidSQLStatementNameError: prepared statement

    asyncpg.exceptions.InvalidSQLStatementNameError: prepared statement "__asyncpg_stmt_37__" does not exist

    • asyncpg version: 0.9.0, 0.10.1
    • PostgreSQL version: 9.6
    • Python version: 3.6
    • Platform: FreeBSD 10.2
    • Do you use pgbouncer?: ~no~ yes
    • Did you install asyncpg with pip?: yes
    • If you built asyncpg locally, which version of Cython did you use?: yes
    • Can the issue be reproduced under both asyncio and uvloop?: yes

    Hi!

    We notice a problem that issue name describes pretty well. How did we get it?

    We have a small aiohttp service which runs a static query to PostgreSQL database on every handled request. Like SELECT foo, bar FROM table ORDER BY baz. No parameters, nothing all. That's the only query we executre to database. We're using connection pool sized from 1 to 4 connections as well, the rest parameters are default (except connection one of course).

    Everything works fine until eventually we starting get the InvalidSQLStatementNameError exception on connection.fetch call. It doesn't happens instantly, it occurs after few hours of service uptime. Sometimes every .fetch call ends with that exception. Sometimes every even only. The number of __asyncpg_stmt_ token is always different, but stable until restart when error occurs.

    We tried to disable statements cache at all, set it to zero. This didn't happened. We thought that 0.10 release will fix that problem. Suddenly, it doesn't. We played around the cache size, connection pool size, connection life time, but all ended with the same issue. Difference may be in dozen of minutes, but all ends similar.

    It's quite hard for me to provide any example to reproduce the problem because everything just works for a quite long time. Except that particular service, which isn't quite different, doesn't uses any hacks, globals, pretty trivial. Just no idea what to try next.

    The last bits of the traceback are here:

          File "/usr/local/lib/python3.6/site-packages/asyncpg/connection.py", line 340, in fetch
            return await self._execute(query, args, 0, timeout)
          File "/usr/local/lib/python3.6/site-packages/asyncpg/connection.py", line 651, in _execute
            return await self._do_execute(query, executor, timeout)
          File "/usr/local/lib/python3.6/site-packages/asyncpg/connection.py", line 672, in _do_execute
            result = await executor(stmt, None)
          File "asyncpg/protocol/protocol.pyx", line 170, in bind_execute (asyncpg/protocol/protocol.c:57679)
        asyncpg.exceptions.InvalidSQLStatementNameError: prepared statement "__asyncpg_stmt_37__" does not exist
    

    Any ideas how to debug this issue? Or workaround it? Would be grateful for any help.

    need more info 
    opened by kxepal 33
  • Compatibility with pgbouncer with pool_mode != session

    Compatibility with pgbouncer with pool_mode != session

    Add session parameter to connection() with default value True. When set to False the client will limit usage of prepared statements to unnamed in a transaction only.

    As a side effect handling of unnamed prepared statements improved irrespective of session value - they are automatically reprepared if something invalidating them occurs.

    opened by aleksey-mashanov 28
  • Batch executemany

    Batch executemany

    Refs #289, this is a rewrite of bind_execute_many():

    • Batch bind args into 4 x 32KB buffers, sent with writelines() once flow control turns green.
    • Send one SYNC at the very last - making use of the implicit transaction for the whole call to executemany().
    • Abort immediately on either client error (with explicit rollback) or server error.
    • ~Support async iterable.~
    • Add executemany() to prepared statement.
    • [x] test_server_failure_during_writes is failing on Windows
    • [x] Fix unstable test_timeout (test_execute.TestExecuteMany)

    This is a breaking change due to the implicit transaction.

    executemany

    The 3-level infinite loop keeps filling buffers and send them under flow control, the only exit is raising StopIteration with 3 possible values:

    • True means there was data sent in the last loop
    • False means the last loop sent nothing
    • exception means the data source raised an exception

    Please note, during the loop, if server-side error is detected before exhausting the data source, it is also considered as StopIteration(True), ending the process early with a SYNC_MESSAGE.

    (I wrote a post about the details of implicit transaction.)

    UPDATED - pgbench results of inserting 1000 rows per query with executemany() on Python 3.6 of 2.2GHz 2015 MacBook Air (best out of 5 runs):

    asyncpg 0.18.2

    710 queries in 30.31 seconds
    Latency: min 341.88ms; max 636.29ms; mean 425.022ms; std: 39.782ms (9.36%)
    Latency distribution: 25% under 401.67ms; 50% under 414.26ms; 75% under 435.37ms; 90% under 478.39ms; 99% under 576.638ms; 99.99% under 636.299ms
    Queries/sec: 23.42
    Rows/sec: 23424.32
    

    This PR:

    4125 queries in 30.02 seconds
    Latency: min 23.14ms; max 734.91ms; mean 72.723ms; std: 49.226ms (67.69%)
    Latency distribution: 25% under 59.958ms; 50% under 65.414ms; 75% under 71.538ms; 90% under 80.95ms; 99% under 175.375ms; 99.99% under 734.912ms
    Queries/sec: 137.39
    Rows/sec: 137389.64
    

    Most branches are covered by tests, overall coverage 85% (1352 miss) -> 85% (1356 miss).

    UPDATED - Python 3.8 of 2.3GHz 2020 MacBook Pro:

    截屏2020-11-24 17 21 26
    opened by fantix 22
  • Awaiting multiple `execute` within a transaction seems to block forever

    Awaiting multiple `execute` within a transaction seems to block forever

    I'm trying to execute a couple of sql queries with arguments, in a context of a transaction.

    I used this code:

     async def execute_many_in_transaction(self, queries_and_args):
        pool = await get_pool() # returns a pool of clients
        async with pool.acquire() as connection:
            async with connection.transaction():
                for query, args in queries_and_args:
                    await connection.execute(
                        query, *args
                    )
    

    When I call it with a list with one tuple [("SQL STATEMENT", [arg1, arg2])] all is good.

    The moment I try to execute more than one, it just hangs there forever. For example:

    [
        ("SQL STATEMENT 1", [arg1, arg2]),
        ("SQL STATEMENT 2", [arg1, arg3]),
    ]
    

    I know that there is execute_many method, but that only seems to work on the same sql statement with multiple lists of arguments.

    Is there something I'm misunderstanding ?

    Thanks !

    need more info 
    opened by seeker89 22
  • asyncpg.exceptions._base.InterfaceError: cannot perform operation: another operation is in progress

    asyncpg.exceptions._base.InterfaceError: cannot perform operation: another operation is in progress

    Hi, here is my code ,please help resovle the problem.thanks a lot.

    from sanic import Sanic
    from sanic.response import json
    import asyncpg
    import asyncio
    import uvloop
    loop = uvloop.new_event_loop()
    app = Sanic()
    app.debug = True
    
    async def initdb_pool():
        dbdict = {"database":"mqtt","user":"mqtt","password":"mqtt123",
                "host":"192.168.25.100","port":5433}
        return await asyncpg.create_pool(**dbdict)
    
    @app.route("/iot/v1.0/app/auth/<user:[A-z0-9]\w+>/<pwd:[A-z0-9]\w+>/")
    async def applogin(request,user,pwd):
        async with engine.acquire() as connection:
            #async with connection.transaction():
            stmt   = await connection.prepare('select key from user_manager_appuser where uname = $1 or email = $2 or phone = $3')
            result =  await stmt.fetchval(user,user,user)
            #    result = await connection.fetchval('select key from user_manager_appuser where uname = $1 or email = $2 or phone = $3',
            #                                        user,user,user)
            if not result:
                return json({'ok':False,'err':'Pwd error'})
            print("result is ",result)
            return json({'ok':True,'data':str(result)})
    
    if __name__ == "__main__":
        engine = loop.run_until_complete(initdb_pool())
        app.run(host="0.0.0.0",port=8000,debug=True)
    
    

     curl "http://127.0.0.1:8000/iot/v1.0/app/auth/abc/123456/"
    Error: cannot perform operation: another operation is in progress
    Exception: Traceback (most recent call last):
      File "/media/yjdwbj/E/py360dev/lib/python3.6/site-packages/sanic/sanic.py", line 194, in handle_request
        response = await response
      File "/opt/python360/lib/python3.6/asyncio/coroutines.py", line 128, in throw
        return self.gen.throw(type, value, traceback)
      File "main.py", line 32, in applogin
        return json({'ok':True,'data':str(result)})
      File "/opt/python360/lib/python3.6/asyncio/coroutines.py", line 109, in __next__
        return self.gen.send(None)
      File "/media/yjdwbj/E/py360dev/lib/python3.6/site-packages/asyncpg/pool.py", line 252, in __aexit__
        await self.pool.release(con)
      File "/opt/python360/lib/python3.6/asyncio/coroutines.py", line 109, in __next__
        return self.gen.send(None)
      File "/media/yjdwbj/E/py360dev/lib/python3.6/site-packages/asyncpg/pool.py", line 184, in release
        await connection.reset()
      File "/opt/python360/lib/python3.6/asyncio/coroutines.py", line 109, in __next__
        return self.gen.send(None)
      File "/media/yjdwbj/E/py360dev/lib/python3.6/site-packages/asyncpg/connection.py", line 411, in reset
        ''')
      File "/opt/python360/lib/python3.6/asyncio/coroutines.py", line 109, in __next__
        return self.gen.send(None)
      File "/media/yjdwbj/E/py360dev/lib/python3.6/site-packages/asyncpg/connection.py", line 170, in execute
        return await self._protocol.query(query, timeout)
      File "asyncpg/protocol/protocol.pyx", line 247, in query (asyncpg/protocol/protocol.c:51830)
      File "asyncpg/protocol/protocol.pyx", line 352, in asyncpg.protocol.protocol.BaseProtocol._ensure_clear_state (asyncpg/protocol/protocol.c:54136)
    asyncpg.exceptions._base.InterfaceError: cannot perform operation: another operation is in progress
    
    need more info 
    opened by yjdwbj 22
  • [RFC] Combine args of `executemany()` in batches

    [RFC] Combine args of `executemany()` in batches

    Is it a good idea to combine all the args in executemany() into a ~single network packet~ single buffer to send (thx Yury) with a list of Bind and Execute command pairs? And, is it a good idea to make this method atomic? Please kindly give me some advices when time. Many thanks!

    • Ends with only one Sync command, that means executemany() will be atomic in an implicit transaction, if not called from an existing transaction.
    • This should reduce round-trip time to database back and forth, with a cost that ...
    • memory usage is no longer smooth - it will have to encode all the bind args into memory before sending to database in a batch.
      • Theoretically it is possible to combine the args into small groups, in order for a balance between reduced RTT and memory usage, as well as to detect an error earlier.
    • Assuming that reducing RTT is helpful, this RFC also added executemany() to prepared statement.

    References:

    • https://www.postgresql.org/docs/10/static/protocol-flow.html#PROTOCOL-FLOW-EXT-QUERY
    • https://www.pgcon.org/2014/schedule/attachments/330_postgres-for-the-wire.pdf
    • https://www.youtube.com/watch?v=qa22SouCr5E
    • https://github.com/MagicStack/asyncpg/issues/36
    opened by fantix 19
  • Cockroachdb connection pooling

    Cockroachdb connection pooling

    • asyncpg version: 0.9.0
    • PostgreSQL version: Cockroachdb beta-20170223
    • Python version: Python 3.6.0
    • Platform: latest OS X
    • Do you use pgbouncer?: no
    • Did you install asyncpg with pip?: yes
    • If you built asyncpg locally, which version of Cython did you use?: 0.25.2
    • Can the issue be reproduced under both asyncio and uvloop?: not related

    I'm trying to use cockroachdb with asyncpg.

    With plain connection it works like a charm. On the other hand using connection pool fails with the following error:

    ERROR:sanic:Traceback (most recent call last):
      File "/Users/krisz/.pyenv/versions/miniconda3-latest/envs/ml3/lib/python3.6/site-packages/sanic/app.py", line 391, in handle_request
        response = await response
      File "/Users/krisz/Workspace/papi/papi/app.py", line 34, in accounts_create
        print(results)
      File "/Users/krisz/.pyenv/versions/miniconda3-latest/envs/ml3/lib/python3.6/site-packages/asyncpg/pool.py", line 257, in __aexit__
        await self.pool.release(con)
      File "/Users/krisz/.pyenv/versions/miniconda3-latest/envs/ml3/lib/python3.6/site-packages/asyncpg/pool.py", line 189, in release
        await connection.reset()
      File "/Users/krisz/.pyenv/versions/miniconda3-latest/envs/ml3/lib/python3.6/site-packages/asyncpg/connection.py", line 412, in reset
        ''')
      File "/Users/krisz/.pyenv/versions/miniconda3-latest/envs/ml3/lib/python3.6/site-packages/asyncpg/connection.py", line 171, in execute
        return await self._protocol.query(query, timeout)
      File "asyncpg/protocol/protocol.pyx", line 255, in query (asyncpg/protocol/protocol.c:56799)
    asyncpg.exceptions.InternalServerError: syntax error at or near "DO"
    

    Caused by the resetting the connection :

    await self.execute('''
                DO $$
                BEGIN
                    PERFORM * FROM pg_listening_channels() LIMIT 1;
                    IF FOUND THEN
                        UNLISTEN *;
                    END IF;
                END;
                $$;
                SET SESSION AUTHORIZATION DEFAULT;
                RESET ALL;
                CLOSE ALL;
                SELECT pg_advisory_unlock_all();
            ''')
    

    I don't know the exact solution, but I'd like to use Cockroach with asyncpg :)

    I've created an issue for cockroach developers too https://github.com/cockroachdb/examples-orms/issues/19

    enhancement 
    opened by kszucs 18
  • Ensure pool connection is released in acquire when cancelled

    Ensure pool connection is released in acquire when cancelled

    Closes #547 and possibly #467

    When wait_for is cancelled, there is a chance that the waited task has already been completed, leaving the connection looking like it is in use. This fix ensures that the connection is returned to the pool in this situation.

    A similar bug was fixed in #468, also with wait_for. The python issue was https://bugs.python.org/issue37658, although this is arguably not a solvable bug in Python itself as it can't know how to undo a completed task.

    opened by aaliddell 17
  • asyncpg.exceptions._base.InterfaceError: cannot call Connection.fetch(): connection has been released back to the pool

    asyncpg.exceptions._base.InterfaceError: cannot call Connection.fetch(): connection has been released back to the pool

    • asyncpg version: 0.26.0
    • PostgreSQL version: 12.0
    • Do you use a PostgreSQL SaaS? If so, which? Can you reproduce : NO, from pg docker the issue with a local PostgreSQL install?: YES,
    • Python version: Python 3.9.13
    • Platform: MAC
    • Do you use pgbouncer?: No
    • Did you install asyncpg with pip?: From tortoise-orm
    • If you built asyncpg locally, which version of Cython did you use?: No
    • Can the issue be reproduced under both asyncio and uvloop?: YES

    I am using Sanic + tortoise-orm,

    the test code as below:

    image

    when runs a query during the Request handler, it runs successfully

    while runs in a background task, it report this error:

    asyncpg.exceptions._base.InterfaceError: cannot call Connection.fetch(): connection has been released back to the pool

    I found when the first time query, through the pool.acquire, got the right <PoolConnectionProxy <asyncpg.connection.Connection object at 0x7fd42102f9e0> 0x7fd3d018e550>

    but when go to the background task query , the pool.acquire, got an "released" <PoolConnectionProxy [released] 0x7fd3d018e550>

    I am confused how can I got an released PoolConnectionProxy from the same code

    [query in the get Request method] query in get

    [query in the background task] query in task

    [the pool connection acquire method] acquire

    [the pool connection release method] release

    opened by iyuhang 0
  • Target session attr (2)

    Target session attr (2)

    Recreating this PR because it seems to have gone silent. The target is to implement the target_session_attribute keyword when connecting to a database cluster. Supported options are

    • any
    • primary
    • standy
    • prefer_standby
    • read-write
    • read-only
    opened by JesseDeLoore 1
  • AsyncPG with twitchAPI error

    AsyncPG with twitchAPI error

    • asyncpg version: 0.27.0
    • PostgreSQL version: 14
    • Do you use a PostgreSQL SaaS? If so, which? Can you reproduce the issue with a local PostgreSQL install?: No
    • Python version: 3.10.2
    • Platform: Windows
    • Do you use pgbouncer?: No
    • Did you install asyncpg with pip?: Yes
    • If you built asyncpg locally, which version of Cython did you use?: No
    • Can the issue be reproduced under both asyncio and uvloop?: Not tested

    Hello! This is not really a bug report but more of a help wanted. I am trying to use the twitchAPI library to listen to stream online events (whenever a streamer goes online). This requires a callback in which i want to access my database to retrieve some information which I need to use (I am making a discord bot btw). Whenever I try to call my database I get the following error:

    (<class 'asyncpg.exceptions.ConnectionDoesNotExistError'>, ConnectionDoesNotExistError('connection was closed in the middle of operation'), None)
    cannot perform operation: another operation is in progress Traceback (most recent call last):
      File "C:\Users\User\Documents\Coding Projects\Project\project\utils\DatabaseHandler.py", line 1683, in get_guilds_by_twitch_account
        async with conn.transaction():
      File "C:\Users\User\Documents\Coding Projects\Project\.venv\lib\site-packages\asyncpg\transaction.py", line 62, in __aenter__
        await self.start()
      File "C:\Users\User\Documents\Coding Projects\Project\.venv\lib\site-packages\asyncpg\transaction.py", line 138, in start
        await self._connection.execute(query)
      File "C:\Users\User\Documents\Coding Projects\Project\.venv\lib\site-packages\asyncpg\connection.py", line 317, in execute
        return await self._protocol.query(query, timeout)
      File "asyncpg\protocol\protocol.pyx", line 338, in query
    RuntimeError: Task <Task pending name='Task-118' coro=<TwitchHandler.on_stream_online() running at C:\Users\User\Documents\Coding Projects\Project\project\extentions\module_interactions\module_socials\TwitchHandler.py:104>> got Future <Future pending cb=[Protocol._on_waiter_completed()]> attached to a different loop
    
    During handling of the above exception, another exception occurred:
    
    Traceback (most recent call last):
      File "C:\Users\User\Documents\Coding Projects\Project\project\extentions\module_interactions\module_socials\TwitchHandler.py", line 104, in on_stream_online
        all_corresponding_guilds = await self.db.get_guilds_by_twitch_account(twitch_account=twitch_account)
      File "C:\Users\User\Documents\Coding Projects\Project\project\utils\DatabaseHandler.py", line 1682, in get_guilds_by_twitch_account
        async with self._pool.acquire() as conn:
      File "C:\Users\User\Documents\Coding Projects\Project\.venv\lib\site-packages\asyncpg\pool.py", line 220, in release
        raise ex
      File "C:\Users\User\Documents\Coding Projects\Project\.venv\lib\site-packages\asyncpg\pool.py", line 210, in release
        await self._con.reset(timeout=budget)
      File "C:\Users\User\Documents\Coding Projects\Project\.venv\lib\site-packages\asyncpg\connection.py", line 1366, in reset
        await self.execute(reset_query, timeout=timeout)
      File "C:\Users\User\Documents\Coding Projects\Project\.venv\lib\site-packages\asyncpg\connection.py", line 317, in execute
        return await self._protocol.query(query, timeout)
      File "asyncpg\protocol\protocol.pyx", line 323, in query
      File "asyncpg\protocol\protocol.pyx", line 707, in asyncpg.protocol.protocol.BaseProtocol._check_state
    asyncpg.exceptions._base.InterfaceError: cannot perform operation: another operation is in progress
    

    Is there a way to fix this issue or somekind of workaround?

    opened by l3keboy 0
  • Connection methods does not accept classic nor unusual arguments supported in other pgsql connection solution

    Connection methods does not accept classic nor unusual arguments supported in other pgsql connection solution

    Hello,

    While trying to configure a Cockroach DB cluster async connection, there seems to be missing support for usual PostgreSQL parameters (or renamed) or supported options not frequently used.

    I don't know why the connection part does not seems to follow other library but it would be nice to at least support all options existing in postgresql client (even renamed ^^).

    For example, the part I am missing to connect is options keyword. It is used to provide complementary informations on the connection (in my case a cluster id to identify exact connection). Guess it is not mandatory by itself but it fells strange to have the issue only on asyncpg :/

    • asyncpg version: Version: 0.26.0
    • PostgreSQL version:
    • Do you use a PostgreSQL SaaS? If so, which? Can you reproduce the issue with a local PostgreSQL install?: Cockroach Cloud
    • Python version: 3.10.7 (venv)
    • Platform: Linux (Docker VSCode python dev container)
    • Do you use pgbouncer?: No
    • Did you install asyncpg with pip?: Yes (using setuptools)
    • If you built asyncpg locally, which version of Cython did you use?:
    • Can the issue be reproduced under both asyncio and uvloop?:
    opened by titouanfreville 7
  • Issuing Advisory Lock

    Issuing Advisory Lock

    • asyncpg version: 0.25.0
    • PostgreSQL version: 12.11
    • Do you use a PostgreSQL SaaS? If so, which? Can you reproduce the issue with a local PostgreSQL install?:
    • Python version: 3.9
    • Platform: Linux
    • Do you use pgbouncer?: No
    • Did you install asyncpg with pip?: yes
    • If you built asyncpg locally, which version of Cython did you use?: No
    • Can the issue be reproduced under both asyncio and uvloop?: NA

    Receiving this error

    asyncio.exceptions.TimeoutError
    [View similar errors](https://link.datadoghq.com/apm/error-tracking?issueId=...471-da7ad0900002)
    
    Traceback (most recent call last):
      File /usr/local/lib/python3.9/site-packages/ddtrace/contrib/asyncpg/patch.py, line 89, in _traced_query
        return await method(*args, **kwargs)
    File asyncpg/protocol/protocol.pyx, line 338, in query
    asyncio.exceptions.TimeoutError
    

    Due to internal query call: SELECT pg_advisory_unlock_all ( ) CLOSE ALL UNLISTEN * RESET ALL

    Happening quite frequently

    opened by jageshmaharjan 1
  • Disconnection from postgres-compatible google cloud AlloyDB instance hangs

    Disconnection from postgres-compatible google cloud AlloyDB instance hangs

    • asyncpg version: 0.26.0
    • PostgreSQL version: Google AlloyDB, postgres 14.4 compatible
    • Do you use a PostgreSQL SaaS? If so, which? Can you reproduce the issue with a local PostgreSQL install?: I cannot reproduce the issue on a local postgres, no.
    • Python version: 3.10
    • Platform: Macos
    • Do you use pgbouncer?: No
    • Did you install asyncpg with pip?: Yes
    • If you built asyncpg locally, which version of Cython did you use?: NA
    • Can the issue be reproduced under both asyncio and uvloop?: I have not tried to reproduce under uvloop

    I am connecting to an AlloyDB instance on Google Cloud (currently, these are available for free testing). I can connect and queries run as expected. However, connection.disconnect() hangs. I cannot reproduce this behavior locally; disconnection happens as expected. Synchronous database work with psycopg2, for example, works as expected.

    This code reproduces the behavior I am seeing (running through the alloydb-auth-proxy as is required):

    import asyncpg
    conn = await asyncpg.connect(ALLOYDB_URI)
    conn.disconnect() # hangs
    

    I'll put in a ticket with Google as well if nothing obvious turns up here.

    opened by seandavi 1
Releases(v0.27.0)
  • v0.27.0(Oct 26, 2022)

    Support Python 3.11 and PostgreSQL 15. This release also drops support for Python 3.6.

    Changes

    • Add arm64 mac and linux wheels (by @ddelange in 7bd6c49f for #954)

    • Add Python 3.11 to the test matrix (by @elprans in 5f908e67 for #948)

    • Exclude .venv from flake8 (#958) (by @jparise in 40b16ea6 for #958)

    • Upgrade to flake8 5.0.4 (from 3.9.2) (#961) (by @jparise in 0e73fec2 for #961)

    • Show an example of a custom Record class (#960) (by @jparise in 84c99bfd for #960)

    • Use the exact type name in Record.repr (#959) (by @jparise in eccdf61a for #959)

    • Drop Python 3.6 support (#940) (by @bryanforbes in bb0cb39d for #940)

    • Test on Python 3.11 and PostgreSQL 15, fix workflow deprecations (#968) (by @elprans in eab7fdf2 for #968)

    Source code(tar.gz)
    Source code(zip)
  • v0.26.0(Jul 7, 2022)

    Changes

    • Add support to use awaitable object in password function. (#889) (by @kwarunek in fb3b6bf7 for #889)

    • Support direct TLS connections (i.e. no STARTTLS) (#923) (by @jackwotherspoon in f2a937d2 for #923)

    Fixes

    • Fix invalid pyproject.toml (#900) (by @Rongronggg9 in eddb649c for #900)

    • Add record_class parameter Pool.fetch and Pool.fetchrow (#896) (by @baltitenger in 2519cf38 for #896)

    • Domain basetypes are introspected (#886) (#887) (by @QuantumTM in cca4a2d3 for #886)

    • Properly handle exceptions raised while handling server auth messages (#862) (by @elprans in bd192623 for #862)

    Source code(tar.gz)
    Source code(zip)
  • v0.25.0(Nov 16, 2021)

    Changes

    • Improve SSL option compatibility in URIs (by @fantix in 383c711e for #827)

    • Add Pool methods to determine its min, max, current and idle size (by @elprans in 603e3868 for #849)

    • Make it possible to specify a statement name in Connection.prepare() (by @elprans in 03a3d18f for #846)

    • Implement support for multirange types (by @elprans in d64a44a1 for #851)

    Fixes

    • Make sure timeout callbacks always get cleaned up (by @elprans in dad26913 for #831)

    • Update __all__ statements to a simpler form that is better supported by typecheckers (by @bschnurr in 0a3ae7f5 for #828)

    • Fix test_timetz_encoding on Python 3.10 (by @elprans in 3a90fef0)

    • Fix a bunch of ResourceWarnings in the test suite (by @elprans in 2f4fe539)

    • Fix SSLContext deprecation warnings (by @elprans in 4d39a052)

    • Fix the description of the database argument to connect() (by @elprans in a2a92374 for #847)

    • Fix parsing of IPv6 addresses in the connection URI (by @elprans in f900b737 for #845)

    • Improve diagnostics of invalid executemany() input (by @elprans in a8fc21e0 for #848)

    Source code(tar.gz)
    Source code(zip)
  • v0.24.0(Aug 10, 2021)

    Changes

    • Drop support for Python 3.5 (#777) (by @and-semakin in da58cd26 for #777)

    • Add support for Python 3.10 (#795) (by @elprans in abf55699 for #795)

    • Add support for asynchronous iterables to copy_records_to_table() (#713) (by @elprans in 1d33ff62 for #713)

    • Add support for coroutine functions as listener callbacks (#802) (by @elprans in 41da093e for #802)

    • Add support for sslcert, sslkey and sslrootcert parameters to DSN (#768) (by @jdobes and @elprans in c674e86a for #768)

    • Add copy_ wrappers to Pool (#661) (by @elprans in a6b0f283 for #661)

    • Add issubset and issuperset methods to the Range type (#563) (by @kdorsel in de07d0ab for #563)

    Fixes

    • Break connection internal circular reference (#774) (by @fantix in d08a9b8b for #774)

    • Make Server Version Extraction More Flexible (#778) (by @Natrinicle in d0761694 for #778)

    Source code(tar.gz)
    Source code(zip)
  • v0.23.0(May 17, 2021)

    Fixes

    • Avoid TypeError in Transaction.__repr__ (#703) (by @BeatButton in d6eea8ed for #703)

    • Feed memoryview to writelines() (#715) (by @fantix in 359a34c4 for #715)

    • Add sslmode=allow support and fix =prefer retry (#720) (by @fantix in 075114c1 for #720)

    • Loosen message test in test_invalid_input (#751) (by @musicinmybrain in bc4127f4 for #751)

    • Support readonly and deferrable for non-serializable transactions (#747) (by @pauldraper in 5cf4089a for #747)

    • Fix asyncpg with Py_DEBUG mode (#719) (by @shadchin in a113d908 for #719)

    • Fix docs/Makefile and docs/_static/theme_overrides.css missing from PyPI package (#708) (by @musicinmybrain in c3060680 for #708)

    Source code(tar.gz)
    Source code(zip)
  • v0.22.0(Feb 10, 2021)

    A new asyncpg release is here.

    Notable additions include Python 3.9 support, support for recently added PostgreSQL types like jsonpath, and last but not least, vastly improved executemany() performance. Importantly, executemany() is also now atomic, which means that either all iterations succeed, or none at all, whereas previously partial results would have remained in place, unless executemany() was called in a transaction.

    There is also the usual assortment of improvements and bugfixes, see the details below.

    This is the last release of asyncpg that supports Python 3.5, which has reached EOL last September.

    Improvements

    • Vastly speedup executemany by batching protocol messages (#295) (by @fantix in 690048db for #295)

    • Allow using custom Record class (by @elprans in db4f1a6c for #577)

    • Add Python 3.9 support (#610) (by @elprans in c05d7260 for #610)

    • Prefer SSL connections by default (#660) (by @elprans in 16183aa0 for #660)

    • Add codecs for a bunch of new builtin types (#665) (by @elprans in b53f0384 for #665)

    • Expose Pool as asyncpg.Pool (#669) (by @rugleb in 0e0eb8d3 for #669)

    • Avoid unnecessary overhead during connection reset (#648) (by @kitogo in ff5da5f9 for #648)

    Fixes

    • Add a workaround for bpo-37658 (by @elprans in 2bac166c for #21894)

    • Fix wrong default transaction isolation level (#622) (by @fantix in 4a627d55 for #622)

    • Fix set_type_codec() to accept standard SQL type names (#619) (by @elprans in 68b40cbf for #619)

    • Ignore custom data codec for internal introspection (#618) (by @fantix in e064f59e for #618)

    • Fix null/NULL quoting in array text encoder (#627) (by @fantix in 92aa8062 for #627)

    • Fix link in connect docstring (#653) (by @samuelcolvin in 8b313bde for #653)

    • Make asyncpg work with pyinstaller (#651) (by @Atem18 in 5ddabb19 for #651)

    • Fix possible AttributeError exception in ConnectionSettings (#632) (by @petriborg in 0d231820 for #632)

    • Prohibit custom codecs on domains (by @elprans in 50f964fc for #457)

    • Raise proper error on anonymous composite input (tuple arguments) (#664) (by @elprans in 7252dbeb for #664)

    • Fix incorrect application of custom codecs in some cases (#662) (by @elprans in 50f65fbb for #662)

    Source code(tar.gz)
    Source code(zip)
  • v0.21.0(Aug 10, 2020)

    Improvements

    • Add support for password functions (useful for RDS IAM auth) (#554) (by Harvey Frye in 1d9457f0 for #554)

    • Add support for connection termination listeners (#525) (by @iomintz in 8141b93c for #525)

    • Update CI matrix, aarch64 builds (#595) (by @Gelbpunkt in ac6a2fcf for #595)

    Fixes

    • Fix possible uninitalized pointer access on unexpected array message data (CVE-2020-17446, by @elprans in 69bcdf5b, reported by @risicle)

    • Fix Connection class _copy_in private method (by @ABCDeath in 7f5c2a24 for #555)

    • Bump pgproto to fix compilation issues (by @elprans in aa67d61b for #565)

    • Improve pool documentation examples (#491) (by @nyurik in 745f8f81 for #491)

    • Update usage.rst (#572) (by @xuedong09 in f5b425ae for #572)

    • Fix links in connection documentation (#584) (by @samuelcolvin in b0813204 for #584)

    • Fix usage documentation for hstore (#515) (by @aaliddell in 39040b3c for #515)

    • Fix compiler warnings (by @elprans in 6cb5ba19)

    Source code(tar.gz)
    Source code(zip)
  • v0.20.0(Nov 21, 2019)

    Improvements

    • Support Python 3.8 (by @1st1 in #504)

    • Support PgBouncer by sending only a single SYNC message per query (by @fvannee in b043fbd3)

    Bug Fixes

    • Handle IP values with prefix in "inet" type as ipaddress.IPvXInterface (by @elprans in 5a4daf71 for #497)

    • Close transport if connection times out (by @hexrain in 926f4833 for #468)

    • Use faster UUID codecs; make UUID decoding/encoding/operations 2-7x faster (by @1st1 in edde3ff4)

    • Use loop.start_tls() to upgrade connections to SSL (by @1st1 in bdba7ce7)

    Build

    • Bump Cython to 0.29.14 (by @1st1 in 7cb31bc6)
    Source code(tar.gz)
    Source code(zip)
  • v0.19.0(Oct 9, 2019)

    Improvements

    • Add support for SCRAM-SHA-256 authentication. (by @jkatz in 2d76f50d)

    • Add PostgreSQL 12 support (by @elprans in 23261532)

    Bug Fixes

    • Remove preexec_fn hack from test cluster management (by @elprans in 36ed11d2)

    • Fix DeprecationWarning in the docstring of copy_from_query() (by @elprans in 482a39ae)

    • Allow specifying the target version when generating the release log (by @elprans in 43a7b213)

    • Check for .flake8 after importing flake8 (by @dotlambda in aaeb7076)

    • Include .flake8 in PyPI tarball (by @dotlambda in 43c6b9ce)

    • fix timezone type label in docs (by @claws in e91e4911)

    • Fix _StatementCache.clear() PS memory leak (by @un-def in f4aa9bf4 for #416)

    • fix for warning_callback not yet defined (by @samuelcolvin in 354d9be5)

    • Fix assertion fail on copy_records_to_table (by Petr Reznikov in ae5a89db)

    • Do not look for a port in a Unix socket domain path (by @Lawouach in b773912d for #419)

    • Unquote connection string components properly (by @elprans in 5513b9d3 for #418)

    • Remove superfluous transaction from a cursor example (by @elprans in 32fccaad for #475)

    Source code(tar.gz)
    Source code(zip)
  • v0.18.2(Nov 10, 2018)

    Bug Fixes

    • Revert "Stop treating ReadyForQuery as a universal result indicator" to fix stability regression. (by @elprans in 04b67480)

    • Correctly ignore all incoming messages after the Terminate message (by @elprans in 787317fb)

    • Properly cleanup connections closed by remote (by @elprans in 4393a159 for #385)

    Source code(tar.gz)
    Source code(zip)
  • v0.18.1(Oct 31, 2018)

  • v0.18.0(Oct 30, 2018)

    Improvements

    • Implement Record.get() (by @elprans in 2b93ee55 for #330)

    • Internal asyncpg errors are now consistently raised as InternalClientError (by @elprans in 263de3ff)

    • Allow mappings as composite type input (by @elprans in eaa2fa1a for #349)

    • Add BitString.to_int() and BitString.from_int() (by @percontation in ffd134e8)

    • Allow 'sslmode' in DSNs (by @percontation in 0304288e)

    • Add support for specifying multiple host addresses when connecting (by @elprans in 1d650ed9 for #257)

    • Low-level protocol implementation has been tweaked and optimized for slightly better performance in certain scenarios (by @elprans in 7a816138 and cc053fe5)

    • Queries with cached statements now generate fewer TCP packets (by @aleksey-mashanov in bf071996)

    • Allow aliasing builtin types by name in set_builtin_type_codec() (by @elprans in 687127ed)

    • Factor out generic PostgreSQL protocol helpers into a separate package (by @1st1 in f0adefc7)

    • Fix tests and enable CI for PostgreSQL 11 (by @elprans in ddb0ec29 and 716fd9d3)

    Bug Fixes

    • Handle and ignore permission errors when attempting to read .pgpass (by @elprans in df7830f0 for #356)

    • Fix decoding of fractional timestamps before Postgres epoch (by @elprans in a7eaf2b3 for #363)

    Source code(tar.gz)
    Source code(zip)
  • v0.17.0(Jul 10, 2018)

    Improvements

    Official support for Python 3.7.

    Bug Fixes

    • Fix garbage collection of connections and emit a ResourceWarning if an unclosed connection is garbage collected. (by @1st1 in d9a236e7 for #323)

    • Raise a clear error if there's a race in pool intialization. (by @1st1 in 3565ef8c for #320)

    • Channel names in Connection.add_listener() and Connection.remove_listener() are now quoted properly. (by @sqwishy in 3e6ade62)

    • Fixed endianness detection on *BSD systems. (by @elprans in 8c83add4 for #313)

    • Fixed handling of large type OIDs. (by @elprans in 2624bdb9 for #300)

    Source code(tar.gz)
    Source code(zip)
  • v0.16.0(Jun 6, 2018)

    Behavior Changes

    Pool.close() now waits until all acquired connections are released. Hence, the below code will now hang indefinitely:

    conn = await pool.acquire()
    await pool.close()
    

    Asyncpg will log a warning if pool.close() takes over 60 seconds to complete. It is advisable to use asyncio.wait_for() to set a timeout.

    Improvements

    • Add support for reading passwords from .pgpass (by @elprans in 55a372fc for #267)

    • Add Connection.is_in_transaction() (#297) (by @bcaudell95 in cf523be7 for #297)

    • Added codec for built-in type tid (by @fantix in cddea9c1)

    • Allow setting custom codecs on domains and enumerated types (by @elprans in 3139322a)

    • Allow passing datetime.date instances as timestamp input (by @elprans in e8cc627a for #288)

    • Implement support for pool connection rotation (by @elprans in 4d209b75 for #291)

    Bug Fixes

    • Prohibit non-iterable containers to be passed as array input (by @elprans in a2fa7b23)

    • Decode numeric zeros with correct scale (by Dmitriy Chugunov in 4124f7d2)

    • Fix handling of OIDs >= 2**31 (by @elprans in 8dd7a6cb for #279)

    • Make Pool.close() wait until all checked out connections are released (by @elprans in 7a0585ac for #290)

    • Fix type codec cache races (by @elprans in 482a1866 for #278)

    • Raise a consistent exception on input encoding errors (by @elprans in 0ddfa466 for #260)

    Source code(tar.gz)
    Source code(zip)
  • v0.15.0(Feb 15, 2018)

    Behavior Changes

    asyncpg no longer uses the common statement cache for explicit prepared statements, so Connection.prepare() always returns a new prepared statement (by @vangheem in a19ce50f).

    Bug Fixes

    • Initialize statement codecs immediately after Prepare (by @elprans in 803c1155 for #241)

    • Fix server version parsing when it contains trailing data (by @elprans in 05dce25f for #250)

    Source code(tar.gz)
    Source code(zip)
  • v0.14.0(Dec 29, 2017)

    Backwards Incompatible Changes

    asyncpg now reports the server version as x.0.y for PostgreSQL 10.0 and later. This is in alignment with how upstream is interpreting the "minor" version component now. (@elprans in 1fa12fe2)

    Improvements

    • Support timeouts in Connection.close() and Pool.release() (by @elprans in bdfdd898 for #220)

    • Invalidate type cache on schema changes affecting statement result. A new Connection.reload_schema_state() method. (by @vitaly-burovoy in b04556e9)

    • Add CrateDB detection (by @mfussenegger in afc1038f)

    Bug Fixes

    • Make prepared statement uid generator global (by @1st1 in 3e43fcf5)

    • Document pgbouncer-related prepared statement breakage (by @elprans in 4a3713f7 for #121)

    • Fix unaligned access issues in host-network byte I/O on ARM (by @elprans in c04576db for #216)

    • Fix issues with timetz type I/O (by @vitaly-burovoy in 7b6c0833)

    • Consistently use postgres/postgres as database/user pair in tests (by @elprans in b4ce7403)

    • Fix Range.__hash__() (by @eumiro in 46f468c9)

    • Guard transaction methods against underlying connection release (by @elprans in 59e2878b for #232)

    Source code(tar.gz)
    Source code(zip)
  • v0.13.0(Oct 20, 2017)

    Backwards Incompatible Changes

    • Drop support for PostgreSQL 9.1 (by @elprans in eaf298b4)

    • Remove the deprecated "binary" parameter from Connection.set_type_codec() (by @elprans in 9ad6625a)

    New Features

    • Add support for PostgreSQL 10 (by @elprans)

    Bug Fixes

    • Document that single fetches can return None (by @khazhyk in 23394c9d)

    • Fix type introspection being very slow on large databases (by @elprans in e0c44ced for #186)

    • Fix copy tests on PostgreSQL < 9.4 (by @elprans in 327058f8)

    • Fix DeprecationWarning in tests (by @elprans in 3f9523c2)

    • Fix intermittent authentication failures against PostgreSQL 10 servers (by @elprans in 498213e6 for #158)

    • Do not attempt to connect to Unix sockets on Windows when host is not set (by @elprans in 8a32fc4b for #184)

    • Guard against incorrect use of resources associated with a connection (by @elprans in 089ac818, 93bf26f3 for #190)

    • Warn when there are active listeners on a connection that is released (by @elprans in d085d2cc for #190)

    • Add a test for cursors on a standby replica. (by @elprans in c484a470 for #66)

    • Clarify the argument/parameter count mismatch exception (by @elprans in b6fe0186 for #178)

    • Fix TypeError on get_attributes when result has no columns (by @fantix in f29de232)

    • protocol: Use try-finally explicitly every time we create a waiter (by @1st1 in 50edd8c8)

    • Call correct parent constructor in InterfaceWarning.__init__ (by @elprans in eec98b0b)

    • Allow grabbing the version information programmatically. (by @s0undt3ch in fa6dbc46)

    • Use the general statement cache for type introspection (by @elprans in 57c9ffdc for #198)

    • Fix issues with inet type I/O (by @elprans in e54f02e6 for #37)

    • Handle inactive connection closes while stored in the pool (by @AmatanHead in 9744adee)

    Source code(tar.gz)
    Source code(zip)
  • v0.12.0(Jul 7, 2017)

    New Features

    • Implement as_dict() method for PostgresMessage. (by @vitaly-burovoy in 90725f1f)

    • Add API for receiving log messages connection.add_log_listener(). (by @vitaly-burovoy and @1st1 in 1b1893df and f072f883 for #144)

    • Add support for "tuple" exchange format for codec overrides. (by @elprans in 0453243b and d27dda72)

    • Implement binary format codec for numeric type. (by @elprans in a5413eb0 for #157)

    Bug Fixes

    • Fix boundary checks for integers. (by @vitaly-burovoy in e0829104 and dfcf1356)

    • Fix handling of NULLs in copy_records_to_table(). (by @elprans in 991b1ae8 for #153)

    • Raise an error when a binary COPY FROM is attempted w/o appropriate codec. (by @elprans in 7aac14e2 for #157)

    • Fix Record.items() to support duplicate keys. See #28 for details. (by @1st1 in 39b390c3 for #28)

    • Fix Record.keys() to correctly return duplicate keys. See #28. (by @1st1 in b06114de for #28)

    • Call notifications listeners using loop.call_soon(). (by @1st1 in 83d08bd8)

    • Make PoolConnectionProxy more dynamic. Fixes #155. (by @1st1 in 6ca1f28b for #155)

    • Make sure set_type_codec() overrides core codec for all formats. (by @elprans in 9c32b86f for #140)

    • New severity "V" field as "severity_en" attribute. (by @vitaly-burovoy in 35bce5a8)

    • Make interval decoding logic match that of psycopg2. (by @elprans in 6b484433 for #150)

    Source code(tar.gz)
    Source code(zip)
  • v0.11.0(May 11, 2017)

    Backwards Incompatible Changes

    • Make timeout param of executemany() a keyword-only kwarg. (by @1st1 in bb326fc0)

    • Prohibit passing non-string instances as text arguments to queries. (by @elprans in ccc5f7a5)

    • connect() and create_pool() no longer accept arbitrary keyword arguments for server settings. Use new server_settings argument for that.

    New Features

    • Add support for COPY IN. (by @elprans in 10d95d43 for #123)

    • Add support for COPY OUT. (by @elprans and @1st1 in 5662d9f5 for #21)

    Bug Fixes

    • Refactor args parsing and config management for connect() and create_pool(). (by @1st1 in f280a566)

    • support postgres-xl version (by songjy in ffb89592)

    • Fix erroneous codec cache invalidation on internal codec aliasing. (by @elprans in df64f554 for #133)

    • Fix potential segfault in text array decoder. (by @elprans in bbd116be)

    • Limit max number of arguments (closes #127). (by @1st1 in 128910e4 for #127)

    Documentation

    • Expand usage documentation, add FAQ. (by @elprans in e4bef17f)
    Source code(tar.gz)
    Source code(zip)
  • v0.10.1(Apr 6, 2017)

    New features

    • Add new Connection parameter: max_cacheable_statement_size. (by @1st1 in b1f87ebe for #115)

    Bug fixes

    • Require command_timeout argument of connect() be greater than 0. (by @1st1 in 7b7ce990)

    • Fix SSL support in connection pool. (by @elprans in b89fb458 for #119)

    • Make timeout in connect() consistent. (by @1st1 in 1d8f5cf1 for #117)

    Source code(tar.gz)
    Source code(zip)
  • v0.10.0(Apr 4, 2017)

    New features

    • Add support for SSL connections. (by @1st1 in 5836a8fc for #25)

    • Add max_cached_statement_lifetime parameter to asyncpg.connect(). (by @1st1 in 10955fdd for #76)

    • Add max_inactive_connection_lifetime parameter to asyncpg.create_pool(). (by @1st1 in a2935aed)

    • Expose some of Connection methods directly on the Pool object. (by @1st1 in 1b3a847c for #39)

    • Allow overriding codecs for builtin types on a connection. (by @elprans in 71de129b for #73)

    • Add support for the text I/O fallback for arrays of unknown types. (by @elprans in a2e53ab0 for #82)

    • Make the Record type available as asyncpg.Record. (by @elprans in d42608f9 for #93)

    • pool: Track connections and prohibit using them after release. (by @1st1 in 3bf6103e)

    • Rollback any active transactions in Connection.reset(). (by @1st1 in 0d19d962)

    • Invalidate statement cache on schema changes affecting statement result. (by @elprans in 749d857a for #72)

    • Use anonymous prepared statements when the cache is off. (by @1st1 in a3e9cb45 for #76)

    • Raise an error if connection is used after being closed. (by @1st1 in 5ddd7fc9)

    • Add Amazon Redshift detection. (by @elprans in 4fdc1dbc for #29)

    • Add rudimentary server capability detection. (by @elprans in e8bb3dc0 for #87)

    Bug fixes

    • Handle exceptions in generators passed to Connection.executemany(). (by @elprans in 8d17eccb for #85)

    • Shield Pool.release() from task cancellation. (by @elprans in 537c8c9f for #97)

    • Fix inclusion of DETAIL and HINT in Postgres messages. (by @elprans in 2e346c12)

    • Implement conistent handling of timeouts. (by @elprans in 1674dece)

    • Make timeout arg of Connection.executemany() a keyword-only arg. (by @1st1 in 424760d3)

    Source code(tar.gz)
    Source code(zip)
  • v0.8.3(Jan 4, 2017)

    Elvis Pranskevichus (9):

    • Increase pool.acquire timeout in tests to avoid failures on Travis
    • Silence a few -Wmaybe-uninitialized warnings
    • Increase test timeouts
    • Attempt to fix spurious failures of test_auth_reject on Windows
    • Use ssize_t for buffer sizes consistently
    • Add infrastructure for multiformat data codecs
    • Implement text array decoder (#60)
    • ci: Make sure wheel builds on macOS start from a clean slate (#61)

    Vitaly Burovoy (3):

    • Make TYPEMAP be stable and avoid extra endline at the end of pgtypes.pxi
    • Make error message be the same as in the "_get_array_shape" function.
    • Add codec tests for "aclitem[]" type
    Source code(tar.gz)
    Source code(zip)
  • v0.8.1(Dec 22, 2016)

    Elvis Pranskevichus (7):

    • Fix Connection.reset() on read-only connections (#48)
    • Add compatibility with PostgreSQL server versions 9.1-9.3
    • travis: Test on all available PostgreSQL versions
    • travis: Also build on Python 3.6
    • Fix setup.py metadata and README rst issues
    • ci: Switch to staged release process
    • travis: Fix documentation deploy on tagged builds (#52)

    Lele Gaifax (1):

    • Fix typo in API documentation

    Yury Selivanov (2):

    • cython: Make async def coroutines awaitable in cythonized coroutines
    • connection: Fix waiter wakeup code in _cancel_current_command
    Source code(tar.gz)
    Source code(zip)
  • v0.7.0(Nov 17, 2016)

    Chris Lamb (1):

    • Fix parsing pg_ctl output in some locales

    Elvis Pranskevichus (9):

    • Fix building of wheels on macOS
    • setup.py: Support --cython-directives properly
    • Decode arrays as lists, handle arrays of composite types (#33)
    • Ignore system and dropped attributes in composite types (#43)
    • Fix handling of CIDR input to "inet" data type (#37)
    • Implement the Connection.executemany() method (#36)

    Yury Selivanov (3):

    • Allow empty Record objects.
    • cluster: Fix how settings args are passed to 'init'
    • Enable 'setup.py test'
    Source code(tar.gz)
    Source code(zip)
  • v0.6.0(Oct 20, 2016)

    Elvis Pranskevichus (5): Merge pull request #24 from johnkingsley/master Make it possible to override cluster connection parameters Move compilation logic to setup.py Add Windows support (#31) asyncpg v0.6.0

    John Kingsley (1): Fixed copy-and-paste error in Line type

    Source code(tar.gz)
    Source code(zip)
  • v0.5.3(Aug 5, 2016)

    • Fix timestamp codec on 32-bit systems
    • Allow any non-trivial Container to be passed as an array argument
    • Fix pool connection failure when a password is present (#16)
    • Various portability and robustness improvements
    Source code(tar.gz)
    Source code(zip)
Owner
magicstack
Any sufficiently advanced software is indistinguishable from magic.
magicstack
A Pythonic, object-oriented interface for working with MongoDB.

PyMODM MongoDB has paused the development of PyMODM. If there are any users who want to take over and maintain this project, or if you just have quest

mongodb 345 Dec 25, 2022
MongoX is an async python ODM for MongoDB which is built on top Motor and Pydantic.

MongoX MongoX is an async python ODM (Object Document Mapper) for MongoDB which is built on top Motor and Pydantic. The main features include: Fully t

Amin Alaee 112 Dec 04, 2022
TileDB-Py is a Python interface to the TileDB Storage Engine.

TileDB-Py TileDB-Py is a Python interface to the TileDB Storage Engine. Quick Links Installation Build Instructions TileDB Documentation Python API re

TileDB, Inc. 149 Nov 28, 2022
MySQL database connector for Python (with Python 3 support)

mysqlclient This project is a fork of MySQLdb1. This project adds Python 3 support and fixed many bugs. PyPI: https://pypi.org/project/mysqlclient/ Gi

PyMySQL 2.2k Dec 25, 2022
Micro ODM for MongoDB

Beanie - is an asynchronous ODM for MongoDB, based on Motor and Pydantic. It uses an abstraction over Pydantic models and Motor collections to work wi

Roman 993 Jan 03, 2023
A simple python package that perform SQL Server Source Control and Auto Deployment.

deploydb Deploy your database objects automatically when the git branch is updated. Production-ready! ⚙️ Easy-to-use 🔨 Customizable 🔧 Installation I

Mert Güvençli 10 Dec 07, 2022
Async ODM (Object Document Mapper) for MongoDB based on python type hints

ODMantic Documentation: https://art049.github.io/odmantic/ Asynchronous ODM(Object Document Mapper) for MongoDB based on standard python type hints. I

Arthur Pastel 732 Dec 31, 2022
A supercharged SQLite library for Python

SuperSQLite: a supercharged SQLite library for Python A feature-packed Python package and for utilizing SQLite in Python by Plasticity. It is intended

Plasticity 703 Dec 30, 2022
Estoult - a Python toolkit for data mapping with an integrated query builder for SQL databases

Estoult Estoult is a Python toolkit for data mapping with an integrated query builder for SQL databases. It currently supports MySQL, PostgreSQL, and

halcyon[nouveau] 15 Dec 29, 2022
New generation PostgreSQL database adapter for the Python programming language

Psycopg 3 -- PostgreSQL database adapter for Python Psycopg 3 is a modern implementation of a PostgreSQL adapter for Python. Installation Quick versio

The Psycopg Team 880 Jan 08, 2023
dbd is a database prototyping tool that enables data analysts and engineers to quickly load and transform data in SQL databases.

dbd: database prototyping tool dbd is a database prototyping tool that enables data analysts and engineers to quickly load and transform data in SQL d

Zdenek Svoboda 47 Dec 07, 2022
A tool to snapshot sqlite databases you don't own

The core here is my first attempt at a solution of this, combining ideas from browser_history.py and karlicoss/HPI/sqlite.py to create a library/CLI tool to (as safely as possible) copy databases whi

Sean Breckenridge 10 Dec 22, 2022
Python PostgreSQL database performance insights. Locks, index usage, buffer cache hit ratios, vacuum stats and more.

Python PG Extras Python port of Heroku PG Extras with several additions and improvements. The goal of this project is to provide powerful insights int

Paweł Urbanek 35 Nov 01, 2022
A Python wheel containing PostgreSQL

postgresql-wheel A Python wheel for Linux containing a complete, self-contained, locally installable PostgreSQL database server. All servers run as th

Michel Pelletier 71 Nov 09, 2022
Pysolr — Python Solr client

pysolr pysolr is a lightweight Python client for Apache Solr. It provides an interface that queries the server and returns results based on the query.

Haystack Search 626 Dec 01, 2022
A simple Python tool to transfer data from MySQL to SQLite 3.

MySQL to SQLite3 A simple Python tool to transfer data from MySQL to SQLite 3. This is the long overdue complimentary tool to my SQLite3 to MySQL. It

Klemen Tusar 126 Jan 03, 2023
Apache Libcloud is a Python library which hides differences between different cloud provider APIs and allows you to manage different cloud resources through a unified and easy to use API

Apache Libcloud - a unified interface for the cloud Apache Libcloud is a Python library which hides differences between different cloud provider APIs

The Apache Software Foundation 1.9k Dec 25, 2022
Make Your Company Data Driven. Connect to any data source, easily visualize, dashboard and share your data.

Redash is designed to enable anyone, regardless of the level of technical sophistication, to harness the power of data big and small. SQL users levera

Redash 22.4k Dec 30, 2022
pandas-gbq is a package providing an interface to the Google BigQuery API from pandas

pandas-gbq pandas-gbq is a package providing an interface to the Google BigQuery API from pandas Installation Install latest release version via conda

Google APIs 348 Jan 03, 2023
Google Sheets Python API v4

pygsheets - Google Spreadsheets Python API v4 A simple, intuitive library for google sheets which gets your work done. Features: Open, create, delete

Nithin Murali 1.4k Dec 31, 2022