Functional interface for concurrent futures, including asynchronous I/O.

Overview

image image image image image image image image image

Futured provides a consistent interface for concurrent functional programming in Python. It wraps any callable to return a concurrent.futures.Future, wraps any async coroutine to return an asyncio.Future, and provides concurrent iterators and context managers for futures.

Usage

threaded, processed

Transform any callable into one which runs in a thread or process pool, and returns a future.

from futured import threaded, processed
import httpx

fetch = threaded(httpx.Client().get)
fetch(url)  # return Future

fs = (fetch(url + path) for path in paths)
threaded.results(fs)  # generate results from futures
threaded.results(fs, timeout=...)  # generate results as completed

fetch.map(urls)  # generate results in order
fetch.map(urls, timeout=...)  # generate results as completed
fetch.mapzip(urls)  # generate (url, result) pairs as completed

Thread and process pool executors may be used as context managers, customized with options, and reused with different callables.

threaded(max_workers=...)(func, ...)
processed(max_workers=...)(func, ...)

futured classes have a waiting context manager which collects results from tasks. Futures can be registered at creation, or appended to the list of tasks.

with threaded.waiting(*fs) as tasks:
    tasks.append(future)
tasks  # list of completed results

futured classes provide a tasks interface which generalizes futures.as_completed and futures.wait, while allowing the set of tasks to be modified, e.g., for retries.

threaded.tasks(fs, timeout=...)  # mutable set of running tasks which iterate as completed

asynced

The same interface works for asyncio.

from futured import asynced
import httpx

fetch = asynced(httpx.AsyncClient().get)
fetch(url)  # return coroutine

asynced.results(fs)  # generate results from futures
asynced.results(fs, timeout=...)  # generate results as completed

fetch.map(urls)  # generate results in order
fetch.map(urls, timeout=...)  # generate results as completed
fetch.mapzip(urls)  # generate (url, result) pairs as completed

asynced provides utilities for calling coroutines from a synchronous context. waiting is similar to trio's nursery, but returns results from a synchronous with block.

asynced.run(async_func, ...)  # call and run until complete
asynced.run(async_gen, ...)  # call and run synchronous iterator
with asynced.waiting(*fs) as tasks:  # concurrent coroutines completed in a block
asynced.tasks(fs, timeout=...)  # mutable set of running tasks which iterate as completed

decorators

Naturally futured wrappers can be used as decorators, but arguments can also be partially bound.

@threaded
def slow():
   ...

fetch = threaded(httpx.Client().get, url)
fetch(params=...)

Methods are supported, as well as a decorated utility for automatically subclassing.

from futured import decorated

FutureClient = decorated(httpx.Client, request=threaded)

 # equivalent to
class FutureClient(httpx.Client):
    request = threaded(httpx.Client.request)

command

command wraps subprocess.Popen to provide a Future compatible interface.

from futured import futured, command

command('ls').result()  # return stdout or raises stderr
command('ls').pipe('wc')  # pipes into next command, or | ('wc',... )
for line in command('ls'):  # iterable lines
command.coroutine('ls')  # return coroutine

futured(command, 'ls')  # supports `map` interface
asynced(command.coroutine, 'ls')  # supports `map` interface with timeout

forked

forked allows iteration in separate child processes.

from futured import forked

for value in forked(values, max_workers=...):
    # in a child process
 # in parent after children have exited

Installation

% pip install futured

Tests

100% branch coverage.

% pytest [--cov]

Changes

1.3

  • Python >=3.7 required
  • Python 3.10 event loop changes
  • Streams replaced with tasks

1.2

  • Python >=3.6 required

1.1

  • Stream completed futures from a pending pool

1.0

  • Executed functions are context managers
  • starmap supported

0.3

  • forked has optional maximum number of workers
  • waiting context manager
  • command pipes (|)
  • distributed.Client support

0.2

  • command.coroutine creates asyncio subprocesses
  • futured.mapzip generates results zipped with arguments
  • asynced.run supports asynchronous iterators
Random Programming Language Project

Crastle Random Programming Language Project Freedom of expression Are you a fan of curly brace languages? Then use curly braces! Not a fan of curly br

DevNugget 2 Dec 23, 2021
An AddOn storing wireguard configuration

Wireguard Database Connector Overview Development Status: 0.1.7 (alpha) First of all, I'd like to thank Jared McKnight for wireguard who inspired me t

Markus Neubauer 3 Dec 30, 2021
Replite - An embeddable REPL powered by JupyterLite

replite An embeddable REPL, powered by JupyterLite. Usage To embed the code cons

Jeremy Tuloup 47 Nov 09, 2022
Processamento da Informação - Disciplina UFABC

Processamento da Informacao Disciplina UFABC, Linguagem de Programação Python - 2021.2 Objetivos Apresentar os fundamentos sobre manipulação e tratame

Melissa Junqueira de Barros Lins 1 Jun 12, 2022
Q-Tracker is originally a High School Project created by Admins of Cirus Lab.

Q-Tracker is originally a High School Project created by Admins of Cirus Lab. It's completly coded in python along with mysql.(Tkinter For GUI)

Adithya Krishnan 2 Nov 14, 2022
Library for RadiaCode-101

RadiaCode Библиотека для работы с дозиметром RadiaCode-101, находится в разработке - API не стабилен и возможны изменения. Пример использования (backe

Maxim Andreev 56 Nov 29, 2022
Run the Tianxunet software on the Xiaoyao Android simulator

Run the Tianxunet software on the Xiaoyao Android simulator, and automatically fill in the answers of English listening on the premise of having answers

1 Feb 13, 2022
A companion web application to connect stash to deovr

stash-vr-companion This is a companion web application to connect stash to deovr. Stash is a self hosted web application to manage your porn collectio

19 Sep 29, 2022
Skip spotify ads by automatically restarting application when ad comes

SpotiByeAds No one likes interruptions! Don't you hate it when you're listening to your favorite jazz track or your EDM playlist and an ad for Old Spi

Partho 287 Dec 29, 2022
A beacon generator using Cobalt Strike and a variety of tools.

Beaconator is an aggressor script for Cobalt Strike used to generate either staged or stageless shellcode and packing the generated shellcode using your tool of choice.

Capt. Meelo 441 Dec 17, 2022
Pokemon sword replay capture

pokemon-sword-replay-capture This is an old version (March 2020) pokemon-sword-replay-capture-mar-2020-version of my Pokemon Replay Capture software.

11 May 15, 2022
Exercise to teach a newcomer to the CLSP grid to set up their environment and run jobs

Exercise to teach a newcomer to the CLSP grid to set up their environment and run jobs

Alexandra 2 May 18, 2022
Store Simulation

Almacenes Para clonar el Repositorio: Vaya a la terminal de Linux o Mac, o a la cmd en Windows y ejecute:

Johan Posada 1 Nov 12, 2021
A simple and easy to use Python's PIP configuration manager, similar to the Arch Linux's Java manager.

PIPCONF - The PIP configuration manager If you need to manage multiple configurations containing indexes and trusted hosts for PIP, this project was m

João Paulo Carvalho 11 Nov 30, 2022
Block fingerprinting for the beacon chain, for client identification & client diversity metrics

blockprint This is a repository for discussion and development of tools for Ethereum block fingerprinting. The primary aim is to measure beacon chain

Sigma Prime 49 Dec 08, 2022
A visidata plugin for parsing f5 ltm/gtm/audit logs

F5 Log Visidata Plugin This plugin supports the default log format for: /var/log/ltm* /var/log/gtm* /var/log/apm* /var/log/audit* It extracts common l

James Deucker 1 Jan 06, 2022
Live tracking, flight database and competition framework

SkyLines SkyLines is a web platform where pilots can share their flights with others after, or even during flight via live tracking. SkyLines is a sor

SkyLines 367 Dec 27, 2022
Plugin to manage site, circuit and device diagrams and documents in Netbox

Netbox Documents Plugin A plugin designed to faciliate the storage of site, circuit and device specific documents within NetBox Note: Netbox v3.2+ is

Jason Yates 38 Dec 24, 2022
A Python version of Canvacord

A copy of canvacord made in python! Installation Run any of these commands in terminal: Mac / Linux pip install canvacord Windows python -m pip insta

10 Mar 28, 2022
List Less Than Ten with python

List Less Than Ten with python

PyLaboratory 0 Feb 07, 2022