Delayed iteration for polling and retries.

Overview

image image image image image image image image image

Does Python need yet another retry / poll library? It needs at least one that isn't coupled to decorators and functions. Decorators prevent the caller from customizing delay options, and organizing the code around functions hinders any custom handling of failures.

Waiter is built around iteration instead, because the foundation of retrying / polling is a slowly executing loop. The resulting interface is both easier to use and more flexible, decoupling the delay algorithms from the application logic.

Usage

creation

Supply a number of seconds to repeat endlessly, or any iterable of seconds.

from waiter import wait

wait(1)                 # 1, 1, 1, 1, ...
wait([1] * 3)           # 1, 1, 1
wait([0.5, 0.5, 60])    # circuit breaker

Iterable delays can express any waiting strategy, and constructors for common algorithms are also provided.

wait.count(1)           # incremental backoff 1, 2, 3, 4, 5, ...
wait(1) + 1             # alternate syntax 1, 2, 3, 4, 5, ...
wait.fibonacci(1)       # 1, 1, 2, 3, 5, ...
wait.polynomial(2)      # 0, 1, 4, 9, 16, ...

wait.exponential(2)     # exponential backoff 1, 2, 4, 8, ...
backoff = wait(1) * 2   # alternate syntax 1, 2, 4, 8, ...
backoff[:3]             # limit attempt count 1, 2, 4
backoff <= 5            # set maximum delay   1, 2, 4, 5, 5, 5, ...
backoff.random(-1, 1)   # add random jitter

iteration

Then simply use the wait object like any iterable, yielding the amount of elapsed time. Timeouts also supported of course.

from waiter import wait, suppress, first

for elapsed in wait(delays):            # first iteration is immediate
    with suppress(exception):           # then each subsequent iteration sleeps as necessary
        ...
        break

for _ in wait(delays, timeout):         # standard convention for ignoring a loop variable
    ...                                 # won't sleep past the timeout
    if ...:
        break

results = (... for _ in wait(delays))   # expressions are even easier
first(predicate, results[, default])    # filter for first true item
assert any(results)                     # perfect for tests too

functions

Yes, functional versions are provided, as well as being trivial to implement.

wait(...).throttle(iterable)                      # generate items from iterable
wait(...).repeat(func, *args, **kwargs)           # generate successive results
wait(...).retry(exception, func, *args, **kwargs) # return first success or re-raise exception
wait(...).poll(predicate, func, *args, **kwargs)  # return first success or raise StopIteration

The decorator variants are simply partial applications of the corresponding methods. Note decorator syntax doesn't support arbitrary expressions.

backoff = wait(0.1) * 2
@backoff.repeating
@backoff.retrying(exception)
@backoff.polling(predicate)

But in the real world:

  • the function may not exist or be succinctly written as a lambda
  • the predicate may not exist or be succinctly written as a lambda
  • logging may be required
  • there may be complex handling of different exceptions or results

So consider the block form, just as decorators don't render with blocks superfluous. Also note wait objects are re-iterable provided their original delays were.

async

Waiters also support async iteration. throttle optionally accepts an async iterable. repeat, retry, and poll optionally accept coroutine functions.

statistics

Waiter objects have a stats attribute for aggregating statistics about the calls made. The base implementation provides total and failure counts. The interface of the stats object itself is considered provisional for now, but can be extended by overriding the Stats class attribute. This also allows customization of the iterable values; elapsed time is the default.

Installation

% pip install waiter

Dependencies

  • multimethod

Tests

100% branch coverage.

% pytest [--cov]

Changes

dev

  • Python >=3.7 required

1.2

  • Python >=3.6 required

1.1

  • Stream from sized groups

1.0

  • Map a function across an iterable in batches

0.6

  • Extensible iterable values and statistics
  • Additional constructors: fibonacci, polynomial, accumulate

0.5

  • Asynchronous iteration

0.4

  • Decorators support methods
  • Iterables can be throttled

0.3

  • Waiters behave as iterables instead of iterators
  • Support for function decorators

0.2

  • suppress context manager for exception handling
  • repeat method for decoupled iteration
  • first function for convenient filtering
Tiny demo site for exploring SameSite=Lax

samesite-lax-demo Background on my blog: Exploring the SameSite cookie attribute for preventing CSRF This repo holds some tools for exploring the impl

Simon Willison 6 Nov 10, 2021
A tool to quickly create codeforces contest directories with templates.

Codeforces Template Tool I created this tool to help me quickly set up codeforces contests/singular problems with templates. Tested for windows, shoul

1 Jun 02, 2022
CDM Device Checker for python

CDM Device Checker for python

zackmark29 79 Dec 14, 2022
Cirq is a Python library for writing, manipulating, and optimizing quantum circuits and running them against quantum computers and simulators

Cirq is a Python library for writing, manipulating, and optimizing quantum circuits and running them against quantum computers and simulators. Install

quantumlib 3.6k Jan 07, 2023
Thinky nature dots with python

Thinky Nature Welcome to my rice dotfiles of my ThinkPad X230 You surely will need to change some configs to fit your files and stuff and maybe tweak

Daniel Mironow 26 Dec 02, 2022
Cylinder volume calculator features the calculations of the volume of a Right /oblique full cylinder

Cylinder-Volume-Calculator Cylinder volume calculator features the calculations of the volume of a Right /oblique full cylinder. Size : 10.5 mb compat

Abhijeet 4 Nov 07, 2022
A project for the Qvault Hackathon, 2022-01-17

musical-octo-engine Steps to run brew install python-tk brew install portaudio

Erik Kristofer Anderson 2 May 17, 2022
Stack-overflow-import - Import arbitrary code from Stack Overflow as Python modules.

StackOverflow Importer Do you ever feel like all you’re doing is copy/pasting from Stack Overflow? Let’s take it one step further. from stackoverflow

Filip Haglund 3.7k Jan 08, 2023
Data Applications Project

DBMS project- Hotel Franchise Data and application project By TEAM Kurukunda Bhargavi Pamulapati Pallavi Greeshma Amaraneni What is this project about

Greeshma 1 Nov 28, 2021
Multtable is a collection of multiplication table generators in various languages.

Multtable Multtable is a collection of multiplication table generators in various languages. This project was created as a joke based on one of my bro

pollen__ 7 Mar 05, 2022
Add all JuliaLang unicode abbreviations to AutoKey.

Autokey Unicode characters Usage This script adds all the unicode character abbreviations supported by Julia to autokey. However, instead of [TAB], th

Randolf Scholz 49 Dec 02, 2022
A Gura parser implementation for Python

Gura parser This repository contains the implementation of a Gura format parser in Python. Installation pip install gura-parser Usage import gura gur

JWare Solutions 19 Jan 25, 2022
A small project of two newbies, who wanted to learn something about Python language programming, via fun way.

HaveFun A small project of two newbies, who wanted to learn something about Python language programming, via fun way. What's this project about? Well.

Patryk Sobczak 2 Nov 24, 2021
flake8 plugin which forbids match statements (PEP 634)

flake8-match flake8 plugin which forbids match statements (PEP 634)

Anthony Sottile 25 Nov 01, 2022
Cairo-math-64x61 - Fixed point 64.61 math library for Cairo / Starknet

Cairo Math 64x61 A fixed point 64.61 math library for Cairo & Starknet Signed 64

Influence 63 Dec 05, 2022
It is convenient to quickly import Python packages from the network.

It is convenient to quickly import Python packages from the network.

zmaplex 1 Jan 18, 2022
A beacon generator using Cobalt Strike and a variety of tools.

Beaconator is an aggressor script for Cobalt Strike used to generate either staged or stageless shellcode and packing the generated shellcode using your tool of choice.

Capt. Meelo 441 Dec 17, 2022
The Great Autoencoder Bake Off

The Great Autoencoder Bake Off The companion repository to a post on my blog. It contains all you need to reproduce the results. Features Currently fe

Tilman Krokotsch 61 Jan 06, 2023
Darkflame Universe Account Manager

Darkflame Universe Account Manager This is a quick and simple web application intended for account creation and management for a DLU instance created

31 Nov 29, 2022
Open source stenotype engine

Plover Bringing stenography to everyone. Homepage Releases Wiki Blog Google Group Discord Chat About Installation Getting help Contributing Donations

Open Steno Project 2k Jan 09, 2023