py.test fixture for benchmarking code

Overview

Overview

docs Documentation Status Join the chat at https://gitter.im/ionelmc/pytest-benchmark
tests
Travis-CI Build Status AppVeyor Build Status Requirements Status
Coverage Status Coverage Status
package

A pytest fixture for benchmarking code. It will group the tests into rounds that are calibrated to the chosen timer.

See calibration and FAQ.

  • Free software: BSD 2-Clause License

Installation

pip install pytest-benchmark

Documentation

For latest release: pytest-benchmark.readthedocs.org/en/stable.

For master branch (may include documentation fixes): pytest-benchmark.readthedocs.io/en/latest.

Examples

But first, a prologue:

This plugin tightly integrates into pytest. To use this effectively you should know a thing or two about pytest first. Take a look at the introductory material or watch talks.

Few notes:

  • This plugin benchmarks functions and only that. If you want to measure block of code or whole programs you will need to write a wrapper function.
  • In a test you can only benchmark one function. If you want to benchmark many functions write more tests or use parametrization <http://docs.pytest.org/en/latest/parametrize.html>.
  • To run the benchmarks you simply use pytest to run your "tests". The plugin will automatically do the benchmarking and generate a result table. Run pytest --help for more details.

This plugin provides a benchmark fixture. This fixture is a callable object that will benchmark any function passed to it.

Example:

def something(duration=0.000001):
    """
    Function that needs some serious benchmarking.
    """
    time.sleep(duration)
    # You may return anything you want, like the result of a computation
    return 123

def test_my_stuff(benchmark):
    # benchmark something
    result = benchmark(something)

    # Extra code, to verify that the run completed correctly.
    # Sometimes you may want to check the result, fast functions
    # are no good if they return incorrect results :-)
    assert result == 123

You can also pass extra arguments:

def test_my_stuff(benchmark):
    benchmark(time.sleep, 0.02)

Or even keyword arguments:

def test_my_stuff(benchmark):
    benchmark(time.sleep, duration=0.02)

Another pattern seen in the wild, that is not recommended for micro-benchmarks (very fast code) but may be convenient:

def test_my_stuff(benchmark):
    @benchmark
    def something():  # unnecessary function call
        time.sleep(0.000001)

A better way is to just benchmark the final function:

def test_my_stuff(benchmark):
    benchmark(time.sleep, 0.000001)  # way more accurate results!

If you need to do fine control over how the benchmark is run (like a setup function, exact control of iterations and rounds) there's a special mode - pedantic:

def my_special_setup():
    ...

def test_with_setup(benchmark):
    benchmark.pedantic(something, setup=my_special_setup, args=(1, 2, 3), kwargs={'foo': 'bar'}, iterations=10, rounds=100)

Screenshots

Normal run:

Screenshot of pytest summary

Compare mode (--benchmark-compare):

Screenshot of pytest summary in compare mode

Histogram (--benchmark-histogram):

Histogram sample

Also, it has nice tooltips.

Development

To run the all tests run:

tox

Credits

Owner
Ionel Cristian Mărieș
Ionel Cristian Mărieș
Argument matchers for unittest.mock

callee Argument matchers for unittest.mock More robust tests Python's mocking library (or its backport for Python 3.3) is simple, reliable, and easy

Karol Kuczmarski 77 Nov 03, 2022
tidevice can be used to communicate with iPhone device

tidevice can be used to communicate with iPhone device

Alibaba 1.8k Jan 08, 2023
automate the procedure of 403 response code bypass

403bypasser automate the procedure of 403 response code bypass Description i notice a lot of #bugbountytips describe how to bypass 403 response code s

smackerdodi2 40 Dec 16, 2022
pywinauto is a set of python modules to automate the Microsoft Windows GUI

pywinauto is a set of python modules to automate the Microsoft Windows GUI. At its simplest it allows you to send mouse and keyboard actions to windows dialogs and controls, but it has support for mo

3.8k Jan 06, 2023
A library to make concurrent selenium tests that automatically download and setup webdrivers

AutoParaSelenium A library to make parallel selenium tests that automatically download and setup webdrivers Usage Installation pip install autoparasel

Ronak Badhe 8 Mar 13, 2022
1st Solution to QQ Browser 2021 AIAC Track 2

1st Solution to QQ Browser 2021 AIAC Track 2 This repository is the winning solution to QQ Browser 2021 AI Algorithm Competition Track 2 Automated Hyp

DAIR Lab 24 Sep 10, 2022
A suite of benchmarks for CPU and GPU performance of the most popular high-performance libraries for Python :rocket:

A suite of benchmarks for CPU and GPU performance of the most popular high-performance libraries for Python :rocket:

Dion Häfner 255 Jan 04, 2023
Android automation project with pytest+appium

Android automation project with pytest+appium

1 Oct 28, 2021
A test fixtures replacement for Python

factory_boy factory_boy is a fixtures replacement based on thoughtbot's factory_bot. As a fixtures replacement tool, it aims to replace static, hard t

FactoryBoy project 3k Jan 05, 2023
A automated browsing experience.

browser-automation This app is an automated browsing technique where one has to enter the required information, it's just like searching for Animals o

Ojas Barawal 3 Aug 04, 2021
WomboAI Art Generator

WomboAI Art Generator Automate AI art generation using wombot.art. Also integrated into SnailBot for you to try out. Setup Install Python Go to the py

nbee 7 Dec 03, 2022
A testing system for catching visual regressions in Web applications.

Huxley Watches you browse, takes screenshots, tells you when they change Huxley is a test-like system for catching visual regressions in Web applicati

Facebook Archive 4.1k Nov 30, 2022
Thin-wrapper around the mock package for easier use with pytest

pytest-mock This plugin provides a mocker fixture which is a thin-wrapper around the patching API provided by the mock package: import os class UnixF

pytest-dev 1.5k Jan 05, 2023
A configurable set of panels that display various debug information about the current request/response.

Django Debug Toolbar The Django Debug Toolbar is a configurable set of panels that display various debug information about the current request/respons

Jazzband 7.3k Jan 02, 2023
fsociety Hacking Tools Pack – A Penetration Testing Framework

Fsociety Hacking Tools Pack A Penetration Testing Framework, you will have every script that a hacker needs. Works with Python 2. For a Python 3 versi

Manisso 8.2k Jan 03, 2023
Selenium Manager

SeleniumManager I'm fed up with always having to struggle unnecessarily when I have to use Selenium on a new machine, so I made this little python mod

Victor Vague 1 Dec 24, 2021
pytest plugin providing a function to check if pytest is running.

pytest-is-running pytest plugin providing a function to check if pytest is running. Installation Install with: python -m pip install pytest-is-running

Adam Johnson 21 Nov 01, 2022
Nokia SR OS automation

Nokia SR OS automation Nokia is one of the biggest vendors of the telecommunication equipment, which is very popular in the Service Provider segment.

Karneliuk.com 7 Jul 23, 2022
Integration layer between Requests and Selenium for automation of web actions.

Requestium is a Python library that merges the power of Requests, Selenium, and Parsel into a single integrated tool for automatizing web actions. The

Tryolabs 1.7k Dec 27, 2022
Let your Python tests travel through time

FreezeGun: Let your Python tests travel through time FreezeGun is a library that allows your Python tests to travel through time by mocking the dateti

Steve Pulec 3.5k Dec 29, 2022