a wrapper around pytest for assessing flakiness and runtime regressions
a cs implementations practice project
First, install depedencies
# from the bubblewrap root
# first, install depdencies:
$ pip3 install -r requirements.txt`# then, invoke with:
$ ./bubblewrap path/to/code
e.g.
$ ./bubblewrap .
or, if you're using add bubblewrap to your $PATH and call with just bubblewrap path/to/code, e.g.:
$ PATH="$HOME/path/to/bubblewrap:$PATH"
This package also has a couple quick demos for testing against:
# from the bubblewrap root
$ ./bubblewrap examples/stable
# from the bubblewrap root
$ ./bubblewrap examples/flake
As of right now, imports in test modules need to import the full path of the module, e.g.:
# this is alright:
from example.one.A import apple
# but this is currently supported:
from A import appleThis'll be changed in the future.
Also, bubblewrap doesn't yet support regression analysis
This project uses Python3.9, pip to manage dependencies, and runs in a virtual environment. Install with the following:
# create + activate venv + install dependencies
# see docs: https://packaging.python.org/guides/installing-using-pip-and-virtual-environments/
$ python3 -m pip install --user virtualenv
$ python3 -m venv env
$ source env/bin/activate
$ pip install -r requirements.txt# note: to deactivate venv, use
$ deactivateThis project uses Black for code formatting:
$ python3 -m black .
The pyproject.toml file includes configuration for Black.
And, the project is linted with flake8
$ python3 -m flake8 path/to/code
Actual unit tests for this project are also run with with pytest. Having installed the pip dependencies (see prior section), run with the following:
# unit tests, ignoring ones in the demo dirs that are designed to flake
$ python3 -m pytest -vv -k 'not flake' .# test coverage report
$ python3 -m coverage run -m pytest -k 'not flake' .
$ coverage report -m