Example of statuses that can be in readme:
Visit my docs for the full documentation, examples and guides.
With this project you get:
- a minimal
setup.py
file - testing with PyTest
- documentation (HTML and PDF) generated using Sphinx
- a CLI entry point
example_project/
|-- docs/
|-- |-- build/
|-- |-- source/
|-- example_project/
|-- |-- __init__.py
|-- |-- __version__.py
|-- |-- example_module.py
|-- tests/
|-- |-- test_data/
|-- | |-- example_class_data.json
|-- | __init__.py
|-- | conftest.py
|-- | test_example_class.py
|-- .env
|-- .gitignore
|-- Pipfile
|-- Pipfile.lock
|-- README.md
|-- setup.py
example_module.py
cli.py
The example_module.py
module contains sample code. tests
folder contains tests using PyTest.
The cli.py
module is referenced in the setup.py
file via the entry_points
definitions:
entry_points={
'console_scripts': ['py-package-template=example_project.cli:main'],
}
Using pipenv. Use --dev flag for pkgs only needed for dev or test. This gives a deterministic build. Note pipenv is a reference implementation recommened by Python. I fully expect pip to eventually implement it internally.
Assuming you have python installed (duh). On Mac (I exclusively code on mac now) I use brew to manage stuff as mac comes with python 2.x but the world has moved on and you MUST use 3.x, latest version at time of writing is 3.10.
Anyway Install pipenv
pip3 install pipenv
pipenv shell
do this from the source folder where Pipfile is present i.e. root folder.
this will also create your virtual env if its not there, i suggest reading up a bit on pipenv (just a quick brush) so you know the fundamentals as its quite different to virtualenv
Make sure that you're in the project's root directory
pipenv install --dev
I find using IPython in command line really useful when I am not in PyCharm IDE. I have included ipython as part of the --dev install above so you should be able to get into it by just doing
❯ ipython
Python 3.10.0 (default, Oct 13 2021, 06:45:00) [Clang 13.0.0 (clang-1300.0.29.3)]
Type 'copyright', 'credits' or 'license' for more information
IPython 7.28.0 -- An enhanced Interactive Python. Type '?' for help.
In [1]:
Pipenv will automatically pickup any environment variables
declared in the .env
file, located in root directory.
For example, adding,
DILLY=VANILLY
Will enable access to this variable from python
os.environ['DILLY']
All test have been written using the PyTest package.
Tests are kept in the tests
folder and can be run from the command line
cd tests
pytest
The conftest.py
module is used by PyTest - I've used it to add fixtures
which is really cool feature of pytest, I recommend.
I used flake8 for linting code.
pipenv run flake8 example_project
And black for formatting.
pipenv run black example_project
And you can use pre-commit to hook it all up (incl docs) so you never have to actually do anything manually by hand.
I think this is very useful. Think of all the times we said ah it might break some import or something but we wont know until we run, sure we can do extensive tests (we should) but this is like being able to do a compile of python and find problems.
it will barf about pandas/numpy etc which doesnt have stubs, so ignore it for now. am using MyPy package. You can configure what it does with mypy.ini options should be in their docs.
Also note Data Science Types is trying to fix above problem - but I have not tried it.
To run mypy do >
pipenv run python -m mypy example_project/*.py
MyPy options for this project can be defined in the mypy.ini
file that MyPy will look for by default. For more information on the full set of options, see the mypy documentation.
Examples of type annotation and type checking for library development can be found in the py_pkg.curves.py
module. This should also be cross-referenced with the improvement to readability (and usability) that this has on package documentation.
❯ pipenv run python -m mypy example_project/*.py
Loading .env environment variables...
example_project/example_module.py:12: error: Skipping analyzing "pandas": found module but no type hints or library stubs
example_project/example_module.py:12: note: See https://mypy.readthedocs.io/en/stable/running_mypy.html#missing-imports
❯ pipenv run flake8 example_project
Loading .env environment variables...
example_project/__version__.py:11:13: W292 no newline at end of file
example_project/cli.py:14:10: E211 whitespace before '('
example_project/cli.py:14:20: W292 no newline at end of file
❯ pipenv run black example_project
Loading .env environment variables...
reformatted example_project/__version__.py
reformatted example_project/cli.py
reformatted example_project/example_module.py
The documentation in the docs
folder has been built using Sphinx.
Its a powerful framework that can be used to generate docs on the fly into various formats.
I generated the initial source using:
sphinx-quickstart docs
And then you'd hope to see auto generated docs based on docstring but sadly
it isnt so auto-magical... I had to add them in manually per module in
docs/source/index.rst
where you see I reference 2 files
modules.rst
and modules_test.rst
And then I did:
cd docs
make html
I like that sphinx ships with make, and we can definitely look to using make as our over-arching tool, plays in nicely with c++.
But alternatively you could also generate the docs by doing:
pipenv run sphinx-build -b html docs/source docs/build/html
Also you obviously should source pipenv shell, and you dont have to actually run pipenv run, but I just show the foolproof way here.
The resulting HTML documentation can be accessed by opening docs/build/html/index.html
in a web browser.
If you are curious you'll see I've had to do quite bit of customisation for the config file
so that it could generate the docs > docs/source/config.py
I also explored creating PDF docs, I think these are really important if wanting to send to end users, and that can be done using an addon called LatEx, but I havent set it up as yet - but I have explored how to do it.
Finally to package this into a wheel! do the following:
pipenv run python setup.py bdist_wheel
This will create build
, example_package.egg-info
and dist
directories.
whl should be in dist
.
Annoyingly, you cant use pipfile for setup.py requirements, so I had to take a shortcut of generate the requirements.txt by doing
pipenv run pip freeze > requirements.txt
and i wrote a func in setup.py to read the file and use the it to generate install_require, so that the generated wheel installs all the dependencies.
But I make persist requirements.txt in git intentionally, so you generate it everytime you want to create a distributable, i suppose it could be made part of a make command.