๐Ÿš€ An end-to-end ML applications using PyTorch, W&B, FastAPI, Docker, Streamlit and Heroku

Overview

Creating an End-to-End ML Application w/ PyTorch

๐Ÿš€ This project was created using the Made With ML boilerplate template. Check it out to start creating your own ML applications.

Overview

  • Why do we need to build end-to-end applications?
    • By building e2e applications, you ensure that your code is organized, tested, testable / interactive and easy to scale-up / assimilate with larger pipelines.
    • If you're someone in industry and are looking to showcase your work to future employers, it's no longer enough to just have code on Jupyter notebooks. ML is just another tool and you need to show that you can use it in conjunction with all the other software engineering disciplines (frontend, backend, devops, etc.). The perfect way to do this is to create end-to-end applications that utilize all these different facets.
  • What are the components of an end-to-end ML application?
    1. Basic experimentation in Jupyter notebooks.
      • We aren't going to completely dismiss notebooks because they're still great tool to iterate quickly. Check out the notebook for our task here โ†’ notebook
    2. Moving our code from notebooks to organized scripts.
      • Once we did some basic development (on downsized datasets), we want to move our code to scripts to reduce technical debt. We'll create functions and classes for different parts of the pipeline (data, model, train, etc.) so we can easily make them robust for different circumstances.
      • We used our own boilerplate to organize our code before moving any of the code from our notebook.
    3. Proper logging and testing for you code.
      • Log key events (preprocessing, training performance, etc.) using the built-in logging library. Also use logging to see new inputs and outputs during prediction to catch issues, etc.
      • You also need to properly test your code. You will add and update your functions and their tests over time but it's important to at least start testing crucial pieces of your code from the beginning. These typically include sanity checks with preprocessing and modeling functions to catch issues early. There are many options for testing Python code but we'll use pytest here.
    4. Experiment tracking.
      • We use Weights and Biases (WandB), where you can easily track all the metrics of your experiment, config files, performance details, etc. for free. Check out the Dashboards page for an overview and tutorials.
      • When you're developing your models, start with simple approaches first and then slowly add complexity. You should clearly document (README, articles and WandB reports) and save your progression from simple to more complex models so your audience can see the improvements. The ability to write well and document your thinking process is a core skill to have in research and industry.
      • WandB also has free tools for hyperparameter tuning (Sweeps) and for data/pipeline/model management (Artifacts).
    5. Robust prediction pipelines.
      • When you actually deploy an ML application for the real world to use, we don't just look at the softmax scores.
      • Before even doing any forward pass, we need to analyze the input and deem if it's within the manifold of the training data. If it's something new (or adversarial) we shouldn't send it down the ML pipeline because the results cannot be trusted.
      • During processes like proprocessing, we need to constantly observe what the model received. For example, if the input has a bunch of unknown tokens than we need to flag the prediction because it may not be reliable.
      • After the forward pass we need to do tests on the model's output as well. If the predicted class has a mediocre test set performance, then we need the class probability to be above some critical threshold. Similarly we can relax the threshold for classes where we do exceptionally well.
    6. Wrap your model as an API.
      • Now we start to modularize larger operations (single/batch predict, get experiment details, etc.) so others can use our application without having to execute granular code. There are many options for this like Flask, Django, FastAPI, etc. but we'll use FastAPI for the ease and performance boost.
      • We can also use a Dockerfile to create a Docker image that runs our API. This is a great way to package our entire application to scale it (horizontally and vertically) depending on requirements and usage.
    7. Create an interactive frontend for your application.
      • The best way to showcase your work is to let others easily play with it. We'll be using Streamlit to very quickly create an interactive medium for our application and use Heroku to serve it (1000 hours of usage per month).
      • This is also a great skill to have because in industry you'll need to create this to show key stakeholders and great to have in documentation as well.

Set up

virtualenv -p python3.6 venv
source venv/bin/activate
pip install -r requirements.txt
pip install torch==1.4.0

Download embeddings

python text_classification/utils.py

Training

python text_classification/train.py \
    --data-url https://raw.githubusercontent.com/madewithml/lessons/master/data/news.csv --lower --shuffle --use-glove

Endpoints

uvicorn text_classification.app:app --host 0.0.0.0 --port 5000 --reload
GOTO: http://localhost:5000/docs

Prediction

Scripts

python text_classification/predict.py --text 'The Canadian government officials proposed the new federal law.'

cURL

curl "http://localhost:5000/predict" \
    -X POST -H "Content-Type: application/json" \
    -d '{
            "inputs":[
                {
                    "text":"The Wimbledon tennis tournament starts next week!"
                },
                {
                    "text":"The Canadian government officials proposed the new federal law."
                }
            ]
        }' | json_pp

Requests

import json
import requests

headers = {
    'Content-Type': 'application/json',
}

data = {
    "experiment_id": "latest",
    "inputs": [
        {
            "text": "The Wimbledon tennis tournament starts next week!"
        },
        {
            "text": "The Canadian minister signed in the new federal law."
        }
    ]
}

response = requests.post('http://0.0.0.0:5000/predict',
                         headers=headers, data=json.dumps(data))
results = json.loads(response.text)
print (json.dumps(results, indent=2, sort_keys=False))

Streamlit

streamlit run text_classification/streamlit.py
GOTO: http://localhost:8501

Tests

pytest

Docker

  1. Build image
docker build -t text-classification:latest -f Dockerfile .
  1. Run container
docker run -d -p 5000:5000 -p 6006:6006 --name text-classification text-classification:latest

Heroku

Set `WANDB_API_KEY` as an environment variable.

Directory structure

text-classification/
โ”œโ”€โ”€ datasets/                           - datasets
โ”œโ”€โ”€ logs/                               - directory of log files
|   โ”œโ”€โ”€ errors/                           - error log
|   โ””โ”€โ”€ info/                             - info log
โ”œโ”€โ”€ tests/                              - unit tests
โ”œโ”€โ”€ text_classification/                - ml scripts
|   โ”œโ”€โ”€ app.py                            - app endpoints
|   โ”œโ”€โ”€ config.py                         - configuration
|   โ”œโ”€โ”€ data.py                           - data processing
|   โ”œโ”€โ”€ models.py                         - model architectures
|   โ”œโ”€โ”€ predict.py                        - prediction script
|   โ”œโ”€โ”€ streamlit.py                      - streamlit app
|   โ”œโ”€โ”€ train.py                          - training script
|   โ””โ”€โ”€ utils.py                          - load embeddings and utilities
โ”œโ”€โ”€ wandb/                              - wandb experiment runs
โ”œโ”€โ”€ .dockerignore                       - files to ignore on docker
โ”œโ”€โ”€ .gitignore                          - files to ignore on git
โ”œโ”€โ”€ CODE_OF_CONDUCT.md                  - code of conduct
โ”œโ”€โ”€ CODEOWNERS                          - code owner assignments
โ”œโ”€โ”€ CONTRIBUTING.md                     - contributing guidelines
โ”œโ”€โ”€ Dockerfile                          - dockerfile to containerize app
โ”œโ”€โ”€ LICENSE                             - license description
โ”œโ”€โ”€ logging.json                        - logger configuration
โ”œโ”€โ”€ Procfile                            - process script for Heroku
โ”œโ”€โ”€ README.md                           - this README
โ”œโ”€โ”€ requirements.txt                    - requirementss
โ”œโ”€โ”€ setup.sh                            - streamlit setup for Heroku
โ””โ”€โ”€ sweeps.yaml                         - hyperparameter wandb sweeps config

Overfit to small subset

python text_classification/train.py \
    --data-url https://raw.githubusercontent.com/madewithml/lessons/master/data/news.csv --lower --shuffle --data-size 0.1 --num-epochs 3

Experiments

  1. Random, unfrozen, embeddings
python text_classification/train.py \
    --data-url https://raw.githubusercontent.com/madewithml/lessons/master/data/news.csv --lower --shuffle
  1. GloVe, frozen, embeddings
python text_classification/train.py \
    --data-url https://raw.githubusercontent.com/madewithml/lessons/master/data/news.csv --lower --shuffle --use-glove --freeze-embeddings
  1. GloVe, unfrozen, embeddings
python text_classification/train.py \
    --data-url https://raw.githubusercontent.com/madewithml/lessons/master/data/news.csv --lower --shuffle --use-glove

Next steps

End-to-end topics that will be covered in subsequent lessons.

  • Utilizing wrappers like PyTorch Lightning to structure the modeling even more while getting some very useful utility.
  • Data / model version control (Artifacts, DVC, MLFlow, etc.)
  • Experiment tracking options (MLFlow, KubeFlow, WandB, Comet, Neptune, etc)
  • Hyperparameter tuning options (Optuna, Hyperopt, Sweeps)
  • Multi-process data loading
  • Dealing with imbalanced datasets
  • Distributed training for much larger models
  • GitHub actions for automatic testing during commits
  • Prediction fail safe techniques (input analysis, class-specific thresholds, etc.)

Helpful docker commands

โ€ข Build image

docker build -t madewithml:latest -f Dockerfile .

โ€ข Run container if using CMD ["python", "app.py"] or ENTRYPOINT [ "/bin/sh", "entrypoint.sh"]

docker run -p 5000:5000 --name madewithml madewithml:latest

โ€ข Get inside container if using CMD ["/bin/bash"]

docker run -p 5000:5000 -it madewithml /bin/bash

โ€ข Run container with mounted volume

docker run -p 5000:5000 -v $PWD:/root/madewithml/ --name madewithml madewithml:latest

โ€ข Other flags

-d: detached
-ti: interative terminal

โ€ข Clean up

docker stop $(docker ps -a -q)     # stop all containers
docker rm $(docker ps -a -q)       # remove all containers
docker rmi $(docker images -a -q)  # remove all images
Owner
Made With ML
Applied ML ยท MLOps ยท Production
Made With ML
Multi-Joint dynamics with Contact. A general purpose physics simulator.

MuJoCo Physics MuJoCo stands for Multi-Joint dynamics with Contact. It is a general purpose physics engine that aims to facilitate research and develo

DeepMind 5.2k Jan 02, 2023
A hybrid framework (neural mass model + ML) for SC-to-FC prediction

The current workflow simulates brain functional connectivity (FC) from structural connectivity (SC) with a neural mass model. Gradient descent is applied to optimize the parameters in the neural mass

Yilin Liu 1 Jan 26, 2022
[ICCV 2021] Deep Hough Voting for Robust Global Registration

Deep Hough Voting for Robust Global Registration, ICCV, 2021 Project Page | Paper | Video Deep Hough Voting for Robust Global Registration Junha Lee1,

57 Nov 28, 2022
Companion code for the paper "Meta-Learning the Search Distribution of Black-Box Random Search Based Adversarial Attacks" by Yatsura et al.

META-RS This is the companion code for the paper "Meta-Learning the Search Distribution of Black-Box Random Search Based Adversarial Attacks" by Yatsu

Bosch Research 7 Dec 09, 2022
Implementation of "StrengthNet: Deep Learning-based Emotion Strength Assessment for Emotional Speech Synthesis"

StrengthNet Implementation of "StrengthNet: Deep Learning-based Emotion Strength Assessment for Emotional Speech Synthesis" https://arxiv.org/abs/2110

RuiLiu 65 Dec 20, 2022
Deep Reinforcement Learning by using an on-policy adaptation of Maximum a Posteriori Policy Optimization (MPO)

V-MPO Simple code to demonstrate Deep Reinforcement Learning by using an on-policy adaptation of Maximum a Posteriori Policy Optimization (MPO) in Pyt

Nugroho Dewantoro 9 Jun 06, 2022
Implementation of Perceiver, General Perception with Iterative Attention, in Pytorch

Perceiver - Pytorch Implementation of Perceiver, General Perception with Iterative Attention, in Pytorch Install $ pip install perceiver-pytorch Usage

Phil Wang 876 Dec 29, 2022
Implementation of Hierarchical Transformer Memory (HTM) for Pytorch

Hierarchical Transformer Memory (HTM) - Pytorch Implementation of Hierarchical Transformer Memory (HTM) for Pytorch. This Deepmind paper proposes a si

Phil Wang 63 Dec 29, 2022
Official implementation of Self-supervised Graph Attention Networks (SuperGAT), ICLR 2021.

SuperGAT Official implementation of Self-supervised Graph Attention Networks (SuperGAT). This model is presented at How to Find Your Friendly Neighbor

Dongkwan Kim 127 Dec 28, 2022
PyTorch code accompanying our paper on Maximum Entropy Generators for Energy-Based Models

Maximum Entropy Generators for Energy-Based Models All experiments have tensorboard visualizations for samples / density / train curves etc. To run th

Rithesh Kumar 135 Oct 27, 2022
Using Tensorflow Object Detection API to detect Waymo open dataset

Waymo-2D-Object-Detection Using Tensorflow Object Detection API to detect Waymo open dataset Result CenterNet Training Loss SSD ResNet Training Loss C

76 Dec 12, 2022
Code for the paper Language as a Cognitive Tool to Imagine Goals in Curiosity Driven Exploration

IMAGINE: Language as a Cognitive Tool to Imagine Goals in Curiosity Driven Exploration This repo contains the code base of the paper Language as a Cog

Flowers Team 26 Dec 22, 2022
FNet Implementation with TensorFlow & PyTorch

FNet Implementation with TensorFlow & PyTorch. TensorFlow & PyTorch implementation of the paper "FNet: Mixing Tokens with Fourier Transforms". Overvie

Abdelghani Belgaid 1 Feb 12, 2022
Download files from DSpace systems (because for some reason DSpace won't let you)

DSpaceDL A tool for downloading files from DSpace items. For some reason, DSpace systems have a dogshit UI, and Universities absolutely LOOOVE to use

Soumitra Shewale 5 Dec 01, 2022
A PyTorch implementation of "Cluster-GCN: An Efficient Algorithm for Training Deep and Large Graph Convolutional Networks" (KDD 2019).

ClusterGCN โ €โ € A PyTorch implementation of "Cluster-GCN: An Efficient Algorithm for Training Deep and Large Graph Convolutional Networks" (KDD 2019). A

Benedek Rozemberczki 697 Dec 27, 2022
A simple configurable bot for sending arXiv article alert by mail

arXiv-newsletter A simple configurable bot for sending arXiv article alert by mail. Prerequisites PyYAML=5.3.1 arxiv=1.4.0 Configuration All config

SXKDZ 21 Nov 09, 2022
Learning Intents behind Interactions with Knowledge Graph for Recommendation, WWW2021

Learning Intents behind Interactions with Knowledge Graph for Recommendation This is our PyTorch implementation for the paper: Xiang Wang, Tinglin Hua

158 Dec 15, 2022
Code implementation for the paper 'Conditional Gaussian PAC-Bayes'.

CondGauss This repository contains PyTorch code for the paper Stochastic Gaussian PAC-Bayes. A novel PAC-Bayesian training method is implemented. Ther

0 Nov 01, 2021
Code for: https://berkeleyautomation.github.io/bags/

DeformableRavens Code for the paper Learning to Rearrange Deformable Cables, Fabrics, and Bags with Goal-Conditioned Transporter Networks. Here is the

Daniel Seita 121 Dec 30, 2022
Official implementation of "Learning Proposals for Practical Energy-Based Regression", 2021.

ebms_proposals Official implementation (PyTorch) of the paper: Learning Proposals for Practical Energy-Based Regression, 2021 [arXiv] [project]. Fredr

Fredrik Gustafsson 10 Oct 22, 2022