Transformers are all you need

In this workshop we will be exploring NLP state of the art transformers, with SOTA models like T5 and BERT, then build a model using HugginFace transformers framework.

Table of Content

The workshop will be divided into four parts

1. Introduction to Transformers as a HYPE
2. Sneak peek to the theory behind Transfomers
3. Quick tour (Huggingface framework)
4. Lab
   • fine tune a translation model

Note that you can always open the notebooks on Google Colab ( No need to install anything ) you just need a stable internet connection :

- fine tune a translation model

2. How to get started

1. Fork this repository
2. Create a branch by your name
3. Go through the notebook and complete all tasks
4. Submit a pull request

Homework exercise

Your task is to fine-tune a classification model

1. Using HuggingFace transformers and datasets.
2. fine tune it to one of the classification task of the GLUE Benchmark(CoLa to be specific).
3. Use a checkpoint from the Hub ("distilbert-base-uncased" for example)
4. Once finished submit a pull request to this repo, make sure to place your .ipynb file in the submissions folder (YOUR_NAME.ipynb)

Useful ressources : text_classification