Self-Supervised Learning

Features

self_supervised offers features like

• modular framework
• support for multi-gpu training using PyTorch Lightning
• easy to use and written in a PyTorch like style
• supports custom backbone models for self-supervised pre-training

Supported Models

• MoCo, 2019
• SimCLR, 2020
• SimSiam, 2021
• Barlow Twins, 2021
• BYOL, 2020
• NNCLR, 2021
• SwaV, 2020
• MocoV2, 2020
• MocoV3, 2021

Supported Loss Function

• NegativeCosineSimilarity
• SwaVLoss

Benchmarks

Currently implemented models and their accuracy on cifar10 and imagenette.

ImageNette

Cifar10

Tutorials

Want to jump to the tutorials and see lightly in action?

• Train MoCo on CIFAR-10
• Train SimCLR on clothing data
• Train SimSiam on satellite images
• Use lightly with custom augmentations

Further Reading

Self-supervised Learning:

• A Simple Framework for Contrastive Learning of Visual Representations (2020)
• Momentum Contrast for Unsupervised Visual Representation Learning (2020)
• Unsupervised Learning of Visual Features by Contrasting Cluster Assignments (2020)
• What Should Not Be Contrastive in Contrastive Learning (2020)