Unofficial Pytorch implementation for Barlow Twins and HSIC_SSL on small datasets (CIFAR10, STL10, and Tiny ImageNet).
Correspondence to:
- Yao-Hung Hubert Tsai (yaohungt@cs.cmu.edu)
A Note on Connecting Barlow Twins with Negative-Samples-Free Contrastive Learning
Yao-Hung Hubert Tsai, Shaojie Bai, Louis-Philippe Morency, and Ruslan Salakhutdinov
I hope this work will be useful for your research 🥰
A large portion of the code is from this repo, which is a great resource for academic development. Note that we do not perform extensive hyper-parameters grid search and hence you may expect a performance boost after tuning some hyper-parameters (e.g., the learning rate).
The official implementation of Barlow Twins can be found here. We have also tried the HSIC_SSL in this official repo and we find similar performance (we tried on ImageNet-1K and CIFAR10) between HSIC_SSL and Barlow Twins' method.
conda install pytorch torchvision cudatoolkit=10.0 -c pytorch
- thop
pip install thop
CIFAR10, STL10, and Tiny_ImageNet.
python main.py --lmbda 0.0078125 --corr_zero --batch_size 128 --feature_dim 128 --dataset cifar10
python linear.py --dataset cifar10 --model_path results/0.0078125_128_128_cifar10_model.pth
python main.py --lmbda 0.0078125 --corr_neg_one --batch_size 128 --feature_dim 128 --dataset cifar10
python linear.py --dataset cifar10 --model_path results/neg_corr_0.0078125_128_128_cifar10_model.pth