This project provides source code for our Mutual Contrastive Learning for Visual Representation Learning (MCL).

Ubuntu 18.04 LTS

Python 3.8 (Anaconda is recommended)

CUDA 11.1

PyTorch 1.7.0

NCCL for CUDA 11.1

CIFAR-100 : download

unzip to the ./data folder

python main_cifar.py --arch resnet32 --number-net 2

More commands for training various architectures can be found in scripts/train_cifar_baseline.sh

python main_cifar.py --arch resnet32  --number-net 2 \
    --alpha 0.1 --gamma 1. --beta 0.1 --lam 1. 

More commands for training various architectures can be found in scripts/train_cifar_mcl.sh

We perform all experiments on a single NVIDIA RTX 3090 GPU (24GB) with three runs.

python main_cifar.py --arch WRN_16_2  --number-net 4 \
    --alpha 0.1 --gamma 1. --beta 0.1 --lam 1. \
    --logit-distill

More commands for training various architectures can be found in scripts/train_cifar_mcl_logit.sh

We perform all experiments on a single NVIDIA RTX 3090 GPU (24GB) with three runs.

• Download the ImageNet dataset to YOUR_IMAGENET_PATH and move validation images to labeled subfolders

  • The script may be helpful.

• Create a datasets subfolder and a symlink to the ImageNet dataset

$ ln -s PATH_TO_YOUR_IMAGENET ./data/

Folder of ImageNet Dataset:

data/ImageNet
├── train
├── val

python main_imagenet.py --arch resnet18  --number-net 2 \
    --alpha 0.1 --gamma 1. --beta 0.1 --lam 1. 

More commands for training various architectures can be found in scripts/train_imagenet_mcl.sh

We perform all experiments on a single NVIDIA Tesla V100 GPU (32GB) with three runs.

python main_imagenet.py --arch resnet18  --number-net 2 \
    --alpha 0.1 --gamma 1. --beta 0.1 --lam 1. 

More commands for training various architectures can be found in scripts/train_imagenet_mcl.sh

We perform all experiments on a single NVIDIA Tesla V100 GPU (32GB) with three runs.

python main_moco_mcl.py \
  -a resnet18 \
  --lr 0.03 \
  --batch-size 256 \
  --number-net 2 \
  --dist-url 'tcp://localhost:10001' \
  --multiprocessing-distributed \
  --world-size 1 \
  --rank 0 \
  --gpu-ids 0,1,2,3,4,5,6,7 

python main_lincls.py \
  -a resnet18 \
  --lr 30.0 \
  --batch-size 256 \
  --pretrained [your checkpoint path]/checkpoint_0199.pth.tar \
  --dist-url 'tcp://localhost:10001' \
  --multiprocessing-distributed \
  --world-size 1 \
  --rank 0 \
  --gpu-ids 0,1,2,3,4,5,6,7 

We perform all experiments on 8 NVIDIA RTX 3090 GPUs with three runs.

@inproceedings{yang2022mcl,
  title={Mutual Contrastive Learning for Visual Representation Learning},
  author={Chuanguang Yang, Zhulin An, Linhang Cai, Yongjun Xu},
  booktitle={Proceedings of the AAAI Conference on Artificial Intelligence},
  year={2022}
}