Contrastive Learning with Non-Semantic Negatives

This repository is the official implementation of Robust Contrastive Learning Using Negative Samples with Diminished Semantics. Contrastive learning utilizes positive pairs which preserve semantic information while perturbing superficial features in the training images. Similarly, we propose to generate negative samples to make the model more robust, where only the superfluous instead of the semantic features are preserved.

Preparation

Install PyTorch and check preprocess/ for ImageNet-100 and ImageNet-Texture preprocessing scripts.

Training

The following code is used to pre-train MoCo-v2 + patch / texture-based NS. The major code is developed with minimal modifications from the official implementation.

python moco-non-sem-neg.py -a resnet50 --lr 0.03 --batch-size 128 --dist-url 'tcp://localhost:10001' \
  --multiprocessing-distributed --world-size 1 --rank 0 \
  --mlp --moco-t 0.2 --aug-plus --cos --moco-k 16384 \
  --robust nonsem --num-nonsem 1 --alpha 2 --epochs 200 --patch-ratio 16 72 \
  --ckpt_path ./ckpts/mocov2_mocok16384_bs128_lr0.03_nonsem_16_72_noaug_nn1_alpha2_epoch200  \
  /path/to/imagenet-100/ 

python moco-non-sem-neg.py -a resnet50 --lr 0.03 --batch-size 128 --dist-url 'tcp://localhost:10001' \
  --multiprocessing-distributed --world-size 1 --rank 0 \
  --mlp --moco-t 0.2 --aug-plus --cos --moco-k 16384 \
  --robust texture_syn --num-nonsem 1 --alpha 2 --epochs 200 \
  --ckpt_path ./ckpts/mocov2_mocok16384_bs128_lr0.03_texture_nn1_alpha2_epoch200 \
  /path/to/imagenet-100-texture/

Change /path/to/imagenet-100/ with the ImageNet-100 dataset directory.
Change --alpha and -moco-k to reproduce results with different configurations.

Linear Evaluation

Run following code is used to reproduce MoCo-v2 + patch-based NS model reported in Table 1.

python main_lincls.py -a resnet50 --lr 10.0 --batch-size 128 --epochs 60 \
  --pretrained ./ckpts/mocov2_mocok16384_bs128_lr0.03_nonsem_16_72_noaug_nn1_alpha2_epoch200/checkpoint_0199.pth.tar \
  --dist-url 'tcp://localhost:10001' --multiprocessing-distributed --world-size 1 --rank 0 \
  --ckpt_path ./ckpts/mocov2_mocok16384_bs128_lr0.03_nonsem_16_72_noaug_nn1_alpha2_epoch200 \
  /path/to/imagenet-100/

Pre-trained Models

You can download pretrained models here:

	moco-k	alpha	ImageNet-100	Corruption	Sketch	Stylized	Rendition	Checkpoints
MoCo-v2	16384	-	77.88±0.28	43.08±0.27	28.24±0.58	16.20±0.55	32.92±0.12	Run1, Run2, Run3
+ Texture	16384	2	77.76±0.17	43.58±0.33	29.11±0.39	16.59±0.17	33.36±0.15	Run1, Run2, Run3
+ Patch	16384	2	79.35±0.12	45.13±0.35	31.76±0.88	17.37±0.19	34.78±0.15	Run1, Run2, Run3
+ Patch	16384	3	75.58±0.52	44.45±0.15	34.03±0.58	18.60±0.26	36.89±0.11	Run1, Run2, Run3
MoCo-v2	8192	-	77.73±0.38	43.22±0.39	28.45±0.36	16.83±0.12	33.19±0.44	Run1, Run2, Run3
+ Patch	8192	2	79.54±0.32	45.48±0.20	33.36±0.45	17.81±0.32	36.31±0.37	Run1, Run2, Run3

Contrastive Learning with Non-Semantic Negatives

Related tags

Overview

Contrastive Learning with Non-Semantic Negatives

Preparation

Training

Linear Evaluation

Pre-trained Models

Owner

Domain Adaptation with Invariant RepresentationLearning: What Transformations to Learn?

SOTA model in CIFAR10

[ICLR 2021] "Neural Architecture Search on ImageNet in Four GPU Hours: A Theoretically Inspired Perspective" by Wuyang Chen, Xinyu Gong, Zhangyang Wang

a basic code repository for basic task in CV(classification,detection,segmentation)

Lab Materials for MIT 6.S191: Introduction to Deep Learning

(CVPR2021) DANNet: A One-Stage Domain Adaptation Network for Unsupervised Nighttime Semantic Segmentation

Official repository for Hierarchical Opacity Propagation for Image Matting

Multitask Learning Strengthens Adversarial Robustness

PyTorch implementation of "Learn to Dance with AIST++: Music Conditioned 3D Dance Generation."

ThunderSVM: A Fast SVM Library on GPUs and CPUs

This repository contains code and data for "On the Multimodal Person Verification Using Audio-Visual-Thermal Data"

This repository contains codes of ICCV2021 paper: SO-Pose: Exploiting Self-Occlusion for Direct 6D Pose Estimation

Tensorflow2.0 🍎🍊 is delicious, just eat it! 😋😋

A Deep learning based streamlit web app which can tell with which bollywood celebrity your face resembles.

End-to-end beat and downbeat tracking in the time domain.

Find the Heart simple Python Game

Analysing poker data from home games with friends

Supplementary code for SIGGRAPH 2021 paper: Discovering Diverse Athletic Jumping Strategies

PSML: A Multi-scale Time-series Dataset for Machine Learning in Decarbonized Energy Grids

Energy consumption estimation utilities for Jetson-based platforms