Contrastive Learning of Image Representations with Cross-Video Cycle-Consistency

Last update: Nov 14, 2022

Related tags

Overview

Contrastive Learning of Image Representations with Cross-Video Cycle-Consistency

This is a official implementation of the CycleContrast introduced in the paper:Contrastive Learning of Image Representations with Cross-Video Cycle-Consistency

Citation

If you find our work useful, please cite:

@article{wu2021contrastive,
  title={Contrastive Learning of Image Representations with Cross-Video Cycle-Consistency},
  author={Wu, Haiping and Wang, Xiaolong},
  journal={arXiv preprint arXiv:2105.06463},
  year={2021}
}

Preparation

Our code is tested on Python 3.7 and Pytorch 1.3.0, please install the environment via

pip install -r requirements.txt

Model Zoo

We provide the model pretrained on R2V2 for 200 epochs.

method	pre-train epochs on R2V2 dataset	ImageNet Top-1 Linear Eval	OTB Precision	OTB Success	UCF Top-1	pretrained model
MoCo	200	53.8	56.1	40.6	80.5	pretrain ckpt
CycleContrast	200	55.7	69.6	50.4	82.8	pretrain ckpt

Run Experiments

Data preparation

Download R2V2 (Random Related Video Views) dataset according to https://github.com/danielgordon10/vince.

The direction structure should be as followed:

CycleContrast
├── cycle_contrast 
├── scripts 
├── utils 
├── data
│   ├── r2v2_large_with_ids 
│   │   ├── train 
│   │   │   ├── --/
│   │   │   ├── -_/
│   │   │   ├── _-/
│   │   │   ├── __/
│   │   │   ├── -0/
│   │   │   ├── _0/
│   │   │   ├── ...
│   │   │   ├── zZ/
│   │   │   ├── zz/
│   │   ├── val
│   │   │   ├── --/
│   │   │   ├── -_/
│   │   │   ├── _-/
│   │   │   ├── __/
│   │   │   ├── -0/
│   │   │   ├── _0/
│   │   │   ├── ...
│   │   │   ├── zZ/
│   │   │   ├── zz/

Unsupervised Pretrain

./scripts/train_cycle.sh

Downstream task - ImageNet linear eval

Prepare ImageNet dataset according to pytorch ImageNet training code.

MODEL_DIR=output/cycle_res50_r2v2_ep200
IMAGENET_DATA=data/ILSVRC/Data/CLS-LOC
./scripts/eval_ImageNet.sh $MODEL_DIR $IMAGENET_DATA

Downstream task - OTB tracking

Transfer to OTB tracking evaluation is based on SiamFC-Pytorch. Please prepare environment and data according to SiamFC-Pytorch

git clone https://github.com/happywu/mmaction2-CycleContrast
# path to your pretrained model, change accordingly
CycleContrast=/home/user/code/CycleContrast
PRETRAIN=${CycleContrast}/output/cycle_res50_r2v2_ep200/checkpoint_0199.pth.tar
cd mmaction2_tracking
./scripts/submit_r2v2_r50_cycle.py ${PRETRAIN}

Downstream task - UCF classification

Transfer to UCF action recognition evaluation is based on AVID-CMA, prepare data and env according to AVID-CMA.

git clone https://github.com/happywu/AVID-CMA-CycleContrast
# path to your pretrained model, change accordingly
CycleContrast=/home/user/code/CycleContrast
PRETRAIN=${CycleContrast}/output/cycle_res50_r2v2_ep200/checkpoint_0199.pth.tar
cd AVID-CMA-CycleContrast 
./scripts/submit_r2v2_r50_cycle.py ${PRETRAIN}

Acknowledgements

The codebase is based on FAIR-MoCo. The OTB tracking evaluation is based on MMAction2, SiamFC-PyTorch and vince. The UCF classification evaluation follows AVID-CMA.

Thank you all for the great open source repositories!

[ICCV'21] Official implementation for the paper Social NCE: Contrastive Learning of Socially-aware Motion Representations

CrowdNav with Social-NCE This is an official implementation for the paper Social NCE: Contrastive Learning of Socially-aware Motion Representations by

125 Dec 23, 2022

PyTorch implementation of SimCLR: A Simple Framework for Contrastive Learning of Visual Representations

1.7k Dec 28, 2022

Supervised Contrastive Learning for Downstream Optimized Sequence Representations

SupCL-Seq 📖 Supervised Contrastive Learning for Downstream Optimized Sequence representations (SupCS-Seq) accepted to be published in EMNLP 2021, ext

18 Oct 21, 2022

《LXMERT: Learning Cross-Modality Encoder Representations from Transformers》(EMNLP 2020)

The Most Important Thing. Our code is developed based on: LXMERT: Learning Cross-Modality Encoder Representations from Transformers

53 Dec 16, 2022

SUPERVISED-CONTRASTIVE-LEARNING-FOR-PRE-TRAINED-LANGUAGE-MODEL-FINE-TUNING - The Facebook paper about fine tuning RoBERTa with contrastive loss

"# SUPERVISED-CONTRASTIVE-LEARNING-FOR-PRE-TRAINED-LANGUAGE-MODEL-FINE-TUNING" i

28 Dec 12, 2022

Self-Learned Video Rain Streak Removal: When Cyclic Consistency Meets Temporal Correspondence

In this paper, we address the problem of rain streaks removal in video by developing a self-learned rain streak removal method, which does not require any clean groundtruth images in the training process.

44 Dec 6, 2022

Cross Quality LFW: A database for Analyzing Cross-Resolution Image Face Recognition in Unconstrained Environments

Cross-Quality Labeled Faces in the Wild (XQLFW) Here, we release the database, evaluation protocol and code for the following paper: Cross Quality LFW

10 Dec 12, 2022

Pytorch Implementation for NeurIPS (oral) paper: Pixel Level Cycle Association: A New Perspective for Domain Adaptive Semantic Segmentation

Pixel-Level Cycle Association This is the Pytorch implementation of our NeurIPS 2020 Oral paper Pixel-Level Cycle Association: A New Perspective for D

87 Oct 19, 2022

Code and models for ICCV2021 paper "Robust Object Detection via Instance-Level Temporal Cycle Confusion".

Robust Object Detection via Instance-Level Temporal Cycle Confusion This repo contains the implementation of the ICCV 2021 paper, Robust Object Detect

69 Oct 13, 2022

Contrastive Learning of Image Representations with Cross-Video Cycle-Consistency

Related tags

Overview

Contrastive Learning of Image Representations with Cross-Video Cycle-Consistency

Citation

Preparation

Model Zoo

Run Experiments

Data preparation

Unsupervised Pretrain

Downstream task - ImageNet linear eval

Downstream task - OTB tracking

Downstream task - UCF classification

Acknowledgements

You might also like...

[ICCV'21] Official implementation for the paper Social NCE: Contrastive Learning of Socially-aware Motion Representations

PyTorch implementation of SimCLR: A Simple Framework for Contrastive Learning of Visual Representations

Supervised Contrastive Learning for Downstream Optimized Sequence Representations

《LXMERT: Learning Cross-Modality Encoder Representations from Transformers》(EMNLP 2020)

SUPERVISED-CONTRASTIVE-LEARNING-FOR-PRE-TRAINED-LANGUAGE-MODEL-FINE-TUNING - The Facebook paper about fine tuning RoBERTa with contrastive loss

Self-Learned Video Rain Streak Removal: When Cyclic Consistency Meets Temporal Correspondence

Cross Quality LFW: A database for Analyzing Cross-Resolution Image Face Recognition in Unconstrained Environments

Pytorch Implementation for NeurIPS (oral) paper: Pixel Level Cycle Association: A New Perspective for Domain Adaptive Semantic Segmentation

Code and models for ICCV2021 paper "Robust Object Detection via Instance-Level Temporal Cycle Confusion".

Releases(v1.0.0)

v1.0.0(Dec 2, 2021)

Owner

Official PyTorch implementation of Synergies Between Affordance and Geometry: 6-DoF Grasp Detection via Implicit Representations

Discriminative Condition-Aware PLDA

Telegram chatbot created with deep learning model (LSTM) and telebot library.

Official pytorch implementation of Rainbow Memory (CVPR 2021)

This repo is customed for VisDrone.

This is a Tensorflow implementation of Learning to See in the Dark in CVPR 2018

Defense-GAN: Protecting Classifiers Against Adversarial Attacks Using Generative Models (published in ICLR2018)

Detecting Potentially Harmful and Protective Suicide-related Content on Twitter

RITA is a family of autoregressive protein models, developed by LightOn in collaboration with the OATML group at Oxford and the Debora Marks Lab at Harvard.

🤗 Paper Style Guide

The project is an official implementation of our CVPR2019 paper "Deep High-Resolution Representation Learning for Human Pose Estimation"

Finite-temperature variational Monte Carlo calculation of uniform electron gas using neural canonical transformation.

This repository holds the code for the paper "Deep Conditional Gaussian Mixture Model forConstrained Clustering".

Labelbox is the fastest way to annotate data to build and ship artificial intelligence applications

Pytorch implementation of AngularGrad: A New Optimization Technique for Angular Convergence of Convolutional Neural Networks

Unofficial implementation of Perceiver IO: A General Architecture for Structured Inputs & Outputs

MMDetection3D is an open source object detection toolbox based on PyTorch

Classic Papers for Beginners and Impact Scope for Authors.

LogDeep is an open source deeplearning-based log analysis toolkit for automated anomaly detection.

Reference PyTorch implementation of "End-to-end optimized image compression with competition of prior distributions"