Contrastive Learning of Image Representations with Cross-Video Cycle-Consistency

Related tags

Deep LearningCycleContrast
Overview

Contrastive Learning of Image Representations with Cross-Video Cycle-Consistency

This is a official implementation of the CycleContrast introduced in the paper:Contrastive Learning of Image Representations with Cross-Video Cycle-Consistency

Citation

If you find our work useful, please cite:

@article{wu2021contrastive,
  title={Contrastive Learning of Image Representations with Cross-Video Cycle-Consistency},
  author={Wu, Haiping and Wang, Xiaolong},
  journal={arXiv preprint arXiv:2105.06463},
  year={2021}
}

Preparation

Our code is tested on Python 3.7 and Pytorch 1.3.0, please install the environment via

pip install -r requirements.txt

Model Zoo

We provide the model pretrained on R2V2 for 200 epochs.

method pre-train epochs on R2V2 dataset ImageNet Top-1 Linear Eval OTB Precision OTB Success UCF Top-1 pretrained model
MoCo 200 53.8 56.1 40.6 80.5 pretrain ckpt
CycleContrast 200 55.7 69.6 50.4 82.8 pretrain ckpt

Run Experiments

Data preparation

Download R2V2 (Random Related Video Views) dataset according to https://github.com/danielgordon10/vince.

The direction structure should be as followed:

CycleContrast
├── cycle_contrast 
├── scripts 
├── utils 
├── data
│   ├── r2v2_large_with_ids 
│   │   ├── train 
│   │   │   ├── --/
│   │   │   ├── -_/
│   │   │   ├── _-/
│   │   │   ├── __/
│   │   │   ├── -0/
│   │   │   ├── _0/
│   │   │   ├── ...
│   │   │   ├── zZ/
│   │   │   ├── zz/
│   │   ├── val
│   │   │   ├── --/
│   │   │   ├── -_/
│   │   │   ├── _-/
│   │   │   ├── __/
│   │   │   ├── -0/
│   │   │   ├── _0/
│   │   │   ├── ...
│   │   │   ├── zZ/
│   │   │   ├── zz/

Unsupervised Pretrain

./scripts/train_cycle.sh

Downstream task - ImageNet linear eval

Prepare ImageNet dataset according to pytorch ImageNet training code.

MODEL_DIR=output/cycle_res50_r2v2_ep200
IMAGENET_DATA=data/ILSVRC/Data/CLS-LOC
./scripts/eval_ImageNet.sh $MODEL_DIR $IMAGENET_DATA

Downstream task - OTB tracking

Transfer to OTB tracking evaluation is based on SiamFC-Pytorch. Please prepare environment and data according to SiamFC-Pytorch

git clone https://github.com/happywu/mmaction2-CycleContrast
# path to your pretrained model, change accordingly
CycleContrast=/home/user/code/CycleContrast
PRETRAIN=${CycleContrast}/output/cycle_res50_r2v2_ep200/checkpoint_0199.pth.tar
cd mmaction2_tracking
./scripts/submit_r2v2_r50_cycle.py ${PRETRAIN}

Downstream task - UCF classification

Transfer to UCF action recognition evaluation is based on AVID-CMA, prepare data and env according to AVID-CMA.

git clone https://github.com/happywu/AVID-CMA-CycleContrast
# path to your pretrained model, change accordingly
CycleContrast=/home/user/code/CycleContrast
PRETRAIN=${CycleContrast}/output/cycle_res50_r2v2_ep200/checkpoint_0199.pth.tar
cd AVID-CMA-CycleContrast 
./scripts/submit_r2v2_r50_cycle.py ${PRETRAIN}

Acknowledgements

The codebase is based on FAIR-MoCo. The OTB tracking evaluation is based on MMAction2, SiamFC-PyTorch and vince. The UCF classification evaluation follows AVID-CMA.

Thank you all for the great open source repositories!

You might also like...
[ICCV'21] Official implementation for the paper Social NCE: Contrastive Learning of Socially-aware Motion Representations
[ICCV'21] Official implementation for the paper Social NCE: Contrastive Learning of Socially-aware Motion Representations

CrowdNav with Social-NCE This is an official implementation for the paper Social NCE: Contrastive Learning of Socially-aware Motion Representations by

VITA lab at EPFL 125 Dec 23, 2022
PyTorch implementation of SimCLR: A Simple Framework for Contrastive Learning of Visual Representations
PyTorch implementation of SimCLR: A Simple Framework for Contrastive Learning of Visual Representations

PyTorch implementation of SimCLR: A Simple Framework for Contrastive Learning of Visual Representations

Thalles Silva 1.7k Dec 28, 2022
Supervised Contrastive Learning for Downstream Optimized Sequence Representations
Supervised Contrastive Learning for Downstream Optimized Sequence Representations

SupCL-Seq 📖 Supervised Contrastive Learning for Downstream Optimized Sequence representations (SupCS-Seq) accepted to be published in EMNLP 2021, ext

Hooman Sedghamiz 18 Oct 21, 2022
《LXMERT: Learning Cross-Modality Encoder Representations from Transformers》(EMNLP 2020)

The Most Important Thing. Our code is developed based on: LXMERT: Learning Cross-Modality Encoder Representations from Transformers

null 53 Dec 16, 2022
SUPERVISED-CONTRASTIVE-LEARNING-FOR-PRE-TRAINED-LANGUAGE-MODEL-FINE-TUNING - The Facebook paper about fine tuning RoBERTa with contrastive loss
SUPERVISED-CONTRASTIVE-LEARNING-FOR-PRE-TRAINED-LANGUAGE-MODEL-FINE-TUNING - The Facebook paper about fine tuning RoBERTa with contrastive loss

"# SUPERVISED-CONTRASTIVE-LEARNING-FOR-PRE-TRAINED-LANGUAGE-MODEL-FINE-TUNING" i

Saeed Lotfi 28 Dec 12, 2022
Self-Learned Video Rain Streak Removal: When Cyclic Consistency Meets Temporal Correspondence
Self-Learned Video Rain Streak Removal: When Cyclic Consistency Meets Temporal Correspondence

In this paper, we address the problem of rain streaks removal in video by developing a self-learned rain streak removal method, which does not require any clean groundtruth images in the training process.

Yang Wenhan 44 Dec 6, 2022
Cross Quality LFW: A database for Analyzing Cross-Resolution Image Face Recognition in Unconstrained Environments

Cross-Quality Labeled Faces in the Wild (XQLFW) Here, we release the database, evaluation protocol and code for the following paper: Cross Quality LFW

Martin Knoche 10 Dec 12, 2022
Pytorch Implementation for NeurIPS (oral) paper: Pixel Level Cycle Association: A New Perspective for Domain Adaptive Semantic Segmentation

Pixel-Level Cycle Association This is the Pytorch implementation of our NeurIPS 2020 Oral paper Pixel-Level Cycle Association: A New Perspective for D

null 87 Oct 19, 2022
Code and models for ICCV2021 paper
Code and models for ICCV2021 paper "Robust Object Detection via Instance-Level Temporal Cycle Confusion".

Robust Object Detection via Instance-Level Temporal Cycle Confusion This repo contains the implementation of the ICCV 2021 paper, Robust Object Detect

Xin Wang 69 Oct 13, 2022
Owner
GitHub Repository
Source for the paper "Universal Activation Function for machine learning"

Universal Activation Function Tensorflow and Pytorch source code for the paper Yuen, Brosnan, Minh Tu Hoang, Xiaodai Dong, and Tao Lu. "Universal acti

4 Dec 03, 2022
Noether Networks: meta-learning useful conserved quantities

Noether Networks: meta-learning useful conserved quantities This repository contains the code necessary to reproduce experiments from "Noether Network

Dylan Doblar 33 Nov 23, 2022
"Learning Free Gait Transition for Quadruped Robots vis Phase-Guided Controller"

PhaseGuidedControl The current version is developed based on the old version of RaiSim series, and possibly requires further modification. It will be

X-Mechanics 12 Oct 21, 2022
Dataloader tools for language modelling

Installation: pip install lm_dataloader Design Philosophy A library to unify lm dataloading at large scale Simple interface, any tokenizer can be inte

5 Mar 25, 2022
Download from Onlyfans.com.

OnlySave: Onlyfans downloader Getting Started: Download the setup executable from the latest release. Install and run. Only works on Windows currently

4 May 30, 2022
Visual Adversarial Imitation Learning using Variational Models (VMAIL)

Visual Adversarial Imitation Learning using Variational Models (VMAIL) This is the official implementation of the NeurIPS 2021 paper. Project website

14 Nov 18, 2022
Inhomogeneous Social Recommendation with Hypergraph Convolutional Networks

Inhomogeneous Social Recommendation with Hypergraph Convolutional Networks This is our Pytorch implementation for the paper: Zirui Zhu, Chen Gao, Xu C

Zirui Zhu 3 Dec 30, 2022
A novel Engagement Detection with Multi-Task Training (ED-MTT) system

A novel Engagement Detection with Multi-Task Training (ED-MTT) system which minimizes MSE and triplet loss together to determine the engagement level of students in an e-learning environment.

Onur Çopur 12 Nov 11, 2022
Jax/Flax implementation of Variational-DiffWave.

jax-variational-diffwave Jax/Flax implementation of Variational-DiffWave. (Zhifeng Kong et al., 2020, Diederik P. Kingma et al., 2021.) DiffWave with

YoungJoong Kim 37 Dec 16, 2022
Miscellaneous and lightweight network tools

Network Tools Collection of miscellaneous and lightweight network tools to simplify daily operations, administration, and troubleshooting of networks.

Nicholas Russo 22 Mar 22, 2022
An Artificial Intelligence trying to drive a car by itself on a user created map

An Artificial Intelligence trying to drive a car by itself on a user created map

Akhil Sahukaru 17 Jan 13, 2022
ONNX Runtime: cross-platform, high performance ML inferencing and training accelerator

ONNX Runtime is a cross-platform inference and training machine-learning accelerator. ONNX Runtime inference can enable faster customer experiences an

Microsoft 8k Jan 04, 2023
Comp445 project - Data Communications & Computer Networks

COMP-445 Data Communications & Computer Networks Change Python version in Conda

Peng Zhao 2 Oct 03, 2022
LSSY量化交易系统

LSSY量化交易系统 该项目是本人3年来研究量化慢慢积累开发的一套系统，属于早期作品慢慢修改而来，仅供学习研究，回测分析，实盘交易部分未公开

55 Oct 04, 2022
Key information extraction from invoice document with Graph Convolution Network

Key Information Extraction from Scanned Invoices Key information extraction from invoice document with Graph Convolution Network Related blog post fro

Phan Hoang 39 Dec 16, 2022
Code for the paper SphereRPN: Learning Spheres for High-Quality Region Proposals on 3D Point Clouds Object Detection, ICIP 2021.

SphereRPN Code for the paper SphereRPN: Learning Spheres for High-Quality Region Proposals on 3D Point Clouds Object Detection, ICIP 2021. Authors: Th

Thang Vu 15 Dec 02, 2022
Safe Control for Black-box Dynamical Systems via Neural Barrier Certificates

Safe Control for Black-box Dynamical Systems via Neural Barrier Certificates Installation Clone the repository: git clone https://github.com/Zengyi-Qi

Zengyi Qin 3 Oct 18, 2022
Do Neural Networks for Segmentation Understand Insideness?

This is part of the code to reproduce the results of the paper Do Neural Networks for Segmentation Understand Insideness? [pdf] by K. Villalobos (*),

biolins 0 Mar 20, 2021
A torch.Tensor-like DataFrame library supporting multiple execution runtimes and Arrow as a common memory format

TorchArrow (Warning: Unstable Prototype) This is a prototype library currently under heavy development. It does not currently have stable releases, an

Facebook Research 536 Jan 06, 2023
ConformalLayers: A non-linear sequential neural network with associative layers

ConformalLayers: A non-linear sequential neural network with associative layers ConformalLayers is a conformal embedding of sequential layers of Convo

Prograf-UFF 5 Sep 28, 2022