Unsupervised Representation Learning by Invariance Propagation

Last update: Jul 06, 2022

Related tags

Deep Learning InvariancePropagation

Overview

Unsupervised Learning by Invariance Propagation

This repository is the official implementation of Unsupervised Learning by Invariance Propagation.

Pretraining on Natual Images

Train on ImageNet

To train the model(s) in the paper, run this command:

python main.py --exp 'your_path' --n_background 4096 --t 0.2 --blur --cos --network 'resnet50' --nonlinearhead 1 --weight_decay 1e-4

Evaluation

To evaluate the model on ImageNet, run:

python -m downstream.linear_classification.linear_classification --gpus '0,1' --exp 'your_exp_path' --pretrained_path 'pretrain_path' --backbone 'resnet50'

Notice that in the paper, to calculate the BFS results, we require to record the id of neighbours of each anchor point. For computational efficiency, we apprximate the BFS results by only concatenating the neighbours of each point, up to L steps. This results may be a little different with the real BFS results due to there exists repeated samples, however it works pretty well, both effectively and efficiently. Pretrained model can be found here.

Train on Cifar

To train the model(s) in cifar10 and cifar100 or svhn, run this command:

# cifar10
python main.py --exp 'your_path' -n_background 4096 --t 0.2 --blur --cos --network 'resnet18_cifar' --nonlinearhead 1 --weight_decay 5e-4 --n_pos 20 --dataset 'cifar10'
# cifar100
python main.py --exp 'your_path' -n_background 4096 --t 0.2 --blur --cos --network 'resnet18_cifar' --nonlinearhead 1 --weight_decay 5e-4 --n_pos 20 --dataset 'cifar100'
# svhn
python main.py --exp 'your_path' -n_background 4096 --t 0.2 --blur --cos --network 'resnet18_cifar' --nonlinearhead 1 --weight_decay 5e-4 --n_pos 20 --dataset 'svhn'

Evaluation

To train the model(s) in cifar10 and cifar100 run this command:

# cifar10
python -m downstream.linear_classification.eval_linear --gpus '0,1' --exp 'your_exp_path' --pretrained_path 'pretrain_path' --backbone 'resnet18_cifar' --dataset 'cifar10'
# cifar100
python -m downstream.linear_classification.eval_linear --gpus '0,1' --exp 'your_exp_path' --pretrained_path 'pretrain_path' --backbone 'resnet18_cifar' --dataset 'cifar100'
# svhn
python -m downstream.linear_classification.eval_linear --gpus '0,1' --exp 'your_exp_path' --pretrained_path 'pretrain_path' --backbone 'resnet18_cifar' --dataset 'svhn'

Pretraining on Defect Classification Dataset

For validate the effectiveness and practicabilities of the proposed algorithms, we can also train and evaluate our method on Defect Detection Dataset.

Train on WM811.

python main.py --gpus '0,1,2' --exp 'output/' --n_background 4096 --t 0.07 --cos --network 'resnet18_wm811' --dataset 'wm811' --nonlinearhead 0 --weight_decay 5e-4

Evaluation

To evaluate the model on WM811, run:

python -m downstream.fine_tune_wm811 --save_folder 'your_output_folder' --model_path 'your_pretrain_model' --model 'resnet18_wm811' --dataset 'wm811' --weight_decay 1e-3 --learning_rate1 0.001 --learning_rate2 0.002 --label_smoothing 0.1 --dropout 0.5

Unsupervised Representation Learning by Invariance Propagation

Related tags

Overview

Unsupervised Learning by Invariance Propagation

Pretraining on Natual Images

Train on ImageNet

Evaluation

Train on Cifar

Evaluation

Pretraining on Defect Classification Dataset

Train on WM811.

Evaluation

Owner

FengWang

This is the official code release for the paper Shape and Material Capture at Home

PyTorch implementation for the paper Pseudo Numerical Methods for Diffusion Models on Manifolds

object recognition with machine learning on Respberry pi

Deep learning algorithms for muon momentum estimation in the CMS Trigger System

Affine / perspective transformation in Pose Estimation with Tensorflow 2

Source code for paper "Deep Superpixel-based Network for Blind Image Quality Assessment"

SMPL-X: A new joint 3D model of the human body, face and hands together

This is the workbook I created while I was studying for the Qiskit Associate Developer exam. I hope this becomes useful to others as it was for me :)

Code for Greedy Gradient Ensemble for Visual Question Answering （ICCV 2021, Oral）

Where2Act: From Pixels to Actions for Articulated 3D Objects

Self-supervised Augmentation Consistency for Adapting Semantic Segmentation (CVPR 2021)

Python library for analysis of time series data including dimensionality reduction, clustering, and Markov model estimation

PyTorch Connectomics: segmentation toolbox for EM connectomics

arxiv-sanity, but very lite, simply providing the core value proposition of the ability to tag arxiv papers of interest and have the program recommend similar papers.

Make your own game in a font!

Interpretable and Generalizable Person Re-Identification with Query-Adaptive Convolution and Temporal Lifting

Autonomous racing with the Anki Overdrive

PyTorch implementation of Deformable Convolution

PyTorch implementation of Off-policy Learning in Two-stage Recommender Systems

Machine learning framework for both deep learning and traditional algorithms