Unsupervised Representation Learning by Invariance Propagation

Last update: Jul 06, 2022

Related tags

Deep Learning InvariancePropagation

Overview

Unsupervised Learning by Invariance Propagation

This repository is the official implementation of Unsupervised Learning by Invariance Propagation.

Pretraining on Natual Images

Train on ImageNet

To train the model(s) in the paper, run this command:

python main.py --exp 'your_path' --n_background 4096 --t 0.2 --blur --cos --network 'resnet50' --nonlinearhead 1 --weight_decay 1e-4

Evaluation

To evaluate the model on ImageNet, run:

python -m downstream.linear_classification.linear_classification --gpus '0,1' --exp 'your_exp_path' --pretrained_path 'pretrain_path' --backbone 'resnet50'

Notice that in the paper, to calculate the BFS results, we require to record the id of neighbours of each anchor point. For computational efficiency, we apprximate the BFS results by only concatenating the neighbours of each point, up to L steps. This results may be a little different with the real BFS results due to there exists repeated samples, however it works pretty well, both effectively and efficiently. Pretrained model can be found here.

Train on Cifar

To train the model(s) in cifar10 and cifar100 or svhn, run this command:

# cifar10
python main.py --exp 'your_path' -n_background 4096 --t 0.2 --blur --cos --network 'resnet18_cifar' --nonlinearhead 1 --weight_decay 5e-4 --n_pos 20 --dataset 'cifar10'
# cifar100
python main.py --exp 'your_path' -n_background 4096 --t 0.2 --blur --cos --network 'resnet18_cifar' --nonlinearhead 1 --weight_decay 5e-4 --n_pos 20 --dataset 'cifar100'
# svhn
python main.py --exp 'your_path' -n_background 4096 --t 0.2 --blur --cos --network 'resnet18_cifar' --nonlinearhead 1 --weight_decay 5e-4 --n_pos 20 --dataset 'svhn'

Evaluation

To train the model(s) in cifar10 and cifar100 run this command:

# cifar10
python -m downstream.linear_classification.eval_linear --gpus '0,1' --exp 'your_exp_path' --pretrained_path 'pretrain_path' --backbone 'resnet18_cifar' --dataset 'cifar10'
# cifar100
python -m downstream.linear_classification.eval_linear --gpus '0,1' --exp 'your_exp_path' --pretrained_path 'pretrain_path' --backbone 'resnet18_cifar' --dataset 'cifar100'
# svhn
python -m downstream.linear_classification.eval_linear --gpus '0,1' --exp 'your_exp_path' --pretrained_path 'pretrain_path' --backbone 'resnet18_cifar' --dataset 'svhn'

Pretraining on Defect Classification Dataset

For validate the effectiveness and practicabilities of the proposed algorithms, we can also train and evaluate our method on Defect Detection Dataset.

Train on WM811.

python main.py --gpus '0,1,2' --exp 'output/' --n_background 4096 --t 0.07 --cos --network 'resnet18_wm811' --dataset 'wm811' --nonlinearhead 0 --weight_decay 5e-4

Evaluation

To evaluate the model on WM811, run:

python -m downstream.fine_tune_wm811 --save_folder 'your_output_folder' --model_path 'your_pretrain_model' --model 'resnet18_wm811' --dataset 'wm811' --weight_decay 1e-3 --learning_rate1 0.001 --learning_rate2 0.002 --label_smoothing 0.1 --dropout 0.5

Unsupervised Representation Learning by Invariance Propagation

Related tags

Overview

Unsupervised Learning by Invariance Propagation

Pretraining on Natual Images

Train on ImageNet

Evaluation

Train on Cifar

Evaluation

Pretraining on Defect Classification Dataset

Train on WM811.

Evaluation

Owner

FengWang

Mini Software that give reminder to drink water as per your weight.

Code for all the Advent of Code'21 challenges mostly written in python

Experiments for Neural Flows paper

A Neural Net Training Interface on TensorFlow, with focus on speed + flexibility

Wenzhou-Kean University AI-LAB

📚 Papermill is a tool for parameterizing, executing, and analyzing Jupyter Notebooks.

StyleSwin: Transformer-based GAN for High-resolution Image Generation

This is the repo of the manuscript "Dual-branch Attention-In-Attention Transformer for speech enhancement"

Data augmentation for NLP, accepted at EMNLP 2021 Findings

PyTorch Implementation for AAAI'21 "Do Response Selection Models Really Know What's Next? Utterance Manipulation Strategies for Multi-turn Response Selection"

Simple PyTorch hierarchical models.

A PyTorch Implementation of the Luna: Linear Unified Nested Attention

A simple AI that will give you si ple task and this is made with python

Roadmap to becoming a machine learning engineer in 2020

3D-CariGAN: An End-to-End Solution to 3D Caricature Generation from Normal Face Photos

🧠 A PyTorch implementation of 'Deep CORAL: Correlation Alignment for Deep Domain Adaptation.', ECCV 2016

Code for the published paper : Learning to recognize rare traffic sign

Face Detection & Age Gender & Expression & Recognition

Large-scale open domain KNOwledge grounded conVERsation system based on PaddlePaddle

This repository contains the source code of an efficient 1D probabilistic model for music time analysis proposed in ICASSP2022 venue.