Official Code for "Constrained Mean Shift Using Distant Yet Related Neighbors for Representation Learning"

Last update: Nov 25, 2022

Related tags

Deep Learning CMSF

Overview

CMSF

Official Code for "Constrained Mean Shift Using Distant Yet Related Neighbors for Representation Learning"

Requirements

Python >= 3.7.6
PyTorch >= 1.4
torchvision >= 0.5.0
faiss-gpu >= 1.6.1

Install PyTorch and ImageNet dataset following the official PyTorch ImageNet training code. We used Python 3.7 for our experiments.

Install PyTorch (pytorch.org)

To run NN and CMSF-KM, you require to install FAISS.

FAISS:

Install FAISS (https://github.com/facebookresearch/faiss/blob/master/INSTALL.md)

Training Self-Supservised CMSF-KM

python self_supervised/train_msf_km.py \
  --cos \
  --weak_strong \
  --learning_rate 0.05 \
  --epochs 200 \
  --arch resnet50 \
  --topk 5 \
  --momentum 0.99 \
  --mem_bank_size 128000 \
  --num_clusters 50000 \
  --checkpoint_path <CHECKPOINT PATH> \
  <DATASET PATH>

Training Self-Supservised CMSF-2Q

python self_supervised/train_msf_2q.py \
  --cos \
  --weak_strong \
  --learning_rate 0.05 \
  --epochs 200 \
  --arch resnet50 \
  --topk 5 \
  --momentum 0.99 \
  --mem_bank_size 128000 \
  --topkp 5 \
  --checkpoint_path <CHECKPOINT PATH> \
  <DATASET PATH>

Training Supservised

Following command can be used to train the CMSF(Supervised Learning)

python supervised/train_sup_msf.py \
  --cos \
  --weak_strong \
  --learning_rate 0.05 \
  --epochs 200 \
  --arch resnet50 \
  --topk 10 \
  --momentum 0.99 \
  --mem_bank_size 128000 \
  --checkpoint_path <CHECKPOINT PATH> \
  <DATASET PATH>

License

This project is under the MIT license.

Official Code for "Constrained Mean Shift Using Distant Yet Related Neighbors for Representation Learning"

Related tags

Overview

CMSF

Requirements

Training Self-Supservised CMSF-KM

Training Self-Supservised CMSF-2Q

Training Supservised

License

Owner

Price-Prediction-For-a-Dream-Home - A machine learning based linear regression trained model for house price prediction.

Fine-Tune EleutherAI GPT-Neo to Generate Netflix Movie Descriptions in Only 47 Lines of Code Using Hugginface And DeepSpeed

Pytorch Performace Tuning, WandB, AMP, Multi-GPU, TensorRT, Triton

Hidden-Fold Networks (HFN): Random Recurrent Residuals Using Sparse Supermasks

Code for Dual Contrastive Learning for Unsupervised Image-to-Image Translation, NTIRE, CVPRW 2021.

Language Models Can See: Plugging Visual Controls in Text Generation

Efficient and intelligent interactive segmentation annotation software

A PyTorch implementation for Unsupervised Domain Adaptation by Backpropagation(DANN), support Office-31 and Office-Home dataset

Awesome AI Learning with +100 AI Cheat-Sheets, Free online Books, Top Courses, Best Videos and Lectures, Papers, Tutorials, +99 Researchers, Premium Websites, +121 Datasets, Conferences, Frameworks, Tools

Generative Flow Networks for Discrete Probabilistic Modeling

Anomaly Localization in Model Gradients Under Backdoor Attacks Against Federated Learning

Official Code for "Constrained Mean Shift Using Distant Yet Related Neighbors for Representation Learning"

"Learning and Analyzing Generation Order for Undirected Sequence Models" in Findings of EMNLP, 2021

Identify the emotion of multiple speakers in an Audio Segment

A Factor Model for Persistence in Investment Manager Performance

Official implementation of "Learning Not to Reconstruct" (BMVC 2021)

Spatial Temporal Graph Convolutional Networks (ST-GCN) for Skeleton-Based Action Recognition in PyTorch

[NeurIPS 2021] Well-tuned Simple Nets Excel on Tabular Datasets

Official implementation of YOGO for Point-Cloud Processing

AI4Good project for detecting waste in the environment