Mae segmentation - Reproduction of semantic segmentation using masked autoencoder (mae)

Last update: Dec 17, 2022

Overview

ADE20k Semantic segmentation with MAE

Getting started

Install the mmsegmentation library and some required packages.

pip install mmcv-full==1.3.0 mmsegmentation==0.11.0
pip install scipy timm==0.3.2

Install apex for mixed-precision training

git clone https://github.com/NVIDIA/apex
cd apex
pip install -v --disable-pip-version-check --no-cache-dir --global-option="--cpp_ext" --global-option="--cuda_ext" ./

Follow the guide in mmseg to prepare the ADE20k dataset.

Fine-tuning for Reproducing Results of MAE ViT-Base

Command:

tools/dist_train.sh configs/mae/upernet_mae_base_12_512_slide_160k_ade20k.py 8 --seed 0  --options model.pretrained=https://dl.fbaipublicfiles.com/mae/pretrain/mae_pretrain_vit_base.pth

Expected results log(paper results: 48.1 mIoU):

+--------+-------+-------+-------+
| Scope  | mIoU  | mAcc  | aAcc  |
+--------+-------+-------+-------+
| global | 48.15 | 58.99 | 83.05 |
+--------+-------+-------+-------+

Evaluation

Command format:

tools/dist_test.sh  <CONFIG_PATH> <CHECKPOINT_PATH> <NUM_GPUS> --eval mIoU

Acknowledgment

This code is built using the mmsegmentation library, Timm library, the Swin repository, XCiT, SETR, BEiT and the MAE repository.

Mae segmentation - Reproduction of semantic segmentation using masked autoencoder (mae)

Related tags

Overview

ADE20k Semantic segmentation with MAE

Getting started

Fine-tuning for Reproducing Results of MAE ViT-Base

Evaluation

Acknowledgment

Owner

Fuzzing the Kernel Using Unicornafl and AFL++

A Neural Net Training Interface on TensorFlow, with focus on speed + flexibility

a basic code repository for basic task in CV(classification,detection,segmentation)

Differentiable scientific computing library

Code to reproduce the experiments in the paper "Transformer Based Multi-Source Domain Adaptation" (EMNLP 2020)

PyTorch implementation of Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasets

AIR^2 for Interaction Prediction

IEEE Winter Conference on Applications of Computer Vision 2022 Accepted

This is the implementation of "SELF SUPERVISED REPRESENTATION LEARNING WITH DEEP CLUSTERING FOR ACOUSTIC UNIT DISCOVERY FROM RAW SPEECH" submitted to ICASSP 2022

Attendance Monitoring with Face Recognition using Python

Materials for upcoming beginner-friendly PyTorch course (work in progress).

Learning to Predict Gradients for Semi-Supervised Continual Learning

DAN: Unfolding the Alternating Optimization for Blind Super Resolution

Real-time object detection on Android using the YOLO network with TensorFlow

The official codes of "Semi-supervised Models are Strong Unsupervised Domain Adaptation Learners".

Transformer part of 12th place solution in Riiid! Answer Correctness Prediction

Learning a mapping from images to psychological similarity spaces with neural networks.

Using a Seq2Seq RNN architecture via TensorFlow to predict future Bitcoin prices

Head2Toe: Utilizing Intermediate Representations for Better OOD Generalization

Denoising Normalizing Flow