Mae segmentation - Reproduction of semantic segmentation using masked autoencoder (mae)

Last update: Dec 17, 2022

Overview

ADE20k Semantic segmentation with MAE

Getting started

Install the mmsegmentation library and some required packages.

pip install mmcv-full==1.3.0 mmsegmentation==0.11.0
pip install scipy timm==0.3.2

Install apex for mixed-precision training

git clone https://github.com/NVIDIA/apex
cd apex
pip install -v --disable-pip-version-check --no-cache-dir --global-option="--cpp_ext" --global-option="--cuda_ext" ./

Follow the guide in mmseg to prepare the ADE20k dataset.

Fine-tuning for Reproducing Results of MAE ViT-Base

Command:

tools/dist_train.sh configs/mae/upernet_mae_base_12_512_slide_160k_ade20k.py 8 --seed 0  --options model.pretrained=https://dl.fbaipublicfiles.com/mae/pretrain/mae_pretrain_vit_base.pth

Expected results log(paper results: 48.1 mIoU):

+--------+-------+-------+-------+
| Scope  | mIoU  | mAcc  | aAcc  |
+--------+-------+-------+-------+
| global | 48.15 | 58.99 | 83.05 |
+--------+-------+-------+-------+

Evaluation

Command format:

tools/dist_test.sh  <CONFIG_PATH> <CHECKPOINT_PATH> <NUM_GPUS> --eval mIoU

Acknowledgment

This code is built using the mmsegmentation library, Timm library, the Swin repository, XCiT, SETR, BEiT and the MAE repository.

Mae segmentation - Reproduction of semantic segmentation using masked autoencoder (mae)

Related tags

Overview

ADE20k Semantic segmentation with MAE

Getting started

Fine-tuning for Reproducing Results of MAE ViT-Base

Evaluation

Acknowledgment

Owner

PyTorch implementation of PSPNet segmentation network

The fundamental package for scientific computing with Python.

RITA is a family of autoregressive protein models, developed by LightOn in collaboration with the OATML group at Oxford and the Debora Marks Lab at Harvard.

Solutions and questions for AoC2021. Merry christmas!

Official repository for ABC-GAN

The repository for the paper "When Do You Need Billions of Words of Pretraining Data?"

Official implementation for "Low-light Image Enhancement via Breaking Down the Darkness"

This repo includes the CUB-GHA (Gaze-based Human Attention) dataset and code of the paper "Human Attention in Fine-grained Classification".

TCTrack: Temporal Contexts for Aerial Tracking (CVPR2022)

PyTorch code for our paper "Attention in Attention Network for Image Super-Resolution"

Repository of Vision Transformer with Deformable Attention

WORD: Revisiting Organs Segmentation in the Whole Abdominal Region

Details about the wide minima density hypothesis and metrics to compute width of a minima

Paper: Cross-View Kernel Similarity Metric Learning Using Pairwise Constraints for Person Re-identification

Hypernetwork-Ensemble Learning of Segmentation Probability for Medical Image Segmentation with Ambiguous Labels

Implementation for "Exploiting Aliasing for Manga Restoration" (CVPR 2021)

Convex optimization for fun and profit.

SwinIR: Image Restoration Using Swin Transformer

AdvStyle - Official PyTorch Implementation

Punctuation Restoration using Transformer Models for High-and Low-Resource Languages