IAST: Instance Adaptive Self-training for Unsupervised Domain Adaptation (ECCV 2020)

Overview

IAST: Instance Adaptive Self-training for Unsupervised Domain Adaptation (ECCV 2020) Tweet

This repo is the official implementation of our paper "Instance Adaptive Self-training for Unsupervised Domain Adaptation". The purpose of this repo is to better communicate with you and respond to your questions. This repo is almost the same with Another-Version, and you can also refer to that version.

Introduction

Abstract

The divergence between labeled training data and unlabeled testing data is a significant challenge for recent deep learning models. Unsupervised domain adaptation (UDA) attempts to solve such a problem. Recent works show that self-training is a powerful approach to UDA. However, existing methods have difficulty in balancing scalability and performance. In this paper, we propose an instance adaptive self-training framework for UDA on the task of semantic segmentation. To effectively improve the quality of pseudo-labels, we develop a novel pseudo-label generation strategy with an instance adaptive selector. Besides, we propose the region-guided regularization to smooth the pseudo-label region and sharpen the non-pseudo-label region. Our method is so concise and efficient that it is easy to be generalized to other unsupervised domain adaptation methods. Experiments on 'GTA5 to Cityscapes' and 'SYNTHIA to Cityscapes' demonstrate the superior performance of our approach compared with the state-of-the-art methods.

IAST Overview

Result

source target device GPU memory mIoU-19 mIoU-16 mIoU-13 model
GTA5 Cityscapes Tesla V100-32GB 18.5 GB 51.88 - - download
GTA5 Cityscapes Tesla T4 6.3 GB 51.20 - - download
SYNTHIA Cityscapes Tesla V100-32GB 18.5 GB - 51.54 57.81 download
SYNTHIA Cityscapes Tesla T4 9.8 GB - 51.24 57.70 download

Setup

1) Envs

  • Pytorch >= 1.0
  • Python >= 3.6
  • cuda >= 9.0

Install python packages

$ pip install -r  requirements.txt

apex : Tools for easy mixed precision and distributed training in Pytorch

git clone https://github.com/NVIDIA/apex
cd apex
pip install -v --no-cache-dir --global-option="--cpp_ext" --global-option="--cuda_ext" ./

2) Download Dataset

Please download the datasets from these links:

Dataset directory should have this structure:

${ROOT_DIR}/data/GTA5/
${ROOT_DIR}/data/GTA5/images
${ROOT_DIR}/data/GTA5/labels

${ROOT_DIR}/data/SYNTHIA_RAND_CITYSCAPES/RAND_CITYSCAPES
${ROOT_DIR}/data/SYNTHIA_RAND_CITYSCAPES/RAND_CITYSCAPES/RGB
${ROOT_DIR}/data/SYNTHIA_RAND_CITYSCAPES/RAND_CITYSCAPES/GT

${ROOT_DIR}/data/cityscapes
${ROOT_DIR}/data/cityscapes/leftImg8bit
${ROOT_DIR}/data/cityscapes/gtFine

3) Download Pretrained Models

We provide pre-trained models. We recommend that you download them and put them in pretrained_models/, which will save a lot of time for training and ensure consistent results.

V100 models

T4 models

(Optional) Of course, if you have plenty of time, you can skip this step and start training from scratch. We also provide these scripts.

Training

Our original experiments are all carried out on Tesla-V100, and there will be a large number of GPU memory usage (batch_size=8). For low GPU memory devices, we also trained on Tesla-T4 to ensure that most people can reproduce the results (batch_size=2).

Start self-training (download the pre-trained models first)

cd code

# GTA5 to Cityscapes (V100)
sh ../scripts/self_training_only/run_gtav2cityscapes_self_traing_only_v100.sh
# GTA5 to Cityscapes (T4)
sh ../scripts/self_training_only/run_gtav2cityscapes_self_traing_only_t4.sh
# SYNTHIA to Cityscapes (V100)
sh ../scripts/self_training_only/run_syn2cityscapes_self_traing_only_v100.sh
# SYNTHIA to Cityscapes (T4)
sh ../scripts/self_training_only/run_syn2cityscapes_self_traing_only_t4.sh

(Optional) Training from scratch

cd code

# GTA5 to Cityscapes (V100)
sh ../scripts/from_scratch/run_gtav2cityscapes_self_traing_v100.sh
# GTA5 to Cityscapes (T4)
sh ../scripts/from_scratch/run_gtav2cityscapes_self_traing_t4.sh
# SYNTHIA to Cityscapes (V100)
sh ../scripts/from_scratch/run_syn2cityscapes_self_traing_v100.sh
# SYNTHIA to Cityscapes (T4)
sh ../scripts/from_scratch/run_syn2cityscapes_self_traing_t4.sh

Evaluation

cd code
python eval.py --config_file  --resume_from 

Support multi-scale testing and flip testing.

# Modify the following parameters in the config file

TEST:
  RESIZE_SIZE: [[1024, 512], [1280, 640], [1536, 768], [1800, 900], [2048, 1024]] 
  USE_FLIP: False 

Citation

Please cite this paper in your publications if it helps your research:

@article{mei2020instance,
  title={Instance Adaptive Self-Training for Unsupervised Domain Adaptation},
  author={Mei, Ke and Zhu, Chuang and Zou, Jiaqi and Zhang, Shanghang},
  booktitle={European Conference on Computer Vision (ECCV)},
  year={2020}
}

Author

Ke Mei, Chuang Zhu

If you have any questions, you can contact me directly.

Owner
CVSM Group - email: [email protected]
Codes of our papers are released in this GITHUB account.
CVSM Group - email: <a href=[email protected]">
Source code for Zalo AI 2021 submission

zalo_ltr_2021 Source code for Zalo AI 2021 submission Solution: Pipeline We use the pipepline in the picture below: Our pipeline is combination of BM2

128 Dec 27, 2022
This repository contains the code for the paper "PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitization"

PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitization News: [2020/05/04] Added EGL rendering option for training data g

Shunsuke Saito 1.5k Jan 03, 2023
Deploying PyTorch Model to Production with FastAPI in CUDA-supported Docker

Deploying PyTorch Model to Production with FastAPI in CUDA-supported Docker A example FastAPI PyTorch Model deploy with nvidia/cuda base docker. Model

Ming 68 Jan 04, 2023
Official implementation of EdiTTS: Score-based Editing for Controllable Text-to-Speech

EdiTTS: Score-based Editing for Controllable Text-to-Speech Official implementation of EdiTTS: Score-based Editing for Controllable Text-to-Speech. Au

Neosapience 98 Dec 25, 2022
Single Image Random Dot Stereogram for Tensorflow

TensorFlow-SIRDS Single Image Random Dot Stereogram for Tensorflow SIRDS is a means to present 3D data in a 2D image. It allows for scientific data di

Greg Peatfield 5 Aug 10, 2022
Molecular Sets (MOSES): A Benchmarking Platform for Molecular Generation Models

Molecular Sets (MOSES): A benchmarking platform for molecular generation models Deep generative models are rapidly becoming popular for the discovery

MOSES 656 Dec 29, 2022
This is the repository for The Machine Learning Workshops, published by AI DOJO

This is the repository for The Machine Learning Workshops, published by AI DOJO. It contains all the workshop's code with supporting project files necessary to work through the code.

AI Dojo 12 May 06, 2022
Code + pre-trained models for the paper Keeping Your Eye on the Ball Trajectory Attention in Video Transformers

Motionformer This is an official pytorch implementation of paper Keeping Your Eye on the Ball: Trajectory Attention in Video Transformers. In this rep

Facebook Research 192 Dec 23, 2022
A PyTorch library and evaluation platform for end-to-end compression research

CompressAI CompressAI (compress-ay) is a PyTorch library and evaluation platform for end-to-end compression research. CompressAI currently provides: c

InterDigital 680 Jan 06, 2023
StableSims is an open-source project aimed at simulating MakerDAO's Dai stablecoin system

StableSims is an open-source project aimed at simulating MakerDAO's Dai stablecoin system, initially used for researching optimal incentive parameters for Liquidations 2.0.

Blockchain at Berkeley 52 Nov 21, 2022
Official implementation of paper "Query2Label: A Simple Transformer Way to Multi-Label Classification".

Introdunction This is the official implementation of the paper "Query2Label: A Simple Transformer Way to Multi-Label Classification". Abstract This pa

Shilong Liu 274 Dec 28, 2022
TAP: Text-Aware Pre-training for Text-VQA and Text-Caption, CVPR 2021 (Oral)

TAP: Text-Aware Pre-training TAP: Text-Aware Pre-training for Text-VQA and Text-Caption by Zhengyuan Yang, Yijuan Lu, Jianfeng Wang, Xi Yin, Dinei Flo

Microsoft 61 Nov 14, 2022
Code, final versions, and information on the Sparkfun Graphical Datasheets

Graphical Datasheets Code, final versions, and information on the SparkFun Graphical Datasheets. Generated Cells After Running Script Example Complete

SparkFun Electronics 102 Jan 05, 2023
Python Library for Signal/Image Data Analysis with Transport Methods

PyTransKit Python Transport Based Signal Processing Toolkit Website and documentation: https://pytranskit.readthedocs.io/ Installation The library cou

24 Dec 23, 2022
WSDM2022 Challenge - Large scale temporal graph link prediction

WSDM 2022 Large-scale Temporal Graph Link Prediction - Baseline and Initial Test Set WSDM Cup Website link Link to this challenge This branch offers A

Deep Graph Library 34 Dec 29, 2022
Team Enigma at ArgMining 2021 Shared Task: Leveraging Pretrained Language Models for Key Point Matching

Team Enigma at ArgMining 2021 Shared Task: Leveraging Pretrained Language Models for Key Point Matching This is our attempt of the shared task on Quan

Manav Nitin Kapadnis 12 Jul 08, 2022
PyTorch implementation for paper Neural Marching Cubes.

NMC PyTorch implementation for paper Neural Marching Cubes, Zhiqin Chen, Hao Zhang. Paper | Supplementary Material (to be updated) Citation If you fin

Zhiqin Chen 109 Dec 27, 2022
NeurIPS 2021, "Fine Samples for Learning with Noisy Labels"

[Official] FINE Samples for Learning with Noisy Labels This repository is the official implementation of "FINE Samples for Learning with Noisy Labels"

mythbuster 27 Dec 23, 2022
Network Pruning That Matters: A Case Study on Retraining Variants (ICLR 2021)

Network Pruning That Matters: A Case Study on Retraining Variants (ICLR 2021)

Duong H. Le 18 Jun 13, 2022
scAR (single-cell Ambient Remover) is a package for data denoising in single-cell omics.

scAR scAR (single cell Ambient Remover) is a package for denoising multiple single cell omics data. It can be used for multiple tasks, such as, sgRNA

19 Nov 28, 2022