VOS: Learning What You Don’t Know by Virtual Outlier Synthesis

Related tags

Deep Learningvos
Overview

VOS

This is the source code accompanying the paper VOS: Learning What You Don’t Know by Virtual Outlier Synthesis by Xuefeng Du, Zhaoning Wang, Mu Cai, and Yixuan Li

The codebase is heavily based on ProbDet and Detectron2.

Dataset Preparation

PASCAL VOC

Download the processed VOC 2007 and 2012 dataset from here.

The VOC dataset folder should have the following structure:

 └── VOC_DATASET_ROOT
     |
     ├── JPEGImages
     ├── voc0712_train_all.json
     └── val_coco_format.json

COCO

Download COCO2017 dataset from the official website.

Download the OOD dataset (json file) when the in-distribution dataset is Pascal VOC from here.

Download the OOD dataset (json file) when the in-distribution dataset is BDD-100k from here.

Put the two processed OOD json files to ./anntoations

The COCO dataset folder should have the following structure:

 └── COCO_DATASET_ROOT
     |
     ├── annotations
        ├── xxx (the original json files)
        ├── instances_val2017_ood_wrt_bdd_rm_overlap.json
        └── instances_val2017_ood_rm_overlap.json
     ├── train2017
     └── val2017

BDD-100k

Donwload the BDD-100k images from the official website.

Download the processed BDD-100k json files from here and here.

The BDD dataset folder should have the following structure:

 └── BDD_DATASET_ROOT
     |
     ├── images
     ├── val_bdd_converted.json
     └── train_bdd_converted.json

OpenImages

Download our OpenImages validation splits here. We created a tarball that contains the out-of-distribution data splits used in our paper for hyperparameter tuning. Do not modify or rename the internal folders as those paths are hard coded in the dataset reader. The OpenImages dataset is created in a similar way following this paper.

The OpenImages dataset folder should have the following structure:

 └── OEPNIMAGES_DATASET_ROOT
     |
     ├── coco_classes
     └── ood_classes_rm_overlap

Before training, modify the dataset address in the ./detection/core/datasets/setup_datasets.py according to your local dataset address.

Visualization of the OOD datasets

The OOD images with respect to different in-distribution datasets can be downloaded from ID-VOC-OOD-COCO, ID-VOC-OOD-openimages, ID-BDD-OOD-COCO, ID-BDD-OOD-openimages.

Training

Firstly, enter the detection folder by running

cd detection

Vanilla Faster-RCNN with VOC as the in-distribution dataset


python train_net.py
--num-gpus 8
--config-file VOC-Detection/faster-rcnn/vanilla.yaml 
--random-seed 0 
--resume

Vanilla Faster-RCNN with BDD as the in-distribution dataset

python train_net.py 
--num-gpus 8 
--config-file BDD-Detection/faster-rcnn/vanilla.yaml 
--random-seed 0 
--resume

VOS on ResNet

python train_net_gmm.py 
--num-gpus 8 
--config-file VOC-Detection/faster-rcnn/vos.yaml 
--random-seed 0 
--resume

VOS on RegNet

Before training using the RegNet as the backbone, download the pretrained RegNet backbone from here.

python train_net_gmm.py 
--num-gpus 8 
--config-file VOC-Detection/faster-rcnn/regnetx.yaml 
--random-seed 0 
--resume

Before training on VOS, change "VOS.STARTING_ITER" and "VOS.SAMPLE_NUMBER" in the config file to the desired numbers in paper.

Evaluation

Evaluation with the in-distribution dataset to be VOC

Firstly run on the in-distribution dataset:

python apply_net.py 
--test-dataset voc_custom_val 
--config-file VOC-Detection/faster-rcnn/vos.yaml 
--inference-config Inference/standard_nms.yaml 
--random-seed 0 
--image-corruption-level 0 
--visualize 0

Then run on the OOD dataset:

python apply_net.py
--test-dataset coco_ood_val 
--config-file VOC-Detection/faster-rcnn/vos.yaml 
--inference-config Inference/standard_nms.yaml 
--random-seed 0 
--image-corruption-level 0 
--visualize 0

Obtain the metrics using:

python voc_coco_plot.py 
--name vos 
--thres xxx 
--energy 1 
--seed 0

Here the threshold is determined according to ProbDet. It will be displayed in the screen as you finish evaluating on the in-distribution dataset.

Evaluation with the in-distribution dataset to be BDD

Firstly run on the in-distribution dataset:

python apply_net.py 
--test-dataset bdd_custom_val 
--config-file VOC-Detection/faster-rcnn/vos.yaml 
--inference-config Inference/standard_nms.yaml 
--random-seed 0 
--image-corruption-level 0 
--visualize 0

Then run on the OOD dataset:

python apply_net.py 
--test-dataset coco_ood_val_bdd 
--config-file VOC-Detection/faster-rcnn/vos.yaml 
--inference-config Inference/standard_nms.yaml 
--random-seed 0 
--image-corruption-level 0 
--visualize 0

Obtain the metrics using:

python bdd_coco_plot.py
--name vos 
--thres xxx 
--energy 1 
--seed 0

Pretrained models

The pretrained models for Pascal-VOC can be downloaded from vanilla and VOS-ResNet and VOS-RegNet.

The pretrained models for BDD-100k can be downloaded from vanilla and VOS-ResNet and VOS-RegNet.

VOS on Classification models

Train on WideResNet

cd classification/CIFAR/ & 
python train_virtual.py 
--start_epoch 40 
--sample_number 1000 
--sample_from 10000 
--select 1 
--loss_weight 0.1 

where "start_epoch" denotes the starting epoch of the uncertainty regularization branch.

"sample_number" denotes the size of the in-distribution queue.

"sample_from" and "select" are used to approximate the likelihood threshold during virtual outlier synthesis.

"loss_weight" denotes the weight of the regularization loss.

Please see Section 3 and Section 4.1 in the paper for details.

Train on DenseNet

cd classification/CIFAR/ &
python train_virtual_dense.py 
--start_epoch 40 
--sample_number 1000 
--sample_from 10000 
--select 1 
--loss_weight 0.1 

Evaluation on different classifiers

cd classification/CIFAR/ & 
python test.py 
--model_name xx 
--method_name xx 
--score energy 
--num_to_avg 10

where "model_name" denotes the model architectures. ("res" denotes the WideResNet and "dense" denotes the DenseNet.)

"method_name" denotes the checkpoint name you are loading.

Pretrained models

We provide the pretrained models using WideResNet and DenseNet with the in-distribution dataset to be CIFAR-10.

Citation

If you found any part of this code is useful in your research, please consider citing our paper:

 @article{du2022vos,
      title={VOS: Learning What You Don’t Know by Virtual Outlier Synthesis}, 
      author={Du, Xuefeng and Wang, Zhaoning and Cai, Mu and Li, Yixuan},
      journal={Proceedings of the International Conference on Learning Representations},
      year={2022}
}
Owner
CS Research Group led by Prof. Sharon Li
This is a collection of all challenges in HKCERT CTF 2021

香港網絡保安新生代奪旗挑戰賽 2021 (HKCERT CTF 2021) This is a collection of all challenges (and writeups) in HKCERT CTF 2021 Challenges ID Chinese name Name Score S

10 Jan 27, 2022
Applying PVT to Semantic Segmentation

Applying PVT to Semantic Segmentation Here, we take MMSegmentation v0.13.0 as an example, applying PVTv2 to SemanticFPN. For details see Pyramid Visio

35 Nov 30, 2022
Cross-Modal Contrastive Learning for Text-to-Image Generation

Cross-Modal Contrastive Learning for Text-to-Image Generation This repository hosts the open source JAX implementation of XMC-GAN. Setup instructions

Google Research 94 Nov 12, 2022
This is the pytorch implementation for the paper: Generalizable Mixed-Precision Quantization via Attribution Rank Preservation, which is accepted to ICCV2021.

GMPQ: Generalizable Mixed-Precision Quantization via Attribution Rank Preservation This is the pytorch implementation for the paper: Generalizable Mix

18 Sep 02, 2022
Official PyTorch implementation of the paper "Graph-based Generative Face Anonymisation with Pose Preservation" in ICIAP 2021

Contents AnonyGAN Installation Dataset Preparation Generating Images Using Pretrained Model Train and Test New Models Evaluation Acknowledgments Citat

Nicola Dall'Asen 10 May 24, 2022
Class activation maps for your PyTorch models (CAM, Grad-CAM, Grad-CAM++, Smooth Grad-CAM++, Score-CAM, SS-CAM, IS-CAM, XGrad-CAM, Layer-CAM)

TorchCAM: class activation explorer Simple way to leverage the class-specific activation of convolutional layers in PyTorch. Quick Tour Setting your C

F-G Fernandez 1.2k Dec 29, 2022
Source code for Zalo AI 2021 submission

zalo_ltr_2021 Source code for Zalo AI 2021 submission Solution: Pipeline We use the pipepline in the picture below: Our pipeline is combination of BM2

128 Dec 27, 2022
DvD-TD3: Diversity via Determinants for TD3 version

DvD-TD3: Diversity via Determinants for TD3 version The implementation of paper Effective Diversity in Population Based Reinforcement Learning. Instal

3 Feb 11, 2022
Combining Diverse Feature Priors

Combining Diverse Feature Priors This repository contains code for reproducing the results of our paper. Paper: https://arxiv.org/abs/2110.08220 Blog

Madry Lab 5 Nov 12, 2022
TLXZoo - Pre-trained models based on TensorLayerX

Pre-trained models based on TensorLayerX. TensorLayerX is a multi-backend AI fra

TensorLayer Community 13 Dec 07, 2022
Reference code for the paper CAMS: Color-Aware Multi-Style Transfer.

CAMS: Color-Aware Multi-Style Transfer Mahmoud Afifi1, Abdullah Abuolaim*1, Mostafa Hussien*2, Marcus A. Brubaker1, Michael S. Brown1 1York University

Mahmoud Afifi 36 Dec 04, 2022
LoveDA: A Remote Sensing Land-Cover Dataset for Domain Adaptive Semantic Segmentation

LoveDA: A Remote Sensing Land-Cover Dataset for Domain Adaptive Semantic Segmentation by Junjue Wang, Zhuo Zheng, Ailong Ma, Xiaoyan Lu, and Yanfei Zh

Payphone 8 Nov 21, 2022
Shared Attention for Multi-label Zero-shot Learning

Shared Attention for Multi-label Zero-shot Learning Overview This repository contains the implementation of Shared Attention for Multi-label Zero-shot

dathuynh 26 Dec 14, 2022
Utilities and information for the signals.numer.ai tournament

dsignals Utilities and information for the signals.numer.ai tournament using eodhistoricaldata.com eodhistoricaldata.com provides excellent historical

Degerhan Usluel 23 Dec 18, 2022
Deep Learning agent of Starcraft2, similar to AlphaStar of DeepMind except size of network.

Introduction This repository is for Deep Learning agent of Starcraft2. It is very similar to AlphaStar of DeepMind except size of network. I only test

Dohyeong Kim 136 Jan 04, 2023
PyTorch code for the paper: FeatMatch: Feature-Based Augmentation for Semi-Supervised Learning

FeatMatch: Feature-Based Augmentation for Semi-Supervised Learning This is the PyTorch implementation of our paper: FeatMatch: Feature-Based Augmentat

43 Nov 19, 2022
Volumetric parameterization of the placenta to a flattened template

placenta-flattening A MATLAB algorithm for volumetric mesh parameterization. Developed for mapping a placenta segmentation derived from an MRI image t

Mazdak Abulnaga 12 Mar 14, 2022
This is the source code for the experiments related to the paper Unsupervised Audio Source Separation Using Differentiable Parametric Source Models

Unsupervised Audio Source Separation Using Differentiable Parametric Source Models This is the source code for the experiments related to the paper Un

30 Oct 19, 2022
Voxel-based Network for Shape Completion by Leveraging Edge Generation (ICCV 2021, oral)

Voxel-based Network for Shape Completion by Leveraging Edge Generation This is the PyTorch implementation for the paper "Voxel-based Network for Shape

10 Dec 04, 2022
Image Captioning on google cloud platform based on iot

Image-Captioning-on-google-cloud-platform-based-on-iot - Image Captioning on google cloud platform based on iot

Shweta_kumawat 1 Jan 20, 2022