PyTorch implementation of VAGAN: Visual Feature Attribution Using Wasserstein GANs

Overview

PyTorch implementation of VAGAN: Visual Feature Attribution Using Wasserstein GANs

This code aims to reproduce results obtained in the paper "Visual Feature Attribution using Wasserstein GANs" (official repo, TensorFlow code)

Description

This repository contains the code to reproduce results for the paper cited above, where the authors presents a novel feature attribution technique based on Wasserstein Generative Adversarial Networks (WGAN). The code works for both synthetic (2D) and real 3D neuroimaging data, you can check below for a brief description of the two datasets.

anomaly maps examples

Here is an example of what the generator/mapper network should produce: ctrl-click on the below image to open the gifv in a new tab (one frame every 50 iterations, left: input, right: anomaly map for synthetic data at iteration 50 * (its + 1)).

anomaly maps examples

Synthetic Dataset

"Data: In order to quantitatively evaluate the performance of the examined visual attribution methods, we generated a synthetic dataset of 10000 112x112 images with two classes, which model a healthy control group (label 0) and a patient group (label 1). The images were split evenly across the two categories. We closely followed the synthetic data generation process described in [31][SubCMap: Subject and Condition Specific Effect Maps] where disease effects were studied in smaller cohorts of registered images. The control group (label 0) contained images with ran- dom iid Gaussian noise convolved with a Gaussian blurring filter. Examples are shown in Fig. 3. The patient images (label 1) also contained the noise, but additionally exhib- ited one of two disease effects which was generated from a ground-truth effect map: a square in the centre and a square in the lower right (subtype A), or a square in the centre and a square in the upper left (subtype B). Importantly, both dis- ease subtypes shared the same label. The location of the off-centre squares was randomly offset in each direction by a maximum of 5 pixels. This moving effect was added to make the problem harder, but had no notable effect on the outcome."

image

ADNI Dataset

Currently we only implemented training on synthetic dataset, we will work on implement training on ADNI dataset asap (but pull requests are welcome as always), we put below ADNI dataset details for sake of completeness.

"We selected 5778 3D T1-weighted MR images from 1288 subjects with either an MCI (label 0) or AD (label 1) diagnosis from the ADNI cohort. 2839 of the images were acquired using a 1.5T magnet, the remainder using a 3T magnet. The subjects are scanned at regular intervals as part of the ADNI study and a number of subjects converted from MCI to AD over the years. We did not use these cor- respondences for training, however, we took advantage of it for evaluation as will be described later. All images were processed using standard operations available in the FSL toolbox [52][Advances in functional and structural MR image analysis and implementation as FSL.] in order to reorient and rigidly register the images to MNI space, crop them and correct for field inhomogeneities. We then skull-stripped the images using the ROBEX algorithm [24][Robust brain extraction across datasets and comparison with publicly available methods]. Lastly, we resampled all images to a resolution of 1.3 mm 3 and nor- malised them to a range from -1 to 1. The final volumes had a size of 128x160x112 voxels."

"Data used in preparation of this article were obtained from the Alzheimers disease Neuroimaging Initiative (ADNI) database (adni.loni.usc.edu). As such, the investigators within the ADNI contributed to the design and implementation of ADNI and/or provided data but did not participate in analysis or writing of this report. A complete listing of ADNI investigators can be found at: http://adni.loni.usc.edu/wp-content/uploads/how_to_apply/ADNI_Acknowledgement_List.pdf"

Usage

Training

To train the WGAN on this task, cd into this repo's src root folder and execute:

$ python train.py

This script takes the following command line options:

  • dataset_root: the root directory where tha dataset is stored, default to '../dataset'

  • experiment: directory in where samples and models will be saved, default to '../samples'

  • batch_size: input batch size, default to 32

  • image_size: the height / width of the input image to network, default to 112

  • channels_number: input image channels, default to 1

  • num_filters_g: number of filters for the first layer of the generator, default to 16

  • num_filters_d: number of filters for the first layer of the discriminator, default to 16

  • nepochs: number of epochs to train for, default to 1000

  • d_iters: number of discriminator iterations per each generator iter, default to 5

  • learning_rate_g: learning rate for generator, default to 1e-3

  • learning_rate_d: learning rate for discriminator, default to 1e-3

  • beta1: beta1 for adam. default to 0.0

  • cuda: enables cuda (store True)

  • manual_seed: input for the manual seeds initializations, default to 7

Running the command without arguments will train the models with the default hyperparamters values (producing results shown above).

Models

We ported all models found in the original repository in PyTorch, you can find all implemented models here: https://github.com/orobix/Visual-Feature-Attribution-Using-Wasserstein-GANs-Pytorch/tree/master/src/models

Useful repositories and code

  • vagan-code: Reposiory for the reference paper from its authors

  • ganhacks: Starter from "How to Train a GAN?" at NIPS2016

  • WassersteinGAN: Code accompanying the paper "Wasserstein GAN"

  • wgan-gp: Pytorch implementation of Paper "Improved Training of Wasserstein GANs".

  • c3d-pytorch: Model used as discriminator in the reference paper

  • Pytorch-UNet: Model used as genertator in this repository

  • dcgan: Model used as discriminator in this repository

.bib citation

cite the paper as follows (copied-pasted it from arxiv for you):

@article{DBLP:journals/corr/abs-1711-08998,
  author    = {Christian F. Baumgartner and
               Lisa M. Koch and
               Kerem Can Tezcan and
               Jia Xi Ang and
               Ender Konukoglu},
  title     = {Visual Feature Attribution using Wasserstein GANs},
  journal   = {CoRR},
  volume    = {abs/1711.08998},
  year      = {2017},
  url       = {http://arxiv.org/abs/1711.08998},
  archivePrefix = {arXiv},
  eprint    = {1711.08998},
  timestamp = {Sun, 03 Dec 2017 12:38:15 +0100},
  biburl    = {http://dblp.org/rec/bib/journals/corr/abs-1711-08998},
  bibsource = {dblp computer science bibliography, http://dblp.org}
}

License

This project is licensed under the MIT License

Copyright (c) 2018 Daniele E. Ciriello, Orobix Srl (www.orobix.com).

Owner
Orobix
Orobix
This provides the R code and data to replicate results in "The USS Trustee’s risky strategy"

USSBriefs2021 This provides the R code and data to replicate results in "The USS Trustee’s risky strategy" by Neil M Davies, Jackie Grant and Chin Yan

1 Oct 30, 2021
Public repository of the 3DV 2021 paper "Generative Zero-Shot Learning for Semantic Segmentation of 3D Point Clouds"

Generative Zero-Shot Learning for Semantic Segmentation of 3D Point Clouds Björn Michele1), Alexandre Boulch1), Gilles Puy1), Maxime Bucher1) and Rena

valeo.ai 15 Dec 22, 2022
Dark Finix: All in one hacking framework with almost 100 tools

Dark Finix - Hacking Framework. Dark Finix is a all in one hacking framework wit

Md. Nur habib 2 Feb 18, 2022
CZU-MHAD: A multimodal dataset for human action recognition utilizing a depth camera and 10 wearable inertial sensors

CZU-MHAD: A multimodal dataset for human action recognition utilizing a depth camera and 10 wearable inertial sensors   In order to facilitate the res

yujmo 11 Dec 12, 2022
Scalable and Elastic Deep Reinforcement Learning Using PyTorch. Please star. 🔥

ElegantRL “小雅”: Scalable and Elastic Deep Reinforcement Learning ElegantRL is developed for researchers and practitioners with the following advantage

AI4Finance Foundation 2.5k Jan 05, 2023
Boundary IoU API (Beta version)

Boundary IoU API (Beta version) Bowen Cheng, Ross Girshick, Piotr Dollár, Alexander C. Berg, Alexander Kirillov [arXiv] [Project] [BibTeX] This API is

Bowen Cheng 177 Dec 29, 2022
Hardware-accelerated DNN model inference ROS2 packages using NVIDIA Triton/TensorRT for both Jetson and x86_64 with CUDA-capable GPU

Isaac ROS DNN Inference Overview This repository provides two NVIDIA GPU-accelerated ROS2 nodes that perform deep learning inference using custom mode

NVIDIA Isaac ROS 62 Dec 14, 2022
Most popular metrics used to evaluate object detection algorithms.

Most popular metrics used to evaluate object detection algorithms.

Rafael Padilla 4.4k Dec 25, 2022
Time-stretch audio clips quickly with PyTorch (CUDA supported)! Additional utilities for searching efficient transformations are included.

Time-stretch audio clips quickly with PyTorch (CUDA supported)! Additional utilities for searching efficient transformations are included.

Kento Nishi 22 Jul 07, 2022
Official implementation of "Towards Good Practices for Efficiently Annotating Large-Scale Image Classification Datasets" (CVPR2021)

Towards Good Practices for Efficiently Annotating Large-Scale Image Classification Datasets This is the official implementation of "Towards Good Pract

Sanja Fidler's Lab 52 Nov 22, 2022
ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generators

ELECTRA Introduction ELECTRA is a method for self-supervised language representation learning. It can be used to pre-train transformer networks using

Google Research 2.1k Dec 28, 2022
Pytorch implementation of Implicit Behavior Cloning.

Implicit Behavior Cloning - PyTorch (wip) Pytorch implementation of Implicit Behavior Cloning. Install conda create -n ibc python=3.8 pip install -r r

Kevin Zakka 49 Dec 25, 2022
A torch implementation of "Pixel-Level Domain Transfer"

Pixel Level Domain Transfer A torch implementation of "Pixel-Level Domain Transfer". based on dcgan.torch. Dataset The dataset used is "LookBook", fro

Fei Xia 260 Sep 02, 2022
Subnet Replacement Attack: Towards Practical Deployment-Stage Backdoor Attack on Deep Neural Networks

Subnet Replacement Attack: Towards Practical Deployment-Stage Backdoor Attack on Deep Neural Networks Official implementation of paper Towards Practic

Xiangyu Qi 8 Dec 30, 2022
Official Codes for Graph Modularity:Towards Understanding the Cross-Layer Transition of Feature Representations in Deep Neural Networks.

Dynamic-Graphs-Construction Official Codes for Graph Modularity:Towards Understanding the Cross-Layer Transition of Feature Representations in Deep Ne

11 Dec 14, 2022
This tool uses Deep Learning to help you draw and write with your hand and webcam.

This tool uses Deep Learning to help you draw and write with your hand and webcam. A Deep Learning model is used to try to predict whether you want to have 'pencil up' or 'pencil down'.

lmagne 169 Dec 10, 2022
Hierarchical User Intent Graph Network for Multimedia Recommendation

Hierarchical User Intent Graph Network for Multimedia Recommendation This is our Pytorch implementation for the paper: Hierarchical User Intent Graph

6 Jan 05, 2023
Compressed Video Action Recognition

Compressed Video Action Recognition Chao-Yuan Wu, Manzil Zaheer, Hexiang Hu, R. Manmatha, Alexander J. Smola, Philipp Krähenbühl. In CVPR, 2018. [Proj

Chao-Yuan Wu 479 Dec 26, 2022
Traditional deepdream with VQGAN+CLIP and optical flow. Ready to use in Google Colab

VQGAN-CLIP-Video cat.mp4 policeman.mp4 schoolboy.mp4 forsenBOG.mp4

23 Oct 26, 2022
SOLOv2 on onnx & tensorRT

SOLOv2.tensorRT: NOTE: code based on WXinlong/SOLO add support to TensorRT inference onnxruntime tensorRT full_dims and dynamic shape postprocess with

47 Nov 26, 2022