This code is for our paper "VTGAN: Semi-supervised Retinal Image Synthesis and Disease Prediction using Vision Transformers"

Related tags

Deep Learninggenerative-adversarial-networksemi-supervised-learningdiabetic-retinopathy-detectiondisease-predictionfundus-image-analysisfluorescein-angiographyvision-transformervision-transformers
Overview

ICCV Workshop 2021 VTGAN

PWC

This code is for our paper "VTGAN: Semi-supervised Retinal Image Synthesis and Disease Prediction using Vision Transformers" which is part of the supplementary materials for ICCV 2021 Workshop on Computer Vision for Automated Medical Diagnosis. The paper has since been accpeted and presented at ICCV 2021 Workshop.

Arxiv Pre-print

https://arxiv.org/abs/2104.06757

CVF ICCVW 2021

https://openaccess.thecvf.com/content/ICCV2021W/CVAMD/html/Kamran_VTGAN_Semi-Supervised_Retinal_Image_Synthesis_and_Disease_Prediction_Using_Vision_ICCVW_2021_paper.html

IEE Xplore ICCVW 2021

https://ieeexplore.ieee.org/document/9607858

Citation

@INPROCEEDINGS{9607858,
  author={Kamran, Sharif Amit and Hossain, Khondker Fariha and Tavakkoli, Alireza and Zuckerbrod, Stewart Lee and Baker, Salah A.},
  booktitle={2021 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW)}, 
  title={VTGAN: Semi-supervised Retinal Image Synthesis and Disease Prediction using Vision Transformers}, 
  year={2021},
  volume={},
  number={},
  pages={3228-3238},
  doi={10.1109/ICCVW54120.2021.00362}
}

Pre-requisite

  • Ubuntu 18.04 / Windows 7 or later
  • NVIDIA Graphics card

Installation Instruction for Ubuntu

sudo add-apt-repository ppa:deadsnakes/ppa
sudo apt install python3.7
  • Install Tensorflow-Gpu version-2.5.0 and Keras version-2.5.0
sudo pip3 install tensorflow-gpu
sudo pip3 install keras
  • Install packages from requirements.txt
sudo pip3 install -r requirements.txt

Dataset download link for Hajeb et al.

https://sites.google.com/site/hosseinrabbanikhorasgani/datasets-1/fundus-fluorescein-angiogram-photographs--colour-fundus-images-of-diabetic-patients
  • Please cite the paper if you use their data
@article{hajeb2012diabetic,
  title={Diabetic retinopathy grading by digital curvelet transform},
  author={Hajeb Mohammad Alipour, Shirin and Rabbani, Hossein and Akhlaghi, Mohammad Reza},
  journal={Computational and mathematical methods in medicine},
  volume={2012},
  year={2012},
  publisher={Hindawi}
}
  • Folder structure for data-preprocessing given below. Please make sure it matches with your local repository.
├── Dataset
|   ├──ABNORMAL
|   ├──NORMAL

Dataset Pre-processing

  • Type this in terminal to run the random_crop.py file
python3 random_crop.py --output_dir=data --input_dim=512 --datadir=Dataset
  • There are different flags to choose from. Not all of them are mandatory.
    '--input_dim', type=int, default=512
    '--n_crops', type=int, default=50
    '--datadir', type=str, required=True, help='path/to/data_directory',default='Dataset'
    '--output_dir', type=str, default='data'

NPZ file conversion

  • Convert all the images to npz format
python3 convert_npz.py --outfile_name=vtgan --input_dim=512 --datadir=data --n_crops=50
  • There are different flags to choose from. Not all of them are mandatory.
    '--input_dim', type=int, default=512
    '--n_crops', type=int, default=50
    '--datadir', type=str, required=True, help='path/to/data_directory',default='data'
    '--outfile_name', type=str, default='vtgan'
    '--n_images', type=int, default=17

Training

  • Type this in terminal to run the train.py file
python3 train.py --npz_file=vtgan --batch=2 --epochs=100 --savedir=VTGAN
  • There are different flags to choose from. Not all of them are mandatory
    '--epochs', type=int, default=100
    '--batch_size', type=int, default=2
    '--npz_file', type=str, default='vtgan', help='path/to/npz/file'
    '--input_dim', type=int, default=512
    '--n_patch', type=int, default=64
    '--savedir', type=str, required=False, help='path/to/save_directory',default='VTGAN'
    '--resume_training', type=str, required=False,  default='no', choices=['yes','no']

License

The code is released under the BSD 3-Clause License, you can read the license file included in the repository for details.

Owner
Sharif Amit Kamran
Interested in Deep learning for Medical Imaging and Computer Vision. Designing robust generative architectures for Ophthalmology and Calcium Imaging.
Sharif Amit Kamran
GitHub Repository
Repository for the semantic WMI loss

Installation: pip install -e . Installing DL2: First clone DL2 in a separate directory and install it using the following commands: git clone https:/

Nick Hoernle 4 Sep 15, 2022
Official code for the ICLR 2021 paper Neural ODE Processes

Neural ODE Processes Official code for the paper Neural ODE Processes (ICLR 2021). Abstract Neural Ordinary Differential Equations (NODEs) use a neura

Cristian Bodnar 50 Oct 28, 2022
Transformer Tracking (CVPR2021)

TransT - Transformer Tracking [CVPR2021] Official implementation of the TransT (CVPR2021) , including training code and trained models. We are revisin

chenxin 465 Jan 06, 2023
Official Implementation and Dataset of "PPR10K: A Large-Scale Portrait Photo Retouching Dataset with Human-Region Mask and Group-Level Consistency", CVPR 2021

Portrait Photo Retouching with PPR10K Paper | Supplementary Material PPR10K: A Large-Scale Portrait Photo Retouching Dataset with Human-Region Mask an

184 Dec 11, 2022
LiDAR R-CNN: An Efficient and Universal 3D Object Detector

LiDAR R-CNN: An Efficient and Universal 3D Object Detector Introduction This is the official code of LiDAR R-CNN: An Efficient and Universal 3D Object

TuSimple 295 Jan 05, 2023
Implementation of UNET architecture for Image Segmentation.

Semantic Segmentation using UNET This is the implementation of UNET on Carvana Image Masking Kaggle Challenge About the Dataset This dataset contains

Anushka agarwal 4 Dec 21, 2021
Implementation of the Transformer variant proposed in "Transformer Quality in Linear Time"

FLASH - Pytorch Implementation of the Transformer variant proposed in the paper Transformer Quality in Linear Time Install $ pip install FLASH-pytorch

Phil Wang 209 Dec 28, 2022
Building Ellee — A GPT-3 and Computer Vision Powered Talking Robotic Teddy Bear With Human Level Conversation Intelligence

Using an object detection and facial recognition system built on MobileNetSSDV2 and Dlib and running on an NVIDIA Jetson Nano, a GPT-3 model, Google Speech Recognition, Amazon Polly and servo motors,

24 Oct 26, 2022
A Machine Teaching Framework for Scalable Recognition

MEMORABLE This repository contains the source code accompanying our ICCV 2021 paper. A Machine Teaching Framework for Scalable Recognition Pei Wang, N

2 Dec 08, 2021
A variational Bayesian method for similarity learning in non-rigid image registration (CVPR 2022)

A variational Bayesian method for similarity learning in non-rigid image registration We provide the source code and the trained models used in the re

daniel grzech 14 Nov 21, 2022
Neighborhood Contrastive Learning for Novel Class Discovery

Neighborhood Contrastive Learning for Novel Class Discovery This repository contains the official implementation of our paper: Neighborhood Contrastiv

Zhun Zhong 56 Dec 09, 2022
Unified Pre-training for Self-Supervised Learning and Supervised Learning for ASR

UniSpeech The family of UniSpeech: UniSpeech (ICML 2021): Unified Pre-training for Self-Supervised Learning and Supervised Learning for ASR UniSpeech-

Microsoft 282 Jan 09, 2023
An experimental technique for efficiently exploring neural architectures.

SMASH: One-Shot Model Architecture Search through HyperNetworks An experimental technique for efficiently exploring neural architectures. This reposit

Andy Brock 478 Aug 04, 2022
DeepMind Alchemy task environment: a meta-reinforcement learning benchmark

The DeepMind Alchemy environment is a meta-reinforcement learning benchmark that presents tasks sampled from a task distribution with deep underlying structure.

DeepMind 188 Dec 25, 2022
Resources for the "Evaluating the Factual Consistency of Abstractive Text Summarization" paper

Evaluating the Factual Consistency of Abstractive Text Summarization Authors: Wojciech Kryściński, Bryan McCann, Caiming Xiong, and Richard Socher Int

Salesforce 165 Dec 21, 2022
Implementation of "Semi-supervised Domain Adaptive Structure Learning"

Semi-supervised Domain Adaptive Structure Learning - ASDA This repo contains the source code and dataset for our ASDA paper. Illustration of the propo

3 Dec 13, 2021
CellRank's reproducibility repository.

CellRank's reproducibility repository We believe that reproducibility is key and have made it as simple as possible to reproduce our results. Please e

Theis Lab 8 Oct 08, 2022
Text-to-SQL in the Wild: A Naturally-Occurring Dataset Based on Stack Exchange Data

SEDE SEDE (Stack Exchange Data Explorer) is new dataset for Text-to-SQL tasks with more than 12,000 SQL queries and their natural language description

Rupert. 83 Nov 11, 2022
A curated list of awesome resources combining Transformers with Neural Architecture Search

A curated list of awesome resources combining Transformers with Neural Architecture Search

Yash Mehta 173 Jan 03, 2023
This is the official repository for evaluation on the NoW Benchmark Dataset. The goal of the NoW benchmark is to introduce a standard evaluation metric to measure the accuracy and robustness of 3D face reconstruction methods from a single image under variations in viewing angle, lighting, and common occlusions.

NoW Evaluation This is the official repository for evaluation on the NoW Benchmark Dataset. The goal of the NoW benchmark is to introduce a standard e

Soubhik Sanyal 71 Dec 30, 2022