Set of models for classifcation of 3D volumes

Overview

Classification models 3D Zoo - Keras and TF.Keras

This repository contains 3D variants of popular CNN models for classification like ResNets, DenseNets, VGG, etc. It also contains weights obtained by converting ImageNet weights from the same 2D models.

This repository is based on great classification_models repo by @qubvel

Architectures:

Installation

pip install classification-models-3D

Examples

Loading model with imagenet weights:
# for keras
from classification_models_3D.keras import Classifiers

# for tensorflow.keras
# from classification_models_3D.tfkeras import Classifiers

ResNet18, preprocess_input = Classifiers.get('resnet18')
model = ResNet18(input_shape=(128, 128, 128, 3), weights='imagenet')

All possible nets for Classifiers.get() method: 'resnet18, 'resnet34', 'resnet50', 'resnet101', 'resnet152', 'seresnet18', 'seresnet34', 'seresnet50', 'seresnet101', 'seresnet152', 'seresnext50', 'seresnext101', 'senet154', 'resnext50', 'resnext101', 'vgg16', 'vgg19', 'densenet121', 'densenet169', 'densenet201', 'inceptionresnetv2', 'inceptionv3', 'mobilenet', 'mobilenetv2'

Convert imagenet weights (2D -> 3D)

Code to convert 2D imagenet weights to 3D variant is available here: convert_imagenet_weights_to_3D_models.py. Weights were obtained with TF2, but works OK with Keras + TF1 as well.

How to choose input shape

If initial 2D model had shape (512, 512, 3) then you can use shape (D, H, W, 3) where D * H * W ~= 512*512, so something like (64, 64, 64, 3) will be ok.

Training with single NVIDIA 1080Ti (11 GB) worked with:

  • DenseNet121, DenseNet169 and ResNet50 with shape (96, 128, 128, 3) and batch size 6
  • DenseNet201 with shape (96, 128, 128, 3) and batch size 5
  • ResNet18 with shape (128, 160, 160, 3) and batch size 6

Related repositories

Unresolved problems

  • There is no DepthwiseConv3D layer in keras, so repo used custom layer from this repo by @alexandrosstergiou which can be slower than native implementation.
  • There is no imagenet weights for 'inceptionresnetv2' and 'inceptionv3'.

Description

This code was used to get 1st place in DrivenData: Advance Alzheimer’s Research with Stall Catchers competition.

More details on ArXiv: https://arxiv.org/abs/2104.01687

Citation

If you find this code useful, please cite it as:

@InProceedings{RSolovyev_2021_stalled,
  author = {Solovyev, Roman and Kalinin, Alexandr A. and Gabruseva, Tatiana},
  title = {3D Convolutional Neural Networks for Stalled Brain Capillary Detection},
  booktitle = {Arxiv: 2104.01687},
  month = {April},
  year = {2021}
}
Comments
  • Update __init__.py

    Update __init__.py

    Using keras 2.9.0, import keras_applications as ka gives the following error:- ModuleNotFoundError: No module named 'keras_applications'

    Instead using from keras import applications as ka works!

    opened by msmuskan 0
  • Pushing current version to PyPI

    Pushing current version to PyPI

    Hello @ZFTurbo,

    if you have time, please push the current updated status (with ConvNeXt) of this repo to PyPI. :)

    Thanks again for the great work and your time!

    Cheers, Dominik

    opened by muellerdo 0
  • Grad cam issue

    Grad cam issue

    Hello ,

    base_model, preprocess_input = Classifiers.get('seresnext50') model = base_model(input_shape=(512, 512, 20, 1 ), weights=None , include_top = False ) x = Flatten()(model.output) x = Dense(1024, activation= 'sigmoid')(x) x = Dense(2, activation= 'sigmoid')(x)

    Trying to train a model , the accuracy is everything resides upto expectation, but the gradcam are quite off from the region of the focus - how the accuracy is good but the grad cam is off the focus of targeted area .

    Using the layer - 'activation-161' as output ref - https://github.com/fitushar/3D-Grad-CAM/blob/master/3DGrad-CAM.ipynb for the gradcam generation code , the results are always at the border of the image.

    opened by ntirupathirao18 0
  • ImportError: cannot import name 'VersionAwareLayers' from 'keras.layers'

    ImportError: cannot import name 'VersionAwareLayers' from 'keras.layers'

    Thank you for the great work.

    I am experiencing the following error over and over, even though I created a brand new tensorflow environment and installed all the necessary libraries in it. Could you please have a look on it and guide me how do I solve this problem? Thank you.

    ImportError: Unable to import 'VersionAwareLayers' from 'keras.layers' (/home/ubuntu/anaconda3/envs/cm_3d/lib/python3.7/site-packages/keras/layers/init.py)

    opened by nasir3843 2
  • 3D DenseNet

    3D DenseNet

    Hello and sorry to bother you beforehand,

    I am currently conducting my master thesis project and I am trying to implement a 3D DenseNet-121 with knee MRIs as input data. While I was searching on how to implement a 3D version of the DenseNet I came across your repository and tried to change it for my application.

    I have some issues regarding my try and I didn't know where else to ask about it and again I am sorry if I am completely of topic asking them here.

    Firstly, my input shapes are (250,320,18,1) and when I give them as input to the 3D DenseNet I developed with stride_size=1 for my Conv_block and pooling_size=(2,2,2) and strides=(2,2,1) for my AveragePooling3D layer in the transition block, the model is constructed properly with the specific input_size, while when I am trying to load a DenseNet121 from classification_models_3d.tfkeras classifiers I am unable to construct it with input_shape(250,320,18,1), stride_size=1 and kernel_size=2. It gives as an error "Negative dimension size... for node pool4_pool/AvgPool3D". Is there a way to specifically define the strides for AvgPool3D layer in the transition block?

    And secondly, I was thinking to load the 3D weights to my 3D DenseNet 121, is there a folder in your repository where I can find your pre-trained weights on imagenet??

    Again thank you for having this repository publicly available and sorry if I am completely of topic asking such things here.

    I look forward for you answer, Kind regards, Anastasis

    opened by alexopoulosanastasis 4
  • What are the limitations on Inceptionv3 input shape?

    What are the limitations on Inceptionv3 input shape?

    I seem to always get this error when I try to create InceptionV3 model no matter what input_shape. What are the limitations on input shape there?

    InvalidArgumentError: Negative dimension size caused by subtracting 3 from 2 for '{{node conv3d_314/Conv3D}} = 
    Conv3D[T=DT_FLOAT, data_format="NDHWC", dilations=[1, 1, 1, 1, 1], padding="VALID", strides=[1, 2, 2, 2, 1]](Placeholder, 
    conv3d_314/Conv3D/ReadVariableOp)' with input shapes: [?,2,17,17,192], [3,3,3,192,320].
    
    opened by mazatov 0
Releases(v1.0.4)
Malmo Collaborative AI Challenge - Team Pig Catcher

The Malmo Collaborative AI Challenge - Team Pig Catcher Approach The challenge involves 2 agents who can either cooperate or defect. The optimal polic

Kai Arulkumaran 66 Jun 29, 2022
This repository contains the code for EMNLP-2021 paper "Word-Level Coreference Resolution"

Word-Level Coreference Resolution This is a repository with the code to reproduce the experiments described in the paper of the same name, which was a

79 Dec 27, 2022
Depth image based mouse cursor visual haptic

Depth image based mouse cursor visual haptic How to run it. Install pyqt5. Install python modules pip install Pillow pip install numpy For illustrati

Xiong Jie 17 Dec 20, 2022
Generic U-Net Tensorflow implementation for image segmentation

Tensorflow Unet Warning This project is discontinued in favour of a Tensorflow 2 compatible reimplementation of this project found under https://githu

Joel Akeret 1.8k Dec 10, 2022
Repo for EMNLP 2021 paper "Beyond Preserved Accuracy: Evaluating Loyalty and Robustness of BERT Compression"

beyond-preserved-accuracy Repo for EMNLP 2021 paper "Beyond Preserved Accuracy: Evaluating Loyalty and Robustness of BERT Compression" How to implemen

Kevin Canwen Xu 10 Dec 23, 2022
DCT-Mask: Discrete Cosine Transform Mask Representation for Instance Segmentation

DCT-Mask: Discrete Cosine Transform Mask Representation for Instance Segmentation This project hosts the code for implementing the DCT-MASK algorithms

Alibaba Cloud 57 Nov 27, 2022
A python implementation of Deep-Image-Analogy based on pytorch.

Deep-Image-Analogy This project is a python implementation of Deep Image Analogy.https://arxiv.org/abs/1705.01088. Some results Requirements python 3

Peng Lu 171 Dec 14, 2022
Api for getting bin info and getting encrypted card details for adyen.

Bin Info And Adyen Cse Enc Python api for getting bin info and getting encrypted

Roldex Stark 8 Dec 30, 2022
U-Net implementation in PyTorch for FLAIR abnormality segmentation in brain MRI

U-Net for brain segmentation U-Net implementation in PyTorch for FLAIR abnormality segmentation in brain MRI based on a deep learning segmentation alg

562 Jan 02, 2023
Deep Reinforcement Learning for Multiplayer Online Battle Arena

MOBA_RL Deep Reinforcement Learning for Multiplayer Online Battle Arena Prerequisite Python 3 gym-derk Tensorflow 2.4.1 Dotaservice of TimZaman Seed R

Dohyeong Kim 32 Dec 18, 2022
This is a simple plugin for Vim that allows you to use OpenAI Codex.

🤖 Vim Codex An AI plugin that does the work for you. This is a simple plugin for Vim that will allow you to use OpenAI Codex. To use this plugin you

Tom Dörr 195 Dec 28, 2022
code for our ECCV-2020 paper: Self-supervised Video Representation Learning by Pace Prediction

Video_Pace This repository contains the code for the following paper: Jiangliu Wang, Jianbo Jiao and Yunhui Liu, "Self-Supervised Video Representation

Jiangliu Wang 95 Dec 14, 2022
Replication attempt for the Protein Folding Model

RGN2-Replica (WIP) To eventually become an unofficial working Pytorch implementation of RGN2, an state of the art model for MSA-less Protein Folding f

Eric Alcaide 36 Nov 29, 2022
Implementation of C-RNN-GAN.

Implementation of C-RNN-GAN. Publication: Title: C-RNN-GAN: Continuous recurrent neural networks with adversarial training Information: http://mogren.

Olof Mogren 427 Dec 25, 2022
[arXiv] What-If Motion Prediction for Autonomous Driving ❓🚗💨

WIMP - What If Motion Predictor Reference PyTorch Implementation for What If Motion Prediction [PDF] [Dynamic Visualizations] Setup Requirements The W

William Qi 96 Dec 29, 2022
Yolov5+SlowFast: Realtime Action Detection Based on PytorchVideo

Yolov5+SlowFast: Realtime Action Detection A realtime action detection frame work based on PytorchVideo. Here are some details about our modification:

WuFan 181 Dec 30, 2022
[ACM MM2021] MGH: Metadata Guided Hypergraph Modeling for Unsupervised Person Re-identification

Introduction This project is developed based on FastReID, which is an ongoing ReID project. Projects BUC In projects/BUC, we implement AAAI 2019 paper

WuYiming 7 Apr 13, 2022
Polyp-PVT: Polyp Segmentation with Pyramid Vision Transformers (arXiv2021)

Polyp-PVT by Bo Dong, Wenhai Wang, Deng-Ping Fan, Jinpeng Li, Huazhu Fu, & Ling Shao. This repo is the official implementation of "Polyp-PVT: Polyp Se

Deng-Ping Fan 102 Jan 05, 2023
This toolkit provides codes to download and pre-process the SLUE datasets, train the baseline models, and evaluate SLUE tasks.

slue-toolkit We introduce Spoken Language Understanding Evaluation (SLUE) benchmark. This toolkit provides codes to download and pre-process the SLUE

ASAPP Research 39 Sep 21, 2022
Unified file system operation experience for different backend

megfile - Megvii FILE library Docs: http://megvii-research.github.io/megfile megfile provides a silky operation experience with different backends (cu

MEGVII Research 76 Dec 14, 2022