The PyTorch re-implement of a 3D CNN Tracker to extract coronary artery centerlines with state-of-the-art (SOTA) performance. (paper: 'Coronary artery centerline extraction in cardiac CT angiography using a CNN-based orientation classifier')

Overview

Coronary Artery Tracking via 3D CNN Classification Pytorch

The PyTorch re-implement of a 3D CNN Tracker to extract coronary artery centerlines with state-of-the-art (SOTA) performance. (paper: 'Coronary artery centerline extraction in cardiac CT angiography using a CNN-based orientation classifier')

Link to paper here.

Key idea

A 3D dilated CNN is trained to predict the most likely direction and radius of an artery at any given point in a CCTA image based on a local image patch. We use a 3D Fibonacci ball to model a CNN Tracker, where the radius of the ball represents the radius of the vessel at the current position, and the points on the ball represent a possible direction of movement.

Starting from a single seed point placed manually or automatically anywhere in a coronary artery, a tracker follows the vessel centerline in two directions using the predictions of the CNN.

Tracking is terminated when no direction can be identified with high certainty.

In order to create a vessel tree automatically, we need to train three neural networks.

  • Firstly, we need to train a centerline net to predict the two directions(d0, d1) of the current position that can be moved and the vessel radius.
  • Secondly, we need to train a neural network to find two entrance points of a coronary artery.
  • The third network is responsible for placing seed points in the image

Architecture of Centerline Net

Layer 1 2 3 4 5 6 7
Kernel width 3 3 3 3 3 1 1
Dilation 1 1 2 4 1 1 1
Channels 32 32 32 32 64 64 D+1
Field width 3 5 9 17 19 19 19

The number of output channels is equal to the number of potential directions in D, plus one channel for radius estimation.

The architecture of seedspint_net and ostiapoint_net are very similar to centerline_net. The only difference is in the output layer: instead of combining classification and regression, the final layer only performs regression.

Installation

To install all the required dependencies:

$ pip install -r requirement.txt

Training

1. Preparing CTA08 dataset

Tip:
CAT08 datasets need to be registered and certified in this website before it can be downloaded. It should be noted that your registration email may not be received by the server of the above website. If you have this problem, download this form, compile it and contact Dr.Theo van Walsum ([email protected]).

  1. Unzip training.tar.gz to:
    Coronary-Artery-Tracking-via-3D-CNN-Classification/
            -data_process_tools/
                -train_data/
                    -dataset00/
                    -dataset01/
                    -dataset02/
                    -dataset03/
                    -dataset04/
                    -dataset05/
                    -dataset06/
                    -dataset07/
  1. Create spacing_info.csv and nii.gz data
python3 creat_spacinginfo_data_tool.py
  1. Create centerline patch data
  • Create no offset samples
python3 centerline_patch_generater_no_offset.py
  • Create samples with offset
python3 centerline_patch_generater_offset.py
  1. Create seeds patch data
  • Create positve samples
python3 seedpoints_patch_generater_postive.py     
  • Create negative sample
python3 seedpoints_patch_generater_negative.py

those scripts will automaticlly create folders

-data_process_tools/
    -patch_data/
         -centerline_patch/
            -no_offset/
                 -point_500_gp_1/
                     -d0/
                     d0_patch_info_500.csv 
                     .
                     .
                     .
                     -d7/
                     d7_patch_info_500.csv
            -offset/
                  -point_500_gp_1/
                     -d0/
                     d0_patch_info_500.csv
                     .
                     .
                     .
                     -d7/
                     d7_patch_info_500.csv
  1. Create osita patch data
  • Create positve samples
python3 ostiapoints_patch_generater_positive.py
  • Create negative sample
python3 ostiapoints_patch_generater_negative.py

It should be noted that 8 samples corresponding to the data will be produced here, and the specific training set and test set division also need to write your own code to divide the data set and generate the train CSV file and val CSV file

2.Training Models

  1. Training centerline net
cd centerline_train_tools/
CUDA_VISIBLE_DEVICES=0 python3 centerline_train_tools.py
  1. Training seedpoints net
cd seedspoints_train_tools/
CUDA_VISIBLE_DEVICES=0 python3 seeds_train_tools.py
  1. Training ostiapoints net
cd ostiapoints_train_tools
CUDA_VISIBLE_DEVICES=0 python3 ostia_train_tools.py 

3.Create coronary artery vessels tree

cd infer_tools_tree/

First, you need to modify settingy.yaml replacing the path inside to the path of the file you saved

python3 vessels_tree_infer.py

The predicted vessel tree is shown in the figure below

The vessels from different seed points are spliced by breadth-first search, and then a complete single vessel is generated by depth-first search

Seedpoints net will generate 200 seed points as shown in the figure below. It can be seen that the seed points are distributed near several coronary arteries

References

@article{wolterink2019coronary,
  title={Coronary artery centerline extraction in cardiac CT angiography using a CNN-based orientation classifier},
  author={Wolterink, Jelmer M and van Hamersvelt, Robbert W and Viergever, Max A and Leiner, Tim Leiner, Ivana},
  journal={Medical image analysis},
  volume={51},
  pages={46--60},
  year={2019},
  publisher={Elsevier}
}
Owner
James
I am an investigator in the SenseTime. My research interests are in 3D Vision and Multiple Object Tracking.
James
🌳 A Python-inspired implementation of the Optimum-Path Forest classifier.

OPFython: A Python-Inspired Optimum-Path Forest Classifier Welcome to OPFython. Note that this implementation relies purely on the standard LibOPF. Th

Gustavo Rosa 30 Jan 04, 2023
CCAFNet: Crossflow and Cross-scale Adaptive Fusion Network for Detecting Salient Objects in RGB-D Images

Code and result about CCAFNet(IEEE TMM) 'CCAFNet: Crossflow and Cross-scale Adaptive Fusion Network for Detecting Salient Objects in RGB-D Images' IEE

zyrant丶 14 Dec 29, 2021
Differentiable Factor Graph Optimization for Learning Smoothers @ IROS 2021

Differentiable Factor Graph Optimization for Learning Smoothers Overview Status Setup Datasets Training Evaluation Acknowledgements Overview Code rele

Brent Yi 60 Nov 14, 2022
Implementation of TabTransformer, attention network for tabular data, in Pytorch

Tab Transformer Implementation of Tab Transformer, attention network for tabular data, in Pytorch. This simple architecture came within a hair's bread

Phil Wang 420 Jan 05, 2023
Avatarify Python - Avatars for Zoom, Skype and other video-conferencing apps.

Avatarify Python - Avatars for Zoom, Skype and other video-conferencing apps.

Ali Aliev 15.3k Jan 05, 2023
Pytorch implementation of AREL

Status: Archive (code is provided as-is, no updates expected) Agent-Temporal Attention for Reward Redistribution in Episodic Multi-Agent Reinforcement

8 Nov 25, 2022
LIMEcraft: Handcrafted superpixel selectionand inspection for Visual eXplanations

LIMEcraft LIMEcraft: Handcrafted superpixel selectionand inspection for Visual eXplanations The LIMEcraft algorithm is an explanatory method based on

MI^2 DataLab 4 Aug 01, 2022
DualGAN-tensorflow: tensorflow implementation of DualGAN

ICCV paper of DualGAN DualGAN: unsupervised dual learning for image-to-image translation please cite the paper, if the codes has been used for your re

Jack Yi 252 Nov 10, 2022
[ICLR 2021] Heteroskedastic and Imbalanced Deep Learning with Adaptive Regularization

Heteroskedastic and Imbalanced Deep Learning with Adaptive Regularization Kaidi Cao, Yining Chen, Junwei Lu, Nikos Arechiga, Adrien Gaidon, Tengyu Ma

Kaidi Cao 29 Oct 20, 2022
DeepMind's software stack for physics-based simulation and Reinforcement Learning environments, using MuJoCo.

dm_control: DeepMind Infrastructure for Physics-Based Simulation. DeepMind's software stack for physics-based simulation and Reinforcement Learning en

DeepMind 3k Dec 31, 2022
This repository contains the source code of our work on designing efficient CNNs for computer vision

Efficient networks for Computer Vision This repo contains source code of our work on designing efficient networks for different computer vision tasks:

Sachin Mehta 386 Nov 26, 2022
Automatic voice-synthetised summaries of latest research papers on arXiv

PaperWhisperer PaperWhisperer is a Python application that keeps you up-to-date with research papers. How? It retrieves the latest articles from arXiv

Valerio Velardo 124 Dec 20, 2022
ActNN: Reducing Training Memory Footprint via 2-Bit Activation Compressed Training

ActNN : Activation Compressed Training This is the official project repository for ActNN: Reducing Training Memory Footprint via 2-Bit Activation Comp

UC Berkeley RISE 178 Jan 05, 2023
CrossNorm and SelfNorm for Generalization under Distribution Shifts (ICCV 2021)

CrossNorm (CN) and SelfNorm (SN) (Accepted at ICCV 2021) This is the official PyTorch implementation of our CNSN paper, in which we propose CrossNorm

100 Dec 28, 2022
Implement the Pareto Optimizer and pcgrad to make a self-adaptive loss for multi-task

multi-task_losses_optimizer Implement the Pareto Optimizer and pcgrad to make a self-adaptive loss for multi-task 已经实验过了,不会有cuda out of memory情况 ##Par

14 Dec 25, 2022
The Multi-Mission Maximum Likelihood framework (3ML)

PyPi Conda The Multi-Mission Maximum Likelihood framework (3ML) A framework for multi-wavelength/multi-messenger analysis for astronomy/astrophysics.

The Multi-Mission Maximum Likelihood (3ML) 62 Dec 30, 2022
minimizer-space de Bruijn graphs (mdBG) for whole genome assembly

rust-mdbg: Minimizer-space de Bruijn graphs (mdBG) for whole-genome assembly rust-mdbg is an ultra-fast minimizer-space de Bruijn graph (mdBG) impleme

Barış Ekim 148 Dec 01, 2022
HDR Video Reconstruction: A Coarse-to-fine Network and A Real-world Benchmark Dataset (ICCV 2021)

Code for HDR Video Reconstruction HDR Video Reconstruction: A Coarse-to-fine Network and A Real-world Benchmark Dataset (ICCV 2021) Guanying Chen, Cha

Guanying Chen 64 Nov 19, 2022
Deep Unsupervised 3D SfM Face Reconstruction Based on Massive Landmark Bundle Adjustment.

(ACMMM 2021 Oral) SfM Face Reconstruction Based on Massive Landmark Bundle Adjustment This repository shows two tasks: Face landmark detection and Fac

BoomStar 51 Dec 13, 2022
Code for the paper Task Agnostic Morphology Evolution.

Task-Agnostic Morphology Optimization This repository contains code for the paper Task-Agnostic Morphology Evolution by Donald (Joey) Hejna, Pieter Ab

Joey Hejna 18 Aug 04, 2022