The PyTorch re-implement of a 3D CNN Tracker to extract coronary artery centerlines with state-of-the-art (SOTA) performance. (paper: 'Coronary artery centerline extraction in cardiac CT angiography using a CNN-based orientation classifier')

Overview

Coronary Artery Tracking via 3D CNN Classification Pytorch

The PyTorch re-implement of a 3D CNN Tracker to extract coronary artery centerlines with state-of-the-art (SOTA) performance. (paper: 'Coronary artery centerline extraction in cardiac CT angiography using a CNN-based orientation classifier')

Link to paper here.

Key idea

A 3D dilated CNN is trained to predict the most likely direction and radius of an artery at any given point in a CCTA image based on a local image patch. We use a 3D Fibonacci ball to model a CNN Tracker, where the radius of the ball represents the radius of the vessel at the current position, and the points on the ball represent a possible direction of movement.

Starting from a single seed point placed manually or automatically anywhere in a coronary artery, a tracker follows the vessel centerline in two directions using the predictions of the CNN.

Tracking is terminated when no direction can be identified with high certainty.

In order to create a vessel tree automatically, we need to train three neural networks.

  • Firstly, we need to train a centerline net to predict the two directions(d0, d1) of the current position that can be moved and the vessel radius.
  • Secondly, we need to train a neural network to find two entrance points of a coronary artery.
  • The third network is responsible for placing seed points in the image

Architecture of Centerline Net

Layer 1 2 3 4 5 6 7
Kernel width 3 3 3 3 3 1 1
Dilation 1 1 2 4 1 1 1
Channels 32 32 32 32 64 64 D+1
Field width 3 5 9 17 19 19 19

The number of output channels is equal to the number of potential directions in D, plus one channel for radius estimation.

The architecture of seedspint_net and ostiapoint_net are very similar to centerline_net. The only difference is in the output layer: instead of combining classification and regression, the final layer only performs regression.

Installation

To install all the required dependencies:

$ pip install -r requirement.txt

Training

1. Preparing CTA08 dataset

Tip:
CAT08 datasets need to be registered and certified in this website before it can be downloaded. It should be noted that your registration email may not be received by the server of the above website. If you have this problem, download this form, compile it and contact Dr.Theo van Walsum ([email protected]).

  1. Unzip training.tar.gz to:
    Coronary-Artery-Tracking-via-3D-CNN-Classification/
            -data_process_tools/
                -train_data/
                    -dataset00/
                    -dataset01/
                    -dataset02/
                    -dataset03/
                    -dataset04/
                    -dataset05/
                    -dataset06/
                    -dataset07/
  1. Create spacing_info.csv and nii.gz data
python3 creat_spacinginfo_data_tool.py
  1. Create centerline patch data
  • Create no offset samples
python3 centerline_patch_generater_no_offset.py
  • Create samples with offset
python3 centerline_patch_generater_offset.py
  1. Create seeds patch data
  • Create positve samples
python3 seedpoints_patch_generater_postive.py     
  • Create negative sample
python3 seedpoints_patch_generater_negative.py

those scripts will automaticlly create folders

-data_process_tools/
    -patch_data/
         -centerline_patch/
            -no_offset/
                 -point_500_gp_1/
                     -d0/
                     d0_patch_info_500.csv 
                     .
                     .
                     .
                     -d7/
                     d7_patch_info_500.csv
            -offset/
                  -point_500_gp_1/
                     -d0/
                     d0_patch_info_500.csv
                     .
                     .
                     .
                     -d7/
                     d7_patch_info_500.csv
  1. Create osita patch data
  • Create positve samples
python3 ostiapoints_patch_generater_positive.py
  • Create negative sample
python3 ostiapoints_patch_generater_negative.py

It should be noted that 8 samples corresponding to the data will be produced here, and the specific training set and test set division also need to write your own code to divide the data set and generate the train CSV file and val CSV file

2.Training Models

  1. Training centerline net
cd centerline_train_tools/
CUDA_VISIBLE_DEVICES=0 python3 centerline_train_tools.py
  1. Training seedpoints net
cd seedspoints_train_tools/
CUDA_VISIBLE_DEVICES=0 python3 seeds_train_tools.py
  1. Training ostiapoints net
cd ostiapoints_train_tools
CUDA_VISIBLE_DEVICES=0 python3 ostia_train_tools.py 

3.Create coronary artery vessels tree

cd infer_tools_tree/

First, you need to modify settingy.yaml replacing the path inside to the path of the file you saved

python3 vessels_tree_infer.py

The predicted vessel tree is shown in the figure below

The vessels from different seed points are spliced by breadth-first search, and then a complete single vessel is generated by depth-first search

Seedpoints net will generate 200 seed points as shown in the figure below. It can be seen that the seed points are distributed near several coronary arteries

References

@article{wolterink2019coronary,
  title={Coronary artery centerline extraction in cardiac CT angiography using a CNN-based orientation classifier},
  author={Wolterink, Jelmer M and van Hamersvelt, Robbert W and Viergever, Max A and Leiner, Tim Leiner, Ivana},
  journal={Medical image analysis},
  volume={51},
  pages={46--60},
  year={2019},
  publisher={Elsevier}
}
Owner
James
I am an investigator in the SenseTime. My research interests are in 3D Vision and Multiple Object Tracking.
James
Deep Learning to Create StepMania SM FIles

StepCOVNet Running Audio to SM File Generator Currently only produces .txt files. Use SMDataTools to convert .txt to .sm python stepmania_note_generat

Chimezie Iwuanyanwu 8 Jan 08, 2023
Simulating Sycamore quantum circuits classically using tensor network algorithm.

Simulating the Sycamore quantum supremacy circuit This repo contains data we have obtained in simulating the Sycamore quantum supremacy circuits with

Feng Pan 46 Nov 17, 2022
Official implementation of "Accelerating Reinforcement Learning with Learned Skill Priors", Pertsch et al., CoRL 2020

Accelerating Reinforcement Learning with Learned Skill Priors [Project Website] [Paper] Karl Pertsch1, Youngwoon Lee1, Joseph Lim1 1CLVR Lab, Universi

Cognitive Learning for Vision and Robotics (CLVR) lab @ USC 134 Dec 06, 2022
Learning Multiresolution Matrix Factorization and its Wavelet Networks on Graphs

Project Learning Multiresolution Matrix Factorization and its Wavelet Networks on Graphs, https://arxiv.org/pdf/2111.01940.pdf. Authors Truong Son Hy

5 Jun 28, 2022
CALVIN - A benchmark for Language-Conditioned Policy Learning for Long-Horizon Robot Manipulation Tasks

CALVIN CALVIN - A benchmark for Language-Conditioned Policy Learning for Long-Horizon Robot Manipulation Tasks Oier Mees, Lukas Hermann, Erick Rosete,

Oier Mees 107 Dec 26, 2022
Official repository of PanoAVQA: Grounded Audio-Visual Question Answering in 360° Videos (ICCV 2021)

Pano-AVQA Official repository of PanoAVQA: Grounded Audio-Visual Question Answering in 360° Videos (ICCV 2021) [Paper] [Poster] [Video] Getting Starte

Heeseung Yun 9 Dec 23, 2022
A curated list of awesome resources combining Transformers with Neural Architecture Search

A curated list of awesome resources combining Transformers with Neural Architecture Search

Yash Mehta 173 Jan 03, 2023
Supplemental Code for "ImpressionNet :A Multi view Approach to Predict Socio Facial Impressions"

Supplemental Code for "ImpressionNet :A Multi view Approach to Predict Socio Facial Impressions" Environment requirement This code is based on Python

Rohan Kumar Gupta 1 Dec 19, 2021
Image-popularity-score - A novel deep regression method for image scoring.

Image-popularity-score - A novel deep regression method for image scoring.

Shoaib ahmed 1 Dec 26, 2021
An implementation of "Learning human behaviors from motion capture by adversarial imitation"

Merel-MoCap-GAIL An implementation of Merel et al.'s paper on generative adversarial imitation learning (GAIL) using motion capture (MoCap) data: Lear

Yu-Wei Chao 34 Nov 12, 2022
Group Activity Recognition with Clustered Spatial Temporal Transformer

GroupFormer Group Activity Recognition with Clustered Spatial-TemporalTransformer Backbone Style Action Acc Activity Acc Config Download Inv3+flow+pos

28 Dec 12, 2022
ADGAN - The Implementation of paper Controllable Person Image Synthesis with Attribute-Decomposed GAN

ADGAN - The Implementation of paper Controllable Person Image Synthesis with Attribute-Decomposed GAN CVPR 2020 (Oral); Pose and Appearance Attributes Transfer;

Men Yifang 400 Dec 29, 2022
Naszilla is a Python library for neural architecture search (NAS)

A repository to compare many popular NAS algorithms seamlessly across three popular benchmarks (NASBench 101, 201, and 301). You can implement your ow

270 Jan 03, 2023
HybVIO visual-inertial odometry and SLAM system

HybVIO A visual-inertial odometry system with an optional SLAM module. This is a research-oriented codebase, which has been published for the purposes

Spectacular AI 320 Jan 03, 2023
Alpha-IoU: A Family of Power Intersection over Union Losses for Bounding Box Regression

Alpha-IoU: A Family of Power Intersection over Union Losses for Bounding Box Regression YOLOv5 with alpha-IoU losses implemented in PyTorch. Example r

Jacobi(Jiabo He) 147 Dec 05, 2022
Combine Tacotron2 and Hifi GAN to generate speech from text

EndToEndTextToSpeech Combine Tacotron2 and Hifi GAN to generate speech from text Download weights Hifi GAN - hifi_gan/checkpoint/ : pretrain 2.5M ste

Phạm Quốc Huy 1 Dec 18, 2021
Learning with Subset Stacking

Learning with Subset Stacking (LESS) LESS is a new supervised learning algorithm that is based on training many local estimators on subsets of a given

S. Ilker Birbil 19 Oct 04, 2022
Super Pix Adv - Offical implemention of Robust Superpixel-Guided Attentional Adversarial Attack (CVPR2020)

Super_Pix_Adv Offical implemention of Robust Superpixel-Guided Attentional Adver

DLight 8 Oct 26, 2022
Adversarial Graph Augmentation to Improve Graph Contrastive Learning

ADGCL : Adversarial Graph Augmentation to Improve Graph Contrastive Learning Introduction This repo contains the Pytorch [1] implementation of Adversa

susheel suresh 62 Nov 19, 2022
Official repository for "Exploiting Session Information in BERT-based Session-aware Sequential Recommendation", SIGIR 2022 short.

Session-aware BERT4Rec Official repository for "Exploiting Session Information in BERT-based Session-aware Sequential Recommendation", SIGIR 2022 shor

Jamie J. Seol 22 Dec 13, 2022