Loss functions for image segmentation

@article{LossOdyssey,
title = {Loss Odyssey in Medical Image Segmentation},
journal = {Medical Image Analysis},
volume = {71},
pages = {102035},
year = {2021},
author = {Jun Ma and Jianan Chen and Matthew Ng and Rui Huang and Yu Li and Chen Li and Xiaoping Yang and Anne L. Martel}
doi = {https://doi.org/10.1016/j.media.2021.102035},
url = {https://www.sciencedirect.com/science/article/pii/S1361841521000815}
}

Take-home message: compound loss functions are the most robust losses, especially for the highly imbalanced segmentation tasks.

Some recent side evidence: the winner in MICCAI 2020 HECKTOR Challenge used DiceFocal loss; the winner and runner-up in MICCAI 2020 ADAM Challenge used DiceTopK loss.

Date	First Author	Title	Conference/Journal
20210330	Suprosanna Shit and Johannes C. Paetzold	clDice - a Novel Topology-Preserving Loss Function for Tubular Structure Segmentation (keras and pytorch)	CVPR 2021
20210318	Xiaoling Hu	Topology-Aware Segmentation Using Discrete Morse Theory arxiv	ICLR 2021
20210211	Hoel Kervadec	Beyond pixel-wise supervision: semantic segmentation with higher-order shape descriptors	Submitted to MIDL 2021
20210210	Rosana EL Jurdi	A Surprisingly Effective Perimeter-based Loss for Medical Image Segmentation	Submitted to MIDL 2021
20201222	Zeju Li	Analyzing Overfitting Under Class Imbalance in Neural Networks for Image Segmentation	TMI
20210129	Nick Byrne	A Persistent Homology-Based Topological Loss Function for Multi-class CNN Segmentation of Cardiac MRI arxiv	STACOM 2020
20201019	Hyunseok Seo	Closing the Gap Between Deep Neural Network Modeling and Biomedical Decision-Making Metrics in Segmentation via Adaptive Loss Functions	TMI
20200929	Stefan Gerl	A Distance-Based Loss for Smooth and Continuous Skin Layer Segmentation in Optoacoustic Images	MICCAI 2020
20200821	Nick Byrne	A persistent homology-based topological loss function for multi-class CNN segmentation of cardiac MRI arxiv	STACOM
20200720	Boris Shirokikh	Universal Loss Reweighting to Balance Lesion Size Inequality in 3D Medical Image Segmentation arxiv (pytorch)	MICCAI 2020
20200708	Gonglei Shi	Marginal loss and exclusion loss for partially supervised multi-organ segmentation (arXiv)	MedIA
20200706	Yuan Lan	An Elastic Interaction-Based Loss Function for Medical Image Segmentation (pytorch) (arXiv)	MICCAI 2020
20200615	Tom Eelbode	Optimization for Medical Image Segmentation: Theory and Practice when evaluating with Dice Score or Jaccard Index	TMI
20200605	Guotai Wang	Noise-robust Dice loss: A Noise-robust Framework for Automatic Segmentation of COVID-19 Pneumonia Lesions from CT Images (pytorch)	TMI
202004	J. H. Moltz	Contour Dice coefficient (CDC) Loss: Learning a Loss Function for Segmentation: A Feasibility Study	ISBI
201912	Yuan Xue	Shape-Aware Organ Segmentation by Predicting Signed Distance Maps (arxiv) (pytorch)	AAAI 2020
201912	Xiaoling Hu	Topology-Preserving Deep Image Segmentation (paper) (pytorch)	NeurIPS
201910	Shuai Zhao	Region Mutual Information Loss for Semantic Segmentation (paper) (pytorch)	NeurIPS 2019
201910	Shuai Zhao	Correlation Maximized Structural Similarity Loss for Semantic Segmentation (paper)	arxiv
201908	Pierre-AntoineGanaye	Removing Segmentation Inconsistencies with Semi-Supervised Non-Adjacency Constraint (paper) (official pytorch)	Medical Image Analysis
201906	Xu Chen	Learning Active Contour Models for Medical Image Segmentation (paper) (official-keras)	CVPR 2019
20190422	Davood Karimi	Reducing the Hausdorff Distance in Medical Image Segmentation with Convolutional Neural Networks (paper) (pytorch)	TMI 201907
20190417	Francesco Caliva	Distance Map Loss Penalty Term for Semantic Segmentation (paper)	MIDL 2019
20190411	Su Yang	Major Vessel Segmentation on X-ray Coronary Angiography using Deep Networks with a Novel Penalty Loss Function (paper)	MIDL 2019
20190405	Boah Kim	Multiphase Level-Set Loss for Semi-Supervised and Unsupervised Segmentation with Deep Learning (paper)	arxiv
201901	Seyed Raein Hashemi	Asymmetric Loss Functions and Deep Densely Connected Networks for Highly Imbalanced Medical Image Segmentation: Application to Multiple Sclerosis Lesion Detection (paper)	IEEE Access
201812	Hoel Kervadec	Boundary loss for highly unbalanced segmentation (paper), (pytorch 1.0)	MIDL 2019
201810	Nabila Abraham	A Novel Focal Tversky loss function with improved Attention U-Net for lesion segmentation (paper) (keras)	ISBI 2019
201809	Fabian Isensee	CE+Dice: nnU-Net: Self-adapting Framework for U-Net-Based Medical Image Segmentation (paper)	arxiv
20180831	Ken C. L. Wong	3D Segmentation with Exponential Logarithmic Loss for Highly Unbalanced Object Sizes (paper)	MICCAI 2018
20180815	Wentao Zhu	Dice+Focal: AnatomyNet: Deep Learning for Fast and Fully Automated Whole-volume Segmentation of Head and Neck Anatomy (arxiv) (pytorch)	Medical Physics
201806	Javier Ribera	Weighted Hausdorff Distance: Locating Objects Without Bounding Boxes (paper), (pytorch)	CVPR 2019
201805	Saeid Asgari Taghanaki	Combo Loss: Handling Input and Output Imbalance in Multi-Organ Segmentation (arxiv) (keras)	Computerized Medical Imaging and Graphics
201709	S M Masudur Rahman AL ARIF	Shape-aware deep convolutional neural network for vertebrae segmentation (paper)	MICCAI 2017 Workshop
201708	Tsung-Yi Lin	Focal Loss for Dense Object Detection (paper), (code)	ICCV, TPAMI
20170711	Carole Sudre	Generalised Dice overlap as a deep learning loss function for highly unbalanced segmentations (paper)	DLMIA 2017
20170703	Lucas Fidon	Generalised Wasserstein Dice Score for Imbalanced Multi-class Segmentation using Holistic Convolutional Networks (paper)	MICCAI 2017 BrainLes
201705	Maxim Berman	The Lovász-Softmax loss: A tractable surrogate for the optimization of the intersection-over-union measure in neural networks (paper), (code)	CVPR 2018
201701	Seyed Sadegh Mohseni Salehi	Tversky loss function for image segmentation using 3D fully convolutional deep networks (paper)	MICCAI 2017 MLMI
201612	Md Atiqur Rahman	Optimizing Intersection-Over-Union in Deep Neural Networks for Image Segmentation (paper)	2016 International Symposium on Visual Computing
201608	Michal Drozdzal	"Dice Loss (without square)" The Importance of Skip Connections in Biomedical Image Segmentation (arxiv)	DLMIA 2016
201606	Fausto Milletari	"Dice Loss (with square)" V-net: Fully convolutional neural networks for volumetric medical image segmentation (arxiv), (caffe code)	International Conference on 3D Vision
201605	Zifeng Wu	TopK loss Bridging Category-level and Instance-level Semantic Image Segmentation (paper)	arxiv
201511	Tom Brosch	"Sensitivity-Specifity loss" Deep Convolutional Encoder Networks for Multiple Sclerosis Lesion Segmentation (paper) (code)	MICCAI 2015
201505	Olaf Ronneberger	"Weighted cross entropy" U-Net: Convolutional Networks for Biomedical Image Segmentation (paper)	MICCAI 2015
201309	Gabriela Csurka	What is a good evaluation measure for semantic segmentation? (paper)	BMVA 2013

Most of the corresponding tensorflow code can be found here.

A collection of loss functions for medical image segmentation

Related tags

Overview

Loss functions for image segmentation

Owner

Jun

Just Go with the Flow: Self-Supervised Scene Flow Estimation

Distributed Evolutionary Algorithms in Python

A visualization tool to show a TensorFlow's graph like TensorBoard

Ranger deep learning optimizer rewrite to use newest components

codebase for "A Theory of the Inductive Bias and Generalization of Kernel Regression and Wide Neural Networks"

Code for EMNLP'21 paper "Types of Out-of-Distribution Texts and How to Detect Them"

The repo contains the code of the ACL2020 paper `Dice Loss for Data-imbalanced NLP Tasks`

Causal Imitative Model for Autonomous Driving

Classification Modeling: Probability of Default

(IEEE TIP 2021) Regularized Densely-connected Pyramid Network for Salient Instance Segmentation

EncT5: Fine-tuning T5 Encoder for Non-autoregressive Tasks

Hippocampal segmentation using the UNet network for each axis

This code provides various models combining dilated convolutions with residual networks

A simple, clean TensorFlow implementation of Generative Adversarial Networks with a focus on modeling illustrations.

Class activation maps for your PyTorch models (CAM, Grad-CAM, Grad-CAM++, Smooth Grad-CAM++, Score-CAM, SS-CAM, IS-CAM, XGrad-CAM, Layer-CAM)

Patch Rotation: A Self-Supervised Auxiliary Task for Robustness and Accuracy of Supervised Models

Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition

This repository contains the exercises and its solution contained in the book "An Introduction to Statistical Learning" in python.

Multi-Agent Reinforcement Learning for Active Voltage Control on Power Distribution Networks (MAPDN)

Training RNNs as Fast as CNNs