Self-Supervised Methods for Noise-Removal

Related tags

Deep LearningSSMNR
Overview

SSMNR | Self-Supervised Methods for Noise Removal

Image denoising is the task of removing noise from an image, which can be formulated as the task of separating the noise signal from the meaningful information in images. Traditionally, this has been addressed both by spatial domain methods and transfer domain methods. However, from around 2016 onwards, image denoising techniques based on neural networks have started to outperfom these methods, with CNN-based denoisers obtaining impressive results.

One limitation to the use of neural-network based denoisers in many applications is the need for extensive, labeled datasets containing both noised images, and ground-truth, noiseless images. In answer to this, multiple works have explored the use of semi-supervised approaches for noise removal, requiring either noised image pairs but no clean target images (Noise2Noise) or, more recently, no additional data than the noised image (Noise2Void). This project aims at studying these approaches for the task of noise removal, and re-implementing them in PyTorch.

This repository contains our code for this task. This code is heavily based on both the original implementation of the Noise2Void article available here, on other implementations and PyTorch/TensorFlow reproducibility challenges here and here, on the U-NET Transformer architecture available here, as well as some base code from our teachers for a project on bird species recognition.

Data

Data used to train and evaluate the algorithm consists mostly in:

No noiseless data was used to train the models.

Usage

To reproduce these results, please start by cloning the repository locally:

git clone https://github.com/bglbrt/SSMNR.git

Then, install the required libraries:

pip install -r requirements.txt

Denoising images (with provided, pre-trained weights)

To denoise an image or multiple images from a specified directory, run:

python main.py --mode denoise --model "model" --images_path "path/to/image/or/dir" --weights "path/to/model/weights"

Provided pre-trained weights are formatted as: "models/model_"+model_name+_+noise_type+sigma+".pth".

Available weights are:

  • weights for the N2V model:
    • models/model_N2V_G5.pth
    • models/model_N2V_G10.pth
    • models/model_N2V_G15.pth
    • models/model_N2V_G25.pth
    • models/model_N2V_G35.pth
    • models/model_N2V_G50.pth
  • weights for the N2VT (N2V with U-NET Transformer) model:
    • models/model_N2V_G5.pth (please contact us to obtain weights)
    • models/model_N2V_G10.pth (please contact us to obtain weights)
    • models/model_N2V_G25.pth (please contact us to obtain weights)

Options available for denoising are:

  • --mode: Training (train), denoising (denoise) or evaluation (eval) mode
    • default: train
  • --images_path: Path to image or directory of images to denoise.
    • default: None
  • --model: Name of model for noise removal
    • default: N2V
  • --n_channels: Number of channels in images - i.e. RGB or Grayscale images
    • default: 3
  • --weights: Path to weights to use for denoising, evaluation, or fine-tuning when training.
    • default: None
  • --slide: Sliding window size for denoising and evaluation
    • default: 32
  • --use_cuda: Use of GPU or CPU
    • default: 32

Evaluation

To evaluate a model using a dataset in a specified directory, run:

python main.py --mode eval --model "model" --images_path "path/to/image/or/dir" --weights "path/to/model/weights"

Note that the data located at path/to/image/or/dir must include a folder named original with noiseless images.

Evaluation methods include:

  • N2V (Noise2Void with trained weights)
  • N2VT (Noise2VoidTransformer with trained weights)
  • BM3D (Block-Matching and 3D Filtering)
  • MEAN (5x5 mean filter)
  • MEDIAN (5x5 median filter)

Provided pre-trained weights for N2V and N2VT are formatted as: "models/model_"+model_name+_+noise_type+sigma+".pth".

Available weights are:

  • weights for the N2V model:
    • models/model_N2V_G5.pth
    • models/model_N2V_G10.pth
    • models/model_N2V_G15.pth
    • models/model_N2V_G25.pth
    • models/model_N2V_G35.pth
    • models/model_N2V_G50.pth
  • weights for the N2VT (N2V with U-NET Transformer) model:
    • models/model_N2V_G5.pth
    • models/model_N2V_G10.pth
    • models/model_N2V_G25.pth

Options available for evaluation are:

  • --mode: Training (train), denoising (denoise) or evaluation (eval) mode
    • default: train
  • --images_path: Path to image or directory of images to evaluate.
    • default: None
  • --model: Name of model for noise removal
    • default: N2V
  • --n_channels: Number of channels in images - i.e. RGB or Grayscale images
    • default: 3
  • --weights: Path to weights to use for denoising, evaluation, or fine-tuning when training.
    • default: None
  • --slide: Sliding window size for denoising and evaluation
    • default: 32
  • --use_cuda: Use of GPU or CPU
    • default: 32

Training

To train weights for the N2V and N2VT models using data located in the data folder, run:

python main.py data "data" --model "N2V" --mode train"

Note that the data folder must contain two folders named train and validation.

Options available for training are:

  • --data: Folder where training and testing data is located.
    • default: data
  • --mode: Training (train), denoising (denoise) or evaluation (eval) mode
    • default: train
  • --model: Name of model for noise removal.
    • default: N2V
  • --n_channels: Number of channels in images - i.e. RGB or Grayscale images
    • default: 3
  • --input_size: Model patches input size
    • default: 64
  • --masking_method: Blind-spot masking method
    • default: UPS
  • --window: Window for blind-spot masking method in UPS
    • default: 5
  • --n_feat: Number of feature maps of the first convolutional layer
    • default: 96
  • --noise_type: Noise type from Gaussian (G), Poisson (P) and Impulse (I)
    • default: G
  • --ratio: Ratio for number of blind-spot pixels in patch
    • default: 1/64
  • --from_pretrained: Train model from pre-trained weights
    • default: False
  • --weights: Path to weights to use for denoising, evaluation, or fine-tuning when training
    • default: None
  • --weights_init_method: Weights initialization method
    • default: kaiming
  • --loss: Loss function for training
    • default: L2
  • --batch_size: Batch size for training data
    • default: 64
  • --epochs: Number of epochs to train the model.
    • default: 300
  • --steps_per_epoch: Number of steps per epoch for training
    • default: 100
  • --sigma: Noise parameter for creating labels - depends on distribution
    • default: 25
  • --lr: Learning rate
    • default: 4e-4
  • --wd: Weight decay for RAdam optimiser
    • default: 1e-4
  • --use_cuda: Use of GPU or CPU
    • default: 32
  • --seed: Random seed
    • default: 1

Required libraries

The files present on this repository require the following libraries (also listed in requirements.txt):

Chunkmogrify: Real image inversion via Segments

Chunkmogrify: Real image inversion via Segments Teaser video with live editing sessions can be found here This code demonstrates the ideas discussed i

David Futschik 112 Jan 04, 2023
HiPAL: A Deep Framework for Physician Burnout Prediction Using Activity Logs in Electronic Health Records

HiPAL Code for KDD'22 Applied Data Science Track submission -- HiPAL: A Deep Framework for Physician Burnout Prediction Using Activity Logs in Electro

Hanyang Liu 4 Aug 08, 2022
ESL: Event-based Structured Light

ESL: Event-based Structured Light Video (click on the image) This is the code for the 2021 3DV paper ESL: Event-based Structured Light by Manasi Mugli

Robotics and Perception Group 29 Oct 24, 2022
Time Dependent DFT in Tamm-Dancoff Approximation

Density Function Theory Program - kspy-tddft(tda) This is an implementation of Time-Dependent Density Functional Theory(TDDFT) using the Tamm-Dancoff

Peter Borthwick 2 Nov 17, 2022
A tool to prepare websites grabbed with wget for local viewing.

makelocal A tool to prepare websites grabbed with wget for local viewing. exapmples After fetching xkcd.com with: wget -r -no-remove-listing -r -N --p

5 Apr 23, 2022
Incremental Transformer Structure Enhanced Image Inpainting with Masking Positional Encoding (CVPR2022)

Incremental Transformer Structure Enhanced Image Inpainting with Masking Positional Encoding by Qiaole Dong*, Chenjie Cao*, Yanwei Fu Paper and Supple

Qiaole Dong 190 Dec 27, 2022
A GUI to automatically create a TOPAS-readable MLC simulation file

Python script to create a TOPAS-readable simulation file descriring a Multi-Leaf-Collimator. Builds the MLC using the data from a 3D .stl file.

Sebastian Schäfer 0 Jun 19, 2022
学习 python3 以来写的一些垃圾玩具……

和东哥做兄弟 Author: chiupam 版权 未经本人同意,仓库内所有资源文件,禁止任何公众号、自媒体、开发者进行任何形式的转载、发布、搬运。 声明 这不是一个开源项目,只是把 GitHub 当作一个代码的存储空间,本项目不接受任何开源要求。 仅用于学习研究,禁止用于商业用途,不能保证其合法性

Chiupam 67 Mar 26, 2022
CyTran: Cycle-Consistent Transformers for Non-Contrast to Contrast CT Translation

CyTran: Cycle-Consistent Transformers for Non-Contrast to Contrast CT Translation We propose a novel approach to translate unpaired contrast computed

Nicolae Catalin Ristea 13 Jan 02, 2023
Example of a Quantum LSTM

Example of a Quantum LSTM

Riccardo Di Sipio 36 Oct 31, 2022
A selection of State Of The Art research papers (and code) on human locomotion (pose + trajectory) prediction (forecasting)

A selection of State Of The Art research papers (and code) on human trajectory prediction (forecasting). Papers marked with [W] are workshop papers.

Karttikeya Manglam 40 Nov 18, 2022
Pytorch implementation for "Density-aware Chamfer Distance as a Comprehensive Metric for Point Cloud Completion" (NeurIPS 2021)

Density-aware Chamfer Distance This repository contains the official PyTorch implementation of our paper: Density-aware Chamfer Distance as a Comprehe

Tong WU 93 Dec 15, 2022
Reproduce ResNet-v2(Identity Mappings in Deep Residual Networks) with MXNet

Reproduce ResNet-v2 using MXNet Requirements Install MXNet on a machine with CUDA GPU, and it's better also installed with cuDNN v5 Please fix the ran

Wei Wu 531 Dec 04, 2022
AEI: Actors-Environment Interaction with Adaptive Attention for Temporal Action Proposals Generation

AEI: Actors-Environment Interaction with Adaptive Attention for Temporal Action Proposals Generation A pytorch-version implementation codes of paper:

11 Dec 13, 2022
Neural-fractal - Create Fractals Using Complex-Valued Neural Networks!

Neural Fractal Create Fractals Using Complex-Valued Neural Networks! Home Page Features Define Dynamical Systems Using Complex-Valued Neural Networks

Amirabbas Asadi 10 Dec 17, 2022
Minimal implementation of Denoised Smoothing: A Provable Defense for Pretrained Classifiers in TensorFlow.

Denoised-Smoothing-TF Minimal implementation of Denoised Smoothing: A Provable Defense for Pretrained Classifiers in TensorFlow. Denoised Smoothing is

Sayak Paul 19 Dec 11, 2022
Implementation for our AAAI2021 paper (Entity Structure Within and Throughout: Modeling Mention Dependencies for Document-Level Relation Extraction).

SSAN Introduction This is the pytorch implementation of the SSAN model (see our AAAI2021 paper: Entity Structure Within and Throughout: Modeling Menti

benfeng 69 Nov 15, 2022
Expressive Power of Invariant and Equivaraint Graph Neural Networks (ICLR 2021)

Expressive Power of Invariant and Equivaraint Graph Neural Networks In this repository, we show how to use powerful GNN (2-FGNN) to solve a graph alig

Marc Lelarge 36 Dec 12, 2022
Transformer model implemented with Pytorch

transformer-pytorch Transformer model implemented with Pytorch Attention is all you need-[Paper] Architecture Self-Attention self_attention.py class

Mingu Kang 12 Sep 03, 2022
Implementation of a Transformer that Ponders, using the scheme from the PonderNet paper

Ponder(ing) Transformer Implementation of a Transformer that learns to adapt the number of computational steps it takes depending on the difficulty of

Phil Wang 65 Oct 04, 2022