Self-Supervised Methods for Noise-Removal

Related tags

Deep LearningSSMNR
Overview

SSMNR | Self-Supervised Methods for Noise Removal

Image denoising is the task of removing noise from an image, which can be formulated as the task of separating the noise signal from the meaningful information in images. Traditionally, this has been addressed both by spatial domain methods and transfer domain methods. However, from around 2016 onwards, image denoising techniques based on neural networks have started to outperfom these methods, with CNN-based denoisers obtaining impressive results.

One limitation to the use of neural-network based denoisers in many applications is the need for extensive, labeled datasets containing both noised images, and ground-truth, noiseless images. In answer to this, multiple works have explored the use of semi-supervised approaches for noise removal, requiring either noised image pairs but no clean target images (Noise2Noise) or, more recently, no additional data than the noised image (Noise2Void). This project aims at studying these approaches for the task of noise removal, and re-implementing them in PyTorch.

This repository contains our code for this task. This code is heavily based on both the original implementation of the Noise2Void article available here, on other implementations and PyTorch/TensorFlow reproducibility challenges here and here, on the U-NET Transformer architecture available here, as well as some base code from our teachers for a project on bird species recognition.

Data

Data used to train and evaluate the algorithm consists mostly in:

No noiseless data was used to train the models.

Usage

To reproduce these results, please start by cloning the repository locally:

git clone https://github.com/bglbrt/SSMNR.git

Then, install the required libraries:

pip install -r requirements.txt

Denoising images (with provided, pre-trained weights)

To denoise an image or multiple images from a specified directory, run:

python main.py --mode denoise --model "model" --images_path "path/to/image/or/dir" --weights "path/to/model/weights"

Provided pre-trained weights are formatted as: "models/model_"+model_name+_+noise_type+sigma+".pth".

Available weights are:

  • weights for the N2V model:
    • models/model_N2V_G5.pth
    • models/model_N2V_G10.pth
    • models/model_N2V_G15.pth
    • models/model_N2V_G25.pth
    • models/model_N2V_G35.pth
    • models/model_N2V_G50.pth
  • weights for the N2VT (N2V with U-NET Transformer) model:
    • models/model_N2V_G5.pth (please contact us to obtain weights)
    • models/model_N2V_G10.pth (please contact us to obtain weights)
    • models/model_N2V_G25.pth (please contact us to obtain weights)

Options available for denoising are:

  • --mode: Training (train), denoising (denoise) or evaluation (eval) mode
    • default: train
  • --images_path: Path to image or directory of images to denoise.
    • default: None
  • --model: Name of model for noise removal
    • default: N2V
  • --n_channels: Number of channels in images - i.e. RGB or Grayscale images
    • default: 3
  • --weights: Path to weights to use for denoising, evaluation, or fine-tuning when training.
    • default: None
  • --slide: Sliding window size for denoising and evaluation
    • default: 32
  • --use_cuda: Use of GPU or CPU
    • default: 32

Evaluation

To evaluate a model using a dataset in a specified directory, run:

python main.py --mode eval --model "model" --images_path "path/to/image/or/dir" --weights "path/to/model/weights"

Note that the data located at path/to/image/or/dir must include a folder named original with noiseless images.

Evaluation methods include:

  • N2V (Noise2Void with trained weights)
  • N2VT (Noise2VoidTransformer with trained weights)
  • BM3D (Block-Matching and 3D Filtering)
  • MEAN (5x5 mean filter)
  • MEDIAN (5x5 median filter)

Provided pre-trained weights for N2V and N2VT are formatted as: "models/model_"+model_name+_+noise_type+sigma+".pth".

Available weights are:

  • weights for the N2V model:
    • models/model_N2V_G5.pth
    • models/model_N2V_G10.pth
    • models/model_N2V_G15.pth
    • models/model_N2V_G25.pth
    • models/model_N2V_G35.pth
    • models/model_N2V_G50.pth
  • weights for the N2VT (N2V with U-NET Transformer) model:
    • models/model_N2V_G5.pth
    • models/model_N2V_G10.pth
    • models/model_N2V_G25.pth

Options available for evaluation are:

  • --mode: Training (train), denoising (denoise) or evaluation (eval) mode
    • default: train
  • --images_path: Path to image or directory of images to evaluate.
    • default: None
  • --model: Name of model for noise removal
    • default: N2V
  • --n_channels: Number of channels in images - i.e. RGB or Grayscale images
    • default: 3
  • --weights: Path to weights to use for denoising, evaluation, or fine-tuning when training.
    • default: None
  • --slide: Sliding window size for denoising and evaluation
    • default: 32
  • --use_cuda: Use of GPU or CPU
    • default: 32

Training

To train weights for the N2V and N2VT models using data located in the data folder, run:

python main.py data "data" --model "N2V" --mode train"

Note that the data folder must contain two folders named train and validation.

Options available for training are:

  • --data: Folder where training and testing data is located.
    • default: data
  • --mode: Training (train), denoising (denoise) or evaluation (eval) mode
    • default: train
  • --model: Name of model for noise removal.
    • default: N2V
  • --n_channels: Number of channels in images - i.e. RGB or Grayscale images
    • default: 3
  • --input_size: Model patches input size
    • default: 64
  • --masking_method: Blind-spot masking method
    • default: UPS
  • --window: Window for blind-spot masking method in UPS
    • default: 5
  • --n_feat: Number of feature maps of the first convolutional layer
    • default: 96
  • --noise_type: Noise type from Gaussian (G), Poisson (P) and Impulse (I)
    • default: G
  • --ratio: Ratio for number of blind-spot pixels in patch
    • default: 1/64
  • --from_pretrained: Train model from pre-trained weights
    • default: False
  • --weights: Path to weights to use for denoising, evaluation, or fine-tuning when training
    • default: None
  • --weights_init_method: Weights initialization method
    • default: kaiming
  • --loss: Loss function for training
    • default: L2
  • --batch_size: Batch size for training data
    • default: 64
  • --epochs: Number of epochs to train the model.
    • default: 300
  • --steps_per_epoch: Number of steps per epoch for training
    • default: 100
  • --sigma: Noise parameter for creating labels - depends on distribution
    • default: 25
  • --lr: Learning rate
    • default: 4e-4
  • --wd: Weight decay for RAdam optimiser
    • default: 1e-4
  • --use_cuda: Use of GPU or CPU
    • default: 32
  • --seed: Random seed
    • default: 1

Required libraries

The files present on this repository require the following libraries (also listed in requirements.txt):

The Incredible PyTorch: a curated list of tutorials, papers, projects, communities and more relating to PyTorch.

This is a curated list of tutorials, projects, libraries, videos, papers, books and anything related to the incredible PyTorch. Feel free to make a pu

Ritchie Ng 9.2k Jan 02, 2023
[ICCV-2021] An Empirical Study of the Collapsing Problem in Semi-Supervised 2D Human Pose Estimation

An Empirical Study of the Collapsing Problem in Semi-Supervised 2D Human Pose Estimation (ICCV 2021) Introduction This is an official pytorch implemen

rongchangxie 42 Jan 04, 2023
Code and models for "Pano3D: A Holistic Benchmark and a Solid Baseline for 360 Depth Estimation", OmniCV Workshop @ CVPR21.

Pano3D A Holistic Benchmark and a Solid Baseline for 360o Depth Estimation Pano3D is a new benchmark for depth estimation from spherical panoramas. We

Visual Computing Lab, Information Technologies Institute, Centre for Reseach and Technology Hellas 50 Dec 29, 2022
Python3 / PyTorch implementation of the following paper: Fine-grained Semantics-aware Representation Enhancement for Self-supervisedMonocular Depth Estimation. ICCV 2021 (oral)

FSRE-Depth This is a Python3 / PyTorch implementation of FSRE-Depth, as described in the following paper: Fine-grained Semantics-aware Representation

77 Dec 28, 2022
Project code for weakly supervised 3D object detectors using wide-baseline multi-view traffic camera data: WIBAM.

WIBAM (Work in progress) Weakly Supervised Training of Monocular 3D Object Detectors Using Wide Baseline Multi-view Traffic Camera Data 3D object dete

Matthew Howe 10 Aug 24, 2022
Code for ViTAS_Vision Transformer Architecture Search

Vision Transformer Architecture Search This repository open source the code for ViTAS: Vision Transformer Architecture Search. ViTAS aims to search fo

46 Dec 17, 2022
Converting CPT to bert form for use

cpt-encoder 将CPT转成bert形式使用 说明 刚刚刷到又出了一种模型:CPT,看论文显示,在很多中文任务上性能比mac bert还好,就迫不及待想把它用起来。 根据对源码的研究,发现该模型在做nlu建模时主要用的encoder部分,也就是bert,因此我将这部分权重转为bert权重类型

黄辉 1 Oct 14, 2021
Integrated Semantic and Phonetic Post-correction for Chinese Speech Recognition

Integrated Semantic and Phonetic Post-correction for Chinese Speech Recognition | paper | dataset | pretrained detection model | Authors: Yi-Chang Che

Yi-Chang Chen 1 Aug 23, 2022
Self-Adaptable Point Processes with Nonparametric Time Decays

NPPDecay This is our implementation for the paper Self-Adaptable Point Processes with Nonparametric Time Decays, by Zhimeng Pan, Zheng Wang, Jeff M. P

zpan 2 Sep 24, 2022
A simplistic and efficient pure-python neural network library from Phys Whiz with CPU and GPU support.

A simplistic and efficient pure-python neural network library from Phys Whiz with CPU and GPU support.

Manas Sharma 19 Feb 28, 2022
This repository is for our EMNLP 2021 paper "Automated Generation of Accurate & Fluent Medical X-ray Reports"

Introduction: X-Ray Report Generation This repository is for our EMNLP 2021 paper "Automated Generation of Accurate & Fluent Medical X-ray Reports". O

no name 36 Dec 16, 2022
Fast Differentiable Matrix Sqrt Root

Official Pytorch implementation of ICLR 22 paper Fast Differentiable Matrix Square Root

YueSong 42 Dec 30, 2022
Deep Structured Instance Graph for Distilling Object Detectors (ICCV 2021)

DSIG Deep Structured Instance Graph for Distilling Object Detectors Authors: Yixin Chen, Pengguang Chen, Shu Liu, Liwei Wang, Jiaya Jia. [pdf] [slide]

DV Lab 31 Nov 17, 2022
PyTorch code for BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation

BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation

Salesforce 1.3k Dec 31, 2022
Repositorio oficial del curso IIC2233 Programación Avanzada 🚀✨

IIC2233 - Programación Avanzada Evaluación Las evaluaciones serán efectuadas por medio de actividades prácticas en clases y tareas. Se calculará la no

IIC2233 @ UC 47 Sep 06, 2022
RuDOLPH: One Hyper-Modal Transformer can be creative as DALL-E and smart as CLIP

[Paper] [Хабр] [Model Card] [Colab] [Kaggle] RuDOLPH 🦌 🎄 ☃️ One Hyper-Modal Transformer can be creative as DALL-E and smart as CLIP Russian Diffusio

AI Forever 232 Jan 04, 2023
LLVIP: A Visible-infrared Paired Dataset for Low-light Vision

LLVIP: A Visible-infrared Paired Dataset for Low-light Vision Project | Arxiv | Abstract It is very challenging for various visual tasks such as image

CVSM Group - email: <a href=[email protected]"> 377 Jan 07, 2023
Breaking the Dilemma of Medical Image-to-image Translation

Breaking the Dilemma of Medical Image-to-image Translation Supervised Pix2Pix and unsupervised Cycle-consistency are two modes that dominate the field

Kid Liet 86 Dec 21, 2022
Annotated, understandable, and visually interpretable PyTorch implementations of: VAE, BIRVAE, NSGAN, MMGAN, WGAN, WGANGP, LSGAN, DRAGAN, BEGAN, RaGAN, InfoGAN, fGAN, FisherGAN

Overview PyTorch 0.4.1 | Python 3.6.5 Annotated implementations with comparative introductions for minimax, non-saturating, wasserstein, wasserstein g

Shayne O'Brien 471 Dec 16, 2022
Explaining Hyperparameter Optimization via PDPs

Explaining Hyperparameter Optimization via PDPs This repository gives access to an implementation of the methods presented in the paper submission “Ex

2 Nov 16, 2022