HINet: Half Instance Normalization Network for Image Restoration

Related tags

Deep LearningHINet
Overview

PWC PWC PWC PWC PWC PWC PWC

HINet: Half Instance Normalization Network for Image Restoration

Liangyu Chen, Xin Lu, Jie Zhang, Xiaojie Chu, Chengpeng Chen

Paper: https://arxiv.org/abs/2105.06086

In this paper, we explore the role of Instance Normalization in low-level vision tasks. Specifically, we present a novel block: Half Instance Normalization Block (HIN Block), to boost the performance of image restoration networks. Based on HIN Block, we design a simple and powerful multi-stage network named HINet, which consists of two subnetworks. With the help of HIN Block, HINet surpasses the state-of-the-art (SOTA) on various image restoration tasks. For image denoising, we exceed it 0.11dB and 0.28 dB in PSNR on SIDD dataset, with only 7.5% and 30% of its multiplier-accumulator operations (MACs), 6.8 times and 2.9 times speedup respectively. For image deblurring, we get comparable performance with 22.5% of its MACs and 3.3 times speedup on REDS and GoPro datasets. For image deraining, we exceed it by 0.3 dB in PSNR on the average result of multiple datasets with 1.4 times speedup. With HINet, we won 1st place on the NTIRE 2021 Image Deblurring Challenge - Track2. JPEG Artifacts, with a PSNR of 29.70.

Network Architecture

arch

Installation

This implementation based on BasicSR which is a open source toolbox for image/video restoration tasks.

python 3.6.9
pytorch 1.5.1
cuda 10.1
git clone https://github.com/megvii-model/HINet
cd HINet
pip install -r requirements.txt
python setup.py develop --no_cuda_ext

Image Restoration Tasks

Image denoise, deblur, derain.

Image Denoise - SIDD dataset (Coming soon)
Image Deblur - GoPro dataset (Click to expand)

  • prepare data

    • mkdir ./datasets/GoPro

    • download the train set in ./datasets/GoPro/train and test set in ./datasets/GoPro/test (refer to MPRNet)

    • it should be like:

      ./datasets/
./datasets/GoPro/
./datasets/GoPro/train/
./datasets/GoPro/train/input/
./datasets/GoPro/train/target/
./datasets/GoPro/test/
./datasets/GoPro/test/input/
./datasets/GoPro/test/target/

    • python scripts/data_preparation/gopro.py

      • crop the train image pairs to 512x512 patches.

  • eval

    • download pretrained model to ./experiments/pretrained_models/HINet-GoPro.pth
    • python basicsr/test.py -opt options/test/REDS/HINet-GoPro.yml

  • train

    • python -m torch.distributed.launch --nproc_per_node=8 --master_port=4321 basicsr/train.py -opt options/train/GoPro/HINet.yml --launcher pytorch
Image Deblur - REDS dataset (Click to expand)

  • prepare data

    • mkdir ./datasets/REDS

    • download the train / val set from train_blur, train_sharp, val_blur, val_sharp to ./datasets/REDS/ and unzip them.

    • it should be like

      ./datasets/
./datasets/REDS/
./datasets/REDS/val/
./datasets/REDS/val/val_blur_jpeg/
./datasets/REDS/val/val_sharp/
./datasets/REDS/train/
./datasets/REDS/train/train_blur_jpeg/
./datasets/REDS/train/train_sharp/

    • python scripts/data_preparation/reds.py

      • flatten the folders and extract 300 validation images.

  • eval

    • download pretrained model to ./experiments/pretrained_models/HINet-REDS.pth
    • python basicsr/test.py -opt options/test/REDS/HINet-REDS.yml

  • train

    • python -m torch.distributed.launch --nproc_per_node=8 --master_port=4321 basicsr/train.py -opt options/train/REDS/HINet.yml --launcher pytorch
Image Derain - Rain13k dataset (Click to expand)

  • prepare data

    • mkdir ./datasets/Rain13k

    • download the train set and test set (refer to MPRNet)

    • it should be like

      ./datasets/
./datasets/Rain13k/
./datasets/Rain13k/train/
./datasets/Rain13k/train/input/
./datasets/Rain13k/train/target/
./datasets/Rain13k/test/
./datasets/Rain13k/test/Test100/
./datasets/Rain13k/test/Rain100H/
./datasets/Rain13k/test/Rain100L/
./datasets/Rain13k/test/Test2800/
./datasets/Rain13k/test/Test1200/

  • eval

    • download pretrained model to ./experiments/pretrained_models/HINet-Rain13k.pth

    • For Test100:

      • python basicsr/test.py -opt options/test/Rain13k/HINet-Test100.yml

    • For Rain100H

      • python basicsr/test.py -opt options/test/Rain13k/HINet-Rain100H.yml

    • For Rain100L

      • python basicsr/test.py -opt options/test/Rain13k/HINet-Rain100L.yml

    • For Test2800

      • python basicsr/test.py -opt options/test/Rain13k/HINet-Test2800.yml

    • For Test1200

      • python basicsr/test.py -opt options/test/Rain13k/HINet-Test1200.yml

  • train

    • python -m torch.distributed.launch --nproc_per_node=8 --master_port=4321 basicsr/train_rain.py -opt options/train/Rain13k/HINet.yml --launcher pytorch

Results

Some of the following results are higher than the original paper as we optimized some hyper-parameters.

NTIRE2021 Deblur Track2 ResultSIDD ResultGoPro Result
REDDS ResultRain13k Result

Citations

If HINet helps your research or work, please consider citing HINet.

@inproceedings{chen2021hinet,
  title={HINet: Half Instance Normalization Network for Image Restoration},
  author={Liangyu Chen and Xin Lu and Jie Zhang and Xiaojie Chu and Chengpeng Chen},
  booktitle={IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops},
  year={2021}
}

Contact

If you have any questions, please contact [email protected] or [email protected] .

Owner
GitHub Repository
StyleGAN-NADA: CLIP-Guided Domain Adaptation of Image Generators

StyleGAN-NADA: CLIP-Guided Domain Adaptation of Image Generators [Project Website] [Replicate.ai Project] StyleGAN-NADA: CLIP-Guided Domain Adaptation

992 Dec 30, 2022
An All-MLP solution for Vision, from Google AI

MLP Mixer - Pytorch An All-MLP solution for Vision, from Google AI, in Pytorch. No convolutions nor attention needed! Yannic Kilcher video Install $ p

Phil Wang 784 Jan 06, 2023
Geometric Sensitivity Decomposition

Geometric Sensitivity Decomposition This repo is the official implementation of A Geometric Perspective towards Neural Calibration via Sensitivity Dec

16 Dec 26, 2022
Implementation of the method described in the Speech Resynthesis from Discrete Disentangled Self-Supervised Representations.

Speech Resynthesis from Discrete Disentangled Self-Supervised Representations Implementation of the method described in the Speech Resynthesis from Di

4 Mar 11, 2022
Food recognition model using convolutional neural network & computer vision

Food recognition model using convolutional neural network & computer vision. The goal is to match or beat the DeepFood Research Paper

Hemanth Chandran 1 Jan 13, 2022
Log4j JNDI inj. vuln scanner

Log-4-JAM - Log 4 Just Another Mess Log4j JNDI inj. vuln scanner Requirements pip3 install requests_toolbelt Usage # make sure target list has http/ht

Ashish Kunwar 66 Nov 09, 2022
Intel® Nervana™ reference deep learning framework committed to best performance on all hardware

DISCONTINUATION OF PROJECT. This project will no longer be maintained by Intel. Intel will not provide or guarantee development of or support for this

Nervana 3.9k Dec 20, 2022
Fairness Metrics: All you need to know

Fairness Metrics: All you need to know Testing machine learning software for ethical bias has become a pressing current concern. Recent research has p

Anonymous2020 1 Jan 17, 2022
blind SQLIpy sebuah alat injeksi sql yang menggunakan waktu sql untuk mendapatkan sebuah server database.

blind SQLIpy Alat blind SQLIpy ini merupakan alat injeksi sql yang menggunakan metode time based blind sql injection metode tersebut membutuhkan waktu

Galih Anggoro Prasetya 4 Feb 24, 2022
Source code for From Stars to Subgraphs

GNNAsKernel Official code for From Stars to Subgraphs: Uplifting Any GNN with Local Structure Awareness Visualizations GNN-AK(+) GNN-AK(+) with Subgra

44 Dec 19, 2022
face property detection pytorch

This is the face property train code of project face-detection-project

i am x 2 Oct 18, 2021
Remote sensing change detection using PaddlePaddle

Change Detection Laboratory Developing and benchmarking deep learning-based remo

Lin Manhui 15 Sep 23, 2022
COPA-SSE contains crowdsourced explanations for the Balanced COPA dataset

COPA-SSE Repository for COPA-SSE: Semi-Structured Explanations for Commonsense Reasoning. COPA-SSE contains crowdsourced explanations for the Balanced

Ana Brassard 5 Jul 31, 2022
TensorFlowOnSpark brings TensorFlow programs to Apache Spark clusters.

TensorFlowOnSpark TensorFlowOnSpark brings scalable deep learning to Apache Hadoop and Apache Spark clusters. By combining salient features from the T

Yahoo 3.8k Jan 04, 2023
This is the source code of the 1st place solution for segmentation task (with Dice 90.32%) in 2021 CCF BDCI challenge.

1st place solution in CCF BDCI 2021 ULSEG challenge This is the source code of the 1st place solution for ultrasound image angioma segmentation task (

Chenxu Peng 30 Nov 22, 2022
RGB-D Local Implicit Function for Depth Completion of Transparent Objects

RGB-D Local Implicit Function for Depth Completion of Transparent Objects [Project Page] [Paper] Overview This repository maintains the official imple

NVIDIA Research Projects 43 Dec 12, 2022
Detection of PCBA defect

Detection_of_PCBA_defect Detection_of_PCBA_defect Use yolov5 to train. $pip install -r requirements.txt Detect.py will detect file(jpg,mp4...) in cu

6 Nov 28, 2022
Film review classification

Film review classification Решение задачи классификации отзывов на фильмы на положительные и отрицательные с помощью рекуррентных нейронных сетей 1. З

Nikita Dukin 3 Jan 21, 2022
PyTorch Code for the paper "VSE++: Improving Visual-Semantic Embeddings with Hard Negatives"

Improving Visual-Semantic Embeddings with Hard Negatives Code for the image-caption retrieval methods from VSE++: Improving Visual-Semantic Embeddings

Fartash Faghri 441 Dec 05, 2022
ObjectDetNet is an easy, flexible, open-source object detection framework

Getting started with the ObjectDetNet ObjectDetNet is an easy, flexible, open-source object detection framework which allows you to easily train, resu

5 Aug 25, 2020