This repository contains the official Pytorch implementation of the paper:

"SAPNet: Segmentation-Aware Progressive Network for Perceptual Contrastive Image Deraining"

• 2021.11.27: The pretrained weight has been uploaded.
• 2021.11.27: The video deraining result at challenging night-time conditions is at Youtube
• 2021.11.17: Arxiv Link is available at https://arxiv.org/abs/2111.08892
• 2021.11.10: This work is accepted and will be presented at WACV 2022.

Deep learning algorithms have recently achieved promising deraining performances on both the natural and synthetic rainy datasets. As an essential low-level pre-processing stage, a deraining network should clear the rain streaks and preserve the fine semantic details. However, most existing methods only consider low-level image restoration. That limits their performances at high-level tasks requiring precise semantic information. To address this issue, in this paper, we present a segmentation-aware progressive network (SAPNet) based upon contrastive learning for single image deraining. We start our method with a lightweight derain network formed with progressive dilated units (PDU). The PDU can significantly expand the receptive field and characterize multi-scale rain streaks without the heavy computation on multi-scale images. A fundamental aspect of this work is an unsupervised background segmentation (UBS) network initialized with ImageNet and Gaussian weights. The UBS can faithfully preserve an image's semantic information and improve the generalization ability to unseen photos. Furthermore, we introduce a perceptual contrastive loss (PCL) and a learned perceptual image similarity loss (LPISL) to regulate model learning. By exploiting the rainy image and groundtruth as the negative and the positive sample in the VGG-16 latent space, we bridge the fine semantic details between the derained image and the groundtruth in a fully constrained manner. Comprehensive experiments on synthetic and real-world rainy images show our model surpasses top-performing methods and aids object detection and semantic segmentation with considerable efficacy.

Click this following link to view the pdf.

From left to right, Rainy, PreNet (CVPR 2019), MSPFN (CVPR 2020), MPRNet (CVPR 2021), SAPNet (Ours)

From left to right, Rainy, PreNet (CVPR 2019), MSPFN (CVPR 2020), MPRNet (CVPR 2021), SAPNet (Ours)

• Windows or Linux
• CUDA 10.0, or CPU
• Python 3.6+
• Pytorch 1.2+
• torchvision 0.4+
• opencv-python
• scikit-image
• numpy

• Download training and testing dataset from either link BaiduYun OneDrive

• Create a new folder called dataset.

• Create sub-folders called train and test under that folder.

• Place the unzipped folders into datasets/train/ (training data) and datasets/test/ (testing data)

Run the following script in terminal

python train.py

Run the following script in terminal

bash main.sh

You can modify the content of main.sh according to your hyperparameters. For example.

python test.py \
      --save_path logs/SAPNet/Model11 \
      --test_data_path datasets/test/Rain100H \
      --output_path results/Rain100H/Model11 \
      --use_dilation True

Please cite the following if you find this repository helpful.

@inproceedings{zheng2022sapnet,
  title={SAPNet: Segmentation-Aware Progressive Network for Perceptual Contrastive Deraining},
  author={Zheng, Shen and Lu, Changjie and Wu, Yuxiong and Gupta, Gaurav},
  booktitle={Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision},
  pages={52--62},
  year={2022}
}

Please reach zhengsh@kean.edu for further questions. You can also open an issue (prefered) or a pull request in this Github repository

This repository is borrowed heavily from PreNet. Thanks for sharing!

• List (important) hyperparameters
• Upload Testing Dataset
• Upload Arxiv Link
• Upload BibTeX
• Add hyperparameter explanation
• Add Dependencies
• Update Code with lpis loss
• Upload Model Architecture Figure
• Upload Visual Comparisons
• Testing on Video
• Upload Pretrained Weight
• Finalize readme