Spatial Attentive Single-Image Deraining with a High Quality Real Rain Dataset 

CVPR 2019
Tianyu Wang 1,2*, Xin Yang 1,2*, Ke Xu1,2, Shaozhe Chen1, Qiang Zhang1, ​ Rynson W.H. Lau ​2†
(* Joint first authors. The corresponding author. )
1Dalian University of Technology, 2City University of Hong Kong

Abstract

Removing rain streaks from a single image has been drawing considerable attention as rain streaks can severely degrade the image quality and affect the performance of existing outdoor vision tasks. While recent CNN-based derainers have reported promising performances, deraining remains an open problem for two reasons. First, existing synthesized rain datasets have only limited realism, in terms of modeling real rain characteristics such as rain shape, direction and intensity. Second, there are no public benchmarks for quantitative comparisons on real rain images, which makes the current evaluation less objective. The core challenge is that real world rain/clean image pairs cannot be captured at the same time. In this paper, we address the single image rain removal problem in two ways. First, we propose a semi-automatic method that incorporates temporal priors and human supervision to generate a high-quality clean image from each input sequence of real rain images. Using this method, we construct a large-scale dataset of ∼29.5K rain/rain-free image pairs that cover a wide range of natural rain scenes. Second, to better cover the stochastic distributions of real rain streaks, we propose a novel SPatial Attentive Network (SPANet) to remove rain streaks in a local-to-global manner. Extensive experiments demonstrate that our network performs favorably against the state-of-the-art deraining methods.

Citation

@InProceedings{Wang_2019_CVPR,
author = {Wang, Tianyu and Yang, Xin and Xu, Ke and Chen, Shaozhe and Zhang, Qiang and Lau, Rynson W.H.},
title = {Spatial Attentive Single-Image Deraining with a High Quality Real Rain Dataset},
booktitle = {The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
month = {June},
year = {2019}
}

Dataset Generation Method

percentile%20range
pipeline

 SPANet

network2
SAB
SARB2
SAM

Figure 2. The architecture of the proposed SPANet (a). It adopts three standard residual blocks (RB) [16] to extract features, four spatial attentive blocks (SAB) to identify rain streaks progressively in four stages, and two residual blocks to reconstruct a clean background. A SAB (b) contains three spatial attentive residual blocks (SARB) (c) and one spatial attentive module (SAM) (d). Dilation convolutions [41] are used in RB and SARB.

 Results

rain
DDN_pre
J_pre
DID_pre
RESCAN_pre
our

Figure 3. Visual comparison of the state-of-the-art CNN-based derainers and the proposed SPANet on some real rain images collected from previous derain papers and the Internet. Note that the state-of-the-art CNN-based derainers are trained on their original dataset.

Methods Rain Images DSC[29]
(ICCV’15)
LP[26]
(CVPR’16)
SILS[14]
(ICCV’17)
Clear[10]
(TIP’17)
DDN[11]
(CVPR’17)
JORDER[40]
(CVPR’17)
DID-MDN[42]
(CVPR’18)
RESCAN[25]
(ECCV18)
Our SPANet
PSNR 32.64 32.33 32.99 33.40 31.31 33.28(34.88) 32.16(35.72) 24.91(28.96) 30.36(35.19) 38.06
SSIM 0.9315 0.9335 0.9475 0.9528 0.9304 0.9414(0.9727) 0.9327(0.9776) 0.8895(0.9457) 0.9553(0.9784) 0.9867

Table 1. Quantitative results of the benchmarked state-of-the-art derainers as well as the proposed SPANet on the proposed test set. The original codes of all these derainers are used for evaluation. We also train CNN based state-of-the-art methods [11, 40, 42, 25] on our dataset, and results are in brackets. Best performance is marked in bold. Note that due to lack of density label of rain streaks in our dataset, we only fine-tune the pre-trained model of DID-MDN [42] without training label classification network.

 Code (Now Available!)

Code : Github

 Dataset (Dataset Now Available!)

Training data* download link: Google DriveBaidu Yun (key: 4fwo)
Testing data download link: Google Drive
T
esting Synthetic Video data with still background: Google Drive

* The training dataset contains 660K+ patches croped from 28.5K rain images.
* Due to the copyright of some videos from the Internet, we cannot public the original videos.  

 Dataset Examples