This is the official PyTorch codes for the paper
Online-Updated High-Order Collaborative Networks for Single Image Deraining
Cong Wang, Jinshan Pan, Xiao-Ming Wu
Single image deraining is an important and challenging task for some downstream artificial intelligence applications such as video surveillance and self-driving systems.
Most of the existing deep-learning-based methods constrain the network to generate derained images but few of them explore features from intermediate layers, different levels, and different modules which are beneficial for rain streaks removal.
In this paper, we propose a high-order collaborative network with multi-scale compact constraints and a bidirectional scale-content similarity mining module to exploit features from deep networks externally and internally for rain streaks removal.
Externally, we design a deraining framework with three sub-networks trained in a collaborative manner, where the bottom network transmits intermediate features to the middle network which also receives shallower rainy features from the top network and sends back features to the bottom network.
Internally, we enforce multi-scale compact constraints on the intermediate layers of deep networks to learn useful features via a Laplacian pyramid.
Further, we develop a bidirectional scale-content similarity mining module to explore features at different scales in a down-to-up and up-to-down manner.
To improve the model performance on real-world images, we propose an online-update learning approach, which uses real-world rainy images to fine-tune the network and update the deraining results in a self-supervised manner.
Extensive experiments demonstrate that our proposed method performs favorably against eleven state-of-the-art methods on five public synthetic datasets and one real-world dataset.
We note that for each dataset, different rain streaks have similar intensity distributions.
However, the distributions of real-world data are significantly different from those of synthetic data, which is probably the reason most of the existing deraining methods trained on Rain100H fail to work on real-world datasets.
Ablation study of the proposed online-update learning approach. (a) Input. (b) Only trained on synthetic data. (c) Directly fine-tuned on real-world images. (d) Online-update learning on real-world images.
1 Create conda environment.
conda create --name supercong
conda activate supercong
conda install pytorch=1.1.0 torchvision=0.3.0 cudatoolkit=9.0 -c pytorch
2 Install other dependencies.
pip install -r requirements.txt
Rain200H&Rain200L, Rain800, Rain1200, Rain1400
bash python train.py
bash python real_train.py
bash python eval.py
bash python show.py
@inproceedings{wang2024msgnn,
author={Cong Wang and Jinshan Pan and Xiao-Ming Wu},
title={Online-updated High-order Collaborative Networks for Image Deraining},
year={2022},
Journal = {Proceedings of the AAAI Conference on Artificial Intelligence (AAAI)},
}
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
Any questions can contact: Cong Wang [supercong94@gmail.com]
ACM MM-2020: Dcsfn: Deep cross-scale fusion network for single image rain removal
ACM MM-2020: Joint self-attention and scale-aggregation for self-calibrated deraining network
IJCAI-2024: Explore Internal and External Similarity for Single Image Deraining with Graph Neural Networks
IEEE TCSVT-2022: Semi-supervised image deraining using knowledge distillation
IEEE ICME-2020: Physical model guided deep image deraining