Reflection Scene Separation From a Single Image

Renjie Wan; Boxin Shi; Haoliang Li; Ling Yu Duan; Alex C. Kot

doi:10.1109/CVPR42600.2020.00247

Reflection Scene Separation From a Single Image

Renjie Wan^*
, Boxin Shi^*
, Haoliang Li
, Ling Yu Duan
, Alex C. Kot

^*Corresponding author for this work

Department of Computer Science

Research output: Chapter in book/report/conference proceeding › Conference proceeding › peer-review

26 Citations (Scopus)

Abstract

For images taken through glass, existing methods focus on the restoration of the background scene by regarding the reflection components as noise. However, the scene reflected by glass surface also contains important information to be recovered, especially for the surveillance or criminal investigations. In this paper, instead of removing reflection components from the mixture image, we aim at recovering reflection scenes from the mixture image. We first propose a strategy to obtain such ground truth and its corresponding input images. Then, we propose a two-stage framework to obtain the visible reflection scene from the mixture image. Specifically, we train the network with a shift-invariant loss which is robust to misalignment between the input and output images. The experimental results show that our proposed method achieves promising results.

Original language	English
Title of host publication	2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
Publisher	IEEE
Pages	2395-2403
Number of pages	9
ISBN (Electronic)	9781728171685
ISBN (Print)	9781728171692
DOIs	https://doi.org/10.1109/CVPR42600.2020.00247
Publication status	Published - 13 Jun 2020
Event	2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2020 - Virtual, Online, United States Duration: 14 Jun 2020 → 19 Jun 2020

Publication series

Name	Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition
Publisher	IEEE
ISSN (Print)	1063-6919
ISSN (Electronic)	2575-7075

Conference

Conference	2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2020
Country/Territory	United States
City	Virtual, Online
Period	14/06/20 → 19/06/20

UN SDGs

This output contributes to the following UN Sustainable Development Goals (SDGs)

SDG 16 Peace, Justice and Strong Institutions

Access to Document

10.1109/CVPR42600.2020.00247

Cite this

@inproceedings{3ab2a0c681e64437ad41a7eac1fa10ef,

title = "Reflection Scene Separation From a Single Image",

abstract = "For images taken through glass, existing methods focus on the restoration of the background scene by regarding the reflection components as noise. However, the scene reflected by glass surface also contains important information to be recovered, especially for the surveillance or criminal investigations. In this paper, instead of removing reflection components from the mixture image, we aim at recovering reflection scenes from the mixture image. We first propose a strategy to obtain such ground truth and its corresponding input images. Then, we propose a two-stage framework to obtain the visible reflection scene from the mixture image. Specifically, we train the network with a shift-invariant loss which is robust to misalignment between the input and output images. The experimental results show that our proposed method achieves promising results.",

author = "Renjie Wan and Boxin Shi and Haoliang Li and Duan, \{Ling Yu\} and Kot, \{Alex C.\}",

note = "This research is supported by the National Research Foundation, Prime Minister{\textquoteright}s Office, Singapore, under the NRFNSFC grant NRF2016NRF-NSFC001-098 and NTU-PKU Joint Research Institute with donation from Ng Teng Fong Charitable Foundation. The research work was done at the Rapid-Rich Object Search (ROSE) Lab, Nanyang Technological University, Singapore. This research is in part supported by the National Natural Science Foundation of China under Grants U1611461 and 61872012, National Key R\&D Program of China (2019YFF0302902), Shenzhen Municipal Science and Technology Program under Grant JCYJ20170818141146428, and Beijing Academy of Artificial Intelligence (BAAI). Publisher Copyright: {\textcopyright} 2020 IEEE; 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2020 ; Conference date: 14-06-2020 Through 19-06-2020",

year = "2020",

month = jun,

day = "13",

doi = "10.1109/CVPR42600.2020.00247",

language = "English",

isbn = "9781728171692",

series = "Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition",

publisher = "IEEE",

pages = "2395--2403",

booktitle = "2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)",

address = "United States",

}

Wan, R, Shi, B, Li, H, Duan, LY & Kot, AC 2020, Reflection Scene Separation From a Single Image. in 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, IEEE, pp. 2395-2403, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2020, Virtual, Online, United States, 14/06/20. https://doi.org/10.1109/CVPR42600.2020.00247

Reflection Scene Separation From a Single Image. / Wan, Renjie; Shi, Boxin; Li, Haoliang et al.
2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2020. p. 2395-2403 (Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition).

Research output: Chapter in book/report/conference proceeding › Conference proceeding › peer-review

TY - GEN

T1 - Reflection Scene Separation From a Single Image

AU - Wan, Renjie

AU - Shi, Boxin

AU - Li, Haoliang

AU - Duan, Ling Yu

AU - Kot, Alex C.

N1 - This research is supported by the National Research Foundation, Prime Minister’s Office, Singapore, under the NRFNSFC grant NRF2016NRF-NSFC001-098 and NTU-PKU Joint Research Institute with donation from Ng Teng Fong Charitable Foundation. The research work was done at the Rapid-Rich Object Search (ROSE) Lab, Nanyang Technological University, Singapore. This research is in part supported by the National Natural Science Foundation of China under Grants U1611461 and 61872012, National Key R&D Program of China (2019YFF0302902), Shenzhen Municipal Science and Technology Program under Grant JCYJ20170818141146428, and Beijing Academy of Artificial Intelligence (BAAI). Publisher Copyright: © 2020 IEEE

PY - 2020/6/13

Y1 - 2020/6/13

N2 - For images taken through glass, existing methods focus on the restoration of the background scene by regarding the reflection components as noise. However, the scene reflected by glass surface also contains important information to be recovered, especially for the surveillance or criminal investigations. In this paper, instead of removing reflection components from the mixture image, we aim at recovering reflection scenes from the mixture image. We first propose a strategy to obtain such ground truth and its corresponding input images. Then, we propose a two-stage framework to obtain the visible reflection scene from the mixture image. Specifically, we train the network with a shift-invariant loss which is robust to misalignment between the input and output images. The experimental results show that our proposed method achieves promising results.

AB - For images taken through glass, existing methods focus on the restoration of the background scene by regarding the reflection components as noise. However, the scene reflected by glass surface also contains important information to be recovered, especially for the surveillance or criminal investigations. In this paper, instead of removing reflection components from the mixture image, we aim at recovering reflection scenes from the mixture image. We first propose a strategy to obtain such ground truth and its corresponding input images. Then, we propose a two-stage framework to obtain the visible reflection scene from the mixture image. Specifically, we train the network with a shift-invariant loss which is robust to misalignment between the input and output images. The experimental results show that our proposed method achieves promising results.

UR - https://www.scopus.com/pages/publications/85094863642

U2 - 10.1109/CVPR42600.2020.00247

DO - 10.1109/CVPR42600.2020.00247

M3 - Conference proceeding

AN - SCOPUS:85094863642

SN - 9781728171692

T3 - Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition

SP - 2395

EP - 2403

BT - 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

PB - IEEE

T2 - 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2020

Y2 - 14 June 2020 through 19 June 2020

ER -

Reflection Scene Separation From a Single Image

Abstract

Publication series

Conference

UN SDGs

Access to Document

Other files and links

Fingerprint

Cite this