Occlusion-Free Scene Recovery via Neural Radiance Fields

Chengxuan Zhu, Renjie Wan, Yunkai Tang, Boxin Shi*

*Corresponding author for this work

Research output: Chapter in book/report/conference proceedingConference proceedingpeer-review

Abstract

Our everyday lives are filled with occlusions that we strive to see through. By aggregating desired background information from different viewpoints, we can easily eliminate such occlusions without any external occlusion-free supervision. Though several occlusion removal methods have been proposed to empower machine vision systems with such ability, their performances are still unsatisfactory due to reliance on external supervision. We propose a novel method, OCC-NeRF, for occlusion removal by adequately considering the benefits of multiple viewing angles, which directly builds a mapping between viewing angles and their corresponding scene details leveraging Neural Radiance Fields (NeRF). We also develop an effective scheme to jointly optimize camera parameters and scene reconstruction when occlusions are present. An additional depth constraint is applied to supervise the entire optimization without labeled external data for training. Our experimental results on existing and newly collected datasets validate the effectiveness of our method.
Original languageEnglish
Title of host publicationProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2023
Place of PublicationVancouver, BC, Canada
PublisherIEEE
Pages20722-20731
ISBN (Electronic)9798350301298
ISBN (Print)9798350301304
DOIs
Publication statusPublished - Jun 2023
Event36th IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2023 - Vancouver, Canada
Duration: 17 Jun 202322 Jun 2023
https://cvpr2023.thecvf.com/
https://openaccess.thecvf.com/CVPR2023
https://ieeexplore.ieee.org/xpl/conhome/10203037/proceeding

Publication series

NameConference on Computer Vision and Pattern Recognition (CVPR)
ISSN (Print)1063-6919
ISSN (Electronic)2575-7075

Conference

Conference36th IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2023
Country/TerritoryCanada
CityVancouver
Period17/06/2322/06/23
Internet address

Fingerprint

Dive into the research topics of 'Occlusion-Free Scene Recovery via Neural Radiance Fields'. Together they form a unique fingerprint.

Cite this