Occlusion-Free Scene Recovery via Neural Radiance Fields

Our everyday lives are filled with occlusions that we strive to see through. By aggregating desired background information from different viewpoints, we can easily eliminate such occlusions without any external occlusion-free supervision. Though several occlusion removal methods have been proposed to empower machine vision systems with such ability, their performances are still unsatisfactory due to reliance on external supervision. We propose a novel method, OCC-NeRF, for occlusion removal by adequately considering the benefits of multiple viewing angles, which directly builds a mapping between viewing angles and their corresponding scene details leveraging Neural Radiance Fields (NeRF). We also develop an effective scheme to jointly optimize camera parameters and scene reconstruction when occlusions are present. An additional depth constraint is applied to supervise the entire optimization without labeled external data for training. Our experimental results on existing and newly collected datasets validate the effectiveness of our method.