NEX+: Novel View Synthesis with Neural Rregularisation over Multi-Plane Images

Wenpeng Xing, Jie Chen*

*Corresponding author for this work

Research output: Chapter in book/report/conference proceedingConference contributionpeer-review

Abstract

We propose Nex+, a neural Multi-Plane Image (MPI) representation with alpha denoising for the task of novel view synthesis (NVS). Overfitting to training data is a common challenge for all learning-based models. We propose a novel
solution for resolving such issue in the context of NVS with signal denoising-motivated operations over the alpha coefficients of the MPI, without any additional requirements for supervision. Nex+ contains a novel 5D Alpha Neural Regulariser (ANR), which favors low-frequency components in the angular domain, i.e., the alpha coefficients’ signal subspace indicating various viewing directions. ANR’s angular low-frequency property derives from its small number of angular encoding levels and output basis. The regularised alpha in Nex+ can model the scene geometry more accurately than Nex, and outperforms other state-of-the-art methods on public datasets for the task of NVS.
Original languageEnglish
Title of host publicationICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
PublisherIEEE
Pages1581-1585
Number of pages5
ISBN (Electronic)9781665405409
ISBN (Print)9781665405416
DOIs
Publication statusPublished - 23 May 2022

Publication series

NameIEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
ISSN (Print)1520-6149
ISSN (Electronic)2379-190X

Scopus Subject Areas

  • Software
  • Signal Processing
  • Electrical and Electronic Engineering

User-Defined Keywords

  • image denoising
  • multi-plane image
  • neural basis learning
  • Novel view synthesis

Fingerprint

Dive into the research topics of 'NEX+: Novel View Synthesis with Neural Rregularisation over Multi-Plane Images'. Together they form a unique fingerprint.

Cite this