NDF: Neural Deformable Fields for Dynamic Human Modelling

Ruiqi Zhang, Jie Chen*

*Corresponding author for this work

Research output: Chapter in book/report/conference proceedingConference proceedingpeer-review

2 Citations (Scopus)

Abstract

We propose Neural Deformable Fields (NDF), a new representation for dynamic human digitization from a multi-view video. Recent works proposed to represent a dynamic human body with shared canonical neural radiance fields which links to the observation space with deformation fields estimations. However, the learned canonical representation is static and the current design of the deformation fields is not able to represent large movements or detailed geometry changes. In this paper, we propose to learn a neural deformable field wrapped around a fitted parametric body model to represent the dynamic human. The NDF is spatially aligned by the underlying reference surface. A neural network is then learned to map pose to the dynamics of NDF. The proposed NDF representation can synthesize the digitized performer with novel views and novel poses with a detailed and reasonable dynamic appearance. Experiments show that our method significantly outperforms recent human synthesis methods.
Original languageEnglish
Title of host publicationComputer Vision – ECCV 2022
Subtitle of host publication17th European Conference, Tel Aviv, Israel, October 23–27, 2022, Proceedings, Part XXXII
EditorsShai Avidan, Gabriel Brostow, Moustapha Cissé, Giovanni Maria Farinella, Tal Hassner
PublisherSpringer Cham
Pages37-52
Number of pages16
Edition1st
ISBN (Electronic)9783031198243
ISBN (Print)9783031198236
DOIs
Publication statusPublished - 12 Nov 2022

Publication series

NameLecture Notes in Computer Science
PublisherSpringer Cham
Volume13692
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

User-Defined Keywords

  • Neural implicit representation
  • Volumetric rendering
  • Novel view synthesis
  • Dynamic motion
  • Human shape and appearance modelling

Fingerprint

Dive into the research topics of 'NDF: Neural Deformable Fields for Dynamic Human Modelling'. Together they form a unique fingerprint.

Cite this