IRCasTRF: Inverse Rendering by Optimizing Cascaded Tensorial Radiance Fields, Lighting, and Materials From Multi-view Images

Wenpeng Xing, Jie Chen*, Ka Chun Cheung, Simon Chong Wee See

*Corresponding author for this work

Research output: Chapter in book/report/conference proceedingConference proceedingpeer-review


We propose an inverse rendering pipeline that simultaneously reconstructs scene geometry, lighting, and spatially-varying material from a set of multi-view images. Specifically, the proposed pipeline involves volume and physics-based rendering, which are performed separately in two steps: exploration and exploitation. During the exploration step, our method utilizes the compactness of neural radiance fields and a flexible differentiable volume rendering technique to learn an initial volumetric field. Here, we introduce a novel cascaded tensorial radiance field method on top of the Canonical Polyadic (CP) decomposition to boost model compactness beyond conventional methods. In the exploitation step, a shading pass that incorporates a differentiable physics-based shading method is applied to jointly optimize the scene's geometry, spatially-varying materials, and lighting, using image reconstruction loss. Experimental results demonstrate that our proposed inverse rendering pipeline, IRCasTRF, outperforms prior works in inverse rendering quality. The final output is highly compatible with downstream applications like scene editing and advanced simulations. Further details are available on the project page:
Original languageEnglish
Title of host publicationMM 2023 - Proceedings of the 31st ACM International Conference on Multimedia
PublisherAssociation for Computing Machinery (ACM)
Number of pages10
ISBN (Print)9798400701085
Publication statusPublished - 27 Oct 2023

Publication series

NameProceedings of the ACM International Conference on Multimedia

User-Defined Keywords

  • Inverse rendering
  • Canonical Polyadic decomposition
  • Neural radiance fields
  • Novel view synthesis
  • Bidirectional reflectance distribution function


Dive into the research topics of 'IRCasTRF: Inverse Rendering by Optimizing Cascaded Tensorial Radiance Fields, Lighting, and Materials From Multi-view Images'. Together they form a unique fingerprint.

Cite this