Deep Spatial-angular Regularization for Light Field Imaging, Denoising, and Super-resolution

Mantang Guo, Junhui Hou, Jing Jin, Jie Chen, Lap-Pui Chau

Research output: Contribution to journalArticlepeer-review


In this paper, we propose a novel learning-based framework for the reconstruction of high-quality LFs from acquisitions via learned coded apertures. The proposed method incorporates the measurement observation into the deep learning framework elegantly to avoid relying entirely on data-driven priors for LF reconstruction. Specifically, we first formulate the compressive LF reconstruction as an inverse problem with an implicit regularization term. Then, we construct the regularization term with a deep efficient spatial-angular separable convolutional sub-network in the form of local and global residual learning to comprehensively explore the signal distribution free from the limited representation ability and inefficiency of deterministic mathematical modeling. Furthermore, we extend this pipeline to LF denoising and spatial super-resolution, which could be considered as variants of coded aperture imaging equipped different degradation matrices. Extensive experimental results demonstrate that the proposed methods outperform state-of-the-art approaches to a significant extent both quantitatively and qualitatively, i.e., the reconstructed LFs not only achieve much higher PSNR/SSIM but also preserve the LF parallax structure better on both real and synthetic LF benchmarks. The code will be publicly available at
Original languageEnglish
Number of pages18
JournalIEEE Transactions on Pattern Analysis and Machine Intelligence
Publication statusE-pub ahead of print - 8 Jun 2021

Scopus Subject Areas

  • Software
  • Computer Vision and Pattern Recognition
  • Computational Theory and Mathematics
  • Artificial Intelligence
  • Applied Mathematics

User-Defined Keywords

  • Apertures
  • Cameras
  • Coded Aperture
  • Deep Learning
  • Denoising
  • Depth
  • Image reconstruction
  • Imaging
  • Light Field
  • Noise reduction
  • Observation Model
  • Optimization
  • Reconstruction algorithms
  • Sensors
  • Spatial Super-resolution


Dive into the research topics of 'Deep Spatial-angular Regularization for Light Field Imaging, Denoising, and Super-resolution'. Together they form a unique fingerprint.

Cite this