Deep nonparametric estimation of intrinsic data structures by chart autoencoders: Generalization error and robustness

Hao Liu, Alex Havrilla, Rongjie Lai*, Wenjing Liao

*Corresponding author for this work

Research output: Contribution to journalJournal articlepeer-review

1 Citation (Scopus)

Abstract

Autoencoders have demonstrated remarkable success in learning low-dimensional latent features of high-dimensional data across various applications. Assuming that data are sampled near a low-dimensional manifold, we employ chart autoencoders, which encode data into low-dimensional latent features on a collection of charts, preserving the topology and geometry of the data manifold. Our paper establishes statistical guarantees on the generalization error of chart autoencoders, and we demonstrate their denoising capabilities by considering n noisy training samples, along with their noise-free counterparts, on a d-dimensional manifold. By training autoencoders, we show that chart autoencoders can effectively denoise the input data with normal noise. We prove that, under proper network architectures, chart autoencoders achieve a squared generalization error in the order of n−[Formula Presented]log4⁡n, which depends on the intrinsic dimension of the manifold and only weakly depends on the ambient dimension and noise level. We further extend our theory on data with noise containing both normal and tangential components, where chart autoencoders still exhibit a denoising effect for the normal component. As a special case, our theory also applies to classical autoencoders, as long as the data manifold has a global parametrization. Our results provide a solid theoretical foundation for the effectiveness of autoencoders, which is further validated through several numerical experiments.

Original languageEnglish
Article number101602
JournalApplied and Computational Harmonic Analysis
Volume68
Early online date12 Oct 2023
DOIs
Publication statusPublished - Jan 2024

Scopus Subject Areas

  • Applied Mathematics

User-Defined Keywords

  • Chart autoencoder
  • Deep learning theory
  • Dimension reduction
  • Generalization error
  • Manifold model

Fingerprint

Dive into the research topics of 'Deep nonparametric estimation of intrinsic data structures by chart autoencoders: Generalization error and robustness'. Together they form a unique fingerprint.

Cite this