Learning Functions Varying along a Central Subspace

Hao Liu, Wenjing Liao*

*Corresponding author for this work

Research output: Contribution to journal β€Ί Journal article β€Ί peer-review

Abstract

Many functions of interest are in a high-dimensional space but exhibit low-dimensional structures. This paper studies regression of an 𝑠-HΓΆlder function 𝑓 in ℝ𝐷 which varies along a central subspace of dimension 𝑑 while 𝑑β‰ͺ𝐷. A direct approximation of 𝑓 in ℝ𝐷 with an πœ€ accuracy requires the number of samples 𝑛 in the order of πœ€βˆ’(2⁒𝑠+𝐷)/𝑠. In this paper, we analyze the generalized contour regression (GCR) algorithm for the estimation of the central subspace and use piecewise polynomials for function approximation. GCR is among the best estimators for the central subspace, but its sample complexity is an open question. In this paper, we partially answer this questions by proving that if a variance quantity is exactly known, GCR leads to a mean squared estimation error of 𝑂⁑(π‘›βˆ’1) for the central subspace. The estimation error of this variance quantity is also given in this paper. The mean squared regression error of 𝑓 is proved to be in the order of (𝑛/log⁑𝑛)βˆ’2⁒𝑠/2⁒𝑠+𝑑, where the exponent depends on the dimension of the central subspace 𝑑 instead of the ambient space 𝐷. This result demonstrates that GCR is effective in learning the low-dimensional central subspace. We also propose a modified GCR with improved efficiency. The convergence rate is validated through several numerical experiments.
Original languageEnglish
Pages (from-to)343-371
Number of pages29
JournalSIAM Journal on Mathematics of Data Science
Volume6
Issue number2
Early online date22 Apr 2024
DOIs
Publication statusPublished - Jun 2024

User-Defined Keywords

  • Central subspace
  • Dimension reduction
  • Nonparametric regression
  • Sample complexity

Fingerprint

Dive into the research topics of 'Learning Functions Varying along a Central Subspace'. Together they form a unique fingerprint.

Cite this