TY - JOUR

T1 - Deep Nets for Local Manifold Learning

AU - CHUI, Charles Kam-Tai

AU - Mhaskar, Hrushikesh N.

N1 - Funding Information:
We would like to thank Professor Tomaso Poggio for his comments and for sharing the manuscript [15] with us. An earlier version of this work is available as a preprint [62] and the authors hold the copyright of the preprint. CC is also associated with the Statistics Department of Stanford University, CA 94305, and his research is partially supported by U.S. ARO Grant # W911NF-15-1-0385, Hong Kong Research Council Grant # 12300917, and Hong Kong Baptist University Grant # HKBU-RC-ICRS/16-17/03. The research of HM is supported in part by ARO Grant W911NF-15-1-0385.

PY - 2018/5/29

Y1 - 2018/5/29

N2 - The problem of extending a function f defined on a training data (Formula presented.) on an unknown manifold X to the entire manifold and a tubular neighborhood of this manifold is considered in this paper. For X embedded in a high dimensional ambient Euclidean space RD, a deep learning algorithm is developed for finding a local coordinate system for the manifold without eigen-decomposition, which reduces the problem to the classical problem of function approximation on a low dimensional cube. Deep nets (or multilayered neural networks) are proposed to accomplish this approximation scheme by using the training data. Our methods do not involve such optimization techniques as back-propagation, while assuring optimal (a priori) error bounds on the output in terms of the number of derivatives of the target function. In addition, these methods are universal, in that they do not require a prior knowledge of the smoothness of the target function, but adjust the accuracy of approximation locally and automatically, depending only upon the local smoothness of the target function. Our ideas are easily extended to solve both the pre-image problem and the out-of-sample extension problem, with a priori bounds on the growth of the function thus extended.

AB - The problem of extending a function f defined on a training data (Formula presented.) on an unknown manifold X to the entire manifold and a tubular neighborhood of this manifold is considered in this paper. For X embedded in a high dimensional ambient Euclidean space RD, a deep learning algorithm is developed for finding a local coordinate system for the manifold without eigen-decomposition, which reduces the problem to the classical problem of function approximation on a low dimensional cube. Deep nets (or multilayered neural networks) are proposed to accomplish this approximation scheme by using the training data. Our methods do not involve such optimization techniques as back-propagation, while assuring optimal (a priori) error bounds on the output in terms of the number of derivatives of the target function. In addition, these methods are universal, in that they do not require a prior knowledge of the smoothness of the target function, but adjust the accuracy of approximation locally and automatically, depending only upon the local smoothness of the target function. Our ideas are easily extended to solve both the pre-image problem and the out-of-sample extension problem, with a priori bounds on the growth of the function thus extended.

KW - deep learning

KW - function approximation

KW - local approximation

KW - manifold learning

KW - neural networks

UR - http://www.scopus.com/inward/record.url?scp=85088309599&partnerID=8YFLogxK

U2 - 10.3389/fams.2018.00012

DO - 10.3389/fams.2018.00012

M3 - Journal article

AN - SCOPUS:85088309599

SN - 2297-4687

VL - 4

JO - Frontiers in Applied Mathematics and Statistics

JF - Frontiers in Applied Mathematics and Statistics

M1 - 12

ER -