TY - JOUR
T1 - A statistical learning approach to modal regression
AU - Feng, Yunlong
AU - Fan, Jun
AU - Suykens, Johan A.K.
N1 - Funding Information:
The authors would like to thank the Action Editor and the reviewers for their constructive suggestions and comments that improved the quality of this paper. The research leading to these results has received funding from the European Research Council under the European Union’s Seventh Framework Programme (FP7/2007-2013) / ERC AdG A-DATADRIVE-B (290923) and ERC AdG E-DUALITY (787960) under the European Union’s Horizon 2020 research and innovation programme. This paper reflects only the authors’ views, the Union is not liable for any use that may be made of the contained information. Research Council KUL: GOA/10/09 MaNet, CoE PFV/10/002 (OPTEC), BIL12/11T; PhD/Postdoc grants. Flemish Government: FWO: projects: G.0377.12 (Structured systems), G.088114N (Tensor based data similarity); PhD/Postdoc grants. IWT: projects: SBO POM (100031); PhD/Postdoc grants. iMinds Medical Information Technologies SBO 2014. Belgian Federal Science Policy Office: IUAP P7/19 (DYSCO, Dynamical systems, control and optimization, 2012-2017). Yunlong Feng also gratefully acknowledges the support of Simons Foundation Collaboration Grant #572064 and the Ralph E. Powe Junior Faculty Enhancement Award by Oak Ridge Associated Universities. The research of Jun Fan was supported in part by the Hong Kong RGC Early Career Schemes 22303518, and the NSF grant of China (No. 11801478). The corresponding author is Jun Fan.
Funding Information:
The authors would like to thank the Action Editor and the reviewers for their constructive suggestions and comments that improved the quality of this paper. The research leading to these results has received funding from the European Research Council under the European Union's Seventh Framework Programme (FP7/2007-2013) / ERC AdG A-DATADRIVEB (290923) and ERC AdG E-DUALITY (787960) under the European Union's Horizon 2020 research and innovation programme. This paper reflects only the authors' views, the Union is not liable for any use that may be made of the contained information. Research Council KUL: GOA/10/09 MaNet, CoE PFV/10/002 (OPTEC), BIL12/11T; PhD/Postdoc grants. Flemish Government: FWO: projects: G.0377.12 (Structured systems), G.088114N (Tensor based data similarity); PhD/Postdoc grants. IWT: projects: SBO POM (100031); PhD/Postdoc grants. iMinds Medical Information Technologies SBO 2014. Belgian Federal Science Policy Office: IUAP P7/19 (DYSCO, Dynamical systems, control and optimization, 2012-2017). Yunlong Feng also gratefully acknowledges the support of Simons Foundation Collaboration Grant #572064 and the Ralph E. Powe Junior Faculty Enhancement Award by Oak Ridge Associated Universities. The research of Jun Fan was supported in part by the Hong Kong RGC Early Career Schemes 22303518, and the NSF grant of China (No. 11801478).
PY - 2020/1
Y1 - 2020/1
N2 - This paper studies the nonparametric modal regression problem systematically from a statistical learning viewpoint. Originally motivated by pursuing a theoretical understanding of the maximum correntropy criterion based regression (MCCR), our study reveals that MCCR with a tending-to-zero scale parameter is essentially modal regression. We show that the nonparametric modal regression problem can be approached via the classical empirical risk minimization. Some efforts are then made to develop a framework for analyzing and implementing modal regression. For instance, the modal regression function is described, the modal regression risk is defined explicitly and its Bayes rule is characterized; for the sake of computational tractability, the surrogate modal regression risk, which is termed as the generalization risk in our study, is introduced. On the theoretical side, the excess modal regression risk, the excess generalization risk, the function estimation error, and the relations among the above three quantities are studied rigorously. It turns out that under mild conditions, function estimation consistency and convergence may be pursued in modal regression as in vanilla regression protocols such as mean regression, median regression, and quantile regression. On the practical side, the implementation issues of modal regression including the computational algorithm and the selection of the tuning parameters are discussed. Numerical validations on modal regression are also conducted to verify our findings.
AB - This paper studies the nonparametric modal regression problem systematically from a statistical learning viewpoint. Originally motivated by pursuing a theoretical understanding of the maximum correntropy criterion based regression (MCCR), our study reveals that MCCR with a tending-to-zero scale parameter is essentially modal regression. We show that the nonparametric modal regression problem can be approached via the classical empirical risk minimization. Some efforts are then made to develop a framework for analyzing and implementing modal regression. For instance, the modal regression function is described, the modal regression risk is defined explicitly and its Bayes rule is characterized; for the sake of computational tractability, the surrogate modal regression risk, which is termed as the generalization risk in our study, is introduced. On the theoretical side, the excess modal regression risk, the excess generalization risk, the function estimation error, and the relations among the above three quantities are studied rigorously. It turns out that under mild conditions, function estimation consistency and convergence may be pursued in modal regression as in vanilla regression protocols such as mean regression, median regression, and quantile regression. On the practical side, the implementation issues of modal regression including the computational algorithm and the selection of the tuning parameters are discussed. Numerical validations on modal regression are also conducted to verify our findings.
KW - Empirical risk minimization
KW - Generalization bounds
KW - Kernel density estimation
KW - Nonparametric modal regression
KW - Statistical learning theory
UR - https://www.jmlr.org/papers/v21/17-068.html
UR - https://www.jmlr.org/papers/v21/
UR - http://www.scopus.com/inward/record.url?scp=85085943745&partnerID=8YFLogxK
M3 - Journal article
AN - SCOPUS:85085943745
SN - 1532-4435
VL - 21
SP - 1
EP - 35
JO - Journal of Machine Learning Research
JF - Journal of Machine Learning Research
ER -