A statistical learning approach to modal regression

Yunlong Feng, Jun Fan*, Johan A.K. Suykens

*Corresponding author for this work

Research output: Contribution to journalJournal articlepeer-review

64 Citations (Scopus)

Abstract

This paper studies the nonparametric modal regression problem systematically from a statistical learning viewpoint. Originally motivated by pursuing a theoretical understanding of the maximum correntropy criterion based regression (MCCR), our study reveals that MCCR with a tending-to-zero scale parameter is essentially modal regression. We show that the nonparametric modal regression problem can be approached via the classical empirical risk minimization. Some efforts are then made to develop a framework for analyzing and implementing modal regression. For instance, the modal regression function is described, the modal regression risk is defined explicitly and its Bayes rule is characterized; for the sake of computational tractability, the surrogate modal regression risk, which is termed as the generalization risk in our study, is introduced. On the theoretical side, the excess modal regression risk, the excess generalization risk, the function estimation error, and the relations among the above three quantities are studied rigorously. It turns out that under mild conditions, function estimation consistency and convergence may be pursued in modal regression as in vanilla regression protocols such as mean regression, median regression, and quantile regression. On the practical side, the implementation issues of modal regression including the computational algorithm and the selection of the tuning parameters are discussed. Numerical validations on modal regression are also conducted to verify our findings.

Original languageEnglish
Pages (from-to)1-35
Number of pages35
JournalJournal of Machine Learning Research
Volume21
Publication statusPublished - Jan 2020

Scopus Subject Areas

  • Software
  • Control and Systems Engineering
  • Statistics and Probability
  • Artificial Intelligence

User-Defined Keywords

  • Empirical risk minimization
  • Generalization bounds
  • Kernel density estimation
  • Nonparametric modal regression
  • Statistical learning theory

Fingerprint

Dive into the research topics of 'A statistical learning approach to modal regression'. Together they form a unique fingerprint.

Cite this