Sparsity and Error Analysis of Empirical Feature-Based Regularization Schemes

Xin Guo, Jun Fan, Ding Xuan Zhou

Research output: Contribution to journalJournal articlepeer-review

6 Citations (Scopus)

Abstract

We consider a learning algorithm generated by a regularization scheme with a concave regularizer for the purpose of achieving sparsity and good learning rates in a least squares regression setting. The regularization is induced for linear combinations of empirical features, constructed in the literatures of kernel principal component analysis and kernel projection machines, based on kernels and samples. In addition to the separability of the involved optimization problem caused by the empirical features, we carry out sparsity and error analysis, giving bounds in the norm of the reproducing kernel Hilbert space, based on a priori conditions which do not require assumptions on sparsity in terms of any basis or system. In particular, we show that as the concave exponent q of the concave regularizer increases to 1, the learning ability of the algorithm improves. Some numerical simulations for both artificial and real MHC-peptide binding data involving the lq regularizer and the SCAD penalty are presented to demonstrate the sparsity and error analysis.
Original languageEnglish
Pages (from-to)3058–3091
Number of pages34
JournalJournal of Machine Learning Research
Volume17
Issue number1
Publication statusPublished - Jan 2016

User-Defined Keywords

  • Sparsity
  • concave regularizer
  • reproducing kernel Hilbert space
  • regularization with empirical features
  • lq-penalty
  • SCAD penalty

Fingerprint

Dive into the research topics of 'Sparsity and Error Analysis of Empirical Feature-Based Regularization Schemes'. Together they form a unique fingerprint.

Cite this