TY - JOUR
T1 - Consistent tuning parameter selection in high dimensional sparse linear regression
AU - Wang, Tao
AU - ZHU, Lixing
N1 - Funding Information:
The research described here was supported by a grant from the Research Council of Hong Kong , and a grant from Hong Kong Baptist University , Hong Kong.
PY - 2011/8
Y1 - 2011/8
N2 - An exhaustive search as required for traditional variable selection methods is impractical in high dimensional statistical modeling. Thus, to conduct variable selection, various forms of penalized estimators with good statistical and computational properties, have been proposed during the past two decades. The attractive properties of these shrinkage and selection estimators, however, depend critically on the size of regularization which controls model complexity. In this paper, we consider the problem of consistent tuning parameter selection in high dimensional sparse linear regression where the dimension of the predictor vector is larger than the size of the sample. First, we propose a family of high dimensional Bayesian Information Criteria (HBIC), and then investigate the selection consistency, extending the results of the extended Bayesian Information Criterion (EBIC), in Chen and Chen (2008) to ultra-high dimensional situations. Second, we develop a two-step procedure, the SIS+AENET, to conduct variable selection in p>n situations. The consistency of tuning parameter selection is established under fairly mild technical conditions. Simulation studies are presented to confirm theoretical findings, and an empirical example is given to illustrate the use in the internet advertising data.
AB - An exhaustive search as required for traditional variable selection methods is impractical in high dimensional statistical modeling. Thus, to conduct variable selection, various forms of penalized estimators with good statistical and computational properties, have been proposed during the past two decades. The attractive properties of these shrinkage and selection estimators, however, depend critically on the size of regularization which controls model complexity. In this paper, we consider the problem of consistent tuning parameter selection in high dimensional sparse linear regression where the dimension of the predictor vector is larger than the size of the sample. First, we propose a family of high dimensional Bayesian Information Criteria (HBIC), and then investigate the selection consistency, extending the results of the extended Bayesian Information Criterion (EBIC), in Chen and Chen (2008) to ultra-high dimensional situations. Second, we develop a two-step procedure, the SIS+AENET, to conduct variable selection in p>n situations. The consistency of tuning parameter selection is established under fairly mild technical conditions. Simulation studies are presented to confirm theoretical findings, and an empirical example is given to illustrate the use in the internet advertising data.
KW - Adaptive Elastic Net
KW - Bayesian information criterion
KW - High dimensionality
KW - Sure independence screening
KW - Tuning parameter selection
KW - Variable selection
UR - http://www.scopus.com/inward/record.url?scp=79956299499&partnerID=8YFLogxK
U2 - 10.1016/j.jmva.2011.03.007
DO - 10.1016/j.jmva.2011.03.007
M3 - Journal article
AN - SCOPUS:79956299499
SN - 0047-259X
VL - 102
SP - 1141
EP - 1151
JO - Journal of Multivariate Analysis
JF - Journal of Multivariate Analysis
IS - 7
ER -