TY - JOUR

T1 - Efficient methods for estimating constrained parameters with applications to regularized (lasso) logistic regression

AU - Tian, Guo Liang

AU - TANG, Man Lai

AU - Fang, Hong Bin

AU - Tan, Ming

N1 - Funding Information:
G.-L. Tian and M. Tan’s research was supported in part by U.S. National Cancer Institute grants CA119758 and CA106767. The work of M.-L. Tang was fully supported by Hong Kong Baptist University grant FRG/06-07/II-20. The research of H.-B. Fang was partially supported by U.S. National Cancer Institute grant CA106767.

PY - 2008/3/15

Y1 - 2008/3/15

N2 - Fitting logistic regression models is challenging when their parameters are restricted. In this article, we first develop a quadratic lower-bound (QLB) algorithm for optimization with box or linear inequality constraints and derive the fastest QLB algorithm corresponding to the smallest global majorization matrix. The proposed QLB algorithm is particularly suited to problems to which the EM-type algorithms are not applicable (e.g., logistic, multinomial logistic, and Cox's proportional hazards models) while it retains the same EM ascent property and thus assures the monotonic convergence. Secondly, we generalize the QLB algorithm to penalized problems in which the penalty functions may not be totally differentiable. The proposed method thus provides an alternative algorithm for estimation in lasso logistic regression, where the convergence of the existing lasso algorithm is not generally ensured. Finally, by relaxing the ascent requirement, convergence speed can be further accelerated. We introduce a pseudo-Newton method that retains the simplicity of the QLB algorithm and the fast convergence of the Newton method. Theoretical justification and numerical examples show that the pseudo-Newton method is up to 71 (in terms of CPU time) or 107 (in terms of number of iterations) times faster than the fastest QLB algorithm and thus makes bootstrap variance estimation feasible. Simulations and comparisons are performed and three real examples (Down syndrome data, kyphosis data, and colon microarray data) are analyzed to illustrate the proposed methods.

AB - Fitting logistic regression models is challenging when their parameters are restricted. In this article, we first develop a quadratic lower-bound (QLB) algorithm for optimization with box or linear inequality constraints and derive the fastest QLB algorithm corresponding to the smallest global majorization matrix. The proposed QLB algorithm is particularly suited to problems to which the EM-type algorithms are not applicable (e.g., logistic, multinomial logistic, and Cox's proportional hazards models) while it retains the same EM ascent property and thus assures the monotonic convergence. Secondly, we generalize the QLB algorithm to penalized problems in which the penalty functions may not be totally differentiable. The proposed method thus provides an alternative algorithm for estimation in lasso logistic regression, where the convergence of the existing lasso algorithm is not generally ensured. Finally, by relaxing the ascent requirement, convergence speed can be further accelerated. We introduce a pseudo-Newton method that retains the simplicity of the QLB algorithm and the fast convergence of the Newton method. Theoretical justification and numerical examples show that the pseudo-Newton method is up to 71 (in terms of CPU time) or 107 (in terms of number of iterations) times faster than the fastest QLB algorithm and thus makes bootstrap variance estimation feasible. Simulations and comparisons are performed and three real examples (Down syndrome data, kyphosis data, and colon microarray data) are analyzed to illustrate the proposed methods.

UR - http://www.scopus.com/inward/record.url?scp=40249087061&partnerID=8YFLogxK

U2 - 10.1016/j.csda.2007.11.007

DO - 10.1016/j.csda.2007.11.007

M3 - Article

AN - SCOPUS:40249087061

SN - 0167-9473

VL - 52

SP - 3528

EP - 3542

JO - Computational Statistics and Data Analysis

JF - Computational Statistics and Data Analysis

IS - 7

ER -