TY - JOUR
T1 - Superlinear Convergence of a General Algorithm for the Generalized Foley-Sammon Discriminant Analysis
AU - Zhang, Lei Hong
AU - Liao, Li Zhi
AU - Ng, Michael K.
N1 - Funding Information:
The authors would like to thank two anonymous referees and the editor for their helpful comments and suggestions on the earlier version of this paper. Research of the second author was supported in part by FRG grants from Hong Kong Baptist University and the Research Grant Council of Hong Kong. Research of the third author was supported in part by RGC grants 7035/04P, 7035/05P and HKBU FRGs.
PY - 2013/6
Y1 - 2013/6
N2 - Linear Discriminant Analysis (LDA) is one of the most efficient statistical approaches for feature extraction and dimension reduction. The generalized Foley-Sammon transform and the trace ratio model are very important in LDA and have received increasing interest. An efficient iterative method has been proposed for the resulting trace ratio optimization problem, which, under a mild assumption, is proved to enjoy both the local quadratic convergence and the global convergence to the global optimal solution (Zhang, L.-H., Liao, L.-Z., Ng, M. K.: SIAM J. Matrix Anal. Appl. 31:1584, 2010). The present paper further investigates the convergence behavior of this iterative method under no assumption. In particular, we prove that the iteration converges superlinearly when the mild assumption is removed. All possible limit points are characterized as a special subset of the global optimal solutions. An illustrative numerical example is also presented.
AB - Linear Discriminant Analysis (LDA) is one of the most efficient statistical approaches for feature extraction and dimension reduction. The generalized Foley-Sammon transform and the trace ratio model are very important in LDA and have received increasing interest. An efficient iterative method has been proposed for the resulting trace ratio optimization problem, which, under a mild assumption, is proved to enjoy both the local quadratic convergence and the global convergence to the global optimal solution (Zhang, L.-H., Liao, L.-Z., Ng, M. K.: SIAM J. Matrix Anal. Appl. 31:1584, 2010). The present paper further investigates the convergence behavior of this iterative method under no assumption. In particular, we prove that the iteration converges superlinearly when the mild assumption is removed. All possible limit points are characterized as a special subset of the global optimal solutions. An illustrative numerical example is also presented.
KW - Dimensionality reduction
KW - Generalized Foley-Sammon transform
KW - Linear discriminant analysis
KW - Superlinear convergence
KW - The trace ratio optimization problem
UR - http://www.scopus.com/inward/record.url?scp=84876877752&partnerID=8YFLogxK
U2 - 10.1007/s10957-011-9832-4
DO - 10.1007/s10957-011-9832-4
M3 - Journal article
AN - SCOPUS:84876877752
SN - 0022-3239
VL - 157
SP - 853
EP - 865
JO - Journal of Optimization Theory and Applications
JF - Journal of Optimization Theory and Applications
IS - 3
ER -