TY - JOUR
T1 - A Self-Consistent-Field Iteration for Orthogonal Canonical Correlation Analysis
AU - Zhang, Lei-Hong
AU - Wang, Li
AU - Bai, Zhaojun
AU - Li, Ren-Cang
N1 - Funding Information:
The work of Lei-Hong Zhang was supported in part by the National Natural Science Foundation of China NSFC-11671246, National Key R&D Program of China (No. 2018YFB0204404) and 2018 Double Innovation Program of Jiangsu Province, China. The work of Li Wang was supported in part by the NSF DMS-2009689. The work of Zhaojun Bai was supported in part by the NSF DMS-1913364. The work of Ren-Cang Li was supported in part by the NSF DMS-1719620 and DMS-2009689.
Publisher Copyright:
© 1979-2012 IEEE.
PY - 2022/2/1
Y1 - 2022/2/1
N2 - We propose an efficient algorithm for solving orthogonal canonical correlation analysis (OCCA) in the form of trace-fractional structure and orthogonal linear projections. Even though orthogonality has been widely used and proved to be a useful criterion for visualization, pattern recognition and feature extraction, existing methods for solving OCCA problem are either numerically unstable by relying on a deflation scheme, or less efficient by directly using generic optimization methods. In this paper, we propose an alternating numerical scheme whose core is the sub-maximization problem in the trace-fractional form with an orthogonality constraint. A customized self-consistent-field (SCF) iteration for this sub-maximization problem is devised. It is proved that the SCF iteration is globally convergent to a KKT point and that the alternating numerical scheme always converges. We further formulate a new trace-fractional maximization problem for orthogonal multiset CCA and propose an efficient algorithm with an either Jacobi-style or Gauss-Seidel-style updating scheme based on the SCF iteration. Extensive experiments are conducted to evaluate the proposed algorithms against existing methods, including real-world applications of multi-label classification and multi-view feature extraction. Experimental results show that our methods not only perform competitively to or better than the existing methods but also are more efficient.
AB - We propose an efficient algorithm for solving orthogonal canonical correlation analysis (OCCA) in the form of trace-fractional structure and orthogonal linear projections. Even though orthogonality has been widely used and proved to be a useful criterion for visualization, pattern recognition and feature extraction, existing methods for solving OCCA problem are either numerically unstable by relying on a deflation scheme, or less efficient by directly using generic optimization methods. In this paper, we propose an alternating numerical scheme whose core is the sub-maximization problem in the trace-fractional form with an orthogonality constraint. A customized self-consistent-field (SCF) iteration for this sub-maximization problem is devised. It is proved that the SCF iteration is globally convergent to a KKT point and that the alternating numerical scheme always converges. We further formulate a new trace-fractional maximization problem for orthogonal multiset CCA and propose an efficient algorithm with an either Jacobi-style or Gauss-Seidel-style updating scheme based on the SCF iteration. Extensive experiments are conducted to evaluate the proposed algorithms against existing methods, including real-world applications of multi-label classification and multi-view feature extraction. Experimental results show that our methods not only perform competitively to or better than the existing methods but also are more efficient.
KW - Canonical correlation analysis
KW - orthogonal multiset canonical correlation analysis
KW - self-consistent-field iteration
UR - http://www.scopus.com/inward/record.url?scp=85122800876&partnerID=8YFLogxK
U2 - 10.1109/TPAMI.2020.3012541
DO - 10.1109/TPAMI.2020.3012541
M3 - Journal article
SN - 0162-8828
VL - 44
SP - 890
EP - 904
JO - IEEE Transactions on Pattern Analysis and Machine Intelligence
JF - IEEE Transactions on Pattern Analysis and Machine Intelligence
IS - 2
ER -