A Self-Consistent-Field Iteration for Orthogonal Canonical Correlation Analysis

Lei-Hong Zhang, Li Wang, Zhaojun Bai, Ren-Cang Li

Research output: Contribution to journalArticlepeer-review

Abstract

We propose an efficient algorithm for solving orthogonal canonical correlation analysis (OCCA) in the form of trace-fractional structure and orthogonal linear projections. Even though orthogonality has been widely used and proved to be a useful criterion for visualization, pattern recognition and feature extraction, existing methods for solving OCCA problem are either numerically unstable by relying on a deflation scheme, or less efficient by directly using generic optimization methods. In this paper, we propose an alternating numerical scheme whose core is the sub-maximization problem in the trace-fractional form with an orthogonality constraint. A customized self-consistent-field (SCF) iteration for this sub-maximization problem is devised. It is proved that the SCF iteration is globally convergent to a KKT point and that the alternating numerical scheme always converges. We further formulate a new trace-fractional maximization problem for orthogonal multiset CCA and propose an efficient algorithm with an either Jacobi-style or Gauss-Seidel-style updating scheme based on the SCF iteration. Extensive experiments are conducted to evaluate the proposed algorithms against existing methods, including real-world applications of multi-label classification and multi-view feature extraction. Experimental results show that our methods not only perform competitively to or better than the existing methods but also are more efficient.

Original languageEnglish
Pages (from-to)890-904
Number of pages15
JournalIEEE Transactions on Pattern Analysis and Machine Intelligence
Volume44
Issue number2
Early online date28 Jul 2020
DOIs
Publication statusPublished - 1 Feb 2022

Scopus Subject Areas

  • Software
  • Artificial Intelligence
  • Applied Mathematics
  • Computer Vision and Pattern Recognition
  • Computational Theory and Mathematics

User-Defined Keywords

  • Canonical correlation analysis
  • orthogonal multiset canonical correlation analysis
  • self-consistent-field iteration

Fingerprint

Dive into the research topics of 'A Self-Consistent-Field Iteration for Orthogonal Canonical Correlation Analysis'. Together they form a unique fingerprint.

Cite this