Realize Generative Yet Complete Latent Representation for Incomplete Multi-View Learning

Hongmin Cai, Weitian Huang, Sirui Yang, Siqi Ding, Yue Zhang, Bin Hu, Fa Zhang, Yiu-ming Cheung*

*Corresponding author for this work

Research output: Contribution to journalJournal articlepeer-review

Abstract

In multi-view environment, it would yield missing observations due to the limitation of the observation process. The most current representation learning methods struggle to explore complete information by lacking either cross-generative via simply filling in missing view data, or solidative via inferring a consistent representation among the existing views. To address this problem, we propose a deep generative model to learn a complete generative latent representation, namely Complete Multi-view Variational Auto-Encoders (CMVAE), which models the generation of the multiple views from a complete latent variable represented by a mixture of Gaussian distributions. Thus, the missing view can be fully characterized by the latent variables and is resolved by estimating its posterior distribution. Accordingly, a novel variational lower bound is introduced to integrate view-invariant information into posterior inference to enhance the solidative of the learned latent representation. The intrinsic correlations between views are mined to seek cross-view generality, and information leading to missing views is fused by view weights to reach solidity. Benchmark experimental results in clustering, classification, and cross-view image generation tasks demonstrate the superiority of CMVAE, while time complexity and parameter sensitivity analyses illustrate the efficiency and robustness. Additionally, application to bioinformatics data exemplifies its practical significance.
Original languageEnglish
Article number10373887
Pages (from-to)3637-3652
Number of pages16
JournalIEEE Transactions on Pattern Analysis and Machine Intelligence
Volume46
Issue number5
Early online date25 Dec 2023
DOIs
Publication statusPublished - May 2024

Scopus Subject Areas

  • Software
  • Artificial Intelligence
  • Applied Mathematics
  • Computer Vision and Pattern Recognition
  • Computational Theory and Mathematics

User-Defined Keywords

  • deep generative models
  • incomplete multi-view problem
  • multi-view learning
  • representation learning
  • Deep generative models

Fingerprint

Dive into the research topics of 'Realize Generative Yet Complete Latent Representation for Incomplete Multi-View Learning'. Together they form a unique fingerprint.

Cite this