GMFAD: Towards Generalized Visual Recognition via Multilayer Feature Alignment and Disentanglement

Haoliang Li*, Shiqi Wang, Renjie Wan, Alex C. Kot

*Corresponding author for this work

Research output: Contribution to journalJournal articlepeer-review

4 Citations (Scopus)


The deep learning based approaches which have been repeatedly proven to bring benefits to visual recognition tasks usually make a strong assumption that the training and test data are drawn from similar feature spaces and distributions. However, such an assumption may not always hold in various practical application scenarios on visual recognition tasks. Inspired by the hierarchical organization of deep feature representation that progressively leads to more abstract features at higher layers of representations, we propose to tackle this problem with a novel feature learning framework, which is called GMFAD, with better generalization capability in a multilayer perceptron manner. We first learn feature representations at the shallow layer where shareable underlying factors among domains (e.g., a subset of which could be relevant for each particular domain) can be explored. In particular, we propose to align the domain divergence between domain pair(s) by considering both inter-dimension and inter-sample correlations, which have been largely ignored by many cross-domain visual recognition methods. Subsequently, to learn more abstract information which could further benefit transferability, we propose to conduct feature disentanglement at the deep feature layer. Extensive experiments based on different visual recognition tasks demonstrate that our proposed framework can learn better transferable feature representation compared with state-of-the-art baselines.

Original languageEnglish
Pages (from-to)1289-1303
Number of pages15
JournalIEEE Transactions on Pattern Analysis and Machine Intelligence
Issue number3
Publication statusPublished - 1 Mar 2022

Scopus Subject Areas

  • Software
  • Artificial Intelligence
  • Applied Mathematics
  • Computer Vision and Pattern Recognition
  • Computational Theory and Mathematics

User-Defined Keywords

  • covariance matrix
  • disentanglement
  • Generalization capability
  • visual recognition


Dive into the research topics of 'GMFAD: Towards Generalized Visual Recognition via Multilayer Feature Alignment and Disentanglement'. Together they form a unique fingerprint.

Cite this