A Unified Framework for Bidirectional Prototype Learning From Contaminated Faces Across Heterogeneous Domains

Meng Pang, Binghui Wang, Siyu Huang, Yiu Ming Cheung, Bihan Wen*

*Corresponding author for this work

Research output: Contribution to journalJournal articlepeer-review

12 Citations (Scopus)


Existing heterogeneous face synthesis (HFS) methods focus on performing accurate image-to-image translation across domains, while they cannot effectively remove the nuisance facial variations such as poses, expressions or occlusions. To address such challenges, this paper studies a new practical heterogeneous prototype learning (HPL) problem. To be specific, given a face image contaminated by facial variations from a source domain, HPL aims to reconstruct the variation-free prototype in a specified target domain. To tackle HPL, we propose a unified and end-to-end framework named bidirectional heterogeneous prototype learning (BHPL). As a bidirectional learning framework, BHPL is able to simultaneously reconstruct the heterogeneous prototypes across source-to-target as well as target-to-source domains. Furthermore, BHPL is capable of learning the identity prototype features for the contaminated face images from both source and target domains in order to perform robust heterogeneous face recognition. BHPL consists of an encoder-decoder structural generator and two dual-task discriminators, which play an adversarial game such that the generator learns the identity prototype feature and generates the cross-domain identity-preserved prototype for each input face image from both domains, and the discriminators accurately predict face identity and distinguish real versus fake prototypes. Empirically studies on multiple heterogeneous face datasets containing facial variations demonstrate the effectiveness of BHPL.

Original languageEnglish
Pages (from-to)1544-1557
Number of pages14
JournalIEEE Transactions on Information Forensics and Security
Publication statusPublished - 1 Apr 2022

Scopus Subject Areas

  • Safety, Risk, Reliability and Quality
  • Computer Networks and Communications

User-Defined Keywords

  • adversarial learning
  • Face synthesis
  • heterogeneous face recognition
  • heterogeneous prototype learning


Dive into the research topics of 'A Unified Framework for Bidirectional Prototype Learning From Contaminated Faces Across Heterogeneous Domains'. Together they form a unique fingerprint.

Cite this