DisP+V: A Unified Framework for Disentangling Prototype and Variation from Single Sample per Person

Meng Pang, Binghui Wang, Mang Ye, Yiu Ming Cheung, Yiran Chen, Bihan Wen*

*Corresponding author for this work

Research output: Contribution to journalJournal articlepeer-review

8 Citations (Scopus)

Abstract

Single sample per person face recognition (SSPP FR) is one of the most challenging problems in FR due to the extreme lack of enrolment data. To date, the most popular SSPP FR methods are the generic learning methods, which recognize query face images based on the so-called prototype plus variation (i.e., P+V) model. However, the classic P+V model suffers from two major limitations: 1) it linearly combines the prototype and variation images in the observational pixel-spatial space and cannot generalize to multiple nonlinear variations, e.g., poses, which are common in face images and 2) it would be severely impaired once the enrolment face images are contaminated by nuisance variations. To address the two limitations, it is desirable to disentangle the prototype and variation in a latent feature space and to manipulate the images in a semantic manner. To this end, we propose a novel disentangled prototype plus variation model, dubbed DisP+V, which consists of an encoder-decoder generator and two discriminators. The generator and discriminators play two adversarial games such that the generator nonlinearly encodes the images into a latent semantic space, where the more discriminative prototype feature and the less discriminative variation feature are disentangled. Meanwhile, the prototype and variation features can guide the generator to generate an identity-preserved prototype and the corresponding variation, respectively. Experiments on various real-world face datasets demonstrate the superiority of our DisP+V model over the classic P+V model for SSPP FR. Furthermore, DisP+V demonstrates its unique characteristics in both prototype recovery and face editing/interpolation.
Original languageEnglish
Pages (from-to)867-881
Number of pages15
JournalIEEE Transactions on Neural Networks and Learning Systems
Volume34
Issue number2
Early online date17 Aug 2021
DOIs
Publication statusPublished - Feb 2023

Scopus Subject Areas

  • Software
  • Artificial Intelligence
  • Computer Networks and Communications
  • Computer Science Applications

User-Defined Keywords

  • Adversarial learning
  • disentangled representation
  • face editing
  • prototype recovery
  • single sample per person

Fingerprint

Dive into the research topics of 'DisP+V: A Unified Framework for Disentangling Prototype and Variation from Single Sample per Person'. Together they form a unique fingerprint.

Cite this