Facial Structure Guided GAN for Identity-preserved Face Image De-occlusion

Yiu Ming Cheung, Mengke Li, Rong Zou

Research output: Chapter in book/report/conference proceedingConference proceedingpeer-review

3 Citations (Scopus)


In some practical scenarios, such as video surveillance and personal identification, we often have to address the recognition problem of occluded faces, where content replacement by serious occlusion with non-face objects always produces partial appearance and ambiguous representation. Under the circumstances, the performance of face recognition algorithms will often deteriorate to a certain degree. In this paper, we therefore address this problem by removing occlusions on face images and present a new two-stage Facial Structure Guided Generative Adversarial Network (FSG-GAN). In Stage I of the FSG-GAN, the variational auto-encoder is used to predict the facial structure. In Stage II, the predicted facial structure and the occluded image are concatenated and fed into a generative adversarial network (GAN) based model to synthesize the de-occlusion face image. In this way, the facial structure knowledge can be transferred to the synthesis network. Especially, in order to enable the occluded face image to be perceived well, the generator in the GAN based synthesis network utilizes the hybrid dilated convolution modules to extend the receptive field. Furthermore, aiming at further eliminating the appearance ambiguity as well as unnatural texture, a multi-receptive fields discriminator is proposed to utilize the features from different levels. Experiments on the benchmark datasets show the efficacy of the proposed FSG-GAN.

Original languageEnglish
Title of host publicationICMR '21: Proceedings of the 2021 International Conference on Multimedia Retrieval
PublisherAssociation for Computing Machinery (ACM)
Number of pages9
ISBN (Electronic)9781450384636
Publication statusPublished - Aug 2021
Event11th ACM International Conference on Multimedia Retrieval, ICMR 2021 - Taipei, Taiwan, Province of China
Duration: 16 Nov 202119 Nov 2021

Publication series

NameProceedings of the International Conference on Multimedia Retrieval


Conference11th ACM International Conference on Multimedia Retrieval, ICMR 2021
Country/TerritoryTaiwan, Province of China

Scopus Subject Areas

  • Computer Graphics and Computer-Aided Design
  • Computer Science Applications
  • Computer Vision and Pattern Recognition
  • Human-Computer Interaction
  • Software

User-Defined Keywords

  • Face de-occlusion
  • Generative adversarial networks
  • Partial face recognition


Dive into the research topics of 'Facial Structure Guided GAN for Identity-preserved Face Image De-occlusion'. Together they form a unique fingerprint.

Cite this