On the reconstruction of face images from deep face templates

Guangcan Mai, Kai Cao, Pong Chi YUEN*, Anil K. Jain

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

23 Citations (Scopus)

Abstract

State-of-the-art face recognition systems are based on deep (convolutional) neural networks. Therefore, it is imperative to determine to what extent face templates derived from deep networks can be inverted to obtain the original face image. In this paper, we study the vulnerabilities of a state-of-the-art face recognition system based on template reconstruction attack. We propose a neighborly de-convolutional neural network (NbNet) to reconstruct face images from their deep templates. In our experiments, we assumed that no knowledge about the target subject and the deep network are available. To train the NbNet reconstruction models, we augmented two benchmark face datasets (VGG-Face and Multi-PIE) with a large collection of images synthesized using a face generator. The proposed reconstruction was evaluated using type-I (comparing the reconstructed images against the original face images used to generate the deep template) and type-II (comparing the reconstructed images against a different face image of the same subject) attacks. Given the images reconstructed from NbNets, we show that for verification, we achieve TAR of 95.20 percent (58.05 percent) on LFW under type-I (type-II) attacks @ FAR of 0.1 percent. Besides, 96.58 percent (92.84 percent) of the images reconstructed from templates of partition fa (fb) can be identified from partition fa in color FERET. Our study demonstrates the need to secure deep templates in face recognition systems.

Original languageEnglish
Article number8338413
Pages (from-to)1188-1202
Number of pages15
JournalIEEE Transactions on Pattern Analysis and Machine Intelligence
Volume41
Issue number5
DOIs
Publication statusPublished - 1 May 2019

Scopus Subject Areas

  • Software
  • Computer Vision and Pattern Recognition
  • Computational Theory and Mathematics
  • Artificial Intelligence
  • Applied Mathematics

User-Defined Keywords

  • deep networks
  • deep templates
  • Face recognition
  • neighborly de-convolutional neural network
  • template reconstruction
  • template security

Fingerprint

Dive into the research topics of 'On the reconstruction of face images from deep face templates'. Together they form a unique fingerprint.

Cite this