Model-Induced Generalization Error Bound for Information-Theoretic Representation Learning in Source-Data-Free Unsupervised Domain Adaptation

Baoyao Yang, Hao-Wei Yeh, Tatsuya Harada, Pong Chi Yuen*

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

Abstract

Many unsupervised domain adaptation (UDA) methods have been developed and have achieved promising results in various pattern recognition tasks. However, most existing methods assume that raw source data are available in the target domain when transferring knowledge from the source to the target domain. Due to the emerging regulations on data privacy, the availability of source data cannot be guaranteed when applying UDA methods in a new domain. The lack of source data makes UDA more challenging, and most existing methods are no longer applicable. To handle this issue, this paper analyzes the cross-domain representations in source-data-free unsupervised domain adaptation (SF-UDA). A new theorem is derived to bound the target-domain prediction error using the trained source model instead of the source data. On the basis of the proposed theorem, information bottleneck theory is introduced to minimize the generalization upper bound of the target-domain prediction error, thereby achieving domain adaptation. The minimization is implemented in a variational inference framework using a newly developed latent alignment variational autoencoder (LA-VAE). The experimental results show good performance of the proposed method in several cross-dataset classification tasks without using source data. Ablation studies and feature visualization also validate the effectiveness of our method in SF-UDA.
Original languageEnglish
Pages (from-to)419-432
Number of pages14
JournalIEEE Transactions on Image Processing
Volume31
Early online date7 Dec 2021
DOIs
Publication statusPublished - Jan 2022

Scopus Subject Areas

  • Software
  • Computer Graphics and Computer-Aided Design

User-Defined Keywords

  • information theory
  • representation learning
  • Unsupervised domain adaptation (UDA)

Fingerprint

Dive into the research topics of 'Model-Induced Generalization Error Bound for Information-Theoretic Representation Learning in Source-Data-Free Unsupervised Domain Adaptation'. Together they form a unique fingerprint.

Cite this