Instance-Dependent Positive and Unlabeled Learning with Labeling Bias Estimation

Chen Gong, Qizhou Wang, Tongliang Liu, Bo Han, Jane J. You, Jian Yang, Dacheng Tao

Research output: Contribution to journalArticlepeer-review


This paper studies instance-dependent Positive and Unlabeled (PU) classification, where whether a positive example will be labeled (indicated by s) is not only related to the class label y, but also depends on the observation x. Therefore, the labeling probability on positive examples is not uniform as previous works assumed, but is biased to some simple or critical data points. To depict the above dependency relationship, a graphical model is built in this paper which further leads to a maximization problem on the induced likelihood function regarding P(s,y|x). By utilizing the well-known EM and Adam optimization techniques, the labeling probability of any positive example P(s=1|y=1,x) as well as the classifier induced by P(y|x) can be acquired. Theoretically, we prove that the critical solution always exists, and is locally unique for linear model if some sufficient conditions are met. Moreover, we upper bound the generalization error for both linear logistic and non-linear network instantiations of our algorithm. Empirically, we compare our method with state-of-the-art instance-independent and instance-dependent PU algorithms on a wide range of synthetic, benchmark and real-world datasets, and the experimental results firmly demonstrate the advantage of the proposed method over the existing PU approaches.

Original languageEnglish
Number of pages16
JournalIEEE Transactions on Pattern Analysis and Machine Intelligence
Publication statusE-pub ahead of print - 23 Feb 2021

Scopus Subject Areas

  • Software
  • Computer Vision and Pattern Recognition
  • Computational Theory and Mathematics
  • Artificial Intelligence
  • Applied Mathematics

User-Defined Keywords

  • Data models
  • Generalization Bound
  • Graphical models
  • Instance-Dependent PU Learning
  • Labeling
  • Labeling Bias
  • Logistics
  • Maximum likelihood estimation
  • Maximum Likelihood Estimation
  • Random variables
  • Solution Uniqueness
  • Training


Dive into the research topics of 'Instance-Dependent Positive and Unlabeled Learning with Labeling Bias Estimation'. Together they form a unique fingerprint.

Cite this