Exploiting counter-examples for active learning with partial labels

Fei Zhang, Yunjie Ye, Lei Feng, Zhongwen Rao, Jieming Zhu, Marcus Kalander, Chen Gong, Jianye Hao, Bo Han*

*Corresponding author for this work

Research output: Contribution to journalJournal articlepeer-review


This paper studies a new problem, active learning with partial labels (ALPL). In this setting, an oracle annotates the query samples with partial labels, relaxing the oracle from the demanding accurate labeling process. To address ALPL, we first build an intuitive baseline that can be seamlessly incorporated into existing AL frameworks. Though effective, this baseline is still susceptible to the overfitting, and falls short of the representative partial-label-based samples during the query process. Drawing inspiration from human inference in cognitive science, where accurate inferences can be explicitly derived from counter-examples (CEs), our objective is to leverage this human-like learning pattern to tackle the overfitting while enhancing the process of selecting representative samples in ALPL. Specifically, we construct CEs by reversing the partial labels for each instance, and then we propose a simple but effective WorseNet to directly learn from this complementary pattern. By leveraging the distribution gap between WorseNet and the predictor, this adversarial evaluation manner could enhance both the performance of the predictor itself and the sample selection process, allowing the predictor to capture more accurate patterns in the data. Experimental results on five real-world datasets and four benchmark datasets show that our proposed method achieves comprehensive improvements over ten representative AL frameworks, highlighting the superiority of WorseNet.

Original languageEnglish
Pages (from-to)3849-3868
Number of pages20
JournalMachine Learning
Issue number6
Early online date8 Jan 2024
Publication statusPublished - Jun 2024

Scopus Subject Areas

  • Software
  • Artificial Intelligence

User-Defined Keywords

  • Active learning
  • Adversarial learning
  • Classification
  • Counter-examples
  • Partial-label learning
  • Weakly-supervised learning


Dive into the research topics of 'Exploiting counter-examples for active learning with partial labels'. Together they form a unique fingerprint.

Cite this