USN: A Robust Imitation Learning Method against Diverse Action Noise

Xingrui Yu, Bo Han, Ivor W. Tsang

Research output: Contribution to journalJournal articlepeer-review


Learning from imperfect demonstrations is a crucial challenge in imitation learning (IL). Unlike existing works that still rely on the enormous effort of expert demonstrators, we consider a more cost-effective option for obtaining a large number of demonstrations. That is, hire annotators to label actions for existing image records in realistic scenarios. However,action noise can occur when annotators are not domain experts or encounter confusing states.In this work, we introduce two particular forms of action noise, i.e.,state-independent and state-dependent action noise. Previous IL methods fail to achieve expert-level performance when the demonstrations contain action noise, especially the state-dependent action noise.To mitigate the harmful effects of action noises, we propose a robust learning paradigm called USN (Uncertainty-aware Sample-selection with Negative learning). The model first estimates the predictive uncertainty for all demonstration data and then selects samples with high loss based on the uncertainty measures. Finally, it updates the model parameters with additional negative learning on the selected samples. Empirical results in Box2Dtasks and Atari games show that USN consistently improves the final rewards of behavioral cloning, online imitation learning, and offline imitation learning methods under various action noises. The ratio of significant improvements is up to 94.44%. Moreover, our method scales to conditional imitation learning with real-world noisy commands in urban driving.
Original languageEnglish
Pages (from-to)1237-1280
Number of pages44
JournalJournal of Artificial Intelligence Research
Publication statusPublished - 21 Apr 2024

Scopus Subject Areas

  • Artificial Intelligence

User-Defined Keywords

  • imitation learning
  • noisy demonstrations
  • soft negative learning
  • uncertainty-aware sample-selection


Dive into the research topics of 'USN: A Robust Imitation Learning Method against Diverse Action Noise'. Together they form a unique fingerprint.

Cite this