TY - GEN
T1 - Variational imitation learning with diverse-quality demonstrations
AU - Tangkaratt, Voot
AU - Han, Bo
AU - Khan, Mohammad Emtiyaz
AU - Sugiyama, Masashi
N1 - Funding Information:
We thank the anonymous reviewers for their constructive feedback. BH was supported by the Early Career Scheme (ECS) through the Research Grants Council of Hong Kong under Grant No.22200720, HKBU Tier-1 Start-up Grant and HKBU CSD Start-up Grant. MS was supported by KAKENHI 17H00757.
PY - 2020/7
Y1 - 2020/7
N2 - Learning from demonstrations can be challenging when the quality of demonstrations is diverse, and even more so when the quality is unknown and there is no additional information to estimate the quality. We propose a new method for imitation learning in such scenarios. We show that simple quality-estimation approaches might fail due to compounding error, and fix this issue by jointly estimating both the quality and reward using a variational approach. Our method is easy to implement within reinforcement-learning frameworks and also achieves state-of-The-Art performance on continuous-control benchmarks. Our work enables scalable and data-efficient imitation learning under more realistic settings than before.
AB - Learning from demonstrations can be challenging when the quality of demonstrations is diverse, and even more so when the quality is unknown and there is no additional information to estimate the quality. We propose a new method for imitation learning in such scenarios. We show that simple quality-estimation approaches might fail due to compounding error, and fix this issue by jointly estimating both the quality and reward using a variational approach. Our method is easy to implement within reinforcement-learning frameworks and also achieves state-of-The-Art performance on continuous-control benchmarks. Our work enables scalable and data-efficient imitation learning under more realistic settings than before.
UR - https://proceedings.mlr.press/v119/tangkaratt20a.html
UR - http://www.scopus.com/inward/record.url?scp=85105288970&partnerID=8YFLogxK
M3 - Conference contribution
AN - SCOPUS:85105288970
T3 - Proceedings of Machine Learning Research
SP - 9349
EP - 9359
BT - Proceedings of the 37th International Conference on Machine Learning, ICML 2020
A2 - Daumé III, Hal
A2 - Singh, Aarti
PB - ML Research Press
T2 - 37th International Conference on Machine Learning, ICML 2020
Y2 - 13 July 2020 through 18 July 2020
ER -