Abstract
Crowdsourcing is a popular solution for large-scale data annotations. So far, various end-to-end deep learning methods have been proposed to improve the practical performance of learning from crowds. Despite their practical effectiveness, most of them have two major limitations--they do not hold learning consistency and suffer from computational inefficiency. In this article, we propose a novel method named UnionNet, which is not only theoretically consistent but also experimentally effective and efficient. Specifically, unlike existing methods that either fit a given label from each annotator independently or fuse all the labels into a reliable one, we concatenate the one-hot encoded vectors of crowdsourced labels provided by all the annotators, which takes all the labeling information as a union and coordinates multiple annotators. In this way, we can directly train an end-to-end deep neural network by maximizing the likelihood of this union with only a parametric transition matrix. We theoretically prove the learning consistency and experimentally show the effectiveness and efficiency of our proposed method.
Original language | English |
---|---|
Pages (from-to) | 10552-10562 |
Number of pages | 11 |
Journal | IEEE Transactions on Neural Networks and Learning Systems |
Volume | 34 |
Issue number | 12 |
Early online date | 29 Apr 2022 |
DOIs | |
Publication status | Published - Dec 2023 |
Scopus Subject Areas
- Software
- Artificial Intelligence
- Computer Networks and Communications
- Computer Science Applications
User-Defined Keywords
- Annotators
- crowdsourcing
- Deep learning
- Labeling
- Learning systems
- Neural networks
- Noise measurement
- noisy labels
- Standards
- Training
- transition matrix