TY - GEN
T1 - Class2Simi
T2 - 38th International Conference on Machine Learning, ICML 2021
AU - Wu, Songhua
AU - Xia, Xiaobo
AU - Liu, Tongliang
AU - Han, Bo
AU - Gong, Mingming
AU - Wang, Nannan
AU - Liu, Haifeng
AU - Niu, Gang
N1 - Funding Information:
SHW, XBX, and TLL were supported by Australian Research Council Project DE-190101473. BH was supported by the RGC Early Career Scheme No. 22200720, NSFC Young Scientists Fund No. 62006202 and HKBU CSD Departmental Incentive Grant. NNW was supported by National Natural Science Foundation of China Grant 61922066, Grant 61876142. GN was supported by JST AIP Acceleration Research Grant Number JPMJCR20U3, Japan. We thank the anonymous reviewers for their constructive comments.
Publisher Copyright:
Copyright © 2021 by the author(s)
PY - 2021/7/18
Y1 - 2021/7/18
N2 - Learning with noisy labels has attracted a lot of attention in recent years, where the mainstream approaches are in pointwise manners. Meanwhile, pairwise manners have shown great potential in supervised metric learning and unsupervised contrastive learning. Thus, a natural question is raised: does learning in a pairwise manner mitigate label noise? To give an affirmative answer, in this paper, we propose a framework called Class2Simi: it transforms data points with noisy class labels to data pairs with noisy similarity labels, where a similarity label denotes whether a pair shares the class label or not. Through this transformation, the reduction of the noise rate is theoretically guaranteed, and hence it is in principle easier to handle noisy similarity labels. Amazingly, DNNs that predict the clean class labels can be trained from noisy data pairs if they are first pretrained from noisy data points. Class2Simi is computationally efficient because not only this transformation is on-the-fly in mini-batches, but also it just changes loss computation on top of model prediction into a pairwise manner. Its effectiveness is verified by extensive experiments.
AB - Learning with noisy labels has attracted a lot of attention in recent years, where the mainstream approaches are in pointwise manners. Meanwhile, pairwise manners have shown great potential in supervised metric learning and unsupervised contrastive learning. Thus, a natural question is raised: does learning in a pairwise manner mitigate label noise? To give an affirmative answer, in this paper, we propose a framework called Class2Simi: it transforms data points with noisy class labels to data pairs with noisy similarity labels, where a similarity label denotes whether a pair shares the class label or not. Through this transformation, the reduction of the noise rate is theoretically guaranteed, and hence it is in principle easier to handle noisy similarity labels. Amazingly, DNNs that predict the clean class labels can be trained from noisy data pairs if they are first pretrained from noisy data points. Class2Simi is computationally efficient because not only this transformation is on-the-fly in mini-batches, but also it just changes loss computation on top of model prediction into a pairwise manner. Its effectiveness is verified by extensive experiments.
UR - http://www.scopus.com/inward/record.url?scp=85161338037&partnerID=8YFLogxK
UR - https://www.proceedings.com/63018.html
M3 - Conference proceeding
AN - SCOPUS:85161338037
T3 - Proceedings of Machine Learning Research
SP - 11285
EP - 11295
BT - Proceedings of the 38th International Conference on Machine Learning, ICML 2021
PB - Mathematical Research Press
Y2 - 18 July 2021 through 24 July 2021
ER -