Contrastive Learning with Boosted Memorization

Zhihan Zhou, Jiangchao Yao, Yanfeng Wang, Bo Han, Ya Zhang

Research output: Chapter in book/report/conference proceedingConference proceedingpeer-review

2 Citations (Scopus)

Abstract

Self-supervised learning has achieved a great success in the representation learning of visual and textual data. However, the current methods are mainly validated on the well-curated datasets, which do not exhibit the real-world long-tailed distribution. Recent attempts to consider self-supervised long-tailed learning are made by rebalancing in the loss perspective or the model perspective, resembling the paradigms in the supervised long-tailed learning. Nevertheless, without the aid of labels, these explorations have not shown the expected significant promise due to the limitation in tail sample discovery or the heuristic structure design. Different from previous works, we explore this direction from an alternative perspective, i.e., the data perspective, and propose a novel Boosted Contrastive Learning (BCL) method. Specifically, BCL leverages the memorization effect of deep neural networks to automatically drive the information discrepancy of the sample views in contrastive learning, which is more efficient to enhance the long-tailed learning in the label-unaware context. Extensive experiments on a range of benchmark datasets demonstrate the effectiveness of BCL over several state-of-the-art methods. Our code is available at https://github.com/MediaBrain-SJTU/BCL.
Original languageEnglish
Title of host publicationProceedings of 39th International Conference on Machine Learning (ICML’22)
EditorsKamalika Chaudhuri, Stefanie Jegelka, Le Song, Csaba Szepesvari, Gang Niu, Sivan Sabato
PublisherML Research Press
Pages27367-27377
Number of pages11
Publication statusPublished - 17 Jul 2022
Event39th International Conference on Machine Learning, ICML 2022 - Baltimore Convention Center , Baltimore, Maryland, United States
Duration: 17 Jul 202223 Jul 2022
https://icml.cc/Conferences/2022

Publication series

NameProceedings of Machine Learning Research
Volume162
ISSN (Print)2640-3498

Conference

Conference39th International Conference on Machine Learning, ICML 2022
Country/TerritoryUnited States
CityBaltimore, Maryland
Period17/07/2223/07/22
Internet address

Fingerprint

Dive into the research topics of 'Contrastive Learning with Boosted Memorization'. Together they form a unique fingerprint.

Cite this