Long-Tailed Visual Recognition via Gaussian Clouded Logit Adjustment

Mengke Li, Yiu Ming Cheung*, Yang Lu

*Corresponding author for this work

Research output: Chapter in book/report/conference proceedingConference proceedingpeer-review

21 Citations (Scopus)


Long-tailed data is still a big challenge for deep neural networks, even though they have achieved great success on balanced data. We observe that vanilla training on longtailed data with crossentropy loss makes the instance-rich head classes severely squeeze the spatial distribution of the tail classes, which leads to difficulty in classifying tail class samples. Furthermore, the original crossentropy loss can only propagate gradient short-lively because the gradient in softmax form rapidly approaches zero as the logit difference increases. This phenomenon is called softmax saturation. It is unfavorable for training on balanced data, but can be utilized to adjust the validity of the samples in long-tailed data, thereby solving the distorted embedding space of long-tailed problems. To this end, this paper proposes the Gaussian clouded logit adjustment by Gaussian perturbation of different class logits with varied amplitude. We define the amplitude of perturbation as cloud size and set relatively large cloud sizes to tail classes. The large cloud size can reduce the softmax saturation and thereby making tail class samples more active as well as enlarging the embedding space. To alleviate the bias in a classifier, we therefore propose the class-based effective number sampling strategy with classifier retraining. Extensive experiments on benchmark datasets validate the superior performance of the proposed method. Source code is available at https://github.com/Keke921/GCLLoss.
Original languageEnglish
Title of host publicationProceedings - 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2022
EditorsLisa O’Conner
Place of PublicationNew Orleans, LA
Number of pages10
ISBN (Electronic)9781665469463
ISBN (Print)9781665469470
Publication statusPublished - 18 Jun 2022
Event35th IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2022 - New Orleans, United States
Duration: 19 Jun 202224 Jun 2022

Publication series

NameProceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition
ISSN (Print)1063-6919
ISSN (Electronic)2575-7075


Conference35th IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2022
Country/TerritoryUnited States
CityNew Orleans

User-Defined Keywords

  • Imbalanced Data Learning
  • Long-tailed Data
  • Deep Neural Networks


Dive into the research topics of 'Long-Tailed Visual Recognition via Gaussian Clouded Logit Adjustment'. Together they form a unique fingerprint.

Cite this