Augmentation Invariant and Instance Spreading Feature for Softmax Embedding

Mang Ye, Jianbing Shen*, Xu Zhang, Pong C. Yuen, Shih Fu Chang

*Corresponding author for this work

Research output: Contribution to journalJournal articlepeer-review

73 Citations (Scopus)


Deep embedding learning plays a key role in learning discriminative feature representations, where the visually similar samples are pulled closer and dissimilar samples are pushed away in the low-dimensional embedding space. This paper studies the unsupervised embedding learning problem by learning such a representation without using any category labels. This task faces two primary challenges: mining reliable positive supervision from highly similar fine-grained classes, and generalizing to unseen testing categories. To approximate the positive concentration and negative separation properties in category-wise supervised learning, we introduce a data augmentation invariant and instance spreading feature using the instance-wise supervision. We also design two novel domain-agnostic augmentation strategies to further extend the supervision in feature space, which simulates the large batch training using a small batch size and the augmented features. To learn such a representation, we propose a novel instance-wise softmax embedding, which directly perform the optimization over the augmented instance features with the binary discrmination softmax encoding. It significantly accelerates the learning speed with much higher accuracy than existing methods, under both seen and unseen testing categories. The unsupervised embedding performs well even without pre-trained network over samples from fine-grained categories. We also develop a variant using category-wise supervision, namely category-wise softmax embedding, which achieves competitive performance over the state-of-of-the-arts, without using any auxiliary information or restrict sample mining.

Original languageEnglish
Pages (from-to)924-939
Number of pages16
JournalIEEE Transactions on Pattern Analysis and Machine Intelligence
Issue number2
Early online date3 Aug 2020
Publication statusPublished - 1 Feb 2022

Scopus Subject Areas

  • Software
  • Computer Vision and Pattern Recognition
  • Computational Theory and Mathematics
  • Artificial Intelligence
  • Applied Mathematics

User-Defined Keywords

  • data augmentation
  • embedding learning
  • instance feature
  • softmax embedding
  • Unsupervised learning


Dive into the research topics of 'Augmentation Invariant and Instance Spreading Feature for Softmax Embedding'. Together they form a unique fingerprint.

Cite this