Cross-Modality Person Re-Identification via Modality-Aware Collaborative Ensemble Learning

Mang Ye, Xiangyuan Lan, Qingming Leng, Jianbing Shen*

*Corresponding author for this work

Research output: Contribution to journalJournal articlepeer-review

145 Citations (Scopus)


Visible thermal person re-identification (VT-ReID) is a challenging cross-modality pedestrian retrieval problem due to the large intra-class variations and modality discrepancy across different cameras. Existing VT-ReID methods mainly focus on learning cross-modality sharable feature representations by handling the modality-discrepancy in feature level. However, the modality difference in classifier level has received much less attention, resulting in limited discriminability. In this paper, we propose a novel modality-aware collaborative ensemble (MACE) learning method with middle-level sharable two-stream network (MSTN) for VT-ReID, which handles the modality-discrepancy in both feature level and classifier level. In feature level, MSTN achieves much better performance than existing methods by capturing sharable discriminative middle-level features in convolutional layers. In classifier level, we introduce both modality-specific and modality-sharable identity classifiers for two modalities to handle the modality discrepancy. To utilize the complementary information among different classifiers, we propose an ensemble learning scheme to incorporate the modality sharable classifier and the modality specific classifiers. In addition, we introduce a collaborative learning strategy, which regularizes modality-specific identity predictions and the ensemble outputs. Extensive experiments on two cross-modality datasets demonstrate that the proposed method outperforms current state-of-the-art by a large margin, achieving rank-1/mAP accuracy 51.64%/50.11% on the SYSU-MM01 dataset, and 72.37%/69.09% on the RegDB dataset.

Original languageEnglish
Article number9107428
Pages (from-to)9387-9399
Number of pages13
JournalIEEE Transactions on Image Processing
Publication statusPublished - 3 Jun 2020

Scopus Subject Areas

  • Software
  • Computer Graphics and Computer-Aided Design

User-Defined Keywords

  • collaborative ensemble learning
  • Cross-modality
  • person re-identification


Dive into the research topics of 'Cross-Modality Person Re-Identification via Modality-Aware Collaborative Ensemble Learning'. Together they form a unique fingerprint.

Cite this