Joint Semantic Preserving Sparse Hashing for Cross-Modal Retrieval

Zhikai Hu, Yiu-ming Cheung*, Mengke Li, Weichao Lan, Donglin Zhang, Qiang Liu

*Corresponding author for this work

Research output: Contribution to journalJournal articlepeer-review

2 Citations (Scopus)

Abstract

Supervised cross-modal hashing has received wide attention in recent years. However, existing methods primarily rely on sample-wise semantic relationships to evaluate the semantic similarity between samples, overlooking the impact of label distribution on enhancing retrieval performance. Moreover, the limited representation capability of traditional dense hash codes hinders the preservation of semantic relationship. To overcome these challenges, we propose a new method, Joint Semantic Preserving Sparse Hashing (JSPSH). Specifically, we introduce a new concept of cluster-wise semantic relationship, which leverages label distribution to indicate which samples are more suitable for clustering. Then, we jointly utilize sample-wise and cluster-wise semantic relationships to supervise the learning of hash codes. In this way, JSPSH preserves both kinds of semantic relationships to ensure that more samples with similar semantics are clustered together, thereby achieving better retrieval results. Furthermore, we utilize high-dimensional sparse hash codes that offer stronger representation capability to preserve such more complex semantics. Finally, an interaction term is introduced in hash functions learning stage to further narrow the gap between modalities. Experimental results on three large-scale datasets demonstrate the effectiveness of JSPSH in achieving superior retrieval performance. Codes are available at https://github.com/hutt94/JSPSH.
Original languageEnglish
Pages (from-to)2989-3002
Number of pages14
JournalIEEE Transactions on Circuits and Systems for Video Technology
Volume34
Issue number4
Early online date22 Aug 2023
DOIs
Publication statusPublished - Apr 2024

Scopus Subject Areas

  • Electrical and Electronic Engineering
  • Media Technology

User-Defined Keywords

  • Codes
  • Encoding
  • Hash functions
  • Quantization (signal)
  • Semantics
  • Sparse matrices
  • Task analysis

Fingerprint

Dive into the research topics of 'Joint Semantic Preserving Sparse Hashing for Cross-Modal Retrieval'. Together they form a unique fingerprint.

Cite this