TY - JOUR
T1 - Cross-Modal Hashing Method with Properties of Hamming Space
T2 - A New Perspective
AU - Hu, Zhikai
AU - Cheung, Yiu-ming
AU - Li, Mengke
AU - Lan, Weichao
N1 - Funding Information:
This work was supported in part by the NSFC/Research Grants Council (RGC) Joint Research Scheme under Grant N_HKBU214/21, in part by the General Research Fund of RGC under Grant 12201321, Grant 12202622 and Grant 12201323, in part by the RGC Senior Research Fellow Scheme under Grant SRFS2324-2S02, in part by the National Natural Science Foundation of China (NSFC) under Grant 62306181, and in part by the Natural Science Foundation of Guangdong Province under Grant 2024A1515010163.
Publisher Copyright:
© 2024 The Authors.
PY - 2024/12
Y1 - 2024/12
N2 - Cross-modal hashing (CMH) has attracted considerable attention in recent
years. Almost all existing CMH methods primarily focus on reducing the
modality gap and semantic gap, i.e., aligning multi-modal features and
their semantics in Hamming space, without taking into account the space
gap, i.e., difference between the real number space and the Hamming
space. In fact, the space gap can affect the performance of CMH methods.
In this paper, we analyze and demonstrate how the space gap affects the
existing CMH methods, which therefore raises two problems: solution
space compression and loss function oscillation. These two problems
eventually cause the retrieval performance deteriorating. Based on these
findings, we propose a novel algorithm, namely Semantic Channel Hashing
(SCH). First, we classify sample pairs into fully semantic-similar,
partially semantic-similar, and semantic-negative ones based on their
similarity and impose different constraints on them, respectively, to
ensure that the entire Hamming space is utilized. Then, we introduce a
semantic channel to alleviate the issue of loss function oscillation.
Experimental results on three public datasets demonstrate that SCH
outperforms the state-of-the-art methods. Furthermore, experimental
validations are provided to substantiate the conjectures regarding
solution space compression and loss function oscillation, offering
visual evidence of their impact on the CMH methods.
AB - Cross-modal hashing (CMH) has attracted considerable attention in recent
years. Almost all existing CMH methods primarily focus on reducing the
modality gap and semantic gap, i.e., aligning multi-modal features and
their semantics in Hamming space, without taking into account the space
gap, i.e., difference between the real number space and the Hamming
space. In fact, the space gap can affect the performance of CMH methods.
In this paper, we analyze and demonstrate how the space gap affects the
existing CMH methods, which therefore raises two problems: solution
space compression and loss function oscillation. These two problems
eventually cause the retrieval performance deteriorating. Based on these
findings, we propose a novel algorithm, namely Semantic Channel Hashing
(SCH). First, we classify sample pairs into fully semantic-similar,
partially semantic-similar, and semantic-negative ones based on their
similarity and impose different constraints on them, respectively, to
ensure that the entire Hamming space is utilized. Then, we introduce a
semantic channel to alleviate the issue of loss function oscillation.
Experimental results on three public datasets demonstrate that SCH
outperforms the state-of-the-art methods. Furthermore, experimental
validations are provided to substantiate the conjectures regarding
solution space compression and loss function oscillation, offering
visual evidence of their impact on the CMH methods.
KW - Codes
KW - Cross-modal retrieval
KW - Hamming distances
KW - Hamming space
KW - hashing
KW - loss oscillation
KW - Oscillators
KW - Semantics
KW - solution space compression
KW - Task analysis
KW - Training
KW - Visualization
UR - http://www.scopus.com/inward/record.url?scp=85191322075&partnerID=8YFLogxK
U2 - 10.1109/TPAMI.2024.3392763
DO - 10.1109/TPAMI.2024.3392763
M3 - Journal article
AN - SCOPUS:85191322075
SN - 0162-8828
VL - 46
SP - 7636
EP - 7650
JO - IEEE Transactions on Pattern Analysis and Machine Intelligence
JF - IEEE Transactions on Pattern Analysis and Machine Intelligence
IS - 12
ER -