TY - JOUR
T1 - OMGH
T2 - Online Manifold-Guided Hashing for Flexible Cross-modal Retrieval
AU - Liu, Xin
AU - Yi, Jinhan
AU - Cheung, Yiu Ming
AU - Xu, Xing
AU - Cui, Zhen
N1 - Funding Information:
This work was supported in part by the Open Project of Zhejiang Lab under Grant 2021KH0AB01, in part by the National Science Foundation of China under Grant 61673185, Grant 61672444, and Grant 61976049, in part by the NSFC/RGC Joint Research Scheme under Grant N HKBU214/21, in part by RGC General Research Fund under Grant RGC/HKBU/12201321, in part by Hong Kong Baptist University under Grant RC-FNRA-IG/18-19/SCI/03 and Grant RC-IRCMs/18-19/SCI/01, in part by the Innovation and Technology Fund of Innovation and Technology Commission of the Government of the Hong Kong SAR under Grant TS/339/18, in part by the National Science Foundation of Fujian Province under Grant 2020J01084, and in part by the Natural Science Foundation of Shandong Province under Grant ZR2020LZH008.
Publisher copyright:
© 2022 IEEE.
PY - 2023/9
Y1 - 2023/9
N2 - Cross-modal hashing hasrecently gained an increasing attention for its efficiency and fast retrieval speed in indexing the multimedia data across different modalities. Nevertheless, the multimedia data points often emerge in a streaming manner, and existing online methods often lack of learning capacity to handle both labeled and unlabeled data.To alleviate these concerns, this paper proposes an Online Manifold-Guided Hashing (OMGH) framework, which can incrementally learn the compact hash code of streaming data while adaptively optimizing the hash function in a streaming manner. To be specific, OMGH first exploits a matrix tri-factorization framework to learn the discriminative hash codes for streaming multi-modal data. Then, an online anchor-based manifold structure is designed to sparsely represent the old data and adaptively guide the hash code learning process, which can wellreduce the complexity in preserving the semantic correlation between the old data and streaming data. Meanwhile, such anchor-based manifold embedding is adaptive to the unsupervised and supervised learning strategies in a flexible way. Besides, an online discrete optimization method is efficiently addressed to incrementally update the hash functions and optimize the hash codes on streaming data points. As a result, the derived hash codes are more semantically meaningful for various online cross-modal retrieval tasks. Extensive experiments verify the advantages of the proposed OMGH model, by achieving and improving the state-of-the-art cross-modal retrieval performances on three benchmark datasets.
AB - Cross-modal hashing hasrecently gained an increasing attention for its efficiency and fast retrieval speed in indexing the multimedia data across different modalities. Nevertheless, the multimedia data points often emerge in a streaming manner, and existing online methods often lack of learning capacity to handle both labeled and unlabeled data.To alleviate these concerns, this paper proposes an Online Manifold-Guided Hashing (OMGH) framework, which can incrementally learn the compact hash code of streaming data while adaptively optimizing the hash function in a streaming manner. To be specific, OMGH first exploits a matrix tri-factorization framework to learn the discriminative hash codes for streaming multi-modal data. Then, an online anchor-based manifold structure is designed to sparsely represent the old data and adaptively guide the hash code learning process, which can wellreduce the complexity in preserving the semantic correlation between the old data and streaming data. Meanwhile, such anchor-based manifold embedding is adaptive to the unsupervised and supervised learning strategies in a flexible way. Besides, an online discrete optimization method is efficiently addressed to incrementally update the hash functions and optimize the hash codes on streaming data points. As a result, the derived hash codes are more semantically meaningful for various online cross-modal retrieval tasks. Extensive experiments verify the advantages of the proposed OMGH model, by achieving and improving the state-of-the-art cross-modal retrieval performances on three benchmark datasets.
KW - Cross-modal hashing
KW - Anchor-based manifold structure
KW - online discrete optimization
KW - streaming data
UR - http://www.scopus.com/inward/record.url?scp=85128660500&partnerID=8YFLogxK
U2 - 10.1109/TMM.2022.3166668
DO - 10.1109/TMM.2022.3166668
M3 - Journal article
SN - 1520-9210
VL - 25
SP - 3811
EP - 3824
JO - IEEE Transactions on Multimedia
JF - IEEE Transactions on Multimedia
ER -