TY - JOUR
T1 - A hybrid probabilistic model for unified collaborative and content-based image tagging
AU - Zhou, Ning
AU - Cheung, Kwok Wai
AU - Qiu, Guoping
AU - Xue, Xiangyang
N1 - This work was supported in part by the HKBU science Faculty Research Student Exchange Program, the 973 Program (No. 2010CB327900, the NSF of China (No. 60873178), and the Shanghai Leading Academic Discipline Project (No. B114). Ning Zhou was with the School of Computer Science, Fudan University, Shanghai, China.
PY - 2011/7
Y1 - 2011/7
N2 - The increasing availability of large quantities of user contributed images with labels has provided opportunities to develop automatic tools to tag images to facilitate image search and retrieval. In this paper, we present a novel hybrid probabilistic model (HPM) which integrates low-level image features and high-level user provided tags to automatically tag images. For images without any tags, HPM predicts new tags based solely on the low-level image features. For images with user provided tags, HPM jointly exploits both the image features and the tags in a unified probabilistic framework to recommend additional tags to label the images. The HPM framework makes use of the tag-image association matrix (TIAM). However, since the number of images is usually very large and user-provided tags are diverse, TIAM is very sparse, thus making it difficult to reliably estimate tag-to-tag co-occurrence probabilities. We developed a collaborative filtering method based on nonnegative matrix factorization (NMF) for tackling this data sparsity issue. Also, an L-1 norm kernel method is used to estimate the correlations between image features and semantic concepts. The effectiveness of the proposed approach has been evaluated using three databases containing 5,000 images with 371 tags, 31,695 images with 5,587 tags, and 269,648 images with 5,018 tags, respectively.
AB - The increasing availability of large quantities of user contributed images with labels has provided opportunities to develop automatic tools to tag images to facilitate image search and retrieval. In this paper, we present a novel hybrid probabilistic model (HPM) which integrates low-level image features and high-level user provided tags to automatically tag images. For images without any tags, HPM predicts new tags based solely on the low-level image features. For images with user provided tags, HPM jointly exploits both the image features and the tags in a unified probabilistic framework to recommend additional tags to label the images. The HPM framework makes use of the tag-image association matrix (TIAM). However, since the number of images is usually very large and user-provided tags are diverse, TIAM is very sparse, thus making it difficult to reliably estimate tag-to-tag co-occurrence probabilities. We developed a collaborative filtering method based on nonnegative matrix factorization (NMF) for tackling this data sparsity issue. Also, an L-1 norm kernel method is used to estimate the correlations between image features and semantic concepts. The effectiveness of the proposed approach has been evaluated using three databases containing 5,000 images with 371 tags, 31,695 images with 5,587 tags, and 269,648 images with 5,018 tags, respectively.
KW - Automatic image tagging
KW - collaborative filtering
KW - feature integration
KW - kernel density estimation
KW - nonnegative matrix factorization
UR - http://www.scopus.com/inward/record.url?scp=85027947980&partnerID=8YFLogxK
U2 - 10.1109/TPAMI.2010.204
DO - 10.1109/TPAMI.2010.204
M3 - Journal article
AN - SCOPUS:85027947980
SN - 0162-8828
VL - 33
SP - 1281
EP - 1294
JO - IEEE Transactions on Pattern Analysis and Machine Intelligence
JF - IEEE Transactions on Pattern Analysis and Machine Intelligence
IS - 7
M1 - 5639019
ER -