Learning topic models by belief propagation

Jia Zeng*, Kwok Wai Cheung, Jiming Liu

*Corresponding author for this work

Research output: Contribution to journalJournal articlepeer-review

48 Citations (Scopus)


Latent Dirichlet allocation (LDA) is an important hierarchical Bayesian model for probabilistic topic modeling, which attracts worldwide interest and touches on many important applications in text mining, computer vision and computational biology. This paper represents the collapsed LDA as a factor graph, which enables the classic loopy belief propagation (BP) algorithm for approximate inference and parameter estimation. Although two commonly used approximate inference methods, such as variational Bayes (VB) and collapsed Gibbs sampling (GS), have gained great success in learning LDA, the proposed BP is competitive in both speed and accuracy, as validated by encouraging experimental results on four large-scale document datasets. Furthermore, the BP algorithm has the potential to become a generic scheme for learning variants of LDA-based topic models in the collapsed space. To this end, we show how to learn two typical variants of LDA-based topic models, such as author-topic models (ATM) and relational topic models (RTM), using BP based on the factor graph representations.

Original languageEnglish
Article number6291721
Pages (from-to)1121-1134
Number of pages14
JournalIEEE Transactions on Pattern Analysis and Machine Intelligence
Issue number5
Publication statusPublished - May 2013

Scopus Subject Areas

  • Software
  • Computer Vision and Pattern Recognition
  • Computational Theory and Mathematics
  • Artificial Intelligence
  • Applied Mathematics

User-Defined Keywords

  • Bayesian networks
  • belief propagation
  • factor graph
  • Gibbs sampling
  • hierarchical Bayesian models
  • Latent Dirichlet allocation
  • Markov random fields
  • message passing
  • topic models
  • variational Bayes


Dive into the research topics of 'Learning topic models by belief propagation'. Together they form a unique fingerprint.

Cite this