Maximum weighted likelihood via rival penalized EM for density mixture clustering with automatic model selection

Yiu Ming CHEUNG*

*Corresponding author for this work

Research output: Contribution to journalJournal articlepeer-review

82 Citations (Scopus)


Expectation-Maximization (EM) algorithm [10] has been extensively used in density mixture clustering problems, but it is unable to perform model selection automatically. This paper, therefore, proposes to learn the model parameters via maximizing a weighted likelihood. Under a specific weight design, we give out a Rival Penalized Expectation-Maximization (RPEM) algorithm, which makes the components in a density mixture compete each other at each time step. Not only are the associated parameters of the winner updated to adapt to an input, but also all rivals' parameters are penalized with the strength proportional to the corresponding posterior density probabilities. Compared to the EM algorithm [10], the RPEM is able to fade out the redundant densities from a density mixture during the learning process. Hence, it can automatically select an appropriate number of densities in density mixture clustering. We experimentally demonstrate its outstanding performance on Gaussian mixtures and color image segmentation problem. Moreover, a simplified version of RPEM generalizes our recently proposed RPCCL algorithm [8] so that it is applicable to elliptical clusters as well with any input proportion. Compared to the existing heuristic RPCL [25] and its variants, this generalized RPCCL (G-RPCCL) circumvents the difficult preselection of the so-called delearning rate. Additionally, a special setting of the G-RPCCL not only degenerates to RPCL and its Type A variant, but also gives a guidance to choose an appropriate delearning rate for them. Subsequently, we propose a stochastic version of RPCL and its Type A variant, respectively, in which the difficult selection problem of delearning rate has been novelly circumvented. The experiments show the promising results of this stochastic implementation. 2005 IEEE.

Original languageEnglish
Pages (from-to)750-761
Number of pages12
JournalIEEE Transactions on Knowledge and Data Engineering
Issue number6
Publication statusPublished - Jun 2005

Scopus Subject Areas

  • Information Systems
  • Computer Science Applications
  • Computational Theory and Mathematics

User-Defined Keywords

  • Cluster number
  • Generalized rival penalization controlled competitive learning
  • Maximum weighted likelihood
  • Rival penalized expectation-maximization algorithm
  • Stochastic implementation


Dive into the research topics of 'Maximum weighted likelihood via rival penalized EM for density mixture clustering with automatic model selection'. Together they form a unique fingerprint.

Cite this