On incremental collaborative appearance model and regional particle filtering for lip region tracking

Xin Liu, Yiu Ming CHEUNG*

*Corresponding author for this work

Research output: Contribution to journalJournal articlepeer-review

1 Citation (Scopus)


Lip region tracking is of crucial importance to the better understanding of visual speech in a computer-aided system. This paper presents an efficient lip region tracking approach in virtue of incremental collaborative appearance model and regional particle filtering. Within this approach, we first learn an incremental weighted appearance model (IWAM) through adaptively updating the time-varying mean and eigenbasis by considering the temporal and spatial weights, and then discriminatively exploit an incremental sparse subspace model (ISSM) by considering the occlusions and background clutters. Accordingly, the collaboration of the IWAM and ISSM leads to a more reliable and flexible lip region representation. Subsequently, we propose a regional particle filtering for motion state estimation, by taking scale size and rotation estimation, reducing the computational load and alleviating the tracking drift into consideration. Furthermore, the affinely warped image patches corresponding to the rank-2-optimal states are employed to incrementally update the IWAM and ISSM synchronously. The extensive experiments conducted on different challenging sequences have shown that the improvement from the proposed framework compared to the state-of-the-art systems, are over 15%, 18% and 20%, on reducing the errors of center location, scale size and rotation angle estimations, respectively.

Original languageEnglish
Pages (from-to)63-80
Number of pages18
JournalIntegrated Computer-Aided Engineering
Issue number1
Publication statusPublished - 2017

Scopus Subject Areas

  • Software
  • Theoretical Computer Science
  • Computer Science Applications
  • Computational Theory and Mathematics
  • Artificial Intelligence

User-Defined Keywords

  • collaborative appearance model
  • Lip region tracking
  • rank-2-optimal states
  • regional particle filtering


Dive into the research topics of 'On incremental collaborative appearance model and regional particle filtering for lip region tracking'. Together they form a unique fingerprint.

Cite this