A local region based approach to lip tracking

Yiu Ming CHEUNG*, Xin Liu, Xinge You

*Corresponding author for this work

Research output: Contribution to journalJournal articlepeer-review

28 Citations (Scopus)


Lip tracking has played a significant role in a lip reading system. In this paper, we present a local region based approach to lip tracking, which consists of two phases: (i) lip contour extraction for the first lip frame, and followed by (ii) lip tracking in the subsequent lip frames. Initially, we construct a localized color active color model provided that the foreground and background regions around the object are locally different in color space. In the first phase, we find a combined semi-ellipse around the lip as the initial evolving curve and compute the localized energies for curve evolution such that the lip image is separated into lip and non-lip regions. Then, we utilize a 16-point deformable model (Wang et al., 2004 [20]) with geometric constraint to achieve lip contour extraction. In the second phase, we present a dynamic selection of the radius of local regions associated with the extracted lip contour of the previous frame to realize lip tracking. The proposed approach not only adapts to the lip movement, but it is also robust against the appearance of teeth, tongue and black hole. Extensive experiments show the efficiency of the proposed lip tracking algorithm in comparison with the existing methods.

Original languageEnglish
Pages (from-to)3336-3347
Number of pages12
JournalPattern Recognition
Issue number9
Publication statusPublished - Sept 2012

Scopus Subject Areas

  • Software
  • Signal Processing
  • Computer Vision and Pattern Recognition
  • Artificial Intelligence

User-Defined Keywords

  • Deformable model
  • Lip tracking
  • Local region
  • Localized color active contour model
  • Semi-ellipse


Dive into the research topics of 'A local region based approach to lip tracking'. Together they form a unique fingerprint.

Cite this