TY - JOUR
T1 - On incremental collaborative appearance model and regional particle filtering for lip region tracking
AU - Liu, Xin
AU - CHEUNG, Yiu Ming
N1 - Funding Information:
The work described in this paper was supported by the National Science Foundation of China (No. 61672444, 61673185), National Science Foundation of Fujian Province (No. 2017J01112), Promotion Program for Young and Middle-aged Teacher in Science and Technology Research (No. ZQN-PY309) of Huaqiao University, and also supported by the Faculty Research Grant of Hong Kong Baptist University (HKBU): FRG2/16-17/051, the Knowledge Transfer Office Grant of HKBU with Project Code: MPCF-004-2017/18, and by the SZSTI (Grant No. JCYJ20160531194006833).
PY - 2017
Y1 - 2017
N2 - Lip region tracking is of crucial importance to the better understanding of visual speech in a computer-aided system. This paper presents an efficient lip region tracking approach in virtue of incremental collaborative appearance model and regional particle filtering. Within this approach, we first learn an incremental weighted appearance model (IWAM) through adaptively updating the time-varying mean and eigenbasis by considering the temporal and spatial weights, and then discriminatively exploit an incremental sparse subspace model (ISSM) by considering the occlusions and background clutters. Accordingly, the collaboration of the IWAM and ISSM leads to a more reliable and flexible lip region representation. Subsequently, we propose a regional particle filtering for motion state estimation, by taking scale size and rotation estimation, reducing the computational load and alleviating the tracking drift into consideration. Furthermore, the affinely warped image patches corresponding to the rank-2-optimal states are employed to incrementally update the IWAM and ISSM synchronously. The extensive experiments conducted on different challenging sequences have shown that the improvement from the proposed framework compared to the state-of-the-art systems, are over 15%, 18% and 20%, on reducing the errors of center location, scale size and rotation angle estimations, respectively.
AB - Lip region tracking is of crucial importance to the better understanding of visual speech in a computer-aided system. This paper presents an efficient lip region tracking approach in virtue of incremental collaborative appearance model and regional particle filtering. Within this approach, we first learn an incremental weighted appearance model (IWAM) through adaptively updating the time-varying mean and eigenbasis by considering the temporal and spatial weights, and then discriminatively exploit an incremental sparse subspace model (ISSM) by considering the occlusions and background clutters. Accordingly, the collaboration of the IWAM and ISSM leads to a more reliable and flexible lip region representation. Subsequently, we propose a regional particle filtering for motion state estimation, by taking scale size and rotation estimation, reducing the computational load and alleviating the tracking drift into consideration. Furthermore, the affinely warped image patches corresponding to the rank-2-optimal states are employed to incrementally update the IWAM and ISSM synchronously. The extensive experiments conducted on different challenging sequences have shown that the improvement from the proposed framework compared to the state-of-the-art systems, are over 15%, 18% and 20%, on reducing the errors of center location, scale size and rotation angle estimations, respectively.
KW - collaborative appearance model
KW - Lip region tracking
KW - rank-2-optimal states
KW - regional particle filtering
UR - http://www.scopus.com/inward/record.url?scp=85039452714&partnerID=8YFLogxK
U2 - 10.3233/ICA-170557
DO - 10.3233/ICA-170557
M3 - Journal article
AN - SCOPUS:85039452714
SN - 1069-2509
VL - 25
SP - 63
EP - 80
JO - Integrated Computer-Aided Engineering
JF - Integrated Computer-Aided Engineering
IS - 1
ER -