TY - JOUR
T1 - Automatic Video Object Segmentation Based on Visual and Motion Saliency
AU - Peng, Qinmu
AU - Cheung, Yiu Ming
N1 - Funding Information:
Manuscript received December 29, 2017; revised April 15, 2019; accepted May 6, 2019. Date of publication May 23, 2019; date of current version November 19, 2019. This work was supported in part by the National Natural Science Foundation of China under Grants 61672444 and 61272366, in part by the Faculty Research Grant of Hong Kong Baptist University (HKBU) under Project FRG2/17-18/082, in part by the KTO Grant of HKBU under Project MPCF-004-2017/18, and in part by the SZSTI under Grants: JCYJ20160531194006833 and JCYJ20180305180637611. The associate editor coordinating the review of this manuscript and approving it for publication was Prof. Xilin Chen. (Corresponding author: Yiu-Ming Cheung.) Q. Peng is with the School of Electronic Information and Communications, Huazhong University of Science and Technology, Wuhan 430074, China, Department of Computer Science, Hong Kong Baptist University, Hong Kong, China, and also with the Shenzhen Research Institute of Huazhong University of Science and Technology, Shenzhen 518055, China (e-mail: [email protected]).
PY - 2019/12
Y1 - 2019/12
N2 - We present an approach to extract the salient object automatically in videos. Given an unannotated video sequence, the proposed method first computes the visual saliency to identify object-like regions in each frame based on the proposed weighted multiple manifold ranking algorithm. We then compute motion cues to estimate the motion saliency and localization prior. Finally, adopting a new energy function, we estimate a superpixel-level object labeling across all frames, where 1) the data term depends on the visual saliency and localization prior, and 2) the smoothness term depends on the constraints in time and space. Compared to the existing counterparts, the proposed approach automatically segments the persistent foreground object meanwhile preserving the potential shape. Experiments show its promising results on the challenging benchmark videos in comparison with the existing counterparts.
AB - We present an approach to extract the salient object automatically in videos. Given an unannotated video sequence, the proposed method first computes the visual saliency to identify object-like regions in each frame based on the proposed weighted multiple manifold ranking algorithm. We then compute motion cues to estimate the motion saliency and localization prior. Finally, adopting a new energy function, we estimate a superpixel-level object labeling across all frames, where 1) the data term depends on the visual saliency and localization prior, and 2) the smoothness term depends on the constraints in time and space. Compared to the existing counterparts, the proposed approach automatically segments the persistent foreground object meanwhile preserving the potential shape. Experiments show its promising results on the challenging benchmark videos in comparison with the existing counterparts.
KW - graph model
KW - manifold ranking
KW - Object segmentation
KW - visual saliency
UR - http://www.scopus.com/inward/record.url?scp=85075647051&partnerID=8YFLogxK
U2 - 10.1109/TMM.2019.2918730
DO - 10.1109/TMM.2019.2918730
M3 - Journal article
AN - SCOPUS:85075647051
SN - 1520-9210
VL - 21
SP - 3083
EP - 3094
JO - IEEE Transactions on Multimedia
JF - IEEE Transactions on Multimedia
IS - 12
M1 - 8721129
ER -