TY - GEN
T1 - Audio-visual speaker recognition via multi-modal correlated neural networks
AU - Geng, Jiajia
AU - Liu, Xin
AU - Cheung, Yiu Ming
N1 - Publisher Copyright:
© 2016 IEEE.
PY - 2017/1/11
Y1 - 2017/1/11
N2 - Multi-modal speaker recognition has received a lot of attention in recent years due to the growing security demands in real applications. In this paper, we present an efficient audiovisual speaker recognition method by fusing face and audio via the multi-modal correlated neural networks. Within our proposed approach, the facial features learned by convolutional neural networks are compatible with audio features at high-level and the heterogeneous multi-modal features can be learned automatically. Accordingly, we propose a correlated neural networks to fuse the face and audio modalities at different level such that the speaker identity can be well identified. The experimental results have shown that our proposed multi-modal speaker recognition approach can produce better performance than single modality, and the feature-level fusion yields comparative and even better results than the decision-level case.
AB - Multi-modal speaker recognition has received a lot of attention in recent years due to the growing security demands in real applications. In this paper, we present an efficient audiovisual speaker recognition method by fusing face and audio via the multi-modal correlated neural networks. Within our proposed approach, the facial features learned by convolutional neural networks are compatible with audio features at high-level and the heterogeneous multi-modal features can be learned automatically. Accordingly, we propose a correlated neural networks to fuse the face and audio modalities at different level such that the speaker identity can be well identified. The experimental results have shown that our proposed multi-modal speaker recognition approach can produce better performance than single modality, and the feature-level fusion yields comparative and even better results than the decision-level case.
UR - http://www.scopus.com/inward/record.url?scp=85013648143&partnerID=8YFLogxK
U2 - 10.1109/WIW.2016.043
DO - 10.1109/WIW.2016.043
M3 - Conference proceeding
AN - SCOPUS:85013648143
T3 - Proceedings - 2016 IEEE/WIC/ACM International Conference on Web Intelligence Workshops, WIW 2016
SP - 123
EP - 128
BT - Proceedings - 2016 IEEE/WIC/ACM International Conference on Web Intelligence Workshops, WIW 2016
PB - IEEE
T2 - 2016 IEEE/WIC/ACM International Conference on Web Intelligence Workshops, WIW 2016
Y2 - 13 October 2016 through 16 October 2016
ER -