Emotion Transfer for 3D Hand Motion using StarGAN

Jacky C.P. Chan; Ana Sabina Irimia; Edmond S.L. Ho

doi:10.2312/cgvc.20201146

Emotion Transfer for 3D Hand Motion using StarGAN

Jacky C.P. Chan, Ana Sabina Irimia, Edmond S.L. Ho^*

^*Corresponding author for this work

Department of Computer Science

Research output: Chapter in book/report/conference proceeding › Conference proceeding › peer-review

2 Citations (Scopus)

Abstract

In this paper, we propose a new data-driven framework for 3D hand motion emotion transfer. Specifically, we first capture high-quality hand motion using VR gloves. The hand motion data is then annotated with the emotion type and converted to images to facilitate the motion synthesis process and the new dataset will be available to the public. To the best of our knowledge, this is the first public dataset with annotated hand motions. We further formulate the emotion transfer for 3D hand motion as an Image-to-Image translation problem, and it is done by adapting the StarGAN framework. Our new framework is able to synthesize new motions, given target emotion type and an unseen input motion. Experimental results show that our framework can produce high quality and consistent hand motions.

Original language	English
Title of host publication	Computer Graphics and Visual Computing, CGVC 2020 - Proceedings
Editors	Panagiotis D. Ritsos, Kai Xu
Publisher	The Eurographics Association
Pages	19-26
Number of pages	8
ISBN (Electronic)	9783038681229
DOIs	https://doi.org/10.2312/cgvc.20201146
Publication status	Published - 10 Sept 2020
Event	2020 Computer Graphics and Visual Computing, CGVC 2020 - Virtual, London, United Kingdom Duration: 10 Sept 2020 → 11 Sept 2020 https://diglib.eg.org/handle/10.2312/2632929

Publication series

Name	Computer Graphics and Visual Computing, CGVC - Proceedings

Conference

Conference	2020 Computer Graphics and Visual Computing, CGVC 2020
Country/Territory	United Kingdom
City	London
Period	10/09/20 → 11/09/20
Internet address	https://diglib.eg.org/handle/10.2312/2632929

Scopus Subject Areas

Computer Vision and Pattern Recognition
Artificial Intelligence
Computer Graphics and Computer-Aided Design
Computer Science Applications

User-Defined Keywords

Emotion
Generative adversarial network
Hand animation
Motion capture
Style transfer

Access to Document

10.2312/cgvc.20201146

Cite this

@inproceedings{7861af160a3246c38e0386cc3204be8a,

title = "Emotion Transfer for 3D Hand Motion using StarGAN",

abstract = "In this paper, we propose a new data-driven framework for 3D hand motion emotion transfer. Specifically, we first capture high-quality hand motion using VR gloves. The hand motion data is then annotated with the emotion type and converted to images to facilitate the motion synthesis process and the new dataset will be available to the public. To the best of our knowledge, this is the first public dataset with annotated hand motions. We further formulate the emotion transfer for 3D hand motion as an Image-to-Image translation problem, and it is done by adapting the StarGAN framework. Our new framework is able to synthesize new motions, given target emotion type and an unseen input motion. Experimental results show that our framework can produce high quality and consistent hand motions.",

keywords = "Emotion, Generative adversarial network, Hand animation, Motion capture, Style transfer",

author = "Chan, {Jacky C.P.} and Irimia, {Ana Sabina} and Ho, {Edmond S.L.}",

note = "We gratefully acknowledge the support of NVIDIA Corporation with the donation of the Titan Xp GPU used for this research. Publisher Copyright: {\textcopyright} 2020 The Author(s) Eurographics Proceedings {\textcopyright} 2020 The Eurographics Association.; 2020 Computer Graphics and Visual Computing, CGVC 2020 ; Conference date: 10-09-2020 Through 11-09-2020",

year = "2020",

month = sep,

day = "10",

doi = "10.2312/cgvc.20201146",

language = "English",

series = "Computer Graphics and Visual Computing, CGVC - Proceedings",

publisher = "The Eurographics Association",

pages = "19--26",

editor = "Ritsos, {Panagiotis D.} and Kai Xu",

booktitle = "Computer Graphics and Visual Computing, CGVC 2020 - Proceedings",

url = "https://diglib.eg.org/handle/10.2312/2632929",

}

Chan, JCP, Irimia, AS & Ho, ESL 2020, Emotion Transfer for 3D Hand Motion using StarGAN. in PD Ritsos & K Xu (eds), Computer Graphics and Visual Computing, CGVC 2020 - Proceedings. Computer Graphics and Visual Computing, CGVC - Proceedings, The Eurographics Association, pp. 19-26, 2020 Computer Graphics and Visual Computing, CGVC 2020, London, United Kingdom, 10/09/20. https://doi.org/10.2312/cgvc.20201146

Emotion Transfer for 3D Hand Motion using StarGAN. / Chan, Jacky C.P.; Irimia, Ana Sabina; Ho, Edmond S.L.
Computer Graphics and Visual Computing, CGVC 2020 - Proceedings. ed. / Panagiotis D. Ritsos; Kai Xu. The Eurographics Association, 2020. p. 19-26 (Computer Graphics and Visual Computing, CGVC - Proceedings).

Research output: Chapter in book/report/conference proceeding › Conference proceeding › peer-review

TY - GEN

T1 - Emotion Transfer for 3D Hand Motion using StarGAN

AU - Chan, Jacky C.P.

AU - Irimia, Ana Sabina

AU - Ho, Edmond S.L.

N1 - We gratefully acknowledge the support of NVIDIA Corporation with the donation of the Titan Xp GPU used for this research. Publisher Copyright: © 2020 The Author(s) Eurographics Proceedings © 2020 The Eurographics Association.

PY - 2020/9/10

Y1 - 2020/9/10

N2 - In this paper, we propose a new data-driven framework for 3D hand motion emotion transfer. Specifically, we first capture high-quality hand motion using VR gloves. The hand motion data is then annotated with the emotion type and converted to images to facilitate the motion synthesis process and the new dataset will be available to the public. To the best of our knowledge, this is the first public dataset with annotated hand motions. We further formulate the emotion transfer for 3D hand motion as an Image-to-Image translation problem, and it is done by adapting the StarGAN framework. Our new framework is able to synthesize new motions, given target emotion type and an unseen input motion. Experimental results show that our framework can produce high quality and consistent hand motions.

AB - In this paper, we propose a new data-driven framework for 3D hand motion emotion transfer. Specifically, we first capture high-quality hand motion using VR gloves. The hand motion data is then annotated with the emotion type and converted to images to facilitate the motion synthesis process and the new dataset will be available to the public. To the best of our knowledge, this is the first public dataset with annotated hand motions. We further formulate the emotion transfer for 3D hand motion as an Image-to-Image translation problem, and it is done by adapting the StarGAN framework. Our new framework is able to synthesize new motions, given target emotion type and an unseen input motion. Experimental results show that our framework can produce high quality and consistent hand motions.

KW - Emotion

KW - Generative adversarial network

KW - Hand animation

KW - Motion capture

KW - Style transfer

UR - http://www.scopus.com/inward/record.url?scp=85103873752&partnerID=8YFLogxK

U2 - 10.2312/cgvc.20201146

DO - 10.2312/cgvc.20201146

M3 - Conference proceeding

AN - SCOPUS:85103873752

T3 - Computer Graphics and Visual Computing, CGVC - Proceedings

SP - 19

EP - 26

BT - Computer Graphics and Visual Computing, CGVC 2020 - Proceedings

A2 - Ritsos, Panagiotis D.

A2 - Xu, Kai

PB - The Eurographics Association

T2 - 2020 Computer Graphics and Visual Computing, CGVC 2020

Y2 - 10 September 2020 through 11 September 2020

ER -

Emotion Transfer for 3D Hand Motion using StarGAN

Abstract

Publication series

Conference

Scopus Subject Areas

User-Defined Keywords

Access to Document

Other files and links

Cite this