Abstract
Embodied Pedagogical Agents (EPAs) possess significant potential for using multimodal interactions to effectively convey social and educational information. While previous studies have examined the impact of EPAs on learning performance, the incorporation of meaningful gestures into EPAs remains relatively unexplored. This study employs a case study approach and systemic functional-multimodal discourse analysis to investigate the multimodal interactions of a human lecturer during teaching and explores the relationship between in-class meaningful gestures and corresponding words. We analyze the alignment between gestures, words, and contextual meanings, with a specific focus on personal pronouns, adjectives, adverbs, and verbs. The findings contribute to the understanding of lecturers' multimodal orchestration in classroom settings. Additionally, we provide a preliminary design guideline for EPAs that integrates meaningful word-synchronized gestures, using multimodal information to enhance students' embodied learning experiences. This research contributes to the advancement of embodied pedagogical agent design and the broader field of multimodal interaction.
Original language | English |
---|---|
Title of host publication | CSCW 2023 Companion |
Subtitle of host publication | Conference Companion Publication of the 2023 Computer Supported Cooperative Work and Social Computing |
Editors | Morgan Ames, Susan Fussell, Eric Gilbert, Vera Liao, Xiaojuan Ma, Xinru Page, Mark Rouncefield, Vivek Singh, Pamela Wisniewski |
Place of Publication | New York |
Publisher | Association for Computing Machinery (ACM) |
Pages | 39-44 |
Number of pages | 6 |
ISBN (Print) | 9798400701290 |
DOIs | |
Publication status | Published - 14 Oct 2023 |
Event | 26th ACM Conference on Computer-Supported Cooperative Work and Social Computing, CSCW 2023 - Minneapolis, United States Duration: 14 Oct 2023 → 18 Oct 2023 https://dl.acm.org/doi/proceedings/10.1145/3584931 (Conference proceedings) |
Publication series
Name | Proceedings of the ACM Conference on Computer Supported Cooperative Work, CSCW |
---|---|
Publisher | Association for Computing Machinery (ACM) |
Conference
Conference | 26th ACM Conference on Computer-Supported Cooperative Work and Social Computing, CSCW 2023 |
---|---|
Country/Territory | United States |
City | Minneapolis |
Period | 14/10/23 → 18/10/23 |
Internet address |
|
Scopus Subject Areas
- Software
- Computer Networks and Communications
- Human-Computer Interaction
User-Defined Keywords
- Gesture
- Human behavior
- Multimodal interaction
- Pedagogical agent
- Word synchronized