Detecting learners’ frustration based on their performance in game-based learning activities

Maya Israel*, Tongxi Liu

*Corresponding author for this work

Research output: Contribution to conferenceConference paperpeer-review


Researchers use various models to detect learners’ emotional states in online instructional settings. Recent work has suggested that detecting emotions is more effective when both physical sensors and interaction logs are used (DeFalco et al., 2018). However, these methods can be invasive and can require costly equipment (Epp et al., 2011). Detecting emotions solely from the interactions between the student and learning system may be more feasible to deploy at scale (Botelho et al., 2017), although these approaches also have limitations. Previous work shows that students’ emotions have been found to correlate with performance (Craig et al., 2004; Pardos et al., 2014). For example, Arroyo’ s (2009) research found that students’ emotions were influenced by their experience with previous problems.
Frustration is a common emotional state during learning, especially when learners experience failure (Bosch & D’Mello, 2017). Additionally, students may transition between emotional states, toggling between engagement, confusion, frustration, and boredom. Thus, learners experience different states of equilibrium and disequilibrium as they engage in problem solving. In this paper, we proposed a model to detect states of frustration when learners engage in game play within Zoombinis, a puzzle-based game that includes scaffolded problem-solving for young learners (Rowe et al., 2017).

Our proposed model for predicting times during gameplay when a learner may experience frustration relied on the interaction logs. Our model considers attempts, time, and outcomes as well as the impact of sequence of attempts (weight of each attempt), as frustration induced by incorrect attempts may fade over time (Walker et al., 1997). We tested this model using one puzzle called Pizza Pass, wherein learners may attempt to solve problems multiple times.

Changes in frustration/cognitive disequilibrium over time were calculated by our model. In one example, the learner played Pizza Pass with three different levels of difficulty. Her frustration increased over time due to the cumulative numbers of incorrect attempts from level 1 to the middle of level 2. Then, frustration level remained stable after she had a correct attempt (seen between the blue line to the end of level 2 in Figure 2). From the beginning of level 3 to the end of game play, frustration seemed lower even though level 3 in the game is more difficult. In this case, this learner’s frustration might transfer to different emotional states, such as anxiety or other negative emotions as the puzzle became harder and without any proper scaffolding.
We acknowledge that studying complex emotions using log data is an oversimplification. With that said, if learning systems can be cued to when students transition between emotional states, those systems can become more responsive to learners’ needs. Our current work focuses on improving our model and examining different emotional states (e.g., frustration, boredom, confusion) as well as ways to triangulate these approaches with other measures including eye tracking and video analysis.


Dive into the research topics of 'Detecting learners’ frustration based on their performance in game-based learning activities'. Together they form a unique fingerprint.

Cite this