Exploring Interpretability in Detecting Frustration and Boredom of Students using Valence-Arousal Space

Abstract

Abstract Recognizing and addressing the complex emotional experiences of students, like frustration and boredom, is key to creating a classroom environment that’s both stimulating and conducive to learning. Traditional methods of emotion detection, which often rely on discrete emotion models, have proven insufficient in capturing the complexity and subtlety of these emotional experiences within academic settings. This research aims to bridge this gap by utilizing computational methods based on the valence and arousal emotions approach, providing a more inclusive view of emotional recognition. The study is further enhanced by the incorporation of Explainable Artificial Intelligence (XAI) techniques and principles of affective computing, which aim to improve the accuracy of emotion detection and prioritize the detection models’ transparency and interpretability. This innovative approach enables the identification of specific facial features and expressions indicative of frustration and boredom, providing a deeper insight into the emotional landscape of students as they engage with educational content. This study utilizes two advanced transfer learning models, VGG16 and MobileNet, trained on the Affectnet dataset to perform regression tasks to predict the emotions of frustration and boredom. The interpretability of these predictions is explored using the LIME method of explainable AI, which helps clarify how the models reach their conclusions. The MobileNet variant model performs well across various regression metrics such as Mean Squared Error (MSE) and Mean Absolute Error (MAE), as well as correlation metrics like Pearson Correlation Coefficient (PCC) and Concordance Correlation Coefficient (CCC). Additionally, the LIME explainer has enhanced the model’s interpretability, which provides insights into the model’s decision-making process. The integration of valence and arousal theory with XAI and affective computing techniques underscores a significant advancement in educational technology, providing educators with actionable insights to tailor interventions effectively. Keywords: Student Engagement, Emotion Detection, Computational Methods, Valence and Arousal Model, Explainable Artificial Intelligence (XAI), Academic Emotions, Frustration and Boredom, Affective Computing Technology, Facial Feature Analysis, Interpretability