Fine-grained Detection of Academic Emotions with Spatial Temporal Graph Attention Networks using Facial Landmarks

Hua Fwa

2022

Abstract

With the incidence of the Covid-19 pandemic, institutions have adopted online learning as the main lesson delivery channel. A common criticism of online learning is that sensing of learners’ affective states such as engagement is lacking which degrades the quality of teaching. In this study, we propose automatic sensing of learners’ affective states in an online setting with web cameras capturing their facial landmarks and head poses. We postulate that the sparsely connected facial landmarks can be modelled using a Graph Neural Network. Using the publicly available in the wild DAiSEE dataset, we modelled both the spatial and temporal dimensions of the facial videos with a deep learning architecture consisting of Graph Attention Networks and Gated Recurrent Units. The ablation study confirmed that the differencing of consecutive frames of facial landmarks and the addition of head poses enhance the detection performance. The results further demonstrated that the model performed well in comparison with other models and more importantly, is suited for implementation on mobile devices with its low computational requirements.

Paper Citation

in Harvard Style

Fwa H. (2022). Fine-grained Detection of Academic Emotions with Spatial Temporal Graph Attention Networks using Facial Landmarks. In Proceedings of the 14th International Conference on Computer Supported Education - Volume 2: CSEDU, ISBN 978-989-758-562-3, pages 27-34. DOI: 10.5220/0010921200003182

in Bibtex Style

@conference{csedu22,
author={Hua Fwa},
title={Fine-grained Detection of Academic Emotions with Spatial Temporal Graph Attention Networks using Facial Landmarks},
booktitle={Proceedings of the 14th International Conference on Computer Supported Education - Volume 2: CSEDU,},
year={2022},
pages={27-34},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0010921200003182},
isbn={978-989-758-562-3},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 14th International Conference on Computer Supported Education - Volume 2: CSEDU,
TI - Fine-grained Detection of Academic Emotions with Spatial Temporal Graph Attention Networks using Facial Landmarks
SN - 978-989-758-562-3
AU - Fwa H.
PY - 2022
SP - 27
EP - 34
DO - 10.5220/0010921200003182