Internal State Estimation Based on Facial Images with Individual Feature Separation and Mixup Augmentation

Ayaka Asaeda, Noriko Takemura

2025

Abstract

In recent years, the opportunity for e-learning and remote work has increased due to the impact of the COVID-19 pandemic. However, issues such as drowsiness and decreased concentration among learners have become apparent, increasing the need to estimate the internal state of learners. Since facial expressions reflect internal states well, they are often utilized in research on state estimation. However, individual differences in facial structure and expression methods can influence the accuracy of these estimations. This study aims to estimate ambiguous internal states such as drowsiness and concentration by considering individual differences based on the Deviation Learning Network (DLN). Such internal states exhibit very subtle and ambiguous changes in facial expressions, making them more difficult to estimate compared to basic emotions. Therefore, this study proposes a model that uses mixup, which is one form of data augmentation, to account for subtle differences in expressions between classes. In the evaluation experiments, facial images of learners during e-learning will be used to estimate their arousal levels in three categories: Asleep, Drowsy, and Awake.

Paper Citation

in Harvard Style

Asaeda A. and Takemura N. (2025). Internal State Estimation Based on Facial Images with Individual Feature Separation and Mixup Augmentation. In Proceedings of the 20th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 2: VISAPP; ISBN 978-989-758-728-3, SciTePress, pages 911-918. DOI: 10.5220/0013161900003912

in Bibtex Style

@conference{visapp25,
author={Ayaka Asaeda and Noriko Takemura},
title={Internal State Estimation Based on Facial Images with Individual Feature Separation and Mixup Augmentation},
booktitle={Proceedings of the 20th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 2: VISAPP},
year={2025},
pages={911-918},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0013161900003912},
isbn={978-989-758-728-3},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 20th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 2: VISAPP
TI - Internal State Estimation Based on Facial Images with Individual Feature Separation and Mixup Augmentation
SN - 978-989-758-728-3
AU - Asaeda A.
AU - Takemura N.
PY - 2025
SP - 911
EP - 918
DO - 10.5220/0013161900003912
PB - SciTePress