Facial Expressions Animation in Sign Language based on Spatio-temporal Centroid

Diego Gonçalves, Maria Baranauskas, Julio Reis, Eduardo Todt

2020

Abstract

Systems that use virtual environments with avatars for information communication are of fundamental importance in contemporary life. They are even more relevant in the context of supporting sign language communication for accessibility purposes. Although facial expressions provide message context and define part of the information transmitted, e.g., irony or sarcasm, facial expressions are usually considered as a static background feature in a primarily gestural language in computational systems. This article proposes a novel parametric model of facial expression synthesis through a 3D avatar representing complex facial expressions leveraging emotion context. Our technique explores interpolation of the base expressions in the geometric animation through centroids control and Spatio-temporal data. The proposed method automatically generates complex facial expressions with controllers that use region parameterization as in manual models used for sign language representation. Our approach to the generation of facial expressions adds emotion to representation, which is a determining factor in defining the tone of a message. This work contributes with the definition of non-manual markers for Sign Languages 3D Avatar and the refinement of the synthesized message in sign languages, proposing a complete model for facial parameters and synthesis using geometric centroid regions interpolation. A dataset with facial expressions was generated using the proposed model and validated using machine learning algorithms. In addition, evaluations conducted with the deaf community showed a positive acceptance of the facial expressions and synthesized emotions.

Paper Citation

in Harvard Style

Gonçalves D., Baranauskas M., Reis J. and Todt E. (2020). Facial Expressions Animation in Sign Language based on Spatio-temporal Centroid.In Proceedings of the 22nd International Conference on Enterprise Information Systems - Volume 2: ICEIS, ISBN 978-989-758-423-7, pages 463-475. DOI: 10.5220/0009344404630475

in Bibtex Style

@conference{iceis20,
author={Diego Gonçalves and Maria Baranauskas and Julio Reis and Eduardo Todt},
title={Facial Expressions Animation in Sign Language based on Spatio-temporal Centroid},
booktitle={Proceedings of the 22nd International Conference on Enterprise Information Systems - Volume 2: ICEIS,},
year={2020},
pages={463-475},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0009344404630475},
isbn={978-989-758-423-7},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 22nd International Conference on Enterprise Information Systems - Volume 2: ICEIS,
TI - Facial Expressions Animation in Sign Language based on Spatio-temporal Centroid
SN - 978-989-758-423-7
AU - Gonçalves D.
AU - Baranauskas M.
AU - Reis J.
AU - Todt E.
PY - 2020
SP - 463
EP - 475
DO - 10.5220/0009344404630475