Speech to Text Recognition for Videogame Controlling with Convolutional Neural Networks

Joaquin Aguirre-Peralta, Marek Rivas-Zavala, Willy Ugarte

2023

Abstract

Disability in people is a reality that has always been present throughout humanity and all nations of the planet are immersed in this reality. Being communication and interaction through technology much more important than ever, people with disabilities are the most affected by having a physical gap. There are still few tools that these people can use to interact more easily with different types of hardware, therefore, we want to provide them a playful and medical tool that can adapt to their needs and allow them to interact a little more with the people around them. From this context, we have decided to focus on people with motor disabilities of the upper limbs and based on this, we propose the use of gamification in the NLP (Natural Language Processing) area, developing a videogame consisting of three voice-operated minigames. This work has 4 stages: analysis (benchmarking), design, development and validation. In the first stage, we elaborated a benchmarking of the models. In the second stage, we describe the implementation of CNNs, together with methods such as gamification and NLP for problem solving. In the third stage, the corresponding mini-games which compose the videogame and its characteristics are described. Finally, in the last stage, the application of the videogame was validated with experts in physiotherapy. Our results show that with the training performed, the prediction of words with noise was improved from 43.49% to 74.50% and of words without noise from 63.87% to 96.36%

Paper Citation

in Harvard Style

Aguirre-Peralta J., Rivas-Zavala M. and Ugarte W. (2023). Speech to Text Recognition for Videogame Controlling with Convolutional Neural Networks. In Proceedings of the 12th International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM, ISBN 978-989-758-626-2, pages 948-955. DOI: 10.5220/0011782900003411

in Bibtex Style

@conference{icpram23,
author={Joaquin Aguirre-Peralta and Marek Rivas-Zavala and Willy Ugarte},
title={Speech to Text Recognition for Videogame Controlling with Convolutional Neural Networks},
booktitle={Proceedings of the 12th International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM,},
year={2023},
pages={948-955},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0011782900003411},
isbn={978-989-758-626-2},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 12th International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM,
TI - Speech to Text Recognition for Videogame Controlling with Convolutional Neural Networks
SN - 978-989-758-626-2
AU - Aguirre-Peralta J.
AU - Rivas-Zavala M.
AU - Ugarte W.
PY - 2023
SP - 948
EP - 955
DO - 10.5220/0011782900003411