Enhancing Gesture Recognition for Sign Language Interpretation in Challenging Environment Conditions: A Deep Learning Approach

Domenico Amalfitano, Vincenzo D’Angelo, Antonio M. Rinaldi, Cristiano Russo, Cristian Tommasino

2023

Abstract

Gesture recognition systems have gained popularity as an effective means of communication, leveraging the simplicity and effectiveness of gestures. With the absence of a universal sign language due to regional variations and limited dissemination in schools and media, there is a need for real-time translation systems to bridge the communication gap. The proposed system aims to translate American Sign Language (ASL), the predominant sign language used by deaf communities in real-time in North America, West Africa, and Southeast Asia. The system utilizes SSD Mobilenet FPN architecture, known for its real-time performance on low-power devices, and leverages transfer learning techniques for efficient training. Data augmentation and preprocessing procedures are applied to improve the quality of training data. The system’s detection capability is enhanced by adapting color space conversions, such as RGB to YCbCr and HSV, to improve the segmentation for varying lighting conditions. Experimental results demonstrate the system’s Accessibility and non-invasiveness, achieving high accuracy in recognizing ASL signs.

Paper Citation

in Harvard Style

Amalfitano D., D’Angelo V., M. Rinaldi A., Russo C. and Tommasino C. (2023). Enhancing Gesture Recognition for Sign Language Interpretation in Challenging Environment Conditions: A Deep Learning Approach. In Proceedings of the 15th International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management - Volume 1: KDIR; ISBN 978-989-758-671-2, SciTePress, pages 395-402. DOI: 10.5220/0012209700003598

in Bibtex Style

@conference{kdir23,
author={Domenico Amalfitano and Vincenzo D’Angelo and Antonio M. Rinaldi and Cristiano Russo and Cristian Tommasino},
title={Enhancing Gesture Recognition for Sign Language Interpretation in Challenging Environment Conditions: A Deep Learning Approach},
booktitle={Proceedings of the 15th International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management - Volume 1: KDIR},
year={2023},
pages={395-402},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0012209700003598},
isbn={978-989-758-671-2},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 15th International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management - Volume 1: KDIR
TI - Enhancing Gesture Recognition for Sign Language Interpretation in Challenging Environment Conditions: A Deep Learning Approach
SN - 978-989-758-671-2
AU - Amalfitano D.
AU - D’Angelo V.
AU - M. Rinaldi A.
AU - Russo C.
AU - Tommasino C.
PY - 2023
SP - 395
EP - 402
DO - 10.5220/0012209700003598
PB - SciTePress