Real-time Arabic Sign Language Recognition based on YOLOv5

Sabrina Aiouez, Anis Hamitouche, Mohamed Belmadoui, Khadidja Belattar, Feryel Souami

2022

Abstract

Sign language is the most common communication mode of deaf and mute community. However, hearing people do not generally know this language. So, an automatic sign langage recognition is required to facilitate and better understand interactions with such people. However, one of the main challlenges in this field is the real-time sign recognition. That is why, deep learning-based object detection models can be used to improve the recognition performance (in terms of time and accuracy). In this paper, we present a real-time system that allows the detection and recognition of hand postures intended for the Arabic sign language alphabet. To do so, we constructed a dataset of 28 Arabic signs containing around 15,000 images acquired with different sizes of hands, lighting conditions, backgrounds and with/without accessories. We then trained and tested different variants of YOLOv5 on the constructed dataset. The conducted experiments on our ArSL real-time recognition system show that the adapted YOLOv5 is more effective than Faster R-CNN detector.

Paper Citation

in Harvard Style

Aiouez S., Hamitouche A., Belmadoui M., Belattar K. and Souami F. (2022). Real-time Arabic Sign Language Recognition based on YOLOv5. In Proceedings of the 2nd International Conference on Image Processing and Vision Engineering - Volume 1: IMPROVE, ISBN 978-989-758-563-0, pages 17-25. DOI: 10.5220/0010979300003209

in Bibtex Style

@conference{improve22,
author={Sabrina Aiouez and Anis Hamitouche and Mohamed Belmadoui and Khadidja Belattar and Feryel Souami},
title={Real-time Arabic Sign Language Recognition based on YOLOv5},
booktitle={Proceedings of the 2nd International Conference on Image Processing and Vision Engineering - Volume 1: IMPROVE,},
year={2022},
pages={17-25},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0010979300003209},
isbn={978-989-758-563-0},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 2nd International Conference on Image Processing and Vision Engineering - Volume 1: IMPROVE,
TI - Real-time Arabic Sign Language Recognition based on YOLOv5
SN - 978-989-758-563-0
AU - Aiouez S.
AU - Hamitouche A.
AU - Belmadoui M.
AU - Belattar K.
AU - Souami F.
PY - 2022
SP - 17
EP - 25
DO - 10.5220/0010979300003209