Indian Sign Language Recognition using Fine-tuned Deep Transfer Learning Model

Chandra Mani Sharma, Kapil Tomar, Ram Krishn Mishra, Vijayaraghavan M. Chariar

2021

Abstract

The World Health Organization (WHO) estimates that over 5% of the world population suffers from hearing impairment. There are over 18 million hearing-impaired people living in India. In some cases, deafness comes from birth, hampering the speech learning capabilities of a child. Therefore, for this type of population, it is difficult to use spoken languages as a medium of communication. Sign languages come to the rescue of such people, providing a medium of expression and communication. However, it is difficult to decode sign language for other people who do not understand it. Computer vision and machine learning may play an important role in understanding what is said using sign language. The Indian sign language (ISL) is very popular in India and in many neighboring countries. It has millions of users. The paper presents a deep convolutional neural network (DCNN) model to recognize various symbols in ISL, belonging to 35 classes. These classes contain cropped images of hand gestures. Unlike other feature selection-based methods, DCNN has the advantage of automatic feature extraction during training. It is called end-to-end learning. A light weight transfer learning architecture makes the model train very fast, giving an accuracy of 100%. Further, a web-based system has been developed that can easily decode these symbols. Experimental results show that the model can classify Indian sign language symbols with accuracy and speed, ideal for real-time applications.

Paper Citation

in Harvard Style

Sharma C., Tomar K., Mishra R. and Chariar V. (2021). Indian Sign Language Recognition using Fine-tuned Deep Transfer Learning Model. In Proceedings of the 1st International Conference on Innovation in Computer and Information Science - Volume 1: ICICIS, ISBN 978-989-758-577-7, pages 63-68. DOI: 10.5220/0010790300003167

in Bibtex Style

@conference{icicis21,
author={Chandra Mani Sharma and Kapil Tomar and Ram Krishn Mishra and Vijayaraghavan M. Chariar},
title={Indian Sign Language Recognition using Fine-tuned Deep Transfer Learning Model},
booktitle={Proceedings of the 1st International Conference on Innovation in Computer and Information Science - Volume 1: ICICIS,},
year={2021},
pages={63-68},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0010790300003167},
isbn={978-989-758-577-7},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 1st International Conference on Innovation in Computer and Information Science - Volume 1: ICICIS,
TI - Indian Sign Language Recognition using Fine-tuned Deep Transfer Learning Model
SN - 978-989-758-577-7
AU - Sharma C.
AU - Tomar K.
AU - Mishra R.
AU - Chariar V.
PY - 2021
SP - 63
EP - 68
DO - 10.5220/0010790300003167