Deep Learning Based Text Translation and Summarization Tool for Hearing Impaired Using Indian Sign Language

Anurag Jha, Kabita Choudhary, Sujala Shetty

2023

Abstract

There have been multiple text conversions emerging with time but there has hardly been any work in the field of sign language. Even in the field of sign language multiple methods have been proposed to convert it into text via image detection, but due to the rarity of sign language corpus not much work has been put into text or speech to sign language. The proposed project intends to create a translation model to convert text or audio into sign language with its designated grammar. The process includes translation of any language to English followed by summarization of a big article or text, removal of stopwords, reordering the grammar form and stemming words into their root form. The translation is performed by mBART model, summarization is performed using BART model, conversion into animation is done via mapping words into a dictionary and replacing words by letters for unknown words. The paper uses HamNoSys (Regina et al., 1989), SiGML, BART, mBART and NLP to form the translation system. The paper aims to establish better means of communication with the deaf, dumb and people with hearing issues.

Paper Citation

in Harvard Style

Jha A., Choudhary K. and Shetty S. (2023). Deep Learning Based Text Translation and Summarization Tool for Hearing Impaired Using Indian Sign Language. In Proceedings of the 12th International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM, ISBN 978-989-758-626-2, pages 426-434. DOI: 10.5220/0011728200003411

in Bibtex Style

@conference{icpram23,
author={Anurag Jha and Kabita Choudhary and Sujala Shetty},
title={Deep Learning Based Text Translation and Summarization Tool for Hearing Impaired Using Indian Sign Language},
booktitle={Proceedings of the 12th International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM,},
year={2023},
pages={426-434},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0011728200003411},
isbn={978-989-758-626-2},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 12th International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM,
TI - Deep Learning Based Text Translation and Summarization Tool for Hearing Impaired Using Indian Sign Language
SN - 978-989-758-626-2
AU - Jha A.
AU - Choudhary K.
AU - Shetty S.
PY - 2023
SP - 426
EP - 434
DO - 10.5220/0011728200003411