Authors:
Anurag Jha
;
Kabita Choudhary
and
Sujala Shetty
Affiliation:
Birla Institute of Technology and Science, Pilani, Dubai Campus, DIAC Dubai 34055, U.A.E.
Keyword(s):
Natural Language Processing (NLP), Natural Language Generation (NLG), Bidirectional Auto-Regressive Transformers (BART), Multilingual Bidirectional Auto-Regressive Transformers (mBART), Signing Gesture Markup Language (SiGML), HamNoSys, Indian Sign Language, Transformer-Based Highlights Extractor (THExt), Automatic Text Summarization (ATS), Machine Translation (MT).
Abstract:
There have been multiple text conversions emerging with time but there has hardly been any work in the field of sign language. Even in the field of sign language multiple methods have been proposed to convert it into text via image detection, but due to the rarity of sign language corpus not much work has been put into text or speech to sign language. The proposed project intends to create a translation model to convert text or audio into sign language with its designated grammar. The process includes translation of any language to English followed by summarization of a big article or text, removal of stopwords, reordering the grammar form and stemming words into their root form. The translation is performed by mBART model, summarization is performed using BART model, conversion into animation is done via mapping words into a dictionary and replacing words by letters for unknown words. The paper uses HamNoSys (Regina et al., 1989), SiGML, BART, mBART and NLP to form the translation s
ystem. The paper aims to establish better means of communication with the deaf, dumb and people with hearing issues.
(More)