A Revolutionary Real-Time Translation Tool for Text, Audio,

and Sign Language from Images

Paula Escudeiro

, Márcia Campos Gouveia

and Nuno Escudeiro

Games, Interaction and Learning Technologies, ISEP, Polytechnic University of Porto,

Rua Dr. António Bernardino Almeida, Porto, Portugal

Keywords: Communication Technology, Inclusive Technology, Real-Time Translation, Optical Text Recognition,

Accessibility, Sign Language, Figure out Application.

Abstract: This paper introduces an innovative approach to overcoming language barriers in effective communication.

Departing from conventional translation tools, the proposed solution presents a groundbreaking technology

facilitating real-time translation by capturing small text portions within images. The Figure Out application,

a transformative solution, stands out as a pioneer in its field, distinguishing itself by offering unmatched

features. Leveraging optical text recognition from captured images, it excels in translating content into text,

audio, or sign language. Notably, this technology goes beyond conventional translations, making information

visually accessible to deaf individuals.

1 INTRODUCTION

Effectively communicating in a non-native language

can be a demanding and even impossible task. Even

with a basic understanding of a foreign language,

maintaining proficiency can be challenging without

regular use. Over time, vocabulary and grammar tend

to fade, making communication even more complex.

This challenge is particularly pronounced when

bridging the gap between different languages,

especially between deaf and non-deaf individuals

(Zheng, et al., 2020).

Translating between spoken and signed languages

is intricate due to differing grammar structures,

creating a demand for robust systems that facilitate

communication between hearing and Deaf

individuals (Stoll, Camgoz, Hadfield, & Bowden,

2019). As such, technology can play a vital role in

addressing these challenges, and one such solution is

the innovative Figure Out application.

Figure Out is a groundbreaking mobile app that

streamlines language translation, offering sign

language translation alongside spoken languages, a

feature not found in any other app. It achieves this by

using the phone's camera to automatically capture

https://orcid.org/0000-0003-2528-572X

https://orcid.org/0000-0003-3932-4486

https://orcid.org/0000-0003-3940-3846

text and provide translations into the desired

language.

The focus is on ensuring inclusivity, accessibility,

and effective communication by providing informa-

tion in the preferred language of individuals.

2 FRAMEWORK

The experience of being a foreigner can be

challenging. However, for individuals who are deaf,

the experience can be even more overwhelming due

to the lack of accessibility and inclusion in many

foreign environments. As Gillovic & McIntosh

(2020) point out, foreign environments are often

“barrier-laden and socially exclusive”, making it

difficult for individuals who are deaf to fully

participate and engage.

To address this issue, it is crucial for foreign

environments to be inclusive and welcoming to all

individuals, regardless of their abilities or language

preferences. As Santa (2020) notes, reducing

uncertainty and providing support to visitors who are

deaf is essential to creating an inclusive foreign

486

Escudeiro, P., Gouveia, M. and Escudeiro, N.

A Revolutionary Real-Time Translation Tool for Text, Audio, and Sign Language from Images.

DOI: 10.5220/0012691400003693

Paper published under CC license (CC BY-NC-ND 4.0)

In Proceedings of the 16th International Conference on Computer Supported Education (CSEDU 2024) - Volume 1, pages 486-493

ISBN: 978-989-758-697-2; ISSN: 2184-5026

environment. This means ensuring that deaf

individuals can access education, services, cultural

sites, and information independently, without the

need for adaptation.

By promoting social inclusion and accessibility,

communication between the deaf community and

others can be greatly improved. As Oliveira et al.

(2019) argue, creating an environment that is

accessible to individuals who are deaf is not only

beneficial for them, but also for society as a whole. It

allows for greater understanding and appreciation of

diversity and promotes a more equitable and inclusive

society.

One tool that can help to facilitate communication

and accessibility for individuals who are deaf is

Figure Out. This mobile application is designed to be

multilingual and allows for translation to Portuguese

Sign Language, making it a valuable resource for deaf

individuals who cannot easily access information. By

enabling translation to Portuguese Sign Language,

Figure Out helps to mediate communication and

create a relationship of equivalence between two

different languages, overcoming communication

barriers (House, 2016).

The deaf community in Portugal uses Portuguese

Sign Language (LGP), which is different from other

sign languages around the world. There is a

communication gap between the deaf and hearing

communities due to the lack of awareness about LGP.

Deaf individuals struggle to understand written text in

Portuguese. A translation model could help bridge

this gap and allow the deaf community to access

information on the internet and other resources.

Machine translation for sign languages has been

overlooked until recently, but it is crucial for

integrating the deaf community into society (Khan,

Abid, & Abid, 2020).

To do so, and by leveraging our expertise in sign

language translation, the Figure Out initiative

endeavours to create a groundbreaking tool that

enhances inclusivity in cultural settings. This tool

aims to cater to the needs of the deaf community

through the Inclusive Cultural Heritage Tourism

(ICHT) project (Escudeiro N. , Escudeiro, Cunha, &

Gouveia, 2022). The ICHT project aims to create an

environment that is welcoming and accommodating

to all visitors, including those with hearing

disabilities (Escudeiro, Gouveia, Cunha, &

Escudeiro, 2022). This will not only benefit the deaf

community but will also create a global strategy for

the cultural sector. As Biddulph & Scheyvens (2017,

2018) suggest, developing an inclusive cultural

environment will open doors to a significant share of

the global market, attracting more visitors and

promoting cultural exchange.

In conclusion, the ICHT project and the FigureOut

app are a step towards creating a more inclusive

society, where everyone can enjoy and appreciate

cultural heritage sites. They will not only benefit the

deaf community but will also create a global strategy

for the cultural sector. By utilizing sign language, we

can create an environment that is accessible to

everyone, regardless of their hearing ability.

3 METHODOLOGY

Figure Out is dedicated to enhancing access to

cultural heritage by offering digital tools and content

in multiple languages, including sign language, to

ensure seamless online and mobile accessibility.

Moreover, at the heart of the Figure Out project

lies a mobile application meticulously crafted to

deliver instant translations of small text segments

captured as images. Functioning seamlessly within an

augmented reality context, users can effortlessly point

their mobile device's camera at the text they wish to

translate. The application rapidly processes visual

data and promptly delivers a translation in the user's

selected language.

The Figure Out app's usability transcends

conventional language translation tools due to the

integration of sign language translation, a pivotal

feature facilitating communication between deaf and

non-deaf individuals across various real-life

scenarios. For instance, envision a deaf individual

navigating a restaurant where the menu is exclusively

available in written form. With the Figure Out app,

they can effortlessly capture the menu items using

their smartphone camera, and the app instantly

translates them into sign language, empowering

informed food choices without reliance on written

text.

Similarly, accessing public transportation

information poses challenges for deaf individuals

reliant on auditory announcements or written signs.

Through the Figure Out app, users can easily capture

bus schedules or train timetables using their

smartphone camera, with the app providing real-time

sign language translations. This capability enables

deaf individuals to navigate public transportation

networks independently, enhancing accessibility and

inclusivity in daily life.

Also, the Figure Out app holds significant

potential for enhancing language learning and

educational experiences, particularly in the realm of

sign language education. By leveraging its real-time

A Revolutionary Real-Time Translation Tool for Text, Audio, and Sign Language from Images

487

sign language translation feature, the app can be

instrumental in facilitating communication between

teachers and deaf students, thereby promoting

inclusivity in educational settings. Moreover, the app

can serve as a valuable tool for teaching sign language

to hearing students, fostering greater awareness and

appreciation of sign language as a language and mode

of communication.

For instance, in a classroom setting, teachers can

utilize the Figure Out app to translate spoken

instructions or lecture content into sign language in

real-time, ensuring that deaf students have equal

access to educational materials. Furthermore, the app

can be integrated into language learning curricula to

provide interactive sign language lessons, allowing

students to practice and improve their signing skills

in an engaging and immersive environment.

By incorporating educational applications into its

scope, Figure Out not only contributes to language

accessibility but also promotes inclusive education

and cultural diversity in schools and learning

institutions.

In addition to translation capabilities, the Figure

Out project introduces a mobile diary tool,

empowering participants to quickly record their

thoughts, actions, and surroundings through their

mobile devices. This feature serves a dual purpose,

enabling users to maintain a personal diary while also

supporting context-based co-creation in various

developmental phases. Users can capture spoken

messages, pictures, videos, or text messages, each

entry accompanied by a wealth of associated data,

including transcriptions, audio files, timestamps, geo-

tags, participant IDs, and message durations. This

data is invaluable for research, fostering a rich source

of information and feedback.

The project's ultimate goal is to implement these

tools within a Living Lab, an open innovation

environment. In a Living Lab, users play an integral

role in the innovation process, ensuring that the tools

are tailored to their needs and experiences in a real-

world setting. This user-centric approach guarantees

that the technology is genuinely effective and user-

friendly.

Figure Out's innovation lies in its capacity to

facilitate sign language translation in real time. This

is a critical feature that distinguishes it from other

translation tools available in the market, which

primarily focus on text or spoken language. The

application's real-time translation capability sets it

apart, making it an invaluable tool for those who rely

on sign language for communication. It addresses a

wide array of scenarios where translation needs arise,

such as reading public transportation signs,

timetables, tickets, receipts, and even restaurant

menus.

It's worth noting that Figure Out builds upon the

foundation of VirtualSign technology. This is a

groundbreaking technology developed to bridge

language gaps, specifically in sign language

communication. It allows written text and spoken

language to be converted into sign language, reducing

the barrier between spoken and sign language. The

technology supports various sign languages, making

it adaptable to users' preferences. One notable feature

is its real-time translation capability, enabling on-the-

fly communication. The user-friendly interface

ensures accessibility for individuals with varying

technical expertise. Virtual Sign has made a

significant impact in education, facilitating access to

educational materials for deaf students. It also

improves communication access in everyday life,

cultural venues, and promotes inclusivity and

accessibility. Figure Out, an extension of VirtualSign,

integrates real-time image-based translation and

mobile diary functionality, providing a

comprehensive solution for the deaf community

(Escudeiro et al., 2011; Escudeiro et al., 2013;

Oliveira et al., 2019).

3.1 Objectives

The Figure Out application's primary goal is to

streamline information access, thereby improving

accessibility, inclusivity, and communication,

particularly between the deaf and non-deaf

communities. With Figure Out, users can access

information in their preferred language, leading to

numerous effects across various societal aspects:

1. Cultural Inclusion: Navigating cultural venues

and accessing information in museums, art

galleries, and other cultural settings can be

challenging for deaf individuals. Figure Out

provides a bridge, enabling them to understand

the information displayed, thus ensuring that

cultural experiences are inclusive and enriching

for all.

2. International Mobility: Tourists and

international students frequently encounter

language barriers when traveling or studying

abroad. Figure Out acts as a language

companion, facilitating access to essential

information and improving the overall

experience of international mobility.

3. Reducing Communication Barriers: With

Figure Out, the divide between the deaf and

non-deaf communities narrows significantly.

Communication between these groups

CSEDU 2024 - 16th International Conference on Computer Supported Education

488

becomes more accessible, fostering

understanding and social cohesion.

To enhance achievement, the Figure Out Project

involves planning, designing, constructing, and

implementing a comprehensive system that integrates

a mobile application and website. This system

facilitates the translation of written words into various

formats, including written, audio, and sign language.

The mobile app records translated words along with

GPS data for analysis. The website complements the

app by offering users access to usage statistics. Thus,

its primary objectives are as follows:

1. Website Development: The creation of a

website to promote the Figure Out tool and

enable users to analyse usage statistics. This

includes insights into translation frequency and

the geographical locations where the

application is most frequently used.

2. Mobile Diary Integration: The project aims to

allow participants to create diary entries,

termed the "Mobile Diary," at any time and

from any location using their mobile phones.

This can be achieved through various means,

such as spoken messages, pictures, videos, or

text messages. A wide range of associated data,

including voice message transcriptions, audio

files, timestamps, geo-tags, participant IDs,

and message durations, will be collected for

analysis.

3. Real-time Data Collection: The Figure Out

system will facilitate real-time data collection,

enabling rapid iterations of the tool and

research methods. Automated messages and

push notifications will be sent to participants to

encourage timely entries and ongoing

participation.

4. Data Analysis and Reporting: The system will

offer capabilities for analysing and reporting

collected data, allowing for a deeper

understanding of user interactions and usage

patterns.

5. User Support and Differentiated Interfaces:

The Figure Out system will cater to four

distinct user types, including administrators

(both technical and managerial roles), content

specialists, mobile diary users, and casual

unregistered browsers. Each user type will have

a tailored interface and language preferences,

with adaptable message strings for easy

translation.

6. User Registration and Content Submission:

User registration will be mandatory for leaving

comments and submitting additional

information. Content specialists will review

and approve or reject submissions. Detailed

guidance on tool usage and available

functionalities will be provided.

7. Data Storage and Retrieval: Submissions of

additional information, such as photos and

videos, will be stored in dedicated directories

on the server, with comprehensive

characteristics stored in the database. The

system will also feature an exploratory

interface for efficient information retrieval.

8. Data Size Management: Restrictions on data

size for videos and high-definition media will

be adjustable based on user profiles, ensuring a

seamless and personalized experience.

The Figure Out Project is poised to deliver a

robust and user-centric system that promotes

translation and interactive communication. With a

multifaceted approach, it addresses various user

needs and preferences, leveraging real-time data

collection and analytical capabilities. By providing

multilingual support and enabling easy data

translation, Figure Out aims to enhance accessibility,

user engagement, and the overall user experience in

both mobile and web environments.

4 TECHNICAL RESOURCES

Acknowledging the need for technical knowledge in

mobile development, a framework with substantial

code reusability for both platforms was selected.

After analysing available options, PhoneGap was

chosen as the preferred solution.

4.1 Frontend Technology

When it comes to fulfilling the front-end

requirements of a software development project, there

are typically two main aspects to consider: the

application and the website. These two components

may have different goals and requirements, but they

both play crucial roles in delivering a cohesive and

user-friendly experience to the end-users.

To begin the application development, we decided

to use Ionic. This open-source SDK is specifically

designed for hybrid mobile application development,

which means that it enables developers to create

mobile apps that can work seamlessly on both

Android and iOS platforms. This is a significant

advantage, as it allows us to reach a wider audience

with our application without having to invest in

separate development efforts for each platform.

A Revolutionary Real-Time Translation Tool for Text, Audio, and Sign Language from Images

489

One of the key benefits of using Ionic is that it

allows developers to create mobile applications with

minimal changes. This saves time and effort, as we

can reuse most of the codebase across different

platforms, while still maintaining a consistent user

interface and experience. Additionally, Ionic also

supports the creation of desktop websites, which

means that we can leverage the same technology and

skills for multiple purposes.

Moving on to the website development, we opted

for Angular as our framework of choice. This

decision was based on several factors, including the

fact that Angular is similar to Ionic in many ways,

which makes it easier and faster for our developers to

work with. Since most of our development team

already had experience with Angular, we could hit the

ground running with our website development efforts.

Angular is a popular open-source framework for

building dynamic web applications. It is based on

TypeScript, a superset of JavaScript that adds static

typing and other features to the language. This makes

it easier to write and maintain complex codebases, as

well as enabling better tooling and refactoring

capabilities. With Angular, we can create responsive

and interactive websites that can adapt to different

screen sizes and user inputs, while still being

performant and scalable.

4.2 Backend Server and Technology

For the project's backend requirements, the team

decided to implement a NodeJS server due to its

widespread use and comprehensive documentation.

This choice enables the establishment of a centralized

system, enhancing control and maintenance of REST

API routes. In terms of data storage, Firebase was

chosen for its reliability, security features, and the

existing expertise within the development team.

4.3 APIs, Libraries and Framework

In order to successfully complete this project, it is

essential to have access to a wide range of APIs,

frameworks, and libraries. These tools will enable us

to create a high-quality final product that meets the

needs and expectations of our clients.

One of the key components of this project is the

creation of a promotional video. To accomplish this,

we will be utilizing Adobe software, which has

proven to be a reliable and effective tool for creating

visually stunning videos that capture the attention of

viewers.

Additionally, we will be leveraging Google's APIs

to capture text with a camera, convert text to audio,

and translate text into different languages. These

APIs are widely regarded as some of the best in the

industry, and we are confident that they will help us

to deliver a final product that is both accurate and

accessible to a diverse audience.

Finally, we will be utilizing the Virtual Sign

translator to translate text into sign language. This

innovative tool uses advanced algorithms and

machine learning techniques to accurately translate

text into sign language, making it accessible to

individuals who are deaf or hard of hearing.

4.4 OCR Integration

The application needed to run on Android and iOS

and include an OCR tool for image to text

recognition. However, integrating an OCR tool with

the chosen framework proved to be difficult. Testing

with different OCR tools showed incompatibilities,

leading to exploring a new framework, Xamarin.

Unfortunately, the available OCR tools were not

compatible with Xamarin.

4.5 Native Development

The team created a Windows Phone application,

which allowed them to conduct a rapid test using

native development, despite it not being a

requirement. The results of this test confirmed that

integrating native capabilities and utilizing an OCR

tool is the most advisable approach. Furthermore, it

provided additional evidence to support the findings

obtained from the OCR search, which consistently

indicated the superiority of native OCR for both

Android and iOS platforms.

4.6 System Architecture

Originally conceived as a centralized web service,

Figure Out ensures continuous access to data and

resources across three logical layers: UI, Server, and

Data. The UI layer engages users through the mobile

app or website, with the web service layer managing

UI requests and translation. The data layer functions

as a persistence mechanism for statistical purposes.

Technology choices were made based on the

team's expertise, utilizing Ionic for the mobile app's

cross-platform support, Node.js and Express for the

backend server, and Firebase for the database,

facilitating push notifications and user management.

Additionally, the VirtualSign API is employed for

converting text to sign language (Figure 1).

CSEDU 2024 - 16th International Conference on Computer Supported Education

490

Figure 1: Logical Architecture.

4.7 The FigureOut App

The Figure Out mobile application is designed to be

user-friendly and intuitive. The prototype mock-ups

have been meticulously crafted to ensure a smooth

and seamless user experience.

The application workflow is straightforward and

easy to follow. Users simply need to capture a photo

using their mobile camera, configure the translation

process if necessary, and proceed to translation. The

app homepage provides direct access to the

translation feature, making it even more convenient

for users to get the translations they need.

In addition to translation, the app also provides

users with access to the About page, which offers

more information about the FigureOut webpage and

other features. This makes it easier for users to learn

more about the app and its capabilities.

When a user starts a translation procedure, the app

directs them to take a picture of the text they wish to

translate using their mobile camera. The application

then identifies and isolates the text area detected in

the photo and presents it together with the

configuration panel. Users can choose to use the

default configuration or reconfigure each translation

according to their specific needs. This involves

choosing the languages pair and the output format.

The app offers three output formats, giving users

plenty of flexibility and options.

For those who prefer typing over capturing images,

an input field is also available. Users can simply type

in the sentence they wish to translate, and the app will

take care of the rest.

Overall, the Figure Out mobile application is a tool

for anyone who needs quick and accurate translations

on the go. Its user-friendly design and intuitive

interface make it easy to use, while its advanced

features and options offer plenty of flexibility and

customization.

Figure 2: Mobile Application.

5 PROJECT EVALUATION AND

QUALITY SCENARIO

CONTROL

The project evaluation adhered to a methodology that

incorporates the assessment of functionality and user

experience to gauge the quality scenario. This

evaluation process was underpinned by a framework

known as QEF (Quantitative Evaluation Framework).

QEF is designed to quantitatively measure the quality

of the system under development (Escudeiro &

Bidarra, 2008; Escudeiro & Escudeiro, 2012; Heidari

& Loucopoulos, 2014).

The quality scenario, structured within the QEF

framework, consists of three key levels: dimensions,

factors, and requirements. Each dimension

encompasses a group of factors, and each factor

comprises a set of specific requirements (Escudeiro &

Bidarra, 2008; Escudeiro & Escudeiro, 2012; Heidari

& Loucopoulos, 2014).

5.1 QEF Dimensions, Factors and

Requirements

The quality assessment of the Figure Out System,

based on the QEF framework and ISO 9126

standards, comprises three dimensions: Functionality

(F), Adaptability (A), and Efficiency (E).

Functionality dimension encompasses four key

factors, which are functional aspects, user interaction,

content quality, and connectivity. The Adaptability

dimension consists of two factors, namely versatility

and maintenance. Efficiency encompasses three

factors: strength, consistency, and integrity

(Escudeiro & Escudeiro, 2012).

The functional factor includes critical aspects

such as language selection, navigation, sign language

avatar reproduction, and translation capabilities.

User interaction focuses on intuitive design and

consistent experiences across Android and iOS

A Revolutionary Real-Time Translation Tool for Text, Audio, and Sign Language from Images

491

platforms, incorporating features like a scoreboard

and statistics view.

Content quality emphasizes organized

information presentation, adherence to grammar

principles, and relevance to the product's purpose.

Connectivity addresses the system's interaction

with external resources, while versatility ensures

compatibility with various platforms and devices.

Maintenance involves regular updates and

hardware upkeep, and strength, consistency, and

integrity contribute to the system's structural

integrity, reliability, and security.

Each requirement is evaluated using predefined

values, minimizing ambiguity in the assessment

process. Some are assessed with binary results (0% or

100%), while others use a Likert scale ranging from

1 to 5. To minimize ambiguity in the evaluation

process, each requirement is clearly defined with its

associated assessment levels.

5.2 Evaluation Methodology

The project team, committed to a comprehensive and

thorough product development process, conducted

multiple assessments at key points in the product

development lifecycle, including alpha testing, beta

testing, and a final evaluation (Escudeiro & Bidarra,

2008). To ensure the product met the highest quality

standards, they employed QEF, which provided a

systematic and objective evaluation based on

questionnaires. These questionnaires were

administered to students in a multimedia program,

ensuring diverse perspectives.

The testers, selected for their experience with

mobile applications, followed a standardized protocol

involving various tasks such as account registration,

viewing promotional content, executing translations,

and providing feedback on their experiences. A

significant majority (73%) of the testers used Android

devices, and the team was confident in their

representativeness for the target audience.

Overall, the project team was pleased with the

results and positive feedback from testers. The

product met predefined quality scenario

requirements, and the team was confident that it was

ready for release.

Testing a new application or software product is

crucial to thoroughly evaluating its features and

functionalities. In this context, actively engaging with

the application's features and functionalities,

including reviewing diary entries, is essential to

comprehensively answering assessment

questionnaires. This framework leverages the degree

of fulfilment of underlying requirements to calculate

a quality value, representing the percentage of

achievement by the current version compared to an

ideal solution.

This approach ensures a clear and quantifiable

assessment of the product's quality throughout its

development lifecycle. By measuring the degree of

fulfilment of underlying requirements, QEF identifies

areas that may need improvement, allowing the team

to address them before releasing the product to the

market. The commitment to delivering a high-quality

product is reinforced through a nuanced

understanding gained by exploring features

thoroughly and engaging with tools during the testing

process.

5.3 Evaluation Results

Throughout the project duration, FigureOut has

demonstrated a significant improvement in overall

quality, aligning with anticipated milestones.

Progress was meticulously evaluated at key stages,

revealing substantial growth over time. A

comprehensive assessment of cumulative quality

across various dimensions yielded positive outcomes:

Alpha testing at 76%, Beta testing at 77%, and a final

assessment at 96%.

Recognizing the critical role of the software

testing process in delivering a high-quality product

that aligns with user requirements, a phased testing

approach was adopted. The marginal quality

improvement from Alpha to Beta testing (76% to

77%) indicated the potential for further refinement in

features and functionality. Encouragingly, significant

enhancements were observed from the Beta to the

final version, notably in the Functionality dimension

with a remarkable 67% improvement. Efficiency

improved by 31%, and Adaptability by 16%.

The positive trend reflects a concerted effort in

prioritizing non-functional requirements to create a

robust platform supporting essential functional

features.

Ensuring product reliability and efficiency,

aligned with user needs, remains a paramount focus.

Acknowledging the continuous nature of evolution

from Alpha to Beta testing and the final version, pride

is taken in significant strides achieved. While

recognizing room for improvement, the commitment

is unwavering in addressing both functional and non-

functional requirements. This dedication ensures the

delivery of a product of the highest quality,

surpassing user expectations for a successful

outcome.

CSEDU 2024 - 16th International Conference on Computer Supported Education

492

6 CONCLUSIONS

Figure Out stands as an assistive technology crafted

with the purpose of reduce communication barriers

commonly faced by everyday users. The technology

offers users access to information in their native

language, which is expected to revolutionize the way

people communicate and interact with each other.

The application has been specifically designed to

foster the inclusion of those for whom sign language

is their primary means of communication. By

integrating national sign languages and International

Sign, the technology ensures that all users can

communicate effectively, regardless of their language

or communication preferences.

As part of its ongoing development, Figure Out is

continually working to enhance the user experience.

This includes the creation of a more appealing visual

identity, which will make the technology more

appealing and user-friendly. Additionally, the

integration of additional sign languages will further

expand the reach of the technology, enabling more

people to benefit from its innovative features.

Overall, Figure Out is a game-changing

technology that has the potential to greatly improve

the daily lives of millions of people around the world,

by contributing to the creation of a more

interconnected and accessible society.

ACKNOWLEDGEMENTS

This work is being developed at the research group

GILT, with the support of FEDER, under the frame

of Portugal 2020, project number POCI-01-0247-

FEDER-069949 and the Erasmus+ Programme,

through project InSign, project number 2019-1-

DE01-KA203-004964.

REFERENCES

Biddulph, R., & Scheyvens, R. (2018). Introducing

inclusive tourism. Tourism Geographies, 583-588.

Escudeiro, N., Escudeiro, P., Cunha, B., & Gouveia, M. C.

(2022). Inclusive Cultural Heritage Tourism. 4th

International Conference on Tourism Technology &

Systems (ICOTTS'22). Santiago: Springer.

Escudeiro, P., & Bidarra, J. (2008). Quantitative Evaluation

Framework (QEF). Revista Ibérica de Sistemas e

Tecnologias de Informação, 16-27.

Escudeiro, P., & Escudeiro, N. (2012). Evaluation of

Serious Games in Mobile Platforms with QEF: QEF

(Quantitative Evaluation Framework). 2012 IEEE

Seventh International Conference on Wireless, Mobile

and Ubiquitous Technology in Education, 268-271.

Escudeiro, P., Escudeiro, N., Reis, R., Barbosa, F. M.,

Bidarra, J., Baltazar, A., & Gouveia, B. (2013). Virtual

Sign Translator. International Conference on

Computer, Networks and Communication Engineering

(pp. 290-292). Beijing: Atlantis Press.

Escudeiro, P., Escudeiro, N., Reis, R., Barbosa, M.,

Bidarra, J., & Gouveia, B. (2011). Automatic Sign

Language Translator Model. Advanced Science Letters,

4, 400-407.

Escudeiro, P., Gouveia, M. C., Cunha, B., & Escudeiro, N.

(2022). Accessibility of National Cultural Heritage to

Deaf Tourists. Proceedings of the 5th International

Conference on Tourism Research (pp. 110-117). Porto:

ACI.

Gillovic, B., & McIntosh, A. (2020). Accessibility and

Inclusive Tourism Development: Current State and

Future Agenda. Sustainability, 1-15.

Heidari, F., & Loucopoulos, P. (2014). Quality evaluation

framework (QEF): Modeling and evaluating quality of

business processes. International Journal of Accounting

Information Systems, 15(3), 193-223.

House, J. (2016). Translation as Communication across

Languages and Cultures. London: Routledge.

Khan, N. S., Abid, A., & Abid, K. (2020). A Novel Natural

Language Processing (NLP)–Based Machine

Translation Model for English to Pakistan Sign

Language Translation. Cognitive Computation, 12,

748-765. doi: 10.1007/s12559-020-09731-7

Oliveira, T., Escudeiro, N., Escudeiro, P., Rocha, E., &

Barbosa, F. M. (2019). The VirtualSign Channel for the

Communication Between Deaf and Hearing Users.

IEEE Revista Iberoamericana de Technologias del

Aprendizaje, 188-195.

Oliveira, T., Escudeiro, P. M., Escudeiro, N. F., & Rocha,

E. (2019). Automatic Sign Language Translation to

Improve Communication. IEEE Global Engineering

Education Conference. Dubai.

Santa, J. G.-S. (2020). Tourism as a lever for a more

inclusive society. Worldwide Hospitality and Tourism

Themes, 731-738.

Scheyvens, R., & Biddulph, R. (2017). Inclusive tourism

development. Tourism Geographies, 589-609.

Stoll, S., Camgoz, N. C., Hadfield, S., & Bowden, R.

(2019). Text2Sign: Towards Sign Language Production

Using Neural Machine Translation and Generative

Adversarial Networks. International Journal of

Computer Vision, 128, 891-908. doi: 10.1007/s11263-

019-01281-2

Zheng, J., Zhao, Z., Chen, M., Chen, J., Wu, C., Chen, Y.,

. . . Tong, Y. (2020). An Improved Sign Language

Translation Model with Explainable Adaptations for

Processing Long Sign Sentences. Computational

Intelligence and Neuroscience. doi:

10.1155/2020/8816125

A Revolutionary Real-Time Translation Tool for Text, Audio, and Sign Language from Images

493