across different contexts have made a trend in recent
years (Krause and Neto, 2021; Lian and Sunar, 2021;
Mubin et al., 2020; Torrado et al., 2019; Grossard
et al., 2017).
Some of the main problems covered by these ap-
plications include ASD diagnosis, adaptive learning
and emotional recognition (Krause and Neto, 2021).
This study will focus on the latter.
According to psychologist Robert Plutchik, there
are 8 basic emotions defined by joy, trust, fear, sur-
prise, sadness, anticipation, anger, and disgust. The
rest are defined as a combination of these primary
emotions and, similarly to the colour palette, a wheel
of emotions is proposed, referred as Plutchik’s Emo-
tion Wheel (Plutchik, 2001).
Children and adolescents with ASD have more
difficulties than typically developing group pairs for
emotion recognition and management (Papoutsi et al.,
2018). Hence, therapy for people with ASD involves
a process of emotion identification and recognition,
and in recent years this has been mediated with the
use of technology (Rashidan et al., 2021).
On the other hand, interactive museums may offer
an interesting option for participation in leisure yet
ordered activities, as they can be ordered in different
itineraries or attractive points depending on the visi-
tors’ interests (Yates et al., 2022). If a child with ASD
gets affected by any stimuli, it is relevant to commu-
nicate their emotions well to their caregiver.
All in all, there is room for investigation within
the context of emotional perception by children with
ASD as an interactive experience. We propose an
adaptation of a user model for profile assignment
based on the user perception when visiting interactive
attractions, the idea is to compare this information to
a pre-evaluation made from experts, to see if they fit.
2 RELATED WORK AND
MOTIVATION
Some notable applications involving the use of smart-
phones as assistance tools in museum visits or emo-
tional recognition for children with ASD are sum-
marised as follows.
ARtis (Vita et al., 2021) is a mobile application
that facilitates the accessibility and visit of muse-
ums for children with ASD and their tutors by over-
lapping virtual content with the surrounding reality.
Through the smartphone camera, it creates a path in-
side a museum to help the user orientate and “experi-
ence” an appealing and interactive visit. It is guided
by a friendly 3D avatar that follows a path guided with
GPS tracking, it allows users to interact with different
points of interest of the museum.
The main goal of ARTis is to provide an experi-
ence centred on the end-user for a tailored cultural
experience that can help to increase and improve so-
cial skills, through a greater sense of self–efficacy
and autonomy(Lorenzo et al., 2019). With the use
of Augmented Reality (AR) and Internet of Things
(IoT) applied in the use of games, pop-up videos and
visual content, it makes the cultural tour more enjoy-
able and motivating, although certain research proto-
cols for usability testing have to be worked on as well
as extension to other contexts (Vita et al., 2021).
Tobias in the Zoo (Carvalho et al., 2015) presents
an AR application in the form of a GameBook to as-
sist children with ASD to recognise and acquire emo-
tions by engaging their attention and motivation. It
shows the story of Tobias, a virtual character who
has adventures during a zoo park visit along 5 sce-
narios. Each stage interacts with virtual animals and
real-world situations which will conduct the children
to become involved on fictional contents associated
with emotions.
In each GameBook chapter, Tobias identifies five
emotions (happiness, anger, sadness, fear and disgust)
and narrates examples of it as a part of the story. At
the end of each chapter, a mini game for emotion as-
sociation is prompted to the user in the form of a ques-
tionnaire based on a situation description and an im-
age of Tobias expressing the emotion. At the end of
the book it has an evaluation of all presented emotions
in a memory game (Carvalho et al., 2015).
Guess What? (Kalantarian et al., 2019) is a mo-
bile charade-style game available in iOS and Android
platforms, made for emotion association training to
children with ASD.
With the use of image pattern recognition, it has
been designed to be a shared experience between the
child, who attempts to enact the prompt shown on the
screen through gestures and facial expressions, and
the parent, who is tasked with guessing the word as-
sociated with the prompt during a 90s game session
(Kalantarian et al., 2020).
All in all, these projects show how the use of
games intended for assistance and emotional learning
can be suited for children with ASD, presenting good
results in terms of their behaviour. However, these
tools work with a learning mechanism where emo-
tions are taught by repetition or common face patterns
but are not necessarily experienced by the users them-
selves.
We propose a tool where users associate their feel-
ings to a given emotion as they go through an interac-
tive experience, so that recognition comes first hand
rather than given by a storybook or character.
HEALTHINF 2023 - 16th International Conference on Health Informatics
508