Inducing Behavior Change in Children with Autism Spectrum Disorders
by Monitoring their Attention
Margarida Lucas da Silva
1,2
, Hugo Silva
3
and Daniel Gonc¸alves
1,2
1
Instituto Superior T
´
ecnico, Av. Rovisco Pais, 1, 1049-001, Lisboa, Portugal
2
INESC-ID, Rua Alves Redol, 9, 1000-029 Lisboa, Portugal
3
Instituto de Telecomunicac¸
˜
oes, Instituto Superior T
´
ecnico,
Av. Rovisco Pais 1, Torre Norte - Piso 10, 1049-001, Lisboa, Portugal
Keywords:
Human Behavior Analysis, Autism Spectrum Disorders, Inducing Behavior Change.
Abstract:
Children with Autism Spectrum Disorders (ASD) generally suffer from disorders which affect multiple be-
havioral aspects, such as communication, emotional awareness, social interaction, lack of attention, among
many others. Modern technologies, are opening up new possibilities for computer-mediated interactions with
increased outcomes, enabling both children and tutors to have a more effective work in the development of
communicative and cognitive skills. In this article we introduce a module implemented in a platform for
human-computer interaction, specifically designed for children with ASD, to control their levels of attention
and test inducing behavior change. This allows us to shape new behaviors and learning strategies both in tutors
and children.
1 INTRODUCTION
The constant evolution in information and communi-
cation technologies, opened a wide range of new pos-
sibilities for helping people with learning disabilities
to have access to new opportunities for learning, en-
tertainment and personal development. Children with
Autism Spectrum Disorders (ASDs) are included in
this group, and although many approaches have been
tested, few have been adopted or successfully inte-
grated in the day-to-day life of the children, their tu-
tors or caregivers. This lack of appropriate tools is
aggravated by the unique specificities of autistic pa-
tients. They have restricted interests and can become
very fixated on them, which sometimes can be used as
motivation tool. Also, general social interaction tools
are usually too complex for them to use, have exces-
sive stimuli, or are considered boring, leading to lack
of interest or loss of focus. This factor makes it im-
possible to make generalizations about what is better
for this kind of patients, creating a practical problem
in software development.
With these issues in mind, we developed a
communicative competence in autistic children; all
the activities are targeted at communication skills
training, and are based in multimedia content: im-
ages, videos, audio and stories. These are activities
familiar to the children, facilitating the platform in-
clusion in their lives. A detailed description of the
platform can be found in (Lucas da Silva et al., 2012).
Some experimental results can be found in (Lucas da
Silva et al., 2011).
With the purpose of keeping children motivated
and engaged in the activities proposed by their tutors,
in particular targeting the improvement and shaping
of new behaviors, we developed an attention manage-
ment module, that provides feedback whenever the
childs behavior is indicative of lack of attention.
Our goal with the attention detection module
was to analyze the child attention while using
perimental evaluation where we automatically try to
recover the user’s attention when he looks away from
the computer by triggering an auditory feedback stim-
uli, and evaluating if this kind of feedback contributes
to have the children more engaged in their activities
with the platform.
2 BACKGROUND
language of the authors) to English language as ex-
131
Lucas da Silva M., Silva H. and Gonçalves D..
Inducing Behavior Change in Children with Autism Spectrum Disorders by Monitoring their Attention.
DOI: 10.5220/0004698401310136
In Proceedings of the International Conference on Physiological Computing Systems (PhyCS-2014), pages 131-136
ISBN: 978-989-758-006-2
Copyright
c
 2014 SCITEPRESS (Science and Technology Publications, Lda.)

changes, since the root of our project was the creation
of a tool to develop new communicative behaviors
in children with ASD. The [e-mail] platform
is now focused on: a) Promoting new communica-
tive behaviors between the child and her peers, tutors
and parents; b) Addressing the individual specificity
of each child through fully adaptable contents and
screens; c) Improving the attention and engagement
through the automated analysis of behavior patterns
and inducing behavior change; and d) Stimulating
new social interaction patterns in the child that might
otherwise not be possible, by using the computer as a
mediator.
Face-to-face interpersonal communication and in-
teraction skills of children with ASD can be very lim-
ited, creating the need for alternative approaches for
their integration in social life. Having a computer as
a mediator breaks several of the barriers which arise
from the child being faced with a human peer. Our
platform provides several tools to address this need,
with the twofold objective of gradually encouraging
socialization and developing communication behav-
iors among children with ASD. Besides the multime-
dia content (images, videos, audio, and stories), that
children can autonomously browse through and use in
their educational experience as we can see in Figure 1,
there is also a message board, and content preference
sharing options associated with each multimedia con-
tent item. These are important contribution of our
work to the field, when compared to existing work,
as they induce the social component in the child.
Figure 1: Picture thumbnails listing in [e-mail].
In the message board (Figure 2), the child can
post messages to her colleagues. The sender child
has a buddy list with names and photos of all counter-
parts, from which the receiver colleague can be cho-
sen from. The message can then be written either
through text, picture exchange, or a combination of
both methods. When the receiver colleague accesses
the platform, he will automatically be prompted that
there are new messages sent to him by others, waiting
Figure 2: Message composition screen in [e-mail].
to be seen in the message board.
For content preference sharing, associated to each
multimedia content item (image, sound or video),
there is a positive and a negative preference option.
These allow the child to express and share with its
counterparts at any time, his/her preference regarding
the individual content item. This information is as-
sociated with the content and shown every time it is
accessed, allowing all children to see the opinion of
their colleagues. Figure 3 illustrates one of the activ-
ities with the content preference-sharing pane on the
right side of the screen.
Figure 3: Message board.
The application can be fully customized with
icons, background images and the contents, that allow
each child to be more effective while using it. The tu-
tor can define a user profile for each child, or use a
default profile; this is done with simple drag and drop
operations. Each user profile may have independent
and distinct UI files, such as the HTML structure files,
the CSS, the background images, the menu icons, and
even the contents shown to the user can be differenti-
ated: the images, videos and audio listing.
These features of [e-mail] have already
shown some positive results in previous work from
our group (Lucas da Silva et al., 2011), where 2 chil-
dren which didn’t talk to each other (despite the fact
PhyCS2014-InternationalConferenceonPhysiologicalComputingSystems
132

that they were colleagues for more than 2 years), be-
gan to bear each other and to show basic interac-
tion traces after starting to use these features of the
platform. Furthermore, children not only accepted
[e-mail] as part of their daily work routines
with the tutor, but also frequently requested the use
of the platform.
3 RELATED WORK
The study and management of attention has been a
highly active, yet poorly understood research topic
over the past years.
In the context of ASD, although attention manage-
ment is of paramount importance to understand how
children can be engaged, few studies are found, es-
pecially in a pervasive computing framework. Most
of the work to date has focused on the control of eye
gaze and eye movement to understand how stimuli are
processed.
This is used so it is possible to detect what inter-
ests the child has, and what strategies we can take to
improve the attention levels, and is especially useful
when the child possess fixation patterns or circum-
scribed interests. Some studies focused on orienting
the object of attention in response to eye gaze with
the purpose of helping draw attention (Leekam et al.,
1997), or giving cues about where the object was lo-
cated, having the purpose of studying the child’s re-
action time (Posner et al., 1984) which has been the
inspiration for a vast number of studies developed
throughout the years. Also, eye tracking was also
used to perform tests of emotion recognitions from
photographs of facial expressions (Pelphrey et al.,
2002), and used in conjunction with Functional mag-
netic resonance imaging (fMRI) (Dalton et al., 2005)
for facial discrimination tasks where researchers con-
cluded that there was a diminished eye region fixation
from subjects with ASD.
Other techniques build upon the idea of finding
differences in experiments settings that are presented
to the children, so that they have to find a single
change introduced in the setting that it was showed,
or by having an object out of context in that setting
(Jones et al., 2008). These studies try to support the
idea that the user suffers the influence of his prefer-
ences in the direction of attention.
In a more recent study from Kinnealey & Pfeif-
fer (Kinnealey et al., 2012), the authors create an in-
stallation of sound-absorbing walls and halogen light-
ing to increased sensory comfort for the users thus af-
fect their engagement in classroom learning activities,
having denoted an increased frequency on attendance.
Other possibilities concern targetting their inter-
ests to keep their focus. Boyd performed a study
where he compared the effects of circumscribed in-
terests to less preferred tangible stimuli on the social
behaviors, and the results showed an increase in social
interactions in tests with circumscried interests (Boyd
et al., 2007), showing that using the target of their in-
terests, helps to keep them attentive and motivated.
Still, many studies present contraditory findings,
highlighting the diversity of disturbances in children
with ASD, and the need to perform further studies
(Ames and Fletcher-Watson, 2010). Our work ex-
plores a new methodology, based on real-time atten-
tion sensing and on the use of immediate feedback to
improve attention and engagement.
4 ATTENTION DETECTION
MODULE
To detect the user attention while using
[e-mail], which will be used to evaluate
the child’s engagement in the platform, we used
the Python OpenCV wrapper
1
. For this feature
the [e-mail] framework uses the computer’s
camera, together with a Haar Feature-based Cascade
Classifier for Object Detection for automated detec-
tion of the face of the child. Using the frontal and
profile face cascades, if the child is not in front of the
computer, or not looking at the screen, the platform
detects it, and records the distraction state in the log
file; on the other hand, when the child returns to the
computer or looks again at the screen, the attention
state is detected and recorded in the log file. Figure 4
illustrates the attention and distraction cases.
For the tests, a log system was implemented in
the [e-mail] platform, which saved the current
user, together with the timestamps in which we looked
at the screen, and in which he looked away. Tak-
ing into account a physiological head movement fre-
quency limit of 0.5Hz (Calais-Germain, 2007; Ber,
1992), the OpenCV was setup to only check for the
head position every 2 seconds.
5 PROCEDURE
Trying to gather attention on children with ASD can
be problematic, since that can cause them stress, mak-
ing them uncomfortable or even become violent. To
1
http://opencv.willowgarage.com/documentation/python
/index.html
InducingBehaviorChangeinChildrenwithAutismSpectrumDisordersbyMonitoringtheirAttention
133

(a) The child is in front
of the computer and
looking at the screen.
(b) The child is in front
of the computer but is not
looking at the screen.
Figure 4: Attention detection while using the
[e-mail] platform.
Table 1: Children evaluated.
Number of Number of Number of
Sessions Children Computers
Sound 12 6 1
No Sound 12 6 1
Total 24 12 2
prevent this, we decided to use a sound that was fa-
miliar for the children, and that was already vali-
dated as acceptable for them for the children, without
any negative reaction. So for our experimental tests
on attention management, we used a recording with
the sentence ”Hi, I am Troc@s”, which is a part of
the welcome sound played whenever the platform is
launched, thus being a familiar and recognizable ele-
ment for which they may respond without triggering
any unexpected or unwanted reactions. Our hypothe-
sis for this part of the work was that when feedback is
provided to the child, he/she will be more attentive.
For our tests, we evaluated 12 children over the
course of 24 sessions, 2 per user, in 2 computers (both
with the same look-and-feel and content configura-
tion): an experimental group of 6 children used the
platform on one computer for a total of 12 sessions,
with the automated attention management module ac-
tivated to try and recover the child’s attention; a con-
trol group of 6 children used the platform on the other
computer for a total of 12 sessions, with the attention
management module disabled. We can see a summary
in Table 1. This setting allows us to assess the im-
pact that the attention detection and real-time acous-
tic feedback has in re-engaging the child and chang-
ing the focus behaviors, after the child’s attention is
diverted.
The sessions are organized and assisted by the tu-
tors, which help the children to perform the tasks that
they think are suitable for them; as previously men-
tioned, all of the tasks are mostly related with commu-
nication development. A statistical occurrence analy-
sis of the logs was performed, to characterize the fre-
quency of distraction events, and the latency between
the detection of a distraction event and the recovery
back to an attentive state.
Table 2: Percentage of sessions in which a child having a
type of distraction.
Momentary Recovered Absent
(%) (%) (%)
Sound 23% 65% 11%
No Sound 42% 54% 3.5%
Table 3: Percentage of sessions in which a child is focused.
Focused
(%)
Sound 50%
No Sound 25%
We classified distraction events into three cate-
gories: a) Momentary: events with less than 3 sec-
onds duration, corresponding to scenarios such as mo-
mentarily looking at a colleague or the tutor; b) Re-
covered: events where the user looks away for more
than 3 seconds, but recovers the attention within up
to 30 seconds; and c) Absent: events where the user
looks away for more than 30 seconds, to perform
tasks that may not be necessarily related with the
[e-mail] platform such as personal hygiene and
others.
6 RESULTS
The metrics to measure the results, that we have
adopted are as follows:
• Focused (%): the percentage of sessions in which
the child begins and ends the session without di-
verting the attention from the computer;
• Momentary (%): the percentage of identifiable
momentary events within the collected data;
• Recovered (%): the percentage of identifiable re-
covered events within the collected data;
• Absent (%): the percentage of identifiable absent
events within the collected data;
As we can see from Table 2 and Table 3, results
show that there are more children focused with the
real-time acoustic feedback, with 50% of the sessions
corresponding to cases where the child always looks
at the screen during a session. Also, there is a lower
number of distractions per session. Another inter-
esting aspect is the fact that momentary distractions
are less frequent in the group that uses the acoustic
PhyCS2014-InternationalConferenceonPhysiologicalComputingSystems
134

feedback. Despite the percentage of the focused chil-
dren being higher, the group with acoustic feedback
exhibits more moments where they are distracted for
more than 3 seconds, the most frequent case being
distractions of up to 30 seconds (65% of the cases).
7 DISCUSSION
With the sound to trigger the child attention, results
show that the users look away from the screen less
times, distracting themselves less often. Also they
have less momentary distractions (less or equal to 3
seconds without looking at the computer screen), and
less absent distractions (more than 30 seconds without
looking at the computer screen). Despite the fact that
there is a higher number of sessions where the child is
focused from start to finish in the experimental group,
in the computer without sound, they regain their at-
tention more quickly, which can result from external
action of the teacher in instructing the child to return
to an attentive state.
Although during a normal session with the plat-
form there might be external factors that we do con-
trol automatically, like absence from the computer to
go to the bathroom, to answer a tutor, or performing a
non computer-mediated task that a tutor told the child
to do during a session, acoustic feedback can be an
important feature to induce a behavior change in the
child toward improved attention. When no feedback
is provided, the use of the computer flows as in a nor-
mal session, and often the teacher might need to sup-
port the child in regaining attention.
8 CONCLUSIONS
In general, children have been able to achieve signifi-
cant outcomes, and shown increased interest in using
the platform, with clear increases in autonomy and
proficiency in its use. Several comments and mani-
festations were registered on this respect, such as: a)
Frequent requests to share the platform with other col-
leagues, tutors, and technicians; b) Requests to use
the platform event out of their assigned schedule to
use it; c) Refuse to attend classes to keep using the
platform; and even d) Interest on behalf of student
without special educational needs to use the platform
to interact with their colleagues with ASD.
From the tutors evaluation, the message board
seems to favor the capability of understanding mes-
sages and orders, and students revealed a higher level
of initiative and autonomy in exchanging messages,
as well as greater capacity of interpreting information
and producing adequate responses; two of the stu-
dents that have not yet developed reading and writing
competences, manifested a growing curiosity in deci-
phering the written code as a communication means.
Automations such as the attention detection module
have also been showing a meaningful positive impact
in the everyday routines performed by the children. In
our attention tests the number of times in which the
children look away from the computer is higher in the
control group (without acoustic feedback). This ex-
perimental analysis allowed us to conclude that the at-
tention detection module would be an important add-
on while evaluating the children using a computer.
Today, society is increasingly supported in so-
cial interaction and communication, which are both
fundamental pillars for the complete integration of
all individuals. In this sense, children with spe-
cial needs present novel and remarkable challenges
due to their condition, in particular in the context
of ASDs. Still, computer-mediated approaches, can
help reshape the learning strategies, induce new be-
haviors, and contribute to the inclusion of these sub-
jects. [e-mail] has been contributing for this
change, by providing tutors with a versatile and easy-
to-use platform to explore the individual specificities
of each user, and by giving children additional tools
to overcome some of their barriers and stimulate new
communicative and social interaction behaviors.
ACKNOWLEDGEMENTS
This work has been partially funded by Fundac¸
˜
ao
Portugal Telecom, ASUS Portugal, and by the
Fundac¸
˜
ao para a Ci
ˆ
encia e Tecnologia (FCT) under
the grant SFRH/BD/65248/2009 and project PEst-
OE/EEI/LA0021/2013. The authors would also like
to acknowledge the special education teacher Carla
Sim
˜
oes (Agrupamento Vertical de Escolas do Vale da
Amoreira) for her support in the results of the prelim-
inary tests, and in the preparation of the new tests to
assess the attention engagement.
REFERENCES
(1992). The Head-Neck Sensory Motor System. Oxford
University Press, USA, 1 edition.
Ames, C. and Fletcher-Watson, S. (2010). A review of
methods in the study of attention in autism. Devel-
opmental Review, 30(1):52–73.
Boyd, B. A., Conroy, M. A., Mancil, G. R., Nakao, T., and
Alter, P. J. (2007). Effects of circumscribed interests
on the social behaviors of children with Autism Spec-
InducingBehaviorChangeinChildrenwithAutismSpectrumDisordersbyMonitoringtheirAttention
135

trum Disorders. Journal of Autism and Developmental
Disorders, 37(8):1550–1561.
Calais-Germain, B. (2007). Anatomy of Movement (Revised
Edition). Eastland Press, revised edition.
Dalton, K. M., Nacewicz, B. M., Johnstone, T., Schaefer,
H. S., Gernsbacher, M. A., Goldsmith, H. H., Alexan-
der, A. L., and Davidson, R. J. (2005). Gaze fixation
and the neural circuitry of face processing in autism.
Nature Neuroscience, 8(4):519–526.
Jones, W., Carr, K., and Klin, A. (2008). Absence of pref-
erential looking to the eyes of approaching adults pre-
dicts level of social disability in 2-year-old toddlers
with autism spectrum disorder. Archives of general
psychiatry, 65(8):946–954.
Kinnealey, M., Pfeiffer, B., Miller, J., Roan, C., Shoener, R.,
and Ellner, M. L. (2012). Effect of classroom modi-
fication on attention and engagement of students with
autism or dyspraxia. The American journal of occu-
pational therapy, 66(5):511–519.
Leekam, S., Baron-Cohen, S., Perrett, D., Milders, M., and
Brown, S. (1997). Eye-direction detection: A disso-
ciation between geometric and joint attention skills in
autism. British Journal of Developmental Psychology,
15(1):77–95.
Lucas da Silva, M., Gonc¸alves, D., Guerreiro, T., and Silva,
H. (2012). A Web-based Application to Address In-
dividual Interests of Children with Autism Spectrum
Disorders. Procedia Computer Science, 14:20–27.
Lucas da Silva, M., Sim
˜
oes, C., Gonc¸alves, D., Guer-
reiro, T., Silva, H., and Botelho, F. (2011). TRO-
CAS: Communication Skills Development in Chil-
dren with Autism Spectrum Disorders via ICT
Human-Computer Interaction INTERACT 2011. vol-
ume 6949 of Lecture Notes in Computer Science,
chapter 103, pages 644–647. Springer Berlin / Hei-
delberg, Berlin, Heidelberg.
Pelphrey, K. A., Sasson, N. J., Reznick, J. S., Paul, G.,
Goldman, B. D., and Piven, J. (2002). Visual scanning
of faces in autism. Journal of autism and developmen-
tal disorders, 32(4):249–261.
Posner, M. I., Walker, J. A., Friedrich, F. J., and Rafal, R. D.
(1984). Effects of parietal injury on covert orienting
of attention. The Journal of neuroscience : the official
journal of the Society for Neuroscience, 4(7):1863–
1874.
PhyCS2014-InternationalConferenceonPhysiologicalComputingSystems
136