A Comparative Study of the Learning Curve of a Novel
Interface for the Deafblind: Implications for
Educational Environments
Nicholas Caporusso
1,2
and Giovanni Perrone
1
1
QIRIS - Quality Innovation Research Instruction Safety, Bari, Italy
2
INTACT Healthcare, Bari, Italy
Abstract. In addition to communication and mobility issues, the blind and the
deafblind suffer from poor social inclusion and illiteracy, and extremely low em-
ployment rates. This is particularly due to lack of access to dedicated educational
resources and to technology for learning.
In this paper, we introduce dbGLOVE, an innovative and low-cost wearable in-
terface based on a tactile communication system especially designed for the deaf-
blind. We compare the performances of dbGLOVE with the state-of-the-art tech-
nology for the blind and the deafblind, i.e., the Braille display.
Results from our study show that dbGLOVE has a faster learning curve, and it
outperforms the Braille display both in accuracy and in speed. Finally, we discuss
the implications of our ndings in regard to learning and we detail the features
that render dbGLOVE particularly suitable for educational purposes.
1 Introduction
According to demographic research [1], the world’s blind population is estimated be-
tween 40 and 45 millions. Every year, 7 million people become blind, that is, two indi-
viduals every 10 seconds (one child per minute) lose their vision. Recent studies antic-
ipate that all blind-related numbers will double by the year 2020 [2]. Although less is
known about the deafblind due to poor statistics, they account for 0.016% of the popula-
tion. Despite being a minority of the world population, they pose additional challenges
in terms of public health and social security, and education. Limited access to assistants,
resources, and technology, prevent deafblind people from achieving autonomous living,
independent mobility, social inclusion, and adequate literacy [3]. Only 10% of children
receive education due to the unavailability and to the cost of both specialized teachers
and technology. As a result, school and working-age blind have very high unemploy-
ment rates (about 75%).
In addition to technologyfor Augmentativeand AlternativeCommunication (AAC),
the deafblind require dedicated systems for getting access to computers, mobile devices,
and to the Internet [3]. In the last decades, a variety of interactive communication tech-
nology have been introduced in the market. Nevertheless, users still rely on the constant
presence of assistants in order to communicate, to learn, and even to read simple text.
Caporusso N. and Perrone G..
A Comparative Study of the Learning Curve of a Novel Interface for the Deafblind: Implications for Educational Environments.
DOI: 10.5220/0004609701150122
In Proceedings of the 2nd International Workshop on Interaction Design in Educational Environments (IDEE-2013), pages 115-122
ISBN: 978-989-8565-65-5
Copyright
c
2013 SCITEPRESS (Science and Technology Publications, Lda.)
Indeed, they need to be autonomous in achieving high quality educational resources
[5]. Assistive technology is crucial for providing people with disabilities and cognitive
impairments with means for social inclusion, access to information, and learning [4].
Systems, such as the Braille display, have been used by the visually impaired for years,
and they are the state-of-the-art devices for communicating and interacting with the out-
side world. Unfortunately, they are extremely expensive. As a result, they are available
in certain communities and schools, only, and they are not affordable by the majority
of target users, who usually have insufficient financial resources. This is especially true
for developing countries, where 90 percent of the blind and deafblind population live.
In this paper, we introduce dbGLOVE, an innovative low-cost wearable device for
interacting with computers and with smartphones [6]. dbGLOVE is a natural interface
based on the Malossi alphabet, a simplified tactile communication system invented by a
deafblind. Both the Malossi and the Braille alphabets were invented by individuals with
sensory impairments, and both rely on prior language training, because they conform to
the syntax and grammar of common verbal languages. However, as the Malossi alpha-
bet does not require sophisticate sensory or cognitive abilities, it is especially suitable
for children education. Also, dbGLOVE is fifteen times cheaper than Braille displays.
Thus, we propose dbGLOVE as a replacement for the Braille alphabet for improving
learning speed and engagement, and for democratizing access to technology and educa-
tion. To this end, we evaluate the performances of dbGLOVE in training users who are
novice of both the language and the device. Specifically, we compare the learning curve
of the Malossi alphabet with a standard Braille display, and we show that dbGLOVE
outperforms the Braille display by 200% both in accuracy and in speed.
2 Related Work
The Braille code is the most famous and adopted system for encoding text in a tactile
form. It utilizes series of raised dots to form letters: each symbol is represented us-
ing a cell consisting of six dots that can be raised or flat in order to obtain different
configurations. As alphabet consists of 6 dots each assuming two values (i.e., raised
or flat), it supports 64 configurations. Nevertheless, it has a very powerful encoding:
there are conventions for associating different meanings to the same configuration, and
for switching between domains (e.g., music, or mathematics). Words are written as se-
quences of adjacent cells. These can be read by people who are blind (or whose sight
is not sufficient for reading printed material) with their fingers, by simply passing the
fingertip over the cells. Usually, teachers, parents, and others who are not visually im-
paired can read Braille dots with their eyes.
Indeed, as Braille is a code by which languages (e.g., English) can be written and
read, Braille readers are required to learn the alphabet, the grammar and the syntax, be-
fore they can communicate. Although studies in the literature demonstrate the efficiency
of the Braille alphabet in encoding information, the Braille alphabet has important lim-
itations from a learning perspective. Specifically, in order to understand Braille, readers
should be able to explore and recognize similarities and differences in objects and mate-
rials. This can be especially difficult in case of cognitive or sensory impairments, which
usually occur in case of deafblindness. In general, discriminating the dots and associ-
116
ating configurations to letters has a long learning curve, and it requires extensive effort
from both the learner and the assistant. Consequently, Braille displays are not suitable
for supporting education of children in their pre-school age and in K-12.
Over the years, several simplified communication systems, such as on-body sign-
ing, have been developed in order to provide children (even infants) and people in their
early stage of blindness with alternative means for basic learning and for interacting
with others. The Malossi alphabet is among the most popular codes based on on-body
signing. It is named after his inventor, an Italian who became deafblind in his early life.
The Malossi system defines an on-hand signing method and a tactile alphabet based on
two types of stimuli: touch and pinch. Figure 1 shows the configuration of the language.
Letters from A to O are distributed over the palm, on the 15 phalanxes from the thumb
to the little finger, in a clockwise fashion. Each phalanx is associated with a different
letter depending on the touch cue (i.e., touch or pinch). Letters from P to Z (excluding
the letter W , which is located close to the proximal phalanxes between the second and
the third metacarpal bones) are distributed over the distal and the proximal phalanxes,
and they are activated when the area is pinched.
G
F
H
I
J
B
Q
C
R
D
S
E
T
A
P
L
V
M
X
N
Y
O
Z
K
U
W
pressed
pinched
Fig.1. Layout of the Malossi alphabet over the palm of the left hand.
Two deafblind individuals communicate using Malossi method as follows: the hand
(usually, the left one) becomes a typewriter for the receiver of the message. As a result,
they type messages on each other’s hand, in turns: the sender writes words by subse-
quently touching and pinching in sequence different parts of the receiver’s phalanxes
that correspond to the alphabet. Then, they exchange their roles in order to achieve
bidirectional communication. This method is used by those who had learned to read
and write prior to becoming deafblind. Nonetheless, as the alphabet is based on simple
touch cues, phalanxes can be associated with elementary objects (e.g., water, mom) and
actions (e.g., eat) in order to achieve functional communication. As a consequence, the
Malossi alphabet is effectively employed with infants. Then, the communication system
can be evolved into an alphabet. Consequently, the speed at which two deafblind people
can communicate using the Malossi alphabet is impressive.
3 dbGLOVE
Research on vibrotactile perception showed that vibration can stimulate the skin in a
117
way that it can induce a large set of sensations ranging from soft displacement to painful
cues (depending on the waveform, and on its intensity and frequency). Nowadays,
miniaturized motors, similar to those employed in smartphones, can be utilized as inex-
pensive means to provide individuals with sophisticated vibrotactile feedback that can
simulate touch cues. These, in turn, can provide users with means for easily recognizing
on-body signing. As a result, vibrotactile interfaces for touch-based communication are
emerging as an alternative to conventional assistive technology for AAC [7], [8].
dbGLOVE is an interactive glove based on the Malossi alphabet. It provides blind
and deafblind people with bidirectional interaction with the computer and computer-
mediated communication with others. The deafblind can wear the device on the left
hand, and they can type messages on their own palm, as on a keyboard. The device in-
corporates an array of sensors and actuators into a pad that can be worn on the palm of
left hand as if it was a glove. This can be connected to a computer, or to a smartphone.
As a result, the deafblind can type on their own hand, instead of that of the receiver.
Input is acquired and processed as a command to the PC (e.g., open application), or as
a message to be displayed to another individual (e.g., I want to eat). Also, dbGLOVE
includes a tactile monitor. So, the deafblind can receive messages in the form of tactile
stimulations, as if someone was typing on their palm. Responses can be received by
the user in the form of vibrotactile stimulation at different intensity and frequency that
simulate touch and pinch cues, as if someone was typing on their hand. As a result, the
device is able to provide the user with bidirectional communication. Figure 2 shows
the device, whose architecture is extensively described in [6].
Fig.2. A prototype of dbGLOVE: the white dots correspond to input/output areas.
4 Experimental Study
In this study, our objective was measuring the learning curve of the Malossi alphabet
with respect to the Braille system by comparing the performance of dbGLOVE with a
standard Braille cell. The main purpose of the experiment was to understand whether
dbGLOVE and the Malossi alphabet can improve learning. To this end, we focused on
output, only, which is the most interesting feature with respect to education: enabling
users to autonomously read documents without requiring the constant presence of a
118
interpreter is a crucial issue. Specifically, considered speed and accuracy over a fixed
number of runs that gave users sufficient training time to learn to associate symbols to
stimuli.
4.1 Experimental Tasks
We designed two similar experimental tasks, one focusing on dbGLOVE and one on the
Braille display. As the experiment focuses on the output layer, in order to realize a com-
parison between the training time required by dbGLOVE with respect to a Braille-based
device, we evaluate subjects’ accuracy and speed in recognizing letters in the form of
vibrotactile or pressure cues, respectively. To this end, in each of the two tasks, partici-
pants were involved in a guided output routine: they were presented with sequences of
letters randomly chosen, and they were required to speak them back to the technician.
We employed single letter instead of using words, as the latter could have introduced
some error in the experiment routine. Specifically, the predictability of the last letters
could have biased the experimental results. Differently from other experiments, as both
the Malossi and the Braille alphabets were given to subjects without prior training,
learning required some time. All the tasks consisted of 30 runs each consisting in 240
seconds, with an inter-run interval of 2 minutes. Each trial had duration of 5 seconds.
Runs were divided into groups of 10, and each group was executed within several days
from one another.
In task I, we evaluated the learning curve of dbGLOVE with respect to the output
function in sending meaningful tactile stimuli to the user. Participants were presented
with sequences of letters represented into a vibrotactile form by the actuators embedded
into the device. The different areas of the hand associated with the letters were stimu-
lated with vibrotactile patterns simulating touch and pinch cues, and participants were
asked to speak the letter to the technician. The objective of the subject was identify-
ing as many letters as possible. The procedure in Task II was exactly the same as that
in Task I. We evaluated the performance of Braille cells in sending perceivable tactile
stimuli to the user. Participants were presented with sequences of letters via a single
Braille cell. They were asked to decode the configuration of the dots, and to speak the
letter back to the experimental technician.
4.2 Participants
13 volunteer participants were recruited for this experiment. They were 5 female and
8 male. All had a normal sight, hearing and tactile sensitivity. Subjects ranged in age
from 18 to 25 with an average of 22. All use computers on a daily basis (1.5-8 hours
usage per day). They were all novice of the Malossi alphabet and of the Braille system.
All had no prior knowledge of the device; two of them had prior experience with vibro-
tactile feedback, as they were involved in other studies about tactile feedback. Subjects
participated on a voluntary basis and they were not paid or rewarded. All subjects were
right-handed as assessed by the Edinburgh inventory [9]. All subjects were prepared
to the experiment by a technician who gave them instructions about the test and the
experimental tasks.
119
4.3 Experimental Setup and Protocol
A device consisting of one piezoelectric Braille cell was utilized in Task II. We em-
ployed an International Building Standard [10] compliant cell (2.5mm for horizontal
and vertical dot-to-dot distance, with a dot diameter of 1.5mm - 1.6mm and a dot height
ranging from 0.6mm to 0.9mm). Its activation was 100 milliseconds, which is compa-
rable with that of vibrotactile actuators, given the timing of the experimental task. Prior
to the experiment, subjects were provided with a preliminary explanation of the Mal-
ossi and the Braille alphabets. Also, before each task, they were given sufficient time
to manipulate the stimulation devices. In each trial, individuals had 5 seconds to speak
the letter back to the technician. We evaluated the training level by comparing speed
(measured in CPM). Moreover, we logged the accuracy in recognizing letters. In this
regard, we associated a trial timeout to an error, as if the wrong letter was recognized.
With respect to the task involving Malossi, we also logged the number of errors due to
germane load, that is, letters biased due to the fact that they are on the same phalanx.
5 Results and Discussion
We evaluatedlanguage proficiencyusing speed and accuracyas the main metrics. Specif-
ically, we adopted the number of Characters Per Minute (CPM) as a standard perfor-
mance measure, and we distinguished correctly written and wrong characters. Also, we
logged the time users spent in recognizing each letter, in order to identify the symbols
that required more effort, and to evaluate the presence of germane load. All the subjects
were able to understand the task. We utilized sequences of characters, as this is an easier
way to measure subjects’ ability in recognizing vibrotactile stimuli in the different areas
of the hand, and to associate touch and pinch cues to letters of the alphabet. Participants
were only required to speak the letter to the technician, who recorded the answer as
correct or wrong. This was to avoid subjects to actually type the letter on a keyboard,
which could have introduced some additional cognitive load.
Figure 3 represents the experimental results about speed and accuracy. In order to
evaluate speed, we calculated the number of letters that subjects were able to process,
that is, the number of answers they were able to give before the run timeout. Subjects
started at very low speed. Initially, participants were able to recognize only a few of
the characters being displayed, with an average of 30.82 and 45.29 letters displayed to
participants using the Braille cell and dbGLOVE, respectively. During the experiment,
we registered an increasing trend in speed. Also, the training effects persists until the
last run, when the performances of dbGLOVE and the Braille cell are 212.17 and 93.24
letters displayed (on average), respectively. Consequently, the improvement is +166.87
and +62.41 with our device and with the Braille cell.
In regard to accuracy, results show an increasing curve in the use of both devices.
Participants using dbGLOVE started at 20.11%, and they increased their performance
by 76.3% after 30 runs, thus, reaching an average accuracy of 96.41%. The training
effect vanished at run 24, when subjects’ performances were over 90%. Conversely, us-
ing the Braille cell, participants found more difficult to decode the letter by reading the
dots. They began with an accuracy of 15.14%, and they improved their performance by
44.01%. In run 30, subjects’ accuracy was 59.15%, -37.26% with respect to dbGLOVE.
120
Speed in recognizing output
0
50
100
150
200
250
1
Speed
Runs
braille
malossi
Accuracy in recognizing output
0
10
20
30
40
50
60
70
80
90
100
1
Accuracy
Runs
braille
malossi
30
30
Fig.3. Experimental results: performances with dbGLOVE and with the Braille cell.
6 Conclusions
Although the Braille alphabet is the most widely adopted system for enabling blind and
deafblind people to read text, it has limitations in terms of technological implementa-
tion and costs. Also, it requires cognitive abilities and sensory perception to be intact.
Conversely, teaching Augmentative and Alternative Communication requires systems
that can cope with developmental, physical, and sensory impairments [3]. In this re-
gard, the Malossi alphabet is extremely intuitive, and it relies on basic touch cues. It
is widely employed with people having cognitive impairments, who cannot learn more
complex communication methods, such as alphabets involving shapes.
In this paper, we compared the Malossi alphabet and the Braille system with the
purpose of evaluating their learning curve. Our main hypothesis was that dbGLOVE is
suitable for substituting Braille displays in everyday interaction, and particularly, dur-
ing early life or in the first stage of deafblindness, that is, when individuals require an
immediate system for basic communication. As shown by our results, the Malossi al-
phabet implemented in dbGLOVE outperforms the Braille alphabet both in speed and
accuracy, in people with no previous training. As a result, dbGLOVE can be utilized as
a substitute of systems implementing the Braille alphabet, especially in circumstances
in which a shorter learning curve is required. The present study has several limitations.
Being an early-stage prototype, we evaluated dbGLOVE with people having normal
vision and hearing. Therefore, our results should be validated with further experiments
with blind and deafblind users, which will be part of our future work.
References
1. Mller, E. Deaf-Blind Child Counts: Issues and Challenges, National Consortium on Deaf-
Blindness, 2006.
2. Dandona L, Dandona R, John R. K. Estimation of blindness in India from 2000 through
2020: implications for the blindness control policy, Natl Med J India. 2001 Nov-
Dec;14(6):327-34.
121
3. Sigafoos, J.; Didden, R.; Schlosser, R.; Green, V.; OReilly, M. & Lancioni, G. A Review
of Intervention Studies on Teaching AAC to Individuals who are Deaf and Blind Journal of
Developmental and Physical Disabilities, Springer US, 2008, 20, 71-99
4. Hersh, M. A. & Johnson, M. A. Assistive Technology for Education, Employment and
Recreation Assistive Technology for Visually Impaired and Blind People, Springer London,
2008, 659-707
5. Al-Salman, A. S. A Bi-directional Bi-Lingual Translation Braille-Text System Journal of
King Saud University - Computer and Information Sciences , 2008, 20, 13 - 29
6. Caporusso, N. A wearable Malossi alphabet interface for deafblind people Proceedings of
the working conference on Advanced visual interfaces, ACM, 2008, 445-448
7. Bark, K., Wheeler, J. W., Premakumar, S. , Cutkosky, M.R. Comparison of Skin Stretch and
Vibrotactile Stimulation for Feedback of Proprioceptive Information. Haptic interfaces for
virtual environment and teleoperator systems, 2008, Pages 71-78.
8. Lam, Amy R. Vibrotactile pattern recognition on the torso with one and two dimensional
displays. PhD Dissertation, 2006. Online at http://dspace.mit.edu/handle/1721.1/36738
9. Oldfield, R. C. (1971). The assessment and analysis of handedness: The Edinburgh inven-
tory. Neuropsychologia, 9, 97-113
10. RNIB Scientific Research Unit Scientific and technological reports ”Braille Cell Dimen-
sions” (online)
122