A Comparative Study of the Learning Curve of a Novel

Interface for the Deafblind: Implications for

Educational Environments

Nicholas Caporusso

1,2

and Giovanni Perrone

QIRIS - Quality Innovation Research Instruction Safety, Bari, Italy

INTACT Healthcare, Bari, Italy

Abstract. In addition to communication and mobility issues, the blind and the

deafblind suffer from poor social inclusion and illiteracy, and extremely low em-

ployment rates. This is particularly due to lack of access to dedicated educational

resources and to technology for learning.

In this paper, we introduce dbGLOVE, an innovative and low-cost wearable in-

terface based on a tactile communication system especially designed for the deaf-

blind. We compare the performances of dbGLOVE with the state-of-the-art tech-

nology for the blind and the deafblind, i.e., the Braille display.

Results from our study show that dbGLOVE has a faster learning curve, and it

outperforms the Braille display both in accuracy and in speed. Finally, we discuss

the implications of our ﬁndings in regard to learning and we detail the features

that render dbGLOVE particularly suitable for educational purposes.

1 Introduction

According to demographic research [1], the world’s blind population is estimated be-

tween 40 and 45 millions. Every year, 7 million people become blind, that is, two indi-

viduals every 10 seconds (one child per minute) lose their vision. Recent studies antic-

ipate that all blind-related numbers will double by the year 2020 [2]. Although less is

known about the deafblind due to poor statistics, they account for 0.016% of the popula-

tion. Despite being a minority of the world population, they pose additional challenges

in terms of public health and social security, and education. Limited access to assistants,

resources, and technology, prevent deafblind people from achieving autonomous living,

independent mobility, social inclusion, and adequate literacy [3]. Only 10% of children

receive education due to the unavailability and to the cost of both specialized teachers

and technology. As a result, school and working-age blind have very high unemploy-

ment rates (about 75%).

In addition to technologyfor Augmentativeand AlternativeCommunication (AAC),

the deafblind require dedicated systems for getting access to computers, mobile devices,

and to the Internet [3]. In the last decades, a variety of interactive communication tech-

nology have been introduced in the market. Nevertheless, users still rely on the constant

presence of assistants in order to communicate, to learn, and even to read simple text.

Caporusso N. and Perrone G..

A Comparative Study of the Learning Curve of a Novel Interface for the Deafblind: Implications for Educational Environments.

DOI: 10.5220/0004609701150122

In Proceedings of the 2nd International Workshop on Interaction Design in Educational Environments (IDEE-2013), pages 115-122

ISBN: 978-989-8565-65-5

 2013 SCITEPRESS (Science and Technology Publications, Lda.)

Indeed, they need to be autonomous in achieving high quality educational resources

[5]. Assistive technology is crucial for providing people with disabilities and cognitive

impairments with means for social inclusion, access to information, and learning [4].

Systems, such as the Braille display, have been used by the visually impaired for years,

and they are the state-of-the-art devices for communicating and interacting with the out-

side world. Unfortunately, they are extremely expensive. As a result, they are available

in certain communities and schools, only, and they are not affordable by the majority

of target users, who usually have insufﬁcient ﬁnancial resources. This is especially true

for developing countries, where 90 percent of the blind and deafblind population live.

In this paper, we introduce dbGLOVE, an innovative low-cost wearable device for

interacting with computers and with smartphones [6]. dbGLOVE is a natural interface

based on the Malossi alphabet, a simpliﬁed tactile communication system invented by a

deafblind. Both the Malossi and the Braille alphabets were invented by individuals with

sensory impairments, and both rely on prior language training, because they conform to

the syntax and grammar of common verbal languages. However, as the Malossi alpha-

bet does not require sophisticate sensory or cognitive abilities, it is especially suitable

for children education. Also, dbGLOVE is ﬁfteen times cheaper than Braille displays.

Thus, we propose dbGLOVE as a replacement for the Braille alphabet for improving

learning speed and engagement, and for democratizing access to technology and educa-

tion. To this end, we evaluate the performances of dbGLOVE in training users who are

novice of both the language and the device. Speciﬁcally, we compare the learning curve

of the Malossi alphabet with a standard Braille display, and we show that dbGLOVE

outperforms the Braille display by 200% both in accuracy and in speed.

2 Related Work

The Braille code is the most famous and adopted system for encoding text in a tactile

form. It utilizes series of raised dots to form letters: each symbol is represented us-

ing a cell consisting of six dots that can be raised or ﬂat in order to obtain different

conﬁgurations. As alphabet consists of 6 dots each assuming two values (i.e., raised

or ﬂat), it supports 64 conﬁgurations. Nevertheless, it has a very powerful encoding:

there are conventions for associating different meanings to the same conﬁguration, and

for switching between domains (e.g., music, or mathematics). Words are written as se-

quences of adjacent cells. These can be read by people who are blind (or whose sight

is not sufﬁcient for reading printed material) with their ﬁngers, by simply passing the

ﬁngertip over the cells. Usually, teachers, parents, and others who are not visually im-

paired can read Braille dots with their eyes.

Indeed, as Braille is a code by which languages (e.g., English) can be written and

read, Braille readers are required to learn the alphabet, the grammar and the syntax, be-

fore they can communicate. Although studies in the literature demonstrate the efﬁciency

of the Braille alphabet in encoding information, the Braille alphabet has important lim-

itations from a learning perspective. Speciﬁcally, in order to understand Braille, readers

should be able to explore and recognize similarities and differences in objects and mate-

rials. This can be especially difﬁcult in case of cognitive or sensory impairments, which

usually occur in case of deafblindness. In general, discriminating the dots and associ-

116

ating conﬁgurations to letters has a long learning curve, and it requires extensive effort

from both the learner and the assistant. Consequently, Braille displays are not suitable

for supporting education of children in their pre-school age and in K-12.

Over the years, several simpliﬁed communication systems, such as on-body sign-

ing, have been developed in order to provide children (even infants) and people in their

early stage of blindness with alternative means for basic learning and for interacting

with others. The Malossi alphabet is among the most popular codes based on on-body

signing. It is named after his inventor, an Italian who became deafblind in his early life.

The Malossi system deﬁnes an on-hand signing method and a tactile alphabet based on

two types of stimuli: touch and pinch. Figure 1 shows the conﬁguration of the language.

Letters from A to O are distributed over the palm, on the 15 phalanxes from the thumb

to the little ﬁnger, in a clockwise fashion. Each phalanx is associated with a different

letter depending on the touch cue (i.e., touch or pinch). Letters from P to Z (excluding

the letter W , which is located close to the proximal phalanxes between the second and

the third metacarpal bones) are distributed over the distal and the proximal phalanxes,

and they are activated when the area is pinched.

pressed

pinched

Fig.1. Layout of the Malossi alphabet over the palm of the left hand.

Two deafblind individuals communicate using Malossi method as follows: the hand

(usually, the left one) becomes a typewriter for the receiver of the message. As a result,

they type messages on each other’s hand, in turns: the sender writes words by subse-

quently touching and pinching in sequence different parts of the receiver’s phalanxes

that correspond to the alphabet. Then, they exchange their roles in order to achieve

bidirectional communication. This method is used by those who had learned to read

and write prior to becoming deafblind. Nonetheless, as the alphabet is based on simple

touch cues, phalanxes can be associated with elementary objects (e.g., water, mom) and

actions (e.g., eat) in order to achieve functional communication. As a consequence, the

Malossi alphabet is effectively employed with infants. Then, the communication system

can be evolved into an alphabet. Consequently, the speed at which two deafblind people

can communicate using the Malossi alphabet is impressive.

3 dbGLOVE

Research on vibrotactile perception showed that vibration can stimulate the skin in a

117

way that it can induce a large set of sensations ranging from soft displacement to painful

cues (depending on the waveform, and on its intensity and frequency). Nowadays,

miniaturized motors, similar to those employed in smartphones, can be utilized as inex-

pensive means to provide individuals with sophisticated vibrotactile feedback that can

simulate touch cues. These, in turn, can provide users with means for easily recognizing

on-body signing. As a result, vibrotactile interfaces for touch-based communication are

emerging as an alternative to conventional assistive technology for AAC [7], [8].

dbGLOVE is an interactive glove based on the Malossi alphabet. It provides blind

and deafblind people with bidirectional interaction with the computer and computer-

mediated communication with others. The deafblind can wear the device on the left

hand, and they can type messages on their own palm, as on a keyboard. The device in-

corporates an array of sensors and actuators into a pad that can be worn on the palm of

left hand as if it was a glove. This can be connected to a computer, or to a smartphone.

As a result, the deafblind can type on their own hand, instead of that of the receiver.

Input is acquired and processed as a command to the PC (e.g., open application), or as

a message to be displayed to another individual (e.g., I want to eat). Also, dbGLOVE

includes a tactile monitor. So, the deafblind can receive messages in the form of tactile

stimulations, as if someone was typing on their palm. Responses can be received by

the user in the form of vibrotactile stimulation at different intensity and frequency that

simulate touch and pinch cues, as if someone was typing on their hand. As a result, the

device is able to provide the user with bidirectional communication. Figure 2 shows

the device, whose architecture is extensively described in [6].

Fig.2. A prototype of dbGLOVE: the white dots correspond to input/output areas.

4 Experimental Study

In this study, our objective was measuring the learning curve of the Malossi alphabet

with respect to the Braille system by comparing the performance of dbGLOVE with a

standard Braille cell. The main purpose of the experiment was to understand whether

dbGLOVE and the Malossi alphabet can improve learning. To this end, we focused on

output, only, which is the most interesting feature with respect to education: enabling

users to autonomously read documents without requiring the constant presence of a

118

interpreter is a crucial issue. Speciﬁcally, considered speed and accuracy over a ﬁxed

number of runs that gave users sufﬁcient training time to learn to associate symbols to

stimuli.

4.1 Experimental Tasks

We designed two similar experimental tasks, one focusing on dbGLOVE and one on the

Braille display. As the experiment focuses on the output layer, in order to realize a com-

parison between the training time required by dbGLOVE with respect to a Braille-based

device, we evaluate subjects’ accuracy and speed in recognizing letters in the form of

vibrotactile or pressure cues, respectively. To this end, in each of the two tasks, partici-

pants were involved in a guided output routine: they were presented with sequences of

letters randomly chosen, and they were required to speak them back to the technician.

We employed single letter instead of using words, as the latter could have introduced

some error in the experiment routine. Speciﬁcally, the predictability of the last letters

could have biased the experimental results. Differently from other experiments, as both

the Malossi and the Braille alphabets were given to subjects without prior training,

learning required some time. All the tasks consisted of 30 runs each consisting in 240

seconds, with an inter-run interval of 2 minutes. Each trial had duration of 5 seconds.

Runs were divided into groups of 10, and each group was executed within several days

from one another.

In task I, we evaluated the learning curve of dbGLOVE with respect to the output

function in sending meaningful tactile stimuli to the user. Participants were presented

with sequences of letters represented into a vibrotactile form by the actuators embedded

into the device. The different areas of the hand associated with the letters were stimu-

lated with vibrotactile patterns simulating touch and pinch cues, and participants were

asked to speak the letter to the technician. The objective of the subject was identify-

ing as many letters as possible. The procedure in Task II was exactly the same as that

in Task I. We evaluated the performance of Braille cells in sending perceivable tactile

stimuli to the user. Participants were presented with sequences of letters via a single

Braille cell. They were asked to decode the conﬁguration of the dots, and to speak the

letter back to the experimental technician.

4.2 Participants

13 volunteer participants were recruited for this experiment. They were 5 female and

8 male. All had a normal sight, hearing and tactile sensitivity. Subjects ranged in age

from 18 to 25 with an average of 22. All use computers on a daily basis (1.5-8 hours

usage per day). They were all novice of the Malossi alphabet and of the Braille system.

All had no prior knowledge of the device; two of them had prior experience with vibro-

tactile feedback, as they were involved in other studies about tactile feedback. Subjects

participated on a voluntary basis and they were not paid or rewarded. All subjects were

right-handed as assessed by the Edinburgh inventory [9]. All subjects were prepared

to the experiment by a technician who gave them instructions about the test and the

experimental tasks.

119

4.3 Experimental Setup and Protocol

A device consisting of one piezoelectric Braille cell was utilized in Task II. We em-

ployed an International Building Standard [10] compliant cell (2.5mm for horizontal

and vertical dot-to-dot distance, with a dot diameter of 1.5mm - 1.6mm and a dot height

ranging from 0.6mm to 0.9mm). Its activation was ∼100 milliseconds, which is compa-

rable with that of vibrotactile actuators, given the timing of the experimental task. Prior

to the experiment, subjects were provided with a preliminary explanation of the Mal-

ossi and the Braille alphabets. Also, before each task, they were given sufﬁcient time

to manipulate the stimulation devices. In each trial, individuals had 5 seconds to speak

the letter back to the technician. We evaluated the training level by comparing speed

(measured in CPM). Moreover, we logged the accuracy in recognizing letters. In this

regard, we associated a trial timeout to an error, as if the wrong letter was recognized.

With respect to the task involving Malossi, we also logged the number of errors due to

germane load, that is, letters biased due to the fact that they are on the same phalanx.

5 Results and Discussion

We evaluatedlanguage proﬁciencyusing speed and accuracyas the main metrics. Specif-

ically, we adopted the number of Characters Per Minute (CPM) as a standard perfor-

mance measure, and we distinguished correctly written and wrong characters. Also, we

logged the time users spent in recognizing each letter, in order to identify the symbols

that required more effort, and to evaluate the presence of germane load. All the subjects

were able to understand the task. We utilized sequences of characters, as this is an easier

way to measure subjects’ ability in recognizing vibrotactile stimuli in the different areas

of the hand, and to associate touch and pinch cues to letters of the alphabet. Participants

were only required to speak the letter to the technician, who recorded the answer as

correct or wrong. This was to avoid subjects to actually type the letter on a keyboard,

which could have introduced some additional cognitive load.

Figure 3 represents the experimental results about speed and accuracy. In order to

evaluate speed, we calculated the number of letters that subjects were able to process,

that is, the number of answers they were able to give before the run timeout. Subjects

started at very low speed. Initially, participants were able to recognize only a few of

the characters being displayed, with an average of 30.82 and 45.29 letters displayed to

participants using the Braille cell and dbGLOVE, respectively. During the experiment,

we registered an increasing trend in speed. Also, the training effects persists until the

last run, when the performances of dbGLOVE and the Braille cell are 212.17 and 93.24

letters displayed (on average), respectively. Consequently, the improvement is +166.87

and +62.41 with our device and with the Braille cell.

In regard to accuracy, results show an increasing curve in the use of both devices.

Participants using dbGLOVE started at 20.11%, and they increased their performance

by 76.3% after 30 runs, thus, reaching an average accuracy of 96.41%. The training

effect vanished at run 24, when subjects’ performances were over 90%. Conversely, us-

ing the Braille cell, participants found more difﬁcult to decode the letter by reading the

dots. They began with an accuracy of 15.14%, and they improved their performance by

44.01%. In run 30, subjects’ accuracy was 59.15%, -37.26% with respect to dbGLOVE.

120

Speed in recognizing output

100

150

200

250

Speed

Runs

braille

malossi

Accuracy in recognizing output

100

Accuracy

Runs

braille

malossi

Fig.3. Experimental results: performances with dbGLOVE and with the Braille cell.

6 Conclusions

Although the Braille alphabet is the most widely adopted system for enabling blind and

deafblind people to read text, it has limitations in terms of technological implementa-

tion and costs. Also, it requires cognitive abilities and sensory perception to be intact.

Conversely, teaching Augmentative and Alternative Communication requires systems

that can cope with developmental, physical, and sensory impairments [3]. In this re-

gard, the Malossi alphabet is extremely intuitive, and it relies on basic touch cues. It

is widely employed with people having cognitive impairments, who cannot learn more

complex communication methods, such as alphabets involving shapes.

In this paper, we compared the Malossi alphabet and the Braille system with the

purpose of evaluating their learning curve. Our main hypothesis was that dbGLOVE is

suitable for substituting Braille displays in everyday interaction, and particularly, dur-

ing early life or in the ﬁrst stage of deafblindness, that is, when individuals require an

immediate system for basic communication. As shown by our results, the Malossi al-

phabet implemented in dbGLOVE outperforms the Braille alphabet both in speed and

accuracy, in people with no previous training. As a result, dbGLOVE can be utilized as

a substitute of systems implementing the Braille alphabet, especially in circumstances

in which a shorter learning curve is required. The present study has several limitations.

Being an early-stage prototype, we evaluated dbGLOVE with people having normal

vision and hearing. Therefore, our results should be validated with further experiments

with blind and deafblind users, which will be part of our future work.

References

1. Mller, E. Deaf-Blind Child Counts: Issues and Challenges, National Consortium on Deaf-

Blindness, 2006.

2. Dandona L, Dandona R, John R. K. Estimation of blindness in India from 2000 through

2020: implications for the blindness control policy, Natl Med J India. 2001 Nov-

Dec;14(6):327-34.

121

3. Sigafoos, J.; Didden, R.; Schlosser, R.; Green, V.; OReilly, M. & Lancioni, G. A Review

of Intervention Studies on Teaching AAC to Individuals who are Deaf and Blind Journal of

Developmental and Physical Disabilities, Springer US, 2008, 20, 71-99

4. Hersh, M. A. & Johnson, M. A. Assistive Technology for Education, Employment and

Recreation Assistive Technology for Visually Impaired and Blind People, Springer London,

2008, 659-707

5. Al-Salman, A. S. A Bi-directional Bi-Lingual Translation Braille-Text System Journal of

King Saud University - Computer and Information Sciences , 2008, 20, 13 - 29

6. Caporusso, N. A wearable Malossi alphabet interface for deafblind people Proceedings of

the working conference on Advanced visual interfaces, ACM, 2008, 445-448

7. Bark, K., Wheeler, J. W., Premakumar, S. , Cutkosky, M.R. Comparison of Skin Stretch and

Vibrotactile Stimulation for Feedback of Proprioceptive Information. Haptic interfaces for

virtual environment and teleoperator systems, 2008, Pages 71-78.

8. Lam, Amy R. Vibrotactile pattern recognition on the torso with one and two dimensional

displays. PhD Dissertation, 2006. Online at http://dspace.mit.edu/handle/1721.1/36738

9. Oldﬁeld, R. C. (1971). The assessment and analysis of handedness: The Edinburgh inven-

tory. Neuropsychologia, 9, 97-113

10. RNIB Scientiﬁc Research Unit Scientiﬁc and technological reports ”Braille Cell Dimen-

sions” (online)

122