Sign-Lingo

Feasibility of a Serious Game for Involving Parents in the Language Development

of Their Deaf or Hearing Impaired Child

Ivor van der Schalk and Marco Spruit

Department of Information and Computing Sciences, Utrecht University, Utrecht, The Netherlands

Keywords: Feasibility Study, Serious Game, Sign Language, Kinect, Language Development.

Abstract: Family involvement plays a critical factor in the language development of a deaf or hearing impaired child.

Hearing parents often have major difficulties in communicating with their child when it is deaf or hearing

impaired. These difficulties often lead to issues in the language development of the child. In this research

we investigate the feasibility of a serious game for involving parents in the language development of their

deaf or hearing impaired child by using sign language together in a fun and engaging way. From the

feasibility analysis we find that such a serious game is feasible and could help deaf and hearing impaired

children to improve their language development.

1 INTRODUCTION

About 3 out of every 1,000 children are born either

deaf or with a detectable level of hearing loss in one

or both ears. More than 90% of these children are

born to hearing parents (NIDCD, 2016). Language

development requires a child to have access to

communication (Marschark, 2001). Unlike children

with no hearing impairment it is difficult or even

impossible for deaf and hearing impaired children to

use a spoken language. These children mainly use

Sign language as communication for language

development.

Hearing parents often have major difficulties in

communicating with their child when it is deaf or

hearing impaired. This obstacle makes it difficult for

parents to be involved in the language development

of the child. Previous research has shown that

Serious games can be used to improve a child’s

language development by using natural language

(Sørensen and Meyer, 2007). This makes one

wonder whether or not a Serious game can also be

used to improve the language development of a deaf

or hearing impaired child by using Sign language.

The Kinect device offers the possibility to create

a fun and engaging Serious game for both the child

and its parents. So far only very limited research has

been performed on Serious games that are intended

to be played together by deaf and hearing impaired

children and their parents.

In this research we conduct a feasibility study for

a Serious game which helps involving parents of

deaf and hearing impaired children to use Sign

language together in a fun and engaging way. The

main goals of the game, which we call Sign-Lingo,

are to improve the child´s language development and

to improve the interaction between the child and its

parents. Through this research it will become clear

how feasible such a Serious game is.

2 RESEARCH METHOD

In this research a feasibility study is performed

based on the CORETEST feasibility study

framework (Meulendijk et al., 2013). In this study

the feasibility of all five aspects are explored. These

aspects are:

 conceptual feasibility

 organizational feasibility

 economic feasibility

 technological feasibility

 societal feasibility

The feasibility of these aspects is explored by

performing a qualitative study. Note that, instead,

Aarnoutse et al. (2016) take a systematic literature

study (SLR) approach to study their application’s

feasibility.

van der Schalk I. and Spruit M.

Sign-Lingo - Feasibility of a Serious Game for Involving Parents in the Language Development of Their Deaf or Hearing Impaired Child.

DOI: 10.5220/0006056701910198

In Proceedings of the 10th International Joint Conference on Biomedical Engineering Systems and Technologies (BIOSTEC 2017), pages 191-198

ISBN: 978-989-758-213-4

191

In this study the initial approach was to explore

the field of Sign language and Serious games. From

a literature study it became clear there is a problem

in the language development of deaf and hearing

impaired children. A feasibility study of a Serious

game in which Sign language is used by deaf and

hearing impaired children to improve their language

development was then conducted. Among the

studied literature are research journals, papers and

books about Sign language, Serious games and

studies about deaf and hearing impaired children.

In the feasibility study several models have been

developed using the modeling tool ArchiMate

(Beauvoir, 2016).

3 CONCEPTUAL FEASIBILITY

3.1 Problem Exploration

In their work Kyle and Woll (1988) describe that the

development of language can only occur when a

child is provided with input it can perceive, and

when the adult and the child are joint partners in

creating communication. The importance of parent

involvement in language learning is supported by the

research of Moeller (2000) in which she shows that

family involvement plays a critical factor in the

language development of deaf and hearing impaired

children, especially those with hearing parents.

The majority of deaf and hearing impaired

children are born to hearing parents who do not

know how to use Sign language to communicate. As

a result, many of these children do not have full

access to language during the early years of life most

critical to language acquisition. Marschark (2001)

states that having parents who can sign well and who

read regularly with their deaf or hearing impaired

child are extremely important factors in the child’s

development of literacy skills. This statement might

explain the findings of Allen (1994) who found that

only 15 percent of white deaf children who

graduated from high school in the United States read

above the sixth-grade level.

Akamatsu, Musselman and Zweibel (2000)

found that fewer than half of the children who use

Sign language in school also sign when they are with

their families, and only a small fraction of those

children are able to carry on normal everyday

conversations with their parents. Marschark (2001)

states that deaf children of hearing parents have

fewer signed or spoken labels for things around

them than hearing children of hearing parents, or

deaf children of deaf parents. Special efforts,

therefore, need to be made to expand the

vocabularies of deaf children of hearing parents

through print, sign and speech.

It is clear there is a problem in the language

development of deaf and hearing impaired children

with hearing parents. There is evidence that the

involvement of parents plays a crucial factor in the

Figure 1: Conceptual model of Sign-Lingo.

HEALTHINF 2017 - 10th International Conference on Health Informatics

192

language development of their deaf or hearing

impaired child but that this involvement is

insufficient due to the parents’ lack in ability to

communicate with their deaf or hearing impaired

child.

3.2 Concept Modeling

In the previous section it has become clear there is a

problem in the language development of deaf and

hearing impaired children with hearing parents. In

this section a conceptual model is presented of the

proposed system. This model visualizes the main

concepts of the system. The model is shown in

Figure 1.

As can be seen in the model, the system contains

nine concepts. These concepts are highlighted in

blue rectangles inside the model. Within the model

we can also find red circles which represent the

attributes of concepts.

Each player has a profile in which general

information of the player is stored. This information

contains the name of the player and its favorite

color.

A player has the option to play practice-mode or

exam-mode. In both game modes the player needs to

guess words by using Sign language.

Practice-mode is meant to be played together by

both the parent and child. The goal of this game

mode is to facilitate family involvement in the

language development of the child. In this game

mode the player who guesses the word first wins the

game.

In their work Michael and Chen (2006) mention

the importance to measure the progress of the player

in a Serious game. The exam-mode is meant be

played by the child. The goal of this game mode is

for the child to demonstrate its acquired vocabulary

knowledge. In this game mode the child needs to

guess multiple words correctly to advance to the

next level. The difficulty of the words given in the

game modes are based on this level.

Tennant and Brown (1998) explain it helps to

fingerspell with a partner when contextual clues are

used. Therefore, when starting a game mode, the

player can select a theme. Examples of themes are

animals, sports or family. Based on the theme the

game presents words that are related to the selected

theme.

From their pilot study Lee et al. (2005) found

that the flow within a game is very important to keep

the child interested in playing. In the Sign-Lingo

game we have therefore included a sign book. This

book contains examples of all the signs of the

Alphabet which can be consulted by the player. This

makes it easier for both the parent and child to

continue playing even when they do not know how

to make a sign.

For every player the number of practice games

played, practice games won and the current level are

stored. The current level is an indication of the

current language level of the child. These statistics

can be used to track the progress and target problem

areas in the language development of the child.

3.3 Process Modeling

Goldin-Meadow and Mayberry (2001) explain that

to teach children to read, the first step is to make

sure they have a language - any language. In case of

a deaf or hearing impaired child this will be a Sign

language. After a language has been obtained a child

needs to learn the concept of mapping between the

language it knows and print. In their research

Padden and Ramsey (2000) identified a teaching

method that they call “chaining”. This method

encourages children to see the relation between print

and Sign language. Chaining consists out of four

steps in which the final step is optional:

Figure 2: Chaining method.

Sign-Lingo - Feasibility of a Serious Game for Involving Parents in the Language Development of Their Deaf or Hearing Impaired Child

193

1. Fingerspell a word.

2. Create the sign that represents the finger

spelled word.

3. Point to the written word.

4. Show a picture which represents the word

(optional).

Fingerspelling is used to express letters from the

Alphabet. Tennant and Brown (1998) explain that

fingerspelling is mainly used in Sign language to

express names of people, places, brand names and

titles. Fingerspelling is performed by creating the

sign with the dominant hand.

In the chaining method

for example a teacher could fingerspell the word ‘d-

o-g’, then create the sign that represents the word

‘dog’ and finally point to the word ‘dog’ written

down on the blackboard. In addition to that the

teacher can show a picture of a dog. This method

creates an understanding between the visual spelling

of a word (e.g. in English) and the Sign language

spelling of the word. This method is presented in

Figure 2 and incorporated in the Sign-Lingo game.

4 ORGANIZATIONAL

FEASIBILITY

4.1 Market Modeling

In the previous section we modeled the system’s

concepts and its interaction with its users. In this

Figure 3: Stakeholder’s relationships of Sign-Lingo.

section the organizational feasibility is assessed by

looking into the market in which the proposed

system is intended to participate. Through this

assessment it becomes clear who the system’s key

players, competitors and potential partners are after

which an organizational framework can be built.

The model presented in Figure 3 shows an

overview of Sign-Lingo system’s stakeholders. This

diagram visualizes the relations between the system

and the stakeholders and how the system behaves in

relation to third-party software.

In the model the red rounded box contains the

rules that need to be incorporated into the system.

The rounded yellow box contains third-party

software that is required for the system to function

properly. The red rectangles in the diagram represent

the plugins which are required for the system to

function. The blue rectangles represent the partners

which are involved in the promotion of the system.

The white rectangle represent the system’s users.

Finally, the purple rectangles represent the

competition of other Sign language games related to

language development.

The system requires hand gesture recognition

rules to recognize letters from the Alphabet in Sign

language. The system makes use of the Microsoft

Kinect SDK which is designed to help recognize

gestures. The system will be built on the Vitruvius

Kinect framework which simplifies Kinect

development.

The system is used by families with deaf or

hearing impaired children. These families can be

reached through elementary schools. The system is

downloadable from the Microsoft Store.

Existing research has been conducted mainly on

the development of Serious games that help a person

to learn Sign language (Escudeiro et al., 2015;

Fisher et al., 2014; Wilson, 2013), but not

specifically on helping a deaf or hearing impaired

child in its language development.

When looking at games related to language

development for deaf or hearing impaired children

we find CopyCat (Lee et al., 2005) and AURIS

(Sarmașik, Serbetcioglu and Kut, 2009) produce

interesting results. What differentiates Sign-Lingo

from these games is the fact that these games are not

based on an existing language development method,

do not facilitate family involvement and do not track

the progress of the player.

4.2 Organization Modeling

In the previous section the key players in the

relevant market were identified. In this section the

HEALTHINF 2017 - 10th International Conference on Health Informatics

194

user roles are identified, and the attitudes towards

the newly proposed system are explored.

Through the literature review several methods

were identified which are used to help children learn

to develop their language. It also became clear

parent involvement plays a crucial role in the

language development of a child.

Figure 4 presents the organizational model of the

Sign-Lingo system. The large rectangles represent

the architectural layers on top of which the Sign-

Lingo system is designed. The yellow rectangles

represent all the stakeholders and their roles.

From the model it becomes clear the Sign-Lingo

system is developed using the Microsoft Kinect

SDK. The system is used by the parent and child

who both have the role of user. It also becomes clear

from the model that the developer needs to

implement hand gesture recognition rules into the

system. In addition to that, these hand gesture

recognition rules are tested by a Sign language

expert to validate whether the Sign language

recognition part of the system performs as expected.

Finally, teachers of elementary schools have the role

as a promoter. Their task is to introduce the system

to families with deaf or hearing impaired children.

5 ECONOMIC FEASIBILITY

5.1 Development Strategy

In the previous section the market in which the Sign-

Lingo system operates was investigated. To reach

out to families who could benefit from the Sign-

Lingo system there are several options to consider.

One option is to promote the system through digital

channels such as application platforms like the

Microsoft Store. Another option is to promote the

system through other channels such as elementary

schools for the deaf and hard hearing.

All partners have different objectives regarding

Sign-Lingo. For example in case of the Microsoft

Store the partner’s interests are clearly commercial.

In the case of elementary schools the interest is

educational. The system will most likely need to

have some relationship with the school’s curriculum

in order for the school to be willing to promote it.

Two different development strategies were

created. These strategies can be run independently

from each other. Below follows an overview of the

two strategies.

1. Digital-channel strategy; the software is

promoted on application platforms such as

the Microsoft Store. In exchange for this

promotion Microsoft wants a percentage of

the selling price for every sold product. In

addition to that a website is made for the

Sign-Lingo system which explains how the

system works and where the system can also

be bought (e.g. Abdat, Spruit and Bos,

2011).

2. Physical-channel strategy; the software is

promoted through elementary schools.

These schools are contacted initially through

Figure 4: Organizational model of Sign-Lingo.

Sign-Lingo - Feasibility of a Serious Game for Involving Parents in the Language Development of Their Deaf or Hearing Impaired Child

195

telephone calls in which the system and the

concept behind it are presented to the

school’s management. Once a school agrees

to promote the system teachers demonstrate

the system to parents and their children and

encourage them to use it.

What both strategies described above share, is the

fact they both require partners in order to be

successful. Without partnerships the system is

unlikely to be used by anyone.

6 TECHNOLOGICAL

FEASIBILITY

6.1 Technological Process Modelling

In this section the activity diagram of the use case

‘Play practice-mode’ of the Sign-Lingo game is

presented. This is the most important and

complicated use case in the system. The diagram is

shown in Figure 5.

This process contains two actors: the player and

Figure 5: Activity diagram of the Sign-Lingo use case ‘Play practice-mode’.

HEALTHINF 2017 - 10th International Conference on Health Informatics

196

the system. Because this is a multiplayer game this

process can be run by two players simultaneously.

Before this process can be initiated the player needs

to have executed the use case ‘View game settings’

in which the player has selected the theme and the

word length. Based on these two parameters the

system selects a random word from the system´s

dictionary and displays a lingo screen which

contains empty squares equal to the length of the

random word. The player can then start to guess the

given word.

Each turn, the player needs to guess the word by

creating a sign gesture that corresponds with a letter.

When the player does not know how to create a

specific sign he can consult the ‘sign book’. The

sign book contains an overview of all the signs of

the Alphabet. When the player was successful in

making a sign the system will recognize this and

check whether the given word contains the signed

letter. If the word contains the signed letter the

system will show the letter to the player on the

correct position in the empty squares. When all the

letters of the word are guessed the system shows a

message to the player which confirms that the word

has been guessed correctly and shows a picture that

represents the guessed word. The player can then

decide whether he wants to play another round or

exit the game.

6.2 Wireframe

Figure 6 contains a wireframe of the Sign-Lingo

game. In this sketch two players are playing the

game in practice-mode and need to guess the word

‘DOG’. In their work Tennant and Brown (1998)

explain clues such as the length of the word are

helpful for a child to improve its receptive skills

which makes this game type well suited for language

development.

As can be seen in the sketch the players can see

each other in the game, this makes it possible for the

players to maintain eye contact. Tennant and Brown

(1998) explain that maintaining eye contact is an

important aspect to understand a signer.

The game keeps track of how many rounds a

player has won. The game also shows a message to

indicate whose turn it is to guess a letter. Once a

letter is guessed correctly it appears on the screen

and its background turns green. When one of the

players do not know how to make a sign it can

consult the sign book which can be found in the

bottom left corner.

Over time, we aim to optimize gameplay using

machine learning techniques for personalization (e.g.

Vleugel, Spruit and Daal, 2010), and even

investigate the added value of factors such as

Figure 6: Wireframe of Sign-Lingo user interface.

playroom environment and its geographical location

and local population density as possible language

variation indicators (Heeringa et al., 2007).

7 SOCIETAL FEASIBILITY

In the final step of the feasibility study we focus on

how the proposed system can contribute to help deaf

and hearing impaired children and their families.

The gains can not only be described in monetary

terms but in educational terms as well. These gains

can be assessed by measuring the language level of

children who use the proposed system and compare

them with children who do not use the system. The

monetization gains can be measured by estimating

how much the education associated to the language

development of a deaf or hearing impaired child

costs and how much of the education can be

replaced by the Sign-Lingo system.

8 CONCLUSIONS

In this paper we presented a feasibility analysis of a

Serious game for involving parents in the language

development of their deaf or hearing impaired

children by using Sign language together in a fun

and engaging way. To assess the feasibility the

conceptual, organizational, economic, technological

and societal feasibility aspects, known as the

CORETEST, have been investigated.

From the CORETEST it became clear there is a

problem with the language development of deaf and

hearing impaired children and a Serious game could

Sign-Lingo - Feasibility of a Serious Game for Involving Parents in the Language Development of Their Deaf or Hearing Impaired Child

197

help these children to improve their language

development. The chaining method which is used to

help a child to improve its language is well suited to

be incorporated in this Serious game.

In future research the proposed system could be

implemented and then tested in a case study with a

sample group. In this case study the language level

of deaf and hearing impaired children who use the

system could be compared with the language level

of deaf and hearing impaired children who do not

use the system to see what the effect of the game is

on the language level of a child.

Finally, we aim to incorporate the lessons

learned into follow-up research on the feasibility of

Serious games for medication adherence in children

and

quality of care in long-term care (e.g. Spruit,

Vroon and Batenburg, 2014).

REFERENCES

Aarnoutse, F., Renes, C., Batenburg, R., & Spruit, M.

(2016). STRIPA: The potential usefulness of a medical

app. In Gasmelseid,T. (Ed.), Advancing

Pharmaceutical Processes and Tools for Improved

Health Outcomes (pp. 114–135). IGI Global.

Abdat, N., Spruit, M., & Bos, M. (2011). Software as a

Service and the Pricing Strategy for Vendors. In

Strader,T. (Ed.), Digital Product Management,

Technology and Practice: Interdisciplinary

Perspectives, Advances in E-Business Research

(AEBR) Book Series (pp. 154–192). IGI Global.

Akamatsu, C. T., Musselman, C., & Zwiebel, A. (2000).

Nature versus nurture in the development of cognition

in deaf people. The deaf child in the family and at

school: Essays in honor of Kathryn P. Meadow-

Orlans, 255-273.

Allen, T. E. (1994). Who are the deaf and hard-of-hearing

students leaving high school and entering

postsecondary education. A Comprehensive

Evaluation of the Postsecondary Educational

Opportunities for Students who are Deaf or Hard of

Hearing, funded by the US Office of Special Education

and Rehabilitative Services. Gallaudet University.

Beauvoir, P. (2016). Archi | Free ArchiMate Modelling

Tool. [online] Archimatetool.com. Available at:

http://www.archimatetool.com/ [Accessed 29 Nov.

2016].

Escudeiro, P., Escudeiro, N., Reis, R., Baltazar, A. B.,

Rodrigues, P., Lopes, J., ... & Bidarra, J. (2015). A

kinect game in the VirtualSign project: training and

learning with gestures. In ARTECH 2015: 7th

International Conference on Digital Arts. Artech

International.

Fisher, J., Ellis, K., Willoughby, L., & Barca, J. (2014).

Taking a User Centred Design Approach for Designing a

System to Teach Sign Language. ACIS.

Goldin-Meadow, S. and Mayberry, R. (2001). How Do

Profoundly Deaf Children Learn to Read? Learning

Disabilities Research and Practice, 16(4), pp.222-229.

Heeringa, W., Nerbonne, J., Bezooijen, R. van, & Spruit,

M. (2007). [Geography and inhabitant numbers as

clarifying factors of variation in the Dutch dialect

region]. Tijdschrift voor Nederlandse taal- en

letterkunde, 123(1), Kwantitatieve benaderingen in de

taal- en letterkunde, 70–82.

Kyle, J. G., Woll, B., & Pullen, G. (1988). Sign language:

The study of deaf people and their language.

Cambridge University Press.

Lee, S., Henderson, V., Brashear, H., Starner, T.,

Hamilton, S., & Hamilton, H. (2005). User-centered

development of a gesture-based American Sign

Language game. In Proceedings of instructional

technology and education of the deaf symposium.

NTID, Rochester, NY.

Marschark, M. (2001). Language Development in

Children Who are Deaf: A Research Synthesis.

FORUM publications.

Meulendijk, M., Drenth-van Maanen, A., Jansen, P.,

Brinkkemper,, S., Numans, M. and Spruit, M. (2013).

Introducing the CORETEST feasibility analysis in

medical informatics: a case study of a decision-

supportive knowledge system in the Dutch primary

care sector. In Miranda, I., Cruz-Cunha, M., &

Gonçalves, P. (Eds.), Handbook of Research on ICTs

for Healthcare and Social Services: Developments and

Applications, pp. 1066–1087, IGI Global.

Michael, D. and Chen, S. (2006). Serious games. Boston,

MA: Thomson Course Technology PTR.

Moeller, M. P. (2000). Early intervention and language

development in children who are deaf and hard of

hearing. Pediatrics, 106(3), e43-e43.

NIDCD (2016). Quick Statistics [NIDCD Health

Information]. Retrieved 7 February 2016, from

http://www.nidcd.nih.gov/health/statistics/pages/quick.

aspx.

Padden, C., & Ramsey, C. (2000). American Sign

Language and reading ability in deaf

children. Language acquisition by eye, 1, 65-89.

Sarmașik, G., Serbetcioglu, B., & Kut, A. (2009)

Computer Aided Education and Training Tool for

Hearing Impaired Children: AURIS.

Sørensen, B. H., & Meyer, B. (2007). Serious Games in

language learning and teaching–a theoretical

perspective. In Proceedings of the 3rd international

conference of the digital games research

association (pp. 559-566).

Spruit, M., Vroon, R., & Batenburg, R. (2014). Towards

healthcare business intelligence in long-term care: an

explorative case study in the Netherlands. Computers

in Human Behavior, 30, 698–707.

Tennant, R. A., & Brown, M. G. (1998). The American

sign language handshape dictionary. Gallaudet

University Press.

Vleugel, A., Spruit, M., & Daal, A. van (2010). Historical

data analysis through data mining from an outsourcing

perspective: the three-phases method. International

Journal of Business Intelligence Research, 1(3), 42–

65.

Wilson, C. (2013). Portable Game Based Instruction of

American Sign Language.

HEALTHINF 2017 - 10th International Conference on Health Informatics

198