MOBILE VIDEOPHONES FOR SIGNED LANGUAGE
COMMUNICATIONS
Kazuo Kamata, Takeaki Shionome, Hideo Yamamoto
Department of Information Science, Utsunomiya University, Utsunomiya, Japan
Keywords: Visual communication, Mobile videophones, Signed language, Quality of service
Abstract: Information and Communication Technologies (ICTs) provide various kinds of information and
communication services. For some people, however, their capabilities affect the accessibility to these
services and terminal equipment. It must be ensured that the information and communication services of the
Information Society are designed to be accessible from all. In this report we show certain results of our on-
going project on signed language communication services by mobile videophones. The mobile videophones
have an interesting feature of portability, but have certain difficulties in signed language production and
perception due to the small size of displays. In this report we show the experimental results for Japanese
Signed Language perception by small sizes of pictures. From the present results and previous experimental
results, we discuss possible information and communication services by a signed language using mobile
videophones from the user’s point of view: How well a user can perform his/her intended information
activities under a particular use situation. This concept leads to another term, User-Based Quality of Service
(UbQoS), which uses measure parameters of user’s attitude to the services at a particular situation, and
includes human and social factors in addition to usual technological ones.
1 INTRODUCTION
ICTs (Information and Communication
Technologies) provide a very wide range of
information and communication services. For some
people, however, their capabilities affect the
accessibility to these services and information and
communication devices (terminal equipment). Deaf
people, for example, can not use the ordinary voice
telephone service, and blind people can not directly
read materials visually displayed. Furthermore,
people with restricted movement experience
difficulties in manipulating terminal equipment. It
must be ensured that the information/communication
services of the Information Society are designed to
be accessible and usable from all (Cullen,
&Robinson, 1997).
The specific aim of this on-going project is
identifying technological, human and social issues
for providing communication services by a signed
language for deaf and hearing people. A signed
language uses body movements, hands, arms, face,
head, and body posture, instead of sound, and
listeners (receivers) use their eyes instead of their
ears to understand what is being said. Therefore, a
signed language has a quite different modality from
the usual spoken language, and is a visual-gestural
language (Schein, & Stewart, 1995). We then need
an appropriate visual communication system for
transmission of a signed language (distant signed
language communication).
In this report, we mainly deal with some mobile
videophones for distant signed language
communications. The mobile videophones have the
feature of portability, and then an individual user can
access to networks and information/communication
services any time, and from anywhere. They have
some potentialities for providing useful
information/communication services by a signed
language.
On the other hand, the mobile videophones have
usually small displays with around two inches. The
display characteristics together with other
technological parameters such as picture quality will
affect the Quality of Service (QoS). In addition to
these technological parameters, others are important
for measuring the quality of service such as user
characteristics for physical, cognitive and language
capabilities including communication situations, and
209
Kamata K., Shionome T. and Yamamoto H. (2004).
MOBILE VIDEOPHONES FOR SIGNED LANGUAGE COMMUNICATIONS.
In Proceedings of the First International Conference on E-Business and Telecommunication Networks, pages 209-214
DOI: 10.5220/0001394402090214
Copyright
c
 SciTePress

social issues that support the services. For example,
when a user uses a mobile videophone, he/she needs
to hold the device by one hand. This use situation
causes an essential restriction: A signer has to use
only one hand for signing instead of usual two-
handed signing. This posture is one of causes of
human stress and fatigue. One-handed signing also
causes difficulties in both production and perception
of Japanese Sign Language (JSL) (Kamata,
Shionome, Yamamoto, & Fischer, 2003). Another
example is a remote signed language interpreting
(video relay) service. The service inherently requires
human and social resources, in addition to
technological one.
In this report we first focus upon the display size
of mobile videophones and show the results for
perception experiments of JSL. We next make
general discussions on information/communication
services by a signed language based on the present
study results and previous experimental results for
one-handed signing (Kamata, Shionome, Yamamoto,
& Fischer, 2003). We further consider the quality of
services from the user’s point of view. We must take
into account those factors which are not considered
for the usual QoS such as user characteristics for
physical, cognitive and language capabilities
including communication situations, and human and
social factors that support the services. We then
introduce another measure, which we name User-
Based Quality of Service (UbQoS), for considering
these factors.
2 TERMINALS, MEDIA,
NETWORKS AND
COMMUNICATION SERVICES
Visual communication systems can provide new
communication opportunities to people those who
use a signed language in their daily lives (Pereira, &
Lindström, 1994). Certain features of
communication systems specify the characteristics
of these communication services.
(1) Network access means including terminal used
by a user. We assume in this discussion that mobile
videophones are used as terminals.
(2) Media type. We can now use various media for
information transmission through communication
systems such as voice, text, still pictures, and motion
pictures (video, or motion images). In this report we
only focus on videophone images (motion images).
(3) Media quality is very important parameter for
the usual QoS. Temporal and spatial resolutions are
important parameters. Another one is the time delay
between the video and its associated voice (audio-
video delay).
(4) Applications: In this report, we focus on the
videophone applications for signed language
communications, such as interpersonal
communication, a remote access to web site
information, and support services in daily life
activities such as entertainment/leisure, work,
education, and emergency.
One of the important issues for a user is whether
or not he/she can perform, in a particular situation,
an intended information/communication activity.
Quality of Service (QoS) is usually measured by
technological parameters such as frame rate and
spatial resolution for transmitted video signals. This
measure, however, does not explicitly include
factors for users and their use environments
(communication situations): Sensory and cognitive
abilities, user preferences, purpose of the activity,
and the environment in which information and
communication activities are performed. We here
show an example of voice on-demand service. Voice
on-demand services (telephone answering service)
with automatic speech recognition (ASR) are widely
used to access to information. Some hard of hearing
and deaf people can not recognise the message nor
do they respond to the message. The use situation is
very similar to that of the ordinary hearing people in
a noisy environment. Hearing people hardly
recognise the voice or speech information in a noisy
environment. From these cases, we can see that the
quality of information and communication activities
also depends on the use environments, or the use
situations.
From these examples, we introduce another term,
User-Based Quality of Service (UbQoS), for taking
into account parameters of human and social issues
that are not considered in the usual QoS. That is,
UbQoS includes parameters for quality or usefulness
of activities that a user really wants to perform in a
particular situation.
In this report, we focus upon deaf people who use
a signed language in their daily lives, and are not
able to use the usual voice/speech
telecommunication services. The aim of our project
as mentioned earlier is identifying fundamental
issues to improve communication environments
where deaf and hearing people are using mobile
videophones. In the study, we consider three types
of videophones commercially available in Japan as
shown in Figure 1. Usual mobile videophones
(Figs.1(a), and1(b)) have display size of around two
inches and weight of around 120g. PDA (Personal
Digital Assistants) type one shown in Figure 1(c)
has a larger display (3.5 inches) and is heavier than
usual ones (280g).
ICETE 2004 - WIRELESS COMMUNICATION SYSTEMS AND NETWORKS
210

We then clarify the communication situation
where they use: (1) Mobile videophones. (2)
Visual/video communication media. (3) A signed
language as a language medium of communication.
(4) Applications in the field of interpersonal
communication (signed language conversation),
video relay, and remote access to information
(Mobile Internet service).
In the following chapters, we consider the
following factors for signed language
communication services using mobile videophones:
(1) Physical interface of mobile videophones for
signed language communication. (2) Language and
cognitive interfaces of one-handed signing. (3)
Cognitive interface for mobile videophones with
small size display. (4) Social and human supports
for applications and services. (5) Value of
communication services.
(a) (b) (c)
Figure 1: Commercially available mobile videophones in
Japan. (a) Folding type with a display of 2.35 inches and
weight of 133g, (b) Straight type with 1.8 inches, and
110g, (c) PDA type with 3.5 inches, and 280g.
3 SIGNED LANGUAGE
PERCEPTION EXPERIMENT
Mobile videophones have small displays with size of
around two inches as shown in Figure 1. In this
chapter, we show certain experimental results for the
perception of signing pictures displayed by small
size screens.
3.1 Display Size and Information
Perception
The display sizes of mobile videophones are around
two inches. These sizes are smaller than usual PDAs.
Other available mobile terminals are notebook PCs
with around 7-10 inch display. Much larger displays
are used for desktop PCs.
One of the most important questions is that how
well we can perceive the information through these
small displays. We make a perception experiment to
clarify this question for JSL.
3.2 Experiment
3.2.1 Procedures
Ten deaf adults (five men, and five women) of ages
of fifties and sixties (M=59.8 years) were joined the
experiment. All participants are prelingual deaf and
use JSL in everyday life.
We first recorded 30 signing sentences on a
digital video recorder (SONY, DSR-300A). A signer
is a prelingual deaf (male) of age of sixties, and is a
member of the same deaf community as participants.
We chose 10 signing motion pictures among 30 ones.
We then generated total number of 50 signing
stimulus (10 sentences x 5 display sizes) using a
video editing software (Ulead, Video Studio 5.0, and
Finalsoft, Simplayer WMP3 TE). The display
picture sizes are 1, 2, 3, 4, and 6 inches.
The experiment consisted of two phases:
Intelligibility and preference tests.
At the first phase, We showed one signing
sentence with a specific picture size on a CRT
(SONY, CPD-E230, 1024x768), and asked every
participant to evaluate subjective intelligibility for
the sentence between 0% (not at all) to 100%
(completely).
The second phase is the evaluation of preference
for signing stimulus with different picture sizes. We
have used Nakaya`s modified version of paired
comparison method (Research Committee of
Sensory Evaluation, 1973). Each participant watches
two signing pictures, say A on one CRT display and
B on the other, with different picture sizes and the
same sentence. A participant can watch each signing
picture repeatedly as many times as he/she wants,
and is asked to compare two signing pictures. The
evaluation scale is a bipolar 7-point rate (A is very
good, A is good, A is slightly good, A and B are the
same, B is slightly good, B is good, B is very good).
3.2.2 Results
The mean values of intelligibility of JSL sentences
for different picture sizes are shown in Table 1, and
are plotted in Figure 2. The error bar in Figure 2
indicates one standard deviation. We have applied t
tests between intelligibility values for different
picture sizes, and the result shows significant
differences between each pair of picture sizes except
two cases: between 3 and 4 inches, and between 4
and 6 inches. We also applied F test to check subject
difference in the group, and the result shows no
significant difference, F(9,90)=1.4698, p=.1713.
MOBILE VIDEOPHONES FOR SIGNED LANGUAGE COMMUNICATIONS
211

Table 1: Intelligibility for signed language sentences with
different picture sizes
Picture
size(inch)
1 2 3 4 6
Intelligi-
bility(%)
24.25 70.25 84.00 92.00 95.50
Figure 2: Intelligibility (mean and one standard deviation)
for JSL sentences with different picture sizes
Table 2: Degrees of preference for different picture sizes
Picture size
(inch)
1 2 3 4 6
Degree of
preference
-1.96 -0.840 0.380 0.960 1.46
Figure 3: Degrees of preference for different picture sizes,
1,2,3,4, and 6 inches
The preference values for five different picture
sizes are shown in Table 2, and Figure 3. We have
applied t tests between values of preference. The
result shows that every difference between
preferences values is statistically significant, p<.01,
except two cases: between 3 and 4 inches, and
between 4 and 6 inches.
4 DISCUSSIONS
The purpose of our study is to clarify design issues
of information/communication services using a
signed language, and mobile videophones. We
discuss these ones together with other experimental
results discussed elsewhere (Kamata, Shionome,
Yamamoto, & Fischer, 2003).
4.1 Display Size
The mean value of intelligibility decreases according
as picture size becomes smaller. The results for t test
show that intelligibility values for picture sizes of 1
and 2 inches are significantly different from others,
and sizes of 3, 4, and 6 inches form a group. These
results lead to a plausible conclusion that picture
size of 3 inches is a critical for a signed language
(JSL) perception in the mobile videophone terminal.
The other experiments on subjective preference
of signing pictures also show that the size of 3
inches is critical. The degrees of preference for
picture sizes of 3, 4, and 6 forms a group and others,
1 and 2 inches, are isolated from those ones.
From these two experimental results, we can say
that real mobile videophones with picture size of
around 2 inches cause certain kinds of cognitive
stress in perceiving signing pictures.
Another videophone is a PDA type mobile
videophone with 3.5-inch display as shown in Figure
1(c). This is allowable picture size from the
experimental results.
4.2 One-Handed Signing
One of the most preferable and useful features of
mobile videophones is the possibility of mobile
access to the network, and the portability of
equipment. Hearing person can use a mobile phone
with either vocal/speech or text communication by
holding it by one hand. This is not the case for those
who intend a signed language communication. When
a user produces signed language, he/she usually
needs to hold the videophone by one hand and to
make signing by the other hand (one-handed
signing). This situation is quite unusual, because
they usually produce a signed language by two
hands.
We have made another experiment to clarify two
questions relating to the use situation (Kamata,
Shionome, Yamamoto, & Fischer, 2003): (1) Can
people make one-handed signing without difficulty?
(2) Can people understand one-handed signing? The
experimental results show that deaf people can do it
without great difficulties, and they may follow rather
common rules for expressing one-handed signing for
two handed signs. On the other hand, hearing people
who do not have much experience of one-handed
signing have difficulties with the production.
Furthermore, a similar situation happens in receiving
(perceiving and understanding) one-handed signing.
Hearing people who do not have much experience
have difficulties in understanding one-handed
signing. Deaf people and some hearing people who
have plenty of one-handed signing experiences can
･ ･ ･ ･ ･
-2.0 -1.0 0.0 +1.0 +2.0
Degree of preference
1 inch 2 3 4 6
100
80
60
40
20
0
Intelligibility (%)
1 2 3 4 6
Picture size (inch)
ICETE 2004 - WIRELESS COMMUNICATION SYSTEMS AND NETWORKS
212

converse with each other by one-handed signing
through the real mobile videophones.
Another issue that we have to take into account is
the posture for holding a mobile videophone during
the one-handed signed language conversation. They
have to keep a certain distance between the body and
the video camera of the videophone so that the
camera can fully capture signing movements. The
posture fundamentally gives a user a considerable
stress that is one of the causes of fatigue.
The use situation for a PDA type mobile
videophone is different. A user holds it by a hand or
put it on somewhere. In the later case, he/she can use
two hands in signing, but its portability is less than
the usual mobile videophones.
4.3 Quality of Signed Language
Communication Services
From the abovementioned discussions, we can say
that the mobile videophones have great benefit for
signed language communication, and on the other
hand have some difficulties in use with physical and
cognitive issues.
In this section, we show certain services by a
signed language, and consider the quality for the
services (User based Quality of Service, UbQoS).
UbQoS can take into account the usual technological
parameters, and other factors: user’s attitude to
services, language and cognitive interface, and
human and social issues for supporting the services.
4.3.1 Distant Signed Language Conversation
The benefit of the use of the mobile videophones is
to make it possible a distant signed language
conversation for deaf and hearing people. Deaf
people need to make real face-to-face conversation
due to the characteristics of signed languages in case
of having no distant communication means. They
now can have a means for signed language
conversation anywhere and any time. The
communication circumstances will be very similar to
those for hearing people with voice/speech
communication by mobile phones.
Although the ISDN videophones provide an
opportunity of distant signed language conversation,
the equipment is not portable, and can not make
mobile access to the network. A portable PC
terminal can be used as a mobile videophone, and
has a larger display than the mobile videophone.
However, the PC is less portable than the mobile
videophone. A PDA type mobile videophone has
some advantageous features except portability: The
display size (3.5 inches) is larger than usual ones
(around 2 inches). It also provides two-handed
signing situation.
Another important issue is the communication fee
for mobile videophone service. Signed language
conversation costs much more than the usual text
and voice/speech communications. This is one of
typical issues for social consideration.
4.3.2 Remote Interpreting Services of Signed
Language
The most advantageous feature of the mobile
videophone is the portability. Certain use situation
for a remote signed language interpreting service
(video relay) is very important for deaf and hearing
people. One of typical situations is the case where
police officers need to communicate with deaf
person, for example, at a traffic accident. It is very
difficult to build a social system to arrange signed
language interpreters at any place and any time.
Remote interpreting services (a relay centre service)
is very attractive. In this application, the portability
of the mobile videophones provides very flexible
communication means in an emergency case.
Although the remote signed language interpreting
service needs human and social resources to manage
it, the service should be considered as a kind of
fundamental social security/welfare service.
4.3.3 Mobile Internet Access
Mobile phones provide Internet services, and a user
can use e-mail and access to Web sites. Deaf people
use mobile e-mail for text communication. Some
mobile videophones have a function of capturing
motion pictures (for example, signing motion
pictures) and storing it, and of sending the stored
data to the party. Although the duration length of
motion pictures for this service is now economically
limited by about fifteen seconds due to the
communication cost. Mobile videophones will
provide, in near future, signed language based e-mail
service. Some simple methods have been discussed
elsewhere (Kamata, Yamamoto, Abe, Fukuda, &
Abe, 2001).
Deaf people can access to the network and get the
desired information from Web sites. The language
medium of information, however, is Japanese in
either text (written language), or speech at almost all
of the Web sites. When one wants to get information
by the medium of a signed language, information
contents must be translated and represented by a
signed language. This needs certain language
processing by either human or machine, and
includes both technological and social factors.
MOBILE VIDEOPHONES FOR SIGNED LANGUAGE COMMUNICATIONS
213

4.3.4 User-Based Quality of Services
We have shown some possible services based on the
mobile videophones and their use situations. The
most important issue is that a user can perform an
intended information activity in a particular situation
using a mobile videophone terminal.
We mainly focus upon signed language
communication with the use of mobile videophones.
Generally speaking the communication environment
for deaf people is restricted, and they usually have to
use a spoken language as an information medium of
communication. Mobile videophones and other
terminals provide new opportunities for signed
language communication. The quality of these new
communication services varies according to some
factors: purpose of use, use situation, information
contents, and other technological parameters.
The mobile videophones have an advantageous
feature of portability and are very attractive, and at
the same time have certain difficulties in use due to
the small display size and others mentioned. There
should exist a trade-off between small display,
portability, purpose of the use, and the value of
service. One example for the trade-off is the signed
language communication in an emergency case
mentioned in the previous section. This service is
important and valuable in a certain situation even if
the picture quality is not fully good enough for
communication. This example shows that the quality
of service from the user’s point of view is not
completely determined only by technological
parameters discussed previously. This situation will
be well discussed by the concept of User-Based
Quality of Service (UbQoS). It reflects user’s
attitude to the service and other factors.
5 CONCLUDING REMARKS
The present study firstly has dealt with the effect of
display size of terminal equipment (mobile
videophones) on information perception in a signed
language, and, secondary, the effect of terminal
function, language and cognitive, and social issues
in signed language communication. Designing of
information/communication services based on a
signed language is very important especially from
the point of social security and welfare. As we have
mentioned in this report, the mobile videophones
have certain difficulties in use for signed language
communication, specifically with physical, and
language and cognitive factors. The measure of
quality for the service has many technological,
human and social factors, and is quite complicated.
We have introduced the term UbQoS for including
these factors. As mentioned in this report, activities
in a particular situation, such as communication with
the party and getting certain information, were very
valuable even if the picture quality of
communication system was not perfect enough. We
need further thorough consideration from various
viewpoints such as technological, human, and social
ones.
REFERENCES
Cullen, K, and Robinson, S. (eds.). 1997.
Telecommunications for older people and disabled
people in Europe - Preparing for the information
society -, IOS Press.
Kamata, K., Yamamoto, H., Abe, Y., Fukuda, M., and
Abe, S., 2001. Picture-Mail System-Means of Signed
Language Communication over Internet-. In HFT’01,
18th International Symposium on Human Factors in
Telecommunication. pp.173-180.
Kamata, K, Shionome, T., Yamamoto, H., and Fischer, S.,
2003. Mobile Videophones and Distant Signed
Language Conversation. In G.Craddock et al. (Eds.).
Assistive Technology-Shaping the Future
(AAATE’03). pp.284-288, IOS Press.
Pereira, L. M., and Lindström, J-I. (Eds.) 1994.
Videotelephony for Disabled and Elderly People,
Commission of the European Communities.
Research Committee of Sensory Evaluation, 1973.
Sensory Evaluation Handbook. JUSE Press.(in
Japanese)
Schein, J. D., and Stewart, D. A., 1995. Language in
Motion – Exploring the Nature of Sign -. Gallaudet
University Press.
ICETE 2004 - WIRELESS COMMUNICATION SYSTEMS AND NETWORKS
214