DEVELOPMENT OF AN EYE GAZE INTERFACE SYSTEM AND

IMPROVEMENT OF CURSOR CONTROL FUNCTION

Tetsuya Yonezawa

Department of Information and Electronics Engineering, Yatsushiro National College of Technology

2627 Hirayama-shinmachi, Yatsushiro-shi, Kumamoto, Japan

Kohichi Ogata, Masashi Nishimura, Kohei Matsumoto

Graduate School of Science and Technology, Kumamoto University

2-39-1 Kurokami, Kumamoto-shi, Kumamoto, Japan

Keywords: Eye gaze, moving average filter, mouse cursor control, eye blink, mouse click function.

Abstract: This paper introduces an eye gaze interface system for controlling a mouse cursor on the computer display.

The system consists of a small video camera to capture an eye image and a computer to detect the eye gaze

from the image and to calculate the position of the cursor to be displayed depending on the detected eye

gaze. In order to develop an easy-to-use system, consideration of involuntary and voluntary eye blink is

necessary for practical use. Improvement of the stability of eye gaze-controlled cursor movement is also

important. In this paper, smooth cursor control using a moving average filter and detection of involuntary

and voluntary eye blink are described. The experiments show the usefulness of the proposed methods for

quick and stable mouse cursor control. In the experiment of cursor pointing accuracy, distances between the

target and the cursor point are about 30 pixels in horizontal direction and 20 pixels in vertical direction.

1 INTRODUCTION

Computers have become popular tools in daily life.

Human computer interfaces using eye gaze have

been developed for user convenience (Hutchinson,

White, Martin, Reichert & Frey, 1989; Kim &

Ramakrishna, 1999; Porta, 2002; Wang & Sung,

2002; Young & Sheena, 1975). Eye gaze human

computer interfaces can provide a useful method for

people with severe motor disabilities to operate a

computer (Cleveland, 1994; Ito, Sudoh & Ifukube,

2000; Kanou, Inoue, Kobayashi, Kawamura &

Nakashima, 2002). However, such systems are so

expensive that the systems are not widely used for

disabled people.

We have developed an eye gaze interface system

with the aim of low cost and easy operation. This

system consists of a small video camera attached on

goggles, a computer with a color image capture

board, and software. In this paper, improvement of

the system to achieve quick and stable mouse cursor

control is described. In Section 2.1, we describe

hardware system. In Section 2.2, we show the

procedure to detect the eye gaze position on the

display from the eye image. In Section 3.1, we

explain a moving average filter for smoothing mouse

cursor movement. In Section 3.2, the experiment on

the moving average filter of mouse cursor movement

and the experimental results are shown. In Sections

4.1 and 4.2, we describe detection of eye blink and

mouse cursor control during eye blink.

2 SYSTEM CONFIGURATION

2.1 Hardware Configuration

The system consists of a small color video camera

(Kyohritsu JPP-CM25F 1/3inch CMOS 0.25

Mpixel) and a desktop computer (CPU: Pentium4

3.2GHz, MEMORY: 1GB, OS: Windows XP) with

an image capture board (Imagination PXC200). The

system is capable of processing 320 x 240 pixels at a

frame rate of 30 fps. Figure 1 shows the small video

camera attached on the goggles. Figure 2 shows an

experimental environment.

281

Yonezawa T., Ogata K., Nishimura M. and Matsumoto K. (2008).

DEVELOPMENT OF AN EYE GAZE INTERFACE SYSTEM AND IMPROVEMENT OF CURSOR CONTROL FUNCTION.

In Proceedings of the International Conference on Signal Processing and Multimedia Applications, pages 281-284

DOI: 10.5220/0001932102810284

 SciTePress

Figure 1: Goggles with a video camera attached.

Figure 2: Environment of eye gaze detection experiment.

The outline of the data processing is as follows:

(1) Filtering in HSI color space to detect the region

having an iris in the captured image frame.

(2) Determining the diameter of a ringshaped

template to be pattern matched with the contour

of the iris from an initial image frame.

(3) Determining the center of the iris as the center of

the ringshaped template after pattern matching.

(4) Mapping the detected center of the iris onto the

position of the mouse cursor on the display.

Through these processes, the user is able to control

the position of the mouse cursor with his/her eye

gaze. The mapping vector used in step (4) is

determined from a prior calibration procedure.

Figure 3 shows two examples of the iris detection.

In each example, the ringshaped template shows a

good match with the contour of the iris. Its center (+)

is also shown in the figure. The mean values of

detected error on the 25 target points on the display

over the five subjects are 1.78 degree in horizontal

direction, 1.82 degree in vertical direction

respectively. These values are almost equal to the

resolution of other tracking system (Yonezawa,

Ogata & Shiratani, 2008).

3 MOUSE CURSOR CONTROL

BY MOVING AVERAGE

3.1 Moving Average Filter

In our system, a pixel on the image frame having eye

image corresponds to about 20 pixels on the

computer display in length. Therefore, fluctuation of

the detected center of the iris can cause difficulty in

controlling the mouse cursor. In order to reduce such

a situation, a moving average filter (MAF) shown as

Eq. (1) was introduced in the system.

∑

−

−=

)(

itx

(1)

where

)(tx is the input data, )(ty is the output data,

and

m is the number of frames to be used.

Figure 3: Examples of detection of the center of the iris

using pattern matching.

3.2 Experimental Setup

In order to evaluate the effects of the moving

average filter on the cursor control, the experimental

setup shown in Figure 4 was used. Illuminance near

the subject’s eyes is about from 250 to 400 lx. An

LCD –display of 17 inch and 1024 x 768 pixels was

used. Five subjects with normal eyesight were

participated in the experiment. The number of

frames used in moving average was 30. The

experimental procedure is as follows:

(1) One of the nine targets shown in Figure 5 is

randomly selected and displayed.

(2) A subject moves the mouse cursor into the target

through his/her eye gaze.

(3) The subject keeps the position of the mouse

cursor on the target for two seconds.

This trial was applied five times to a randomly

selected new target for each subject. Total time for

the five trials and the standard deviation of the

mouse cursor movement for each subject were

evaluated.

Figure 6 shows the total time for the five trials

averaged over the five subjects. The effect of

moving average filter is shown as the reduction of

time for the mouse cursor control. Table 1 shows the

standard deviations of the mouse cursor movement

within the each target for two seconds averaged over

the trials and the subjects. The standard deviations

for the case using the moving average filter are less

than those for the case without the filter in both

directions.

Because the moving average filter was useful for

the mouse cursor control, the number of frames used

in filtering was evaluated to obtain the optimum

number. The experimental procedure was the same

as used in the previous experiment and 10, 20, 30,

and 40 frames were used in the filtering. Two

subjects with normal eyesight were participated in

the experiment.

Figure 7 shows the total time for the five trials

averaged over the two subjects. Table 2 shows the

SIGMAP 2008 - International Conference on Signal Processing and Multimedia Applications

282

standard deviations of the mouse cursor movement

averaged over the trials and the subjects. As shown

in the figure and the table, the moving average filter

with 20 frames is most effective for quick and stable

cursor control. Therefore the filtering function is

installed on our interface system.

4 DETECTION OF EYE BLINK

AND MOUSE CLICK

FUNCTION

4.1 Detection of Eye Blink

Involuntary eye blink is inevitable while use of the

system. During the eye blink, it is difficult to detect

the contour of the iris because of occlusion caused

by eyelid. To avoid failure in the detection of the

contour and cursor control, a detection algorithm

must be considered. In this section, the outline of the

algorithm is described.

Figure 4: Experimental environment.

Figure 5: Nine target points on the display.

Figure 6: The total time for pointing the mouse cursor onto

the five targets on the screen by eye gaze.

Figure 7: The total time for pointing the mouse cursor onto

the five targets on the screen using MAF by eye-gaze.

Table 1: The standard deviation of the mouse cursor

movement within the each target.

Table 2: The standard deviation of the mouse cursor

movement within the each target.

The beginning of the eye blink is defined as the

situation that the difference of vertical length of the

iris region between consecutive two image frames is

ten or more pixels. The end of the eye blink is

defined as the situation that the vertical length of the

iris region returns the level of the beginning of eye

blink with a margin of ten pixels. The duration of

eye blink is used to distinguish between voluntary

and involuntary eye blink. The duration of 300 ms is

used as the threshold based on the preliminary

experiment. The value of the threshold is adjustable

through a GUI window of the system for user

convenience. Because the detection result is used as

a trigger for holding the position of the mouse cursor

during eye blink, erroneous movement of the mouse

cursor during eye blink can be reduced. Moreover,

the detection method allows us to apply voluntary

eye blink to a mouse click function.

The system holds the position of the cursor during

eye blink. Furthermore, for mouse click, the system

holds the position of the cursor for 20 frames (660

ms) after the end of eye blink to prevent unsteady

cursor movement in a transition state.

4.2 Experiments

The experiments for evaluating the detection method

for eye blink and the mouse click function were

done for the five subjects with normal eyesight. The

moving average filter shown in Section 3.2 was used

during the experiments. The experimental setup and

the procedure are almost the same as in Section 3.2.

In the experiments, each subject was requested to

move mouse cursor to each target area of 60 x 60

pixels and to make a click through voluntary eye

blink. This trial was repeated nine times with a

randomly selected target among nine targets shown

DEVELOPMENT OF AN EYE GAZE INTERFACE SYSTEM AND IMPROVEMENT OF CURSOR CONTROL

FUNCTION

283

in Figure 8. The target points in the figure are

located inside of the measurement area in Figure 5.

Table 3 shows a comparison of average distance

between the target and the mouse cursor for the five

subjects. Average values over the subjects show that

the detection method is more effective in vertical

direction. Table 4 shows average of the standard

deviation of mouse cursor movement every one

second while the experiments. Reduction of the

deviation, i.e., fluctuation of the cursor point on the

computer display is clearly seen in both directions.

Subjects A and B are skilled users, and the others

are beginners. Table 3 suggests that skilled users are

able to control the mouse cursor with a high

accuracy through eye gaze using our interface

system.

Figure 8: Nine target points on the display.

Table 3: The average distance between the target and the

mouse cursor.

Table 4: The standard deviation of the movement of the

mouse cursor.

5 CONCLUSIONS

In this paper, smooth cursor control using a moving

average filter and detection of involuntary and

voluntary eye blink were proposed and evaluated for

developing an easy-to-use eye gaze interface system.

The experimental results showed the usefulness of

the proposed methods for quick and stable mouse

cursor control.

ACKNOWLEDGEMENTS

We wish to acknowledge Kazuyuki Shiratani and

Daisuke Kido at Graduate School of Science and

Technology, Kumamoto University, who have

contributed their efforts and talents in developing a

prototype system.

REFERENCES

Cleveland, N. R. (1994). Eyegaze human-computer

interface for people with disabilities. First Automation

Technology and Human Performance Conference,

http://www.lctinc.com/doc/cathuniv.htm

Hutchinson, T. E., White, K. P. Jr., Martin, W. N.,

Reichert, K. C., & Frey, L. A. (1989). Human-

computer interaction using eye-gaze input. IEEE

Transactions on Systems, Man, and Cybernetics, 19,

(6), 1527-1534.

Itoh, K., Sudoh, Y. & Ifukube, T. (2000). Eye gaze

communication system for people with severe physical

disabilities. IEICE Trans. on Information & Systems,

83-DI, (5), 495-503.

Kanou, N., Inoue, M., Kobayashi, Y., Kawamura, T. &

Nakashima, K. (2002). Detection of the eye-focused

character on moving syllabary by Neural Networks for

ALS patients. IEEJ, 122-C, (5), 744-752.

Kim, K., & Ramakrishna, R. S. (1999). Vision-based eye-

gaze tracking for human computer interface,. IEEE

SMC '99 Conference Proceedings , 324-329.

Porta, M. (2002). Vision-based user interfaces: Methods

and applications. International Journal of Human-

Compute Studies, 57, 27-73.

Wang, J. & Sung, E. (2002). Study on eye gaze estimation.

IEEE Transactions on Systems, Man, and Cybernetics-

Part B:Cybernetics, 32, (3), 332-350.

Yonezawa, T., Ogata, K., & Shiratani, K. (2008). An

algorithm for detecting the center of the iris using a

color image and improvements of its processing speed.

IEEJ, 128-C, (2), 253-259.

Young, L. & Sheena, D. (1975). Survey of eye movement

recording methods. Behavior Research Methods and

Instrumentation, 7, (5), 397-429.

SIGMAP 2008 - International Conference on Signal Processing and Multimedia Applications

284