Table 1: Accuracy for each one of the eye movement cate-
gories.
Eyes Eyes Gaze shift Gaze shift
closed open to left to right
Accuracy 84.31% 98.2% 85.89% 82.84%
3.2 Detection of Reactions to the Sound
As commented before, the great contribution that we
can provide to the audiologist is a proper detection of
the eye movements associated with reactions to the
auditory stimuli. Since the video sequences have a
frame rate of 25 FPS we know for certain that a re-
action will last more than one frame, this is why we
are not concerned about obtaining a high success rate
in classification, because if a typical reaction lasts be-
tween 10 and 15 frames, the misclassification of one
frame will not affect the proper detection of the reac-
tion.
For this experiment, we consider that a status can
be established when three or more consecutive frames
receive the same category in classification. Results
are detailed in Table 2, where we evaluate the agree-
ment between the methodology and the audiologists
based on the number of reactions to the stimuli de-
tected by each one of them. The agreement between
the methodology and the audiologists is complete
(100% of agreement) for the video sequences eval-
uated in this test.
Table 2: Evaluation in the detection of reactions to the
sound. Results are expressed in number of reactions.
Gaze shift Gaze shift
to left to right
Expected Detected Expected Detected
Video 1 17 17 15 15
Video 2 17 17 21 21
Video 3 20 20 18 18
Agreement 100% 100%
4 CONCLUSIONS
This work proposes a novel methodology for the de-
tection and identification of eye gestural reactions to
the auditory stimuli in order to facilitate the hearing
assessment of patients when no cooperation exists.
This task is accomplished using information about
the color distribution of the sclera. The results ob-
tained in this first approach point out the suitability
of the method for the detection of these specific kind
of reactions. Besides, we want to highlight the fact
that is methodology is an initial proposal where we
have classified the eye movements into the four cate-
gories established by the audiologists, but it is impor-
tant to note that the methodology is generic proposal
that would allow us to model or to characterize other
eye movements.
Several experiments were conducted in order to
obtain the final methodology. Four of them were ori-
ented to choose the best method for different steps of
the methodology: study of three different techniques
for pupil detection (Section 2.3), survey about differ-
ent interest operators for the selection of candidate
points (Section 2.4.1), analysis about the suitability
of the reference points (Section 2.4.2) and analysis of
the suitability of the candidate points chosen as eyes’
corners (Section 2.4.3). However, in order to sum-
marize, we only detail here a final experiment which
analyzes the behavior of the final methodology.
It must be highlighted that the proposed methodol-
ogy is a noninvasive method. In contrast to other eye
tracking methods, this solution that not require special
devices, or that the patients wear special glasses or
other contact elements neither a change in the typical
procedure of the audiologists. This is highly impor-
tant, since the need of an scenario as natural as possi-
ble has been highlighted by the audiologist in order to
not alter as much as we can the environment in order
to not to influence the natural responses of the patient.
Despite of the promising results, there still exist
some points that will be attempt as future works. First
of all, we need to obtain more video sequences in or-
der to increase our dataset and, thus, extend the ex-
perimental results with a more complete survey. This
is a complicated step, since it is not easy to obtain the
necessary permissions to record this special patients.
We can also consider the integration of this methodol-
ogy with the information provided by the audiometer.
The methodology could also be considered in other
domains such as the diagnosis of nystagmus, a condi-
tion of involuntary eye movement that may result in
reduced or limited vision.
The final contribution of this work might be very
interesting for the audiologist community since it is
a novel method for the detection of eye based ges-
tural reactions. This methodology will facilitate the
hearing assessment of patients with sever cognitive
decline or other communication difficulties, patients
that can not be evaluated following a standard proce-
dure. A proper hearing assessment of these patients
is more difficult to conduct, but it is very important to
solve this issue since a proper evaluation may help to
treat the hearing loss and improve the quality of life
of these patients.
DetectionandCharacterizationoftheSclera-EvaluationofEyeGesturalReactionstoAuditoryStimuli
319