Analysis of the Use of the Maintenance Documentation Using

Eye-Tracking: A Pilot Study

F. Paris

, R. Casanova

, M. L. Bergeonneau

and D. R. Mestre

Institute of Movement Sciences, CNRS, Aix-Marseille University, 163 Av. De Luminy, Marseille, France

Airbus Helicopters, Aéroport International Marseille Provence, Marignane, France

Keywords: Eye-Tracking, Aircraft Maintenance, Real-World Research, Ergonomics, Expertise.

Abstract: Maintenance tasks have an essential role in the safety of helicopters throughout their life. It is performed by

aircraft maintenance technicians (AMT). Two major types of operations in the maintenance activity are

information intake/processing and motor actions. This extended abstract will present a new work-analysis

methodology, using a mobile eye-tracking system to measure the use of maintenance documentation during

the preparation and execution of the maintenance task.

1 INTRODUCTION

Helicopters maintenance, which is performed by

aircraft maintenance technicians (AMT) is a critical

process that involves the use of visual information

obtained from aircraft maintenance manuals. In this

context maintenance documentation is important to

ensure the security of the whole process. In order to

understand the maintenance activity, we need to

collect data on the use of these documents by the

AMT in the real work environment with minimum

interference. Eye-tracking is a behavioral

measurement tool adapted to real-world conditions

that provides insights about information intake and a

view on the action.

The aim of this preliminary, observational study

is to validate a new methodology using eye tracking

to capture the operator’s viewing of documentation

(task prescription) during a helicopter maintenance

task.

2 LITERATURE REVIEW

Maintenance has an essential role in keeping the

airworthiness of the helicopter. Aeronautical

maintenance is part of the framework of industrial

https://orcid.org/0000-0001-9127-2041

https://orcid.org/0000-0002-7101-8043

https://orcid.org/0000-0002-0399-4747

maintenance and is considered critical for both the

operator and the system. Criticality is explained by a

high level of uncertainty and variability in

maintenance operations. The execution of all the

tasks requires a trained workforce (Grusenmeyer,

2014; Souza et al., 2021), aircraft maintenance

technicians. In order to have a high level of

consistency and reliability in the outcome of

maintenance activities, it is prescribed by a set of

rules, procedures and standards.

Procedural documents are technical documents

guiding the operator’s actions toward the task

requirements (Leplat, 2008). However, as part of an

aeronautical maintenance study Zafiharimalala

(2011) mentioned the lack of data on the actual use of

the maintenance documentation.

In this context eye-tracking appears to be an

interesting tool to better understand the visual

behavior of the AMTs. This tool measures eye

movement and provides metrics on gaze fixation. In

particular, the spatial and temporal pattern of

fixations in the environment provides information

about attentional processes. Eye tracking is used to

analyze how the operator selects the sources of visual

information essential to carry out the work activity

(Duchowski, 2002; Souza et al., 2021). To measure

visual attention, the area of interest (AOI) (Jacob &

Karn 2003) classification combines all the fixations

Paris, F., Casanova, R., Bergeonneau, M. and Mestre, D.

Analysis of the Use of the Maintenance Documentation Using Eye-Tracking: A Pilot Study.

DOI: 10.5220/0011958400003622

In Proceedings of the 1st International Conference on Cognitive Aircraft Systems (ICCAS 2022), pages 57-60

ISBN: 978-989-758-657-6

 2023 by SCITEPRESS – Science and Technology Publications, Lda. Under CC license (CC BY-NC-ND 4.0)

made in a defined area of interest (an object for

example). Visual attention distribution analysis was

previously used in several analyses of the activity

under ecological conditions (Henderson, 2003; Land,

2006) or highlighting information gathering

strategies for in the cockpit by the pilot-co-pilot pair

(Hutchins et al., 2013).

A mobile eye tracking tool in the form of glasses

has the advantage of being minimally invasive for the

operator. The realization of eye-tracking

measurements is based in part on a stage camera for

allowing to reference eye movements in relation to

the environment. This camera has an egocentric view

of the action environment, offering a significant gain

of information compared to a fixed stage camera: the

camera is closest to the operator and moves with him.

The operator hands are visible and when handling

objects, objects are located at maximum arm distance

(Ren & Philipose, 2009). Eye-tracking is a behavioral

measurement tool adapted to the real (mobile)

condition, providing cues to the information intake

(here the maintenance documentation) but also a view

on the action (stages of the maintenance procedure).

Our study proposes a new methodology to

evaluate information intake in the maintenance

documentation, depending on the progress in the

operation (stages).

3 METHODOLOGY

3.1 Procedure

During this pilot study, the aircraft maintenance

technician (AMT) performed an inspection of the

component parts of the right rear landing gear brake

unit on a H215/225 helicopter. The operation consists

of 3 phases: disassembly, inspection, assembly. The

task is divided into stages that are bounded by

milestones. Milestones are specific steps of the task.

The sequence of milestones cannot be performed in

any other order. The whole maintenance operation is

presented in Figure 1. It is broken down into 3 phases

and 9 stages, delimited by 8 milestones. The first

phase is the removal and disassembly phase: to reach

the break, the operator lifts the helicopter on jacks and

removes a pin, then removes the wheel (Stage I & II).

The braking unit is extracted from the helicopter

(Stage III) and all its components are removed

(Milestone 4). The second phase is an inspection of

the brake components (Stage V). The third phase is

an assembly and installation phase: the brake unit is

reassembled and put back on the helicopter (Stage

VI), then the wheel and the pin are installed and

finally the helicopter is put back on its wheels (Stage

VII & VIII).

The entire procedure is detailed in a 21-page

aircraft maintenance manual in paper format. During

the experiment, there was no time constraint and all

the required tools were available.

We studied the use of the maintenance document-

tation during the course of the operation. We captured

the time and duration of each phase and stage and in

the same way the time and duration of consulting of the

documentation (hereafter called viewing).

3.2 Material

The AMT’s eye gaze data was collected using an eye-

tracking system Tobii® Pro Glasses V2 (100Hz,

1080p, 60fps). Among the data flow provided by this

device, we used the egocentric video from the scene

camera and the eye-tracking data.

3.3 Data Analysis & Processing

All fixations were manually mapped on 21 AOIs

corresponding to 21 pages of aircraft maintenance

manual documentation using Tobii Pro Lab 1.152

analysis software. The milestones timestamps

(described above in the procedure), were manually

extracted from the scene video. Since Tobii Pro Lab

provides a set of predicates for each AOI for each

acquisition timestamp, we only combined the 21

Figure 1: Decomposition of the maintenance operation: The numbers on the upper part are the milestones of the operation,

color zones represent the stages between two milestones. Bottom: the three main phases of the operation are represented. The

colors used on the figure emphasize the association of pairs of disassembly/assembly stages. The middle phase is the

inspection.

ICCAS 2022 - International Conference on Cognitive Aircraft Systems

Table 1: Relative and absolute duration of execution times and viewing by stage.

Figure 2: Evolution of document consultation (cumulated) as a function of time. The colored stripes represent stages delimited

by milestones (described in Figure 1). The gray areas represent the periods gazing at the document. The black curve represents

the cumulated percentage of time past gazing at the document.

predicates corresponding to the pages into a single

predicate identifying the action of viewing the

document over time. Milestone timestamps are used

to describe the evolution of the of the task progress in

the time dimension. All data processing outside Tobii

Pro Lab was performed on Matlab®.

4 RESULTS

The operation lasted 1 hour 36 minutes and 33

seconds with a total viewing time of 09 minutes and

27 seconds. Table 1 provides detailed information on

the cumulative percentage of time and viewing at

each stage.

The data are presented in Figure 2, with the

evolution of cumulative viewing time on the timeline

with the viewing occurrence represented by gray bars.

Fixations in the documentation separated by less than

1 second were grouped in the same bar.

The end of the disassembly phase (Milestones 4)

occur at 41.7% of the total time while 67.0% of the

total time is spent viewing the document. The

inspection time took 2.8% (Stage V) with 1.6% of the

time spent viewing the document. Assembly time

(from Milestones 5 to the end) took 55.4% of the total

time and 32.5% of the time spent viewing the. More

than 55% of the time spent viewing the

documentation occurs during stages I and II and after

15 minutes and 24 seconds: 1/6 of the total operation

time. 19.4% of the stage I duration is devoted to

document consultation; this ratio subsequently

increases to 34.9% in stage II. From stage III

onwards, it is less than 11.0 %.

5 DISCUSSION

In this study, eye-tracking was used to capture gaze

behavior as a function of time and stages in a

maintenance task.

The analysis revealed an asymmetrical

distribution of phases and stages: the inspection phase

(Stage V) is not centered (41.7% to 44.6%). The

Stage Metrics

Viewing the document

Stage

Duration

(mm:ss)

% / total

Duration

(mm:ss)

% / total

% /stage

duration

I 09:18 9.6% 01:48 19.1% 19.4%

II 10:22 10.7% 03:37 38.2% 34.9%

III 09:14 9.6% 00:00 0.0% 0.0%

IV 11:25 11.8% 00:55 9.7% 8.1%

V 02:44 2.8% 00:03 0.5% 1.6%

VI 20:16 21.0% 00:54 9.6% 4.5%

VII 16:09 16.7% 00:51 8.9% 5.2%

VIII 10:33 10.9% 01:10 12.3% 11.0%

IX 06:33 6.8% 00:10 1.7% 2.5%

Analysis of the Use of the Maintenance Documentation Using Eye-Tracking: A Pilot Study

disassembly phase is shorter than the assembly phase.

In more detail, the assembly/disassembly stages are

not totally symmetrical: for example, unscrewing

does not require any torque wrench, while it is

mandatory in the assembly phase.

Nonetheless, the ratio of time spent viewing the

documentation is higher in the disassembly phase. In

addition, the cumulative time spent viewing the

document is mostly at the beginning of the

disassembly phase. The two longest phases occur at

the beginning of the operation (Stage I & II)

corresponding to more than 55% of the total viewing

time, while short reading sequences occur in the

following stages.

This consultation pattern reveals an important

information acquisition phase at the beginning of the

operation. This suggests that the operator carries out

an initial preparatory organization and viewing on the

whole documentation and reads specific information

on critical points during each stage (ex. finding a

force momentum torque). We suggest that the quality

of this preparatory phase has a direct impact on the

good and successful progress of the whole operation.

From this first instantiation of our methodology,

we conclude that it allows us to demonstrate 1) the

unbalanced repartition of the different phases of the

operation 2) and a specific pattern of consultation of

the procedural documentation. Our methodology will

allow us to test the generalization of these patterns to

a larger population and to test specific profiles

(novice & expert).

ACKNOWLEDGEMENTS

Florence Paris’s work is supported by a doctoral grant

from ANRT (l’Association Nationale de la Recherche

et de la Technologie) under N° IPAH13430, in

agreement between Airbus Helicopters and Aix-

Marseille University.

REFERENCES

Duchowski, A. T., 2002. A breadth-first survey of eye-

tracking applications. (Springer, Ed.) Behavior

Research Methods, Instruments, & Computers, 34(4),

455-470.

Grusenmeyer, C. 2014. Maintenance: organizational

modes, activities and health and safety. Use of a French

national survey and in-situ analyses. (Elsevier, Ed.)

Accident Analysis & Prevention, 187-199.

Henderson, J. M. 2003. Human gaze control during real-

world scene perception. (Elsevier, Ed.) Trends in

cognitive sciences, 7(11), 498-504.

Hutchins, E., Weibel, N., Emmenegger, C., Fouse, A., &

Holder, B. (2013). An integrative approach to

understanding flight crew activity. (C. SAGE

Publications Sage CA: Los Angeles, Ed.) Journal of

Cognitive Engineering and Decision Making, 7(4),

353-376.

Jacob, R. J., & Karn, K. S. (2003). Eye tracking in human-

computer interaction and usability research: Ready to

deliver the promises. In The mind's eye (pp. 573-605).

North-Holland.

Land, M. F. (2006). Eye movements and the control of

actions in everyday life. (Elsevier, Ed.) Progress in

retinal and eye research, 25(3), 296-324.

Leplat, J. (2008). Eléments pour l'étude des documents

prescripteurs. In J. Leplat, Le Travail humain (pp. 93-

130). Presses Universitaires de France.

Ren, X., & Philipose, M. (2009). Egocentric recognition of

handled objects: Benchmark and analysis. IEEE

Computer Society Conference on Computer Vision and

Pattern Recognition Workshops, (pp. 1-8).

Souza, M. L., Pereira-Guizzo, C. S., Santos, A. A. B., &

León-Rubio, J. M. (2021). Visual attention of experts

and novices to a critical industrial maintenance task. (S.

Brasil, Ed.) Gestão & Produção, 28.

Zafiharimalala, H. (2011). Étude ergonomique pour la

consultation sur écran de petite taille de la

documentation de maintenance aéronautique (Doctoral

dissertation, Université Toulouse le Mirail-Toulouse

II).

ICCAS 2022 - International Conference on Cognitive Aircraft Systems