Comprehensive Study on Fighter Pilot Attention and Vigilance

Monitoring

Alberto Calvo C

ordoba

1,2 a

, Mar

ıa Rivas Vidal

1,2 b

, Ana Mar

ıa Sollars Castellano

1 c

and Botyuu Oscar Sipele Siale

1 d

Indra Factor

ıa Tecnol

ogica S.U., Avenida de Brueselas 35, 28108, Alcobendas (Madrid), Spain

Universidad Polit

ecnica de Madrid, Centre for Automation and Robotics (UPM-CSIC), Escuela T

ecnica Superior de

Ingenieros Industriales, C. de Jos

e Guti

errez Abascal 2, 28006, Madrid, Spain

Keywords:

Avionics, Situational Awareness, Crew Monitoring, Perception, Sensing Technologies.

Abstract:

Modern air warfare demands a holistic approach to address the evolving complexities of all ﬁghter systems.

With a focus on enhancing pilot performance, multi-task automation provides solutions that allow pilots to

concentrate on critical aspects of the ﬂight and the mission. Recognizing the need for task automation tailored

to individual pilot capabilities, functional state monitoring has become one of the most valuable areas of

research. Speciﬁcally, monitoring pilot attention and vigilance capacities are pivotal factors in achieving the

ﬁrst layer of situational awareness (perception).

This study explores the adverse effects on pilots’ perception of their environment, including issues such as

drowsiness, physical and mental fatigue, visual inattention, attentional tunnelling, and attentional entropy.

Furthermore, it investigates broader conditions such as stress and workload which have a general inﬂuence

on pilot attention. Moreover, with the primary objective of providing a comprehensive overview of how the

perception of pilots can be effectively evaluated, this work integrates insights from biomedical sensors. By

analysing how aviators’ perception is impaired because the inﬂuence of deleterious factors cited above, this

study contributes to the development of tailored solutions aimed at mitigating risks associated with reduced

attention and vigilance.

As a conclusion, this paper sketches a conceptual map illustrating the interconnections between perception-

related conditions, with the aim of serving as a road map for researchers and practitioners and facilitating a

deeper understanding of complex relationships within the proposed framework.

1 INTRODUCTION

The criticality of modern air warfare puts increas-

ing pressure on pilot performance and responsibil-

ities. As a result, higher levels of automation are

required to cope with the demands of the mission.

However, there have been instances where automa-

tion has also proved detrimental to pilot performance

(Gouraud et al., 2017).

Therefore, it is essential that this automation can

be adaptable, with the pilot as the central axis of reg-

ulation. In this way, knowing the functional status of

https://orcid.org/0000-0002-7772-2824

https://orcid.org/0009-0004-0363-8162

https://orcid.org/0009-0000-6698-0820

https://orcid.org/0000-0002-2030-3805

the pilot would allow the system to perform the spe-

ciﬁc actions and tasks required at each moment. This

condition can be altered by various factors, includ-

ing psychological, cognitive, and behavioural aspects

(Martins, 2016; Zhang et al., 2020).

In this context, understanding these alterations is

crucial for optimizing pilot well-being and mission

outcomes. Speciﬁcally, conditions related to per-

ception are intrinsic aspects of pilot performance, as

they directly affect their ability to maintain situational

awareness and make critical decisions in difﬁcult en-

vironments. This situational awareness can be deﬁned

based on three different layers that explains how hu-

mans handle and process information in dynamic en-

vironments (Endsley, 1995):

1. The perception of key information relevant to the

current state of the environment;

118

Córdoba, A., Vidal, M., Castellano, A. and Siale, B.

Comprehensive Study on Fighter Pilot Attention and Vigilance Monitoring.

DOI: 10.5220/0013035400004562

Paper published under CC license (CC BY-NC-ND 4.0)

In Proceedings of the 2nd International Conference on Cognitive Aircraft Systems (ICCAS 2024), pages 118-125

ISBN: 978-989-758-724-5

2. The understanding of the signiﬁcance this infor-

mation has according to the current goals;

3. The projection of the future status to prevent pos-

sible situations.

In the aviation context, a paired state of these con-

ditions with environment is crucial to ensure safety,

decision-making, performance, risk mitigation and

adaptability, among others. This study will focus par-

ticularly on the ﬁrst of them, perception, and how it

is computed based on the integration of different sys-

tems together with the assessment of pilot’s functional

state.

To this end, monitoring pilots’ health and cogni-

tive conditions is one of the fundamental pillars when

designing a user-adapted automation system. In this

case, these monitoring techniques require the use of

biosensors, which measure human physiology, cog-

nition, and behaviour to perform speciﬁc evaluations

related to the health or performance of the individual.

Some examples used for this purpose are electrocar-

diograms (ECGs), electroencephalograms (EEGs), or

eye-trackers.

The primary objectives of this article are to ex-

plore the complexities of pilot condition impairment,

focusing speciﬁcally on vigilance, attention, and other

perception-related factors. By reviewing the latest re-

search ﬁndings and theoretical frameworks, this paper

aims to provide a comprehensive understanding of the

interaction between these conditions and pilot percep-

tion, ultimately contributing to the development of ef-

fective strategies to improve pilot well-being and per-

formance. Finally, the correlation between the condi-

tions and the way they are measured, through biosen-

sors, will be analysed.

2 THEORETICAL FRAMEWORK

2.1 Hypovigilance

Sustained attention, or vigilance, refers to the state in

which attention must be maintained over time (Britan-

nica, nd). A degraded state of this condition could be

said to represent a state of hypovigilance (Sahayadhas

et al., 2015).

Further conceptualization of this condition is pro-

vided in (Abbas and Alsheddy, 2021), emphasizing

its association with cognitive and visual inattention,

drowsiness and fatigue. The study is framed in the

automotive industry, where hypovigilance detection

systems use analysis of driver behaviour and physi-

ological measures to identify signs of reduced alert-

ness. Thus, drawing parallels, hypovigilance poses

signiﬁcant challenges to ﬁghter pilot performance as

it compromises the ability to maintain perception and

respond effectively to dynamic ﬂight conditions. Pos-

sible causes of hypovigilance, including distractions,

boredom, sleep disturbances, and fatigue, directly af-

fect the alertness and attentional resources of the pi-

lot, and are closely related to other conditions where

information processing is involved.

2.2 Fatigue, Drowsiness, and Sleep

Fatigue and hypovigilance contribute to the degrada-

tion of pilot alertness and performance in distinct yet

interrelated ways. While hypovigilance represents a

broader state of reduced vigilance and attentional re-

sources, fatigue manifests as a speciﬁc physiological

state characterized by decreased mental or physical

performance capability due to sleep loss, prolonged

wakefulness, circadian rhythm disturbances, or work-

load (ICAO, nd). This condition, often accompanied

by sensations of tiredness and frustration, can arise

from prolonged physical exertion or engagement in

monotonous tasks, both of which are prevalent in the

demanding aviation environment (Hooda et al., 2022).

When pilots experience fatigue, their ability to

maintain optimal perception levels is compromised.

Both civilian and military pilots are susceptible to

the adverse effects of fatigue, which encompass di-

minished cognitive function, slower reaction times,

and higher error rates, compared to a well-rested

state (Caldwell et al., 2009). Furthermore, when dis-

cussing fatigue among ﬁghter pilots, the risk of fa-

tigue is heightened, given the demanding nature of

their tasks and the inherent physiological and psycho-

logical stress associated with combat operations, es-

pecially on prolonged ﬂights (Ohrui et al., 2008).

On the other hand, distinguishing between task-

related and sleep-related fatigue provides valuable in-

sights into the mechanisms underlying pilot fatigue

and its impact on performance (May and Baldwin,

2009). Sleep-related fatigue is associated with insuf-

ﬁcient sleep or operating during periods of the circa-

dian rhythm when sleep is usually occurring. In con-

trast, task-related fatigue, resulting from prolonged

participation in demanding activities or exposure to

monotonous tasks, is tied to the task itself and its as-

sociated environmental factors such as temperature or

humidity. In addition, it can exacerbate sleep-related

fatigue, further compromising alertness and cognitive

function of the pilot (Harding et al., 2019; Imtiaz,

2021; Kang et al., 2015).

As each type of fatigue represents a distinct phys-

iological state, with unique implications for pilot per-

formance (Borghini et al., 2014), it has been consid-

Comprehensive Study on Fighter Pilot Attention and Vigilance Monitoring

119

ered relevant for this study to differentiate it into the

following concepts: mental fatigue and drowsiness

(or sleepiness). Thus, based on the deﬁnition of the

International Civil Aviation Organization (ICAO, nd),

mental fatigue could be referred to as the reduction

in performance capacity resulting from a prolonged

workload or high task demands, while drowsiness

could be associated with alterations in sleep patterns

(Chowdhury et al., 2018; Guede-Fernandez et al.,

2019; Raﬁd et al., 2020).

While mental fatigue is more associated with a de-

crease in cognitive performance due to sustained men-

tal effort, drowsiness is more related to a physiolog-

ical drive toward sleep. A person experiencing men-

tal fatigue may be able to maintain concentration and

performance for a time through compensatory effort.

However, as the state progresses toward drowsiness,

this becomes increasingly difﬁcult and performance

begins to decline signiﬁcantly, reﬂecting the increas-

ing need for sleep and the body’s natural preparation

for this transition (Borghini et al., 2014). Therefore,

the main difference between the two states would be

that short rest decreases mental fatigue, while it ag-

gravates drowsiness (Stancin et al., 2021).

Considering the above-mentioned, drowsiness can

be described as a physiological state in which the

body is in transition from wakefulness to a sleep-

ing state (Ngxande et al., 2017). According to this

deﬁnition, drowsiness is frequently experienced by

pilots during long-duration missions due to circa-

dian rhythm disturbance, loss or interruption of sleep,

and prolonged wakefulness; as well as during non-

demanding tasks or monotonous activities that could

end up in boredom. Performing sophisticated aircraft

operations under reduced alertness is primarily asso-

ciated with severe aircraft accidents (Board, 2018; M.

R. Rosekind, K. B. Gregory, E. L. Co, D. L. Miller

and Dinges, 2000), and drowsiness has been identi-

ﬁed as a contributing factor to such accidents (et Al,

2012).

Another signiﬁcant aspect of drowsiness is its

close relationship with the state of sleep, to such an

extent that they can be considered an evolution of

states. That is, sleep could be deﬁned as the circadian

state that emerges in the ﬁnal phases of drowsiness,

marked by the appearance of partial or total suspen-

sion of consciousness, muscular inhibition, and re-

duced responsiveness to external stimuli. Moreover,

the main driver of this continuum can be considered

the level of alertness; a decreasing alertness that con-

verges in the ﬁnal state: sleep (Albadawi et al., 2022).

Falling into a sleep state could compromise safety if

it occurs at an inappropriate time, such as when pilot-

ing an aircraft. Furthermore, in the case of a ﬁghter

aircraft, the likelihood of a sleep episode can be in-

ﬂuenced by several factors associated with the circa-

dian rhythm disruptions, such as participating in pro-

longed missions without rest or night missions, and

could even result in pilots experiencing what is known

as Shift Work Sleep Disorder (Zou et al., 2022).

2.3 Inattention-Related Conditions

Having deﬁned primarily physiological conditions

that affect attention and vigilance, the next step will

be to analyse inattention as manifested through vari-

ous cognitive and behavioural processes, illustrating

the intricate ways in which attention can be dete-

riorated in demanding environments. For example,

attention may be unintentionally diverted from the

task at hand as thoughts wander to unrelated mat-

ters. This phenomenon often occurs during periods

of monotony or boredom and shows how cognitive

resources may inadvertently divert from the demands

of the task, what is often referred to as mind wander-

ing (Smallwood and Schooler, 2015). This detour of

attention can also be abrupt, due to a sudden, unex-

pected, overwhelming stimulus. At such moments, a

rapid physical and mental response (startle effect) is

produced (Deniel et al., 2023), which causes the at-

tention to be momentarily disconnected from the cur-

rent task (Diarra et al., 2023).

On the other hand, perseveration, another facet of

cognitive inattention (Dehais et al., 2019), highlights

the challenges in adapting to shifting circumstances.

When individuals have difﬁculty redirecting their at-

tention in response to changes in situations, they may

become locked into outdated mental frameworks, im-

peding their ability to interact effectively with new

information (Dehais et al., 2010). Perseveration is

intrinsically related to one aspect of behavioural at-

tention, attentional tunnelling. In this state, atten-

tional resources are largely used to select and process

subjectively relevant information, and attention is ar-

guably channelled (Baddeley, 1972). In contrast, fo-

cusing on many stimuli at once could lead to a failure

to focus attention on relevant information. This lack

of focus may manifest itself in another behavioural

condition known as attentional entropy, due to high

entropy in the attentional pathway (Ayala et al., 2023).

2.4 Hypervigilance and

Performance-Related Conditions

Hypervigilance is closely related to these attention-

related conditions, as it involves an abnormal increase

in attention to threat-related stimuli and difﬁculty dis-

engaging attention from such stimuli (Kimble et al.,

ICCAS 2024 - International Conference on Cognitive Aircraft Systems

120

2014; Zawilinski, 2020). Thus, in the face of high-

priority targets, hypervigilance may lead to ampliﬁed

attention or increased scanning speed, eliciting atyp-

ical alertness. In addition, hypervigilance has been

associated with disorders such as depression, anxi-

ety, or post-traumatic stress disorder, among others.

The latter is intrinsically related to stress, a common

phenomenon among aircraft pilots, who face vari-

ous sources of physical and psychological demands

(stressors) in their work environment, such as per-

sistent noise, uncomfortable temperature, or lighting

conditions. In the case of ﬁghter pilots, not only

are these stressors exacerbated by the nature of the

aircraft, but others are added, such as psychologi-

cal stress arising from combat and mission-speciﬁc

operations (Sullivan-Kwantes et al., 2021). Mental

workload can act as one of these stressors, but it is

a condition in itself derived from high task-related

demands compared to available cognitive resources

(Gaillard, 1993). Furthermore, sustained stress and

mental workload can lead to mental fatigue and de-

creased performance (Holm et al., 2009; Kunasegaran

et al., 2023).

3 HOLISTIC ANALYSIS

3.1 Perception Framework

The previous section has shown that pilot perception

is affected by a multifaceted set of conditions and

their complex interactions. This section holistically

addresses each of these states to understand the fac-

tors inﬂuencing pilot well-being and operational ef-

fectiveness comprehensively.

Figure 1 describes the theoretical framework of

the subject of this paper, showing the main factors that

alter perception:

• Tiredness may negatively impact pilot attention,

leading to decreased alertness and increased risk

of errors during ﬂight;

• Boredom could lead to decreased vigilance and

reduced focus on the pilot;

• Sleepiness and circadian rhythm alterations re-

quire early detection before the mission begins.

They may impair pilot attention, resulting in re-

duced alertness and susceptibility to errors, which

can pose signiﬁcant risks to ﬂight safety;

• Sleepiness and circadian rhythm alterations re-

quire early detection before the mission begins.

They may impair pilot attention, resulting in re-

duced alertness and susceptibility to errors, which

can pose signiﬁcant risks to ﬂight safety;

• Inattention could signiﬁcantly compromise pilot

vigilance, potentially leading to missed critical

cues or hazards, and increasing the likelihood of

errors or accidents during ﬂight. Distractions, sur-

prises, extreme focus, and dispersion are the lead-

ing causes of inattention that are unrelated to the

other factors;

• Performance refers to the relationships between

fatigue, mental workload, and stress, as the con-

tinuity of the different demands and stressors that

could occur during the mission generates men-

tal fatigue by overloading their mental capabil-

ities. Complex task demands, high-stakes situ-

ations, and the need for rapid decision-making

can contribute to lapses in attention and vigilance

among ﬁghter pilots.

All these factors could potentially compromise the

ability of the pilot to maintain perception and respond

effectively to critical events during ﬂight, leading to a

state of hypo/hyper-vigilance. This could correspond

to the expected output that the proposed pilot health

monitoring system would be waiting for to assist the

pilot, guide their attention, reduce their tasks, or ap-

ply the adequate action according to their current at-

tentional capabilities.

Figure 1: Proposed cognitive model for perception.

On the other hand, there may arise situations

where the pilot becomes incapacitated to fulﬁl his du-

ties appropriately. Fighter cockpits are highly com-

plex and dangerous environments where pilots must

specially maintain their orientation, body functions

and consciousness, to avoid spatial disorientation and

sensory illusions, alterations in respiratory, thermal,

glucose or hydration levels, and loss of conscious-

ness. In this extreme case, the aircraft should take

control.

Comprehensive Study on Fighter Pilot Attention and Vigilance Monitoring

121

Therefore, according to our study, it could be pos-

sible to address three levels of perception assessment

considering the physiology, behaviour, and cognition

of the pilot: alertness, performance, and incapaci-

tation. The next step is to conceptualise the future

ﬁghter cockpit pilot monitoring system and how to

evaluate the functional state of the pilot based on per-

ception.

3.2 Pilot Monitoring System

In-ﬂight pilot condition monitoring is a signiﬁcant

concern for countries to guide future advances in

aerospace engineering and promote ﬂight safety and

efﬁciency (Shaw and Harrell, 2023). To this end, it is

necessary to integrate physiological sensors and other

devices (such as cameras or microphones) into the

aircraft to provide a multi-modal monitoring system.

For instance, previous studies have shown that pulse

oximetry, respiratory gas exchange, ECG or EEG are

among the most promising (Shaw and Harrell, 2023).

Following these as baselines, the sensors proposed to

compose this monitoring system for the deﬁned pilot

perception framework are detailed in the correspon-

dence matrix (Table 1).

By analyzing this matrix, two clusters of sensors

can be identiﬁed: The ﬁrst cluster encompasses sen-

sors with broad applicability across various condi-

tions. For example, ocular sensors (eye-trackers), of-

fer versatile implementation across different condi-

tions, enabling assessment of pilot performance in di-

verse physical and cognitive contexts (Nemcova et al.,

2021). Similarly, cardiac technologies, such as ECGs

and photoplethysmograms (PPGs), provide biomed-

ical indicators such as heart rate variability, aiding

in understanding autonomic nervous system regula-

tion due to varying stress levels, physical fatigue,

mental workload, and changes in cognitive demand.

These sensors have also proven useful in predicting

drowsiness, sleep, vigilance levels and mind wander-

ing (Ad

ao Martins et al., 2021; Burrell et al., 2016;

Lohani et al., 2019). Additionally, all these condi-

tions can be detected by recording the electrical ac-

tivity of the brain, for example through non-invasive

sensors such as EEG, and examining its parameters

and their variability (Kumar and Bhuvaneswari, 2012;

Stancin et al., 2021). Electrodermal activity (EDA)

sensors search for a physical response to these states

by measuring variances in skin conductance response

and conductance levels (Khushaba et al., 2011). Fur-

thermore, respiratory sensors, such as oxygen and car-

bon dioxide concentration and ﬂow meters, provide

different gas exchange metrics, including respiratory

rate and volume. These parameters are strongly re-

lated to fatigue and monitoring drowsiness (Chowd-

hury et al., 2018). Motor sensors (electromyograms,

cameras, and accelerometers) also contribute to this

cluster by detecting body positioning and body seg-

ment neuromuscular activities (In-Ho Choi and Yong-

Guk Kim, 2014).

The second cluster comprises to sensors tailored

to speciﬁc conditions. Acoustic sensors (micro-

phones), for instance, capture vigilance or sleep-

related states through voice interactions (Kaur and

Singh, 2023). On the other hand, thermal activity sen-

sors predict core body temperature, offering insights

into emotional states, arousal levels, and stress reac-

tions triggered by cockpit environmental conditions

(Shetty et al., 2015; Trujillo, 1998). Lastly, glucome-

ters focus on detecting blood glucose concentration

and predicting trends in glucose levels, with impli-

cations for assessing physical fatigue (Velasco et al.,

2022).

In essence, the true power of pilot monitoring

lies not in the capabilities of individual sensors, but

in their collective integration. Each sensor brings

unique insights, but it’s their combined data that of-

fers a comprehensive understanding of pilot states.

This comprehensive monitoring approach, facilitated

by the collective integration of sensors, is a signiﬁcant

step towards enhancing aviation safety and efﬁciency,

providing reassurance in the thoroughness of our pro-

posed system.

4 DISCUSSION

In the rapidly evolving landscape of modern air war-

fare, developing automation systems tailored to the

functional state of the pilot is urgently necessary.

These systems, designed to enhance pilot and mis-

sion performance, must address the complex inter-

actions of impairments that deteriorate pilot condi-

tions inﬂuenced by psychological, cognitive, and be-

havioural factors. Understanding these relationships

is crucial for developing effective strategies to design

pilot monitoring systems that aim to improve pilot

well-being and mission outcomes.

This study presents a comprehensive framework

that organizes the ﬁrst layer of situational awareness

– perception - within the context of future cockpits.

By breaking down perception into three fundamen-

tal pillars - alertness, performance, and capability – a

structured approach to evaluate and comprehend the

factors integral to pilot condition can be provided. As

for the level of alertness, given that vigilance is es-

sentially the result of sustained attention, the causes

related to its absence have been highlighted as the

ICCAS 2024 - International Conference on Cognitive Aircraft Systems

122

Table 1: Relevance matrix between conditions and monitoring system components. X indicates condition measured by sensor.

CONDITIONS

SENSORS V D SL ST PF MF MWL MW AT AE

Acoustic X X X

Cardiac X X X X X X X X

Cerebral X X X X X X X X

Electrodermal X X X X X X X

Glucose X

Motor X X X X X

Ocular X X X X X X X X X X

Respiratory X X X X X X X

Thermal X X X

V: Vigilance; D: Drowsiness; SL: Sleep; ST: Stress; PF: Physical Fatigue; MF: Mental Fatigue;

MWL: Mental Workload; MW: Mind Wandering; AT: Attentional Tunnelling; AE: Attentional Entropy

ﬁrst to be monitored. Drawing from the literature,

this study has identiﬁed and uniﬁed attention disor-

ders: surprises and distractions for cognitive con-

ditions; and extreme focus and dispersion for be-

havioural. As secondary disorders, startle effect and

mind wandering have been reported as cognitive con-

ditions; and attentional tunnelling, perseveration, and

attentional entropy as behavioural ones. In terms of

physiological conditions, drowsiness has been identi-

ﬁed as the primary one caused by boredom, circadian

rhythm alterations, fatigue, and, in extreme cases,

falling asleep. Based on this premise, the leading re-

ported causes of impaired vigilance capabilities are

inattention, tiredness, boredom, and circadian rhythm

alteration.

Furthermore, a positive balance between stress,

mental workload, and fatigue is critical for pilot per-

formance. Such conditions are inﬂuenced by complex

tasks demands, high-risk situations, and the extended

need for rapid decision-making. Therefore, pilot per-

formance is assumed to decrease since the beginning

of the mission. Finally, physical disturbances lead-

ing to incapacitation (such as loss of consciousness,

spatial disorientation, or body malfunctions) are con-

sidered within the last pillar of perception.

The primary function of the monitoring system is

to track the evolution of pilot state, based on their sit-

uational awareness, and to determine the appropriate

level of task automation. To achieve this, this study

has identiﬁed a set of sensors that can cover the con-

ditions related to the ﬁrst layer of situational aware-

ness. Table 1 outlines the correlation between these

conditions and the necessary sensors for their mea-

surement and evaluation. It is important to note the

need for a multi-modal approach that can differen-

tiate between conditions and provide a comprehen-

sive assessment of those that occur simultaneously.

Through this holistic monitoring approach, while vi-

sion is the primary mode of (to evaluate the visual

attention and vigilance capabilities of the pilot), other

sensory channels are also utilized to address complex

and speciﬁc cognitive states.

While this study provides valuable insights into

pilot cognition and behaviour, it acknowledges its cur-

rent limitation in not fully exploring the broader con-

text involving a myriad of conditions and possible cir-

cumstances. Additionally, the framework is still at a

preliminary stage, awaiting experimentation and com-

putation to further reﬁne the proposed perception as-

sessment. This presents an inspiring opportunity for

future research and development in this ﬁeld.

Future research to overcome these limitations

will require creating a complete model of the pilot

functional state alongside corresponding experimen-

tal protocols regarding pilot perception capabilities

over a multi-modal approach. It would also be valu-

able to consider incorporating the other two layers of

situational awareness - relevance and anticipation -

thereby complementing the scope of this study. In the

future, this reﬁned framework may serve as a foun-

dational basis for developing advanced machine and

deep learning models aimed at detecting, differenti-

ating, and predicting the various states constituting

the situational awareness framework in pilots. More-

over, these models could focus on the interpretation of

their results, ensuring clarity on how and why speciﬁc

states are classiﬁed or predicted.

5 CONCLUSIONS

In conclusion, the ﬁndings of this study shed light on

a holistic approach of pilot’s perception of their sur-

roundings. By conducting an integrated analysis of

the physiological, cognitive, and behavioural condi-

tions involved, as well as their relationship, this study

Comprehensive Study on Fighter Pilot Attention and Vigilance Monitoring

123

has built a conceptual framework regarding key fac-

tors inﬂuencing pilot perception. Furthermore, by ex-

amining the correlation between sensors and condi-

tions, the need for a multi-modal approach has been

highlighted. This will provide means to quantify the

different perception levels identiﬁed, improving the

ability to effectively address and assess them. Over-

all, this study contributes to expanding the literature

on condition monitoring and underscores its impor-

tance in the military aviation ﬁeld.

ACKNOWLEDGEMENTS

This publication was co-funded by the European

Union under Grant Agreement No 101103592. Views

and opinions expressed are however those of the au-

thor(s) only and do not necessarily reﬂect those of the

European Union or the European Commission. Nei-

ther the European Union nor the granting authority

can be held responsible for them.

REFERENCES

Abbas, Q. and Alsheddy, A. (2021). A Methodological

Review on Prediction of Multi-Stage Hypovigilance

Detection Systems Using Multimodal Features. IEEE

Access, 9:47530–47564.

ao Martins, N. R., Annaheim, S., Spengler, C. M., and

Rossi, R. M. (2021). Fatigue Monitoring Through

Wearables: A State-of-the-Art Review. Frontiers in

Physiology, 12(December).

Albadawi, Y., Takruri, M., and Awad, M. (2022). A Review

of Recent Developments in Driver Drowsiness Detec-

tion Systems. Sensors, 22(5):2069.

Ayala, N., Zafar, A., Kearns, S., Irving, E., Cao, S., and

Niechwiej-Szwedo, E. (2023). The effects of task dif-

ﬁculty on gaze behaviour during landing with visual

ﬂight rules in low-time pilots. Journal of eye move-

ment research, 16(1).

Baddeley, A. D. (1972). Selective attention and perfor-

mance in dangerous environments. British Journal of

Psychology, 63(4):537–546.

Board, N. T. S. (2018). Most Wanted List of Transportation

Safety Improvements: Reduce Fatigue-Related Acci-

dents.

Borghini, G., Astolﬁ, L., Vecchiato, G., Mattia, D., and Ba-

biloni, F. (2014). Measuring neurophysiological sig-

nals in aircraft pilots and car drivers for the assessment

of mental workload, fatigue and drowsiness. Neuro-

science & Biobehavioral Reviews, 44:58–75.

Britannica (n.d). Vigilance deﬁnition.

Burrell, C., Love, R. J., Stergiopoulos, S., and Centre, D.

T. R. (2016). Integrated Physiological Monitoring.

Defence Research and Development Canada Scientiﬁc

Report, (DRDC-RDDC-2016-R207):1–54.

Caldwell, J. A., Mallis, M. M., Caldwell, J. L., Paul, M. A.,

Miller, J. C., and Neri, D. F. (2009). Fatigue Counter-

measures in Aviation. Aviation, Space, and Environ-

mental Medicine, 80(1):29–59.

Chowdhury, A., Shankaran, R., Kavakli, M., and Haque,

M. M. (2018). Sensor Applications and Physiological

Features in Drivers’ Drowsiness Detection: A Review.

IEEE Sensors Journal, 18(8):3055–3067.

Dehais, F., Hodgetts, H. M., Causse, M., Behrend, J., Du-

rantin, G., and Tremblay, S. (2019). Momentary

lapse of control: A cognitive continuum approach

to understanding and mitigating perseveration in hu-

man error. Neuroscience & Biobehavioral Reviews,

100:252–262.

Dehais, F., Tessier, C., Christophe, L., and Reuzeau, F.

(2010). The Perseveration Syndrome in the Pilot’s Ac-

tivity: Guidelines and Cognitive Countermeasures. In

Palanque, P., Vanderdonckt, J., and Winckler, M., ed-

itors, Human Error, Safety and Systems Development,

pages 68–80, Berlin, Heidelberg. Springer Berlin Hei-

delberg.

Deniel, J., Dupuy, M., Duchevet, A., Matton, N., Imbert, J.-

P., and Causse, M. (2023). An in-depth examination of

mental incapacitation and startle reﬂex: A ﬂight simu-

lator study. In Harris, D. and Li, W.-C., editors, Engi-

neering Psychology and Cognitive Ergonomics, pages

46–59, Cham. Springer Nature Switzerland.

Diarra, M., Marchitto, M., Bressolle, M.-C., Baccino, T.,

and Drai-Zerbib, V. (2023). A narrative review of the

interconnection between pilot acute stress, startle, and

surprise effects in the aviation context: Contribution

of physiological measurements .

Endsley, M. R. (1995). Toward a Theory of Situation

Awareness in Dynamic Systems. Human Factors,

37(1):32–64.

et Al, J. O. (2012). Evaluating the Effectiveness of Sched-

ule Changes for Air Trafﬁce Service (ATS) Providers:

Controller Alertness and Fatigue Monitoring Study.

Gaillard, A. (1993). Comparing the concepts of mental load

and stress. Ergonomics, 36(9):991–1005.

Gouraud, J., Delorme, A., and Berberian, B. (2017). Au-

topilot, Mind Wandering, and the Out of the Loop Per-

formance Problem. Frontiers in Neuroscience, 11.

Guede-Fernandez, F., Fernandez-Chimeno, M., Ramos-

Castro, J., and Garcia-Gonzalez, M. A. (2019). Driver

Drowsiness Detection Based on Respiratory Signal

Analysis. IEEE Access, 7:81826–81838.

Harding, E. C., Franks, N. P., and Wisden, W. (2019). The

Temperature Dependence of Sleep. Frontiers in Neu-

roscience, 13.

Holm, A., Lukander, K., Korpela, J., Sallinen, M., and

uller, K. M. I. (2009). Estimating Brain Load from

the EEG. TheScientiﬁcWorldJOURNAL, 9:973791.

Hooda, R., Joshi, V., and Shah, M. (2022). A compre-

hensive review of approaches to detect fatigue using

machine learning techniques. Chronic Diseases and

Translational Medicine, 8(1):26–35.

ICAO (n.d.). Fatigue Management.

Imtiaz, S. A. (2021). A Systematic Review of Sensing

ICCAS 2024 - International Conference on Cognitive Aircraft Systems

124

Technologies for Wearable Sleep Staging. Sensors,

21(5):1562.

In-Ho Choi and Yong-Guk Kim (2014). Head pose and gaze

direction tracking for detecting a drowsy driver. In

2014 International Conference on Big Data and Smart

Computing (BIGCOMP), pages 241–244. IEEE.

Kang, D.-H., Jeon, J.-K., and Lee, J.-H. (2015). Effects

of low-frequency electrical stimulation on cumulative

fatigue and muscle tone of the erector spinae. Journal

of Physical Therapy Science, 27(1):105–108.

Kaur, K. and Singh, P. (2023). Trends in speech emo-

tion recognition: a comprehensive survey. Multimedia

Tools and Applications.

Khushaba, R. N., Kodagoda, S., Lal, S., and Dissanayake,

G. (2011). Driver Drowsiness Classiﬁcation Using

Fuzzy Wavelet-Packet-Based Feature-Extraction Al-

gorithm. IEEE Transactions on Biomedical Engineer-

ing, 58(1):121–131.

Kimble, M., Boxwala, M., Bean, W., Maletsky, K., Halper,

J., Spollen, K., and Fleming, K. (2014). The impact

of hypervigilance: Evidence for a forward feedback

loop. Journal of Anxiety Disorders, 28(2):241–245.

Kumar, J. S. and Bhuvaneswari, P. (2012). Analysis of

Electroencephalography (EEG) Signals and Its Cate-

gorization–A Study. Procedia Engineering, 38:2525–

2536.

Kunasegaran, K., Ismail, A. M. H., Ramasamy, S., Gnanou,

J. V., Caszo, B. A., and Chen, P. L. (2023). Un-

derstanding mental fatigue and its detection: a com-

parative analysis of assessments and tools. PeerJ,

11:e15744.

Lohani, M., Payne, B. R., and Strayer, D. L. (2019). A

Review of Psychophysiological Measures to Assess

Cognitive States in Real-World Driving. Frontiers in

Human Neuroscience, 13.

M. R. Rosekind, K. B. Gregory, E. L. Co, D. L. Miller and

Dinges, D. F. (2000). Crew Factors in Flight Opera-

tions XII: A Survey of Sleep Quantity and Quality in

On-Board Crew Rest Facilities.

Martins, A. P. G. (2016). A review of important cognitive

concepts in aviation. Aviation, 20(2):65–84.

May, J. F. and Baldwin, C. L. (2009). Driver fatigue:

The importance of identifying causal factors of fatigue

when considering detection and countermeasure tech-

nologies. Transportation Research Part F: Trafﬁc Psy-

chology and Behaviour, 12(3):218–224.

Nemcova, A., Svozilova, V., Bucsuhazy, K., Smisek, R.,

Mezl, M., Hesko, B., Belak, M., Bilik, M., Maxera,

P., Seitl, M., Dominik, T., Semela, M., Sucha, M.,

and Kolar, R. (2021). Multimodal Features for De-

tection of Driver Stress and Fatigue: Review. IEEE

Transactions on Intelligent Transportation Systems,

22(6):3214–3233.

Ngxande, M., Tapamo, J.-R., and Burke, M. (2017). Driver

drowsiness detection using behavioral measures and

machine learning techniques: A review of state-of-

art techniques. In 2017 Pattern Recognition Associ-

ation of South Africa and Robotics and Mechatronics

(PRASA-RobMech), pages 156–161. IEEE.

Ohrui, N., Kanazawa, F., Takeuchi, Y., Otsuka, Y., Tarui,

H., and Miyamoto, Y. (2008). Urinary Catecholamine

Responses in F-15 Pilots: Evaluation of the Stress In-

duced by Long-Distance Flights. Military Medicine,

173(6):594–598.

Raﬁd, A.-U.-I., Raha Niloy, A., Chowdhury, A. I., and

Sharmin, N. (2020). A Brief Review on Different

Driver’s Drowsiness Detection Techniques. Interna-

tional Journal of Image, Graphics and Signal Process-

ing, 12(3):41–50.

Sahayadhas, A., Sundaraj, K., Murugappan, M., and Pala-

niappan, R. (2015). Physiological signal based detec-

tion of driver hypovigilance using higher order spec-

tra. Expert Systems with Applications, 42(22):8669–

8677.

Shaw, D. M. and Harrell, J. W. (2023). Integrating phys-

iological monitoring systems in military aviation : a

brief narrative review of its importance , opportunities

, and risks ABSTRACT. Ergonomics, 0(0):1–13.

Shetty, J., Lawson, C. P., and Shahneh, A. Z. (2015). Sim-

ulation for temperature control of a military aircraft

cockpit to avoid pilot’s thermal stress. CEAS Aero-

nautical Journal, 6(2):319–333.

Smallwood, J. and Schooler, J. W. (2015). The Science of

Mind Wandering: Empirically Navigating the Stream

of Consciousness. Annual Review of Psychology,

66(Volume 66, 2015):487–518.

Stancin, I., Cifrek, M., and Jovic, A. (2021). A Review of

EEG Signal Features and Their Application in Driver

Drowsiness Detection Systems. Sensors, 21(11):3786.

Sullivan-Kwantes, W., Cramer, M., Bouak, F., and Good-

man, L. (2021). Environmental Stress in Military Set-

tings. In Handbook of Military Sciences, pages 1–27.

Springer International Publishing, Cham.

Trujillo, A. (1998). Pilot mental workload with predictive

system status information. In Proceedings Fourth An-

nual Symposium on Human Interaction with Complex

Systems, pages 73–80.

Velasco, J. M., Botella-Serrano, M., S

anchez-S

anchez, A.,

Aramendi, A., Mart

ınez, R., Maqueda, E., Garnica,

O., Contador, S., Lanchares, J., and Hidalgo, J. I.

(2022). Evaluating the Inﬂuence of Mood and Stress

on Glycemic Variability in People with T1DM Using

Glucose Monitoring Sensors and Pools. Diabetology,

3(2):268–275.

Zawilinski, L. (2020). Hypervigilance, pages 2101–2103.

Springer International Publishing, Cham.

Zhang, T., Yang, J., Liang, N., Pitts, B. J., Prakah-Asante,

K., Curry, R., Duerstock, B., Wachs, J. P., and Yu,

D. (2020). Physiological Measurements of Situation

Awareness: A Systematic Review. Human Factors,

65(5):737–758.

Zou, H., Zhou, H., Yan, R., Yao, Z., and Lu, Q. (2022).

Chronotype, circadian rhythm, and psychiatric dis-

orders: Recent evidence and potential mechanisms.

Frontiers in Neuroscience, 16.

Comprehensive Study on Fighter Pilot Attention and Vigilance Monitoring

125