Automated Planning for Military Airline Controller Training Scenarios

Romain Goutiere, Domitile Lourdeaux

and Sylvain Lagrue

Alliance Sorbonne Universit

e, Universit

e de Technologie de Compi

egne, CNRS, Heudiasyc UMR 7253,

CS 60 319, 60203 Compi

egne Cedex, France

Keywords:

ANML, Automated Scenario Planning, Interactive Storytelling, Alternative Scenario Generation, HTN.

Abstract:

In this paper we focus on the generation of scenarios for military airline controller training in a virtual environ-

ment. We are using a planning system mixing temporal planning and hierarchical task networks based on the

ANML planning language and allowing a better representation of both narrative and pedagogical objectives.

We also propose and test a method to automatically generate potential alternative plans to the initially planned

scenario, reaching the same objectives, to make it more robust.

1 INTRODUCTION

The objective of Interactive Storytelling (IS) is to

create multimedia systems in which users can interact

and inﬂuence, in interactive time (time interval be-

tween users actions), the evolution of the narrative.

One of the most active research areas in IS is the

automated generation of scenarios. The objective

of generative approaches is to reduce the authoring

bottleneck (difﬁculties for an author to design com-

plex scenarios with multiple storylines and interac-

tions) during the design of the interactive narration

system. The scenario is thus automatically gener-

ated before and/or dynamically during the simulation,

opening the way to adaptations of the scenario during

the simulation and to a greater variety in the proposed

content. Planning is nowadays a widespread method

for automatic scenario generation. However, its use in

IS requires some adaptations compared to more clas-

sical application frameworks and several challenges

remain. Porteous (Porteous, 2016) highlights in par-

ticular: (1) the planner’s ability to respond to narrative

content, (2) interactive time control over plan gener-

ation, (3) user interactivity which requires the system

to be resilient, and (4) planning on qualitative criteria

other than pure optimization. Two planning methods

are mainly used in IS. Classical planning, based on

the use of heuristics and bringing a strong variability

due to the generative power of this method. And hier-

archical task networks (HTN), consisting in a suc-

cessive decomposition of the problem and bringing a

https://orcid.org/0000-0002-3354-7294

https://orcid.org/0000-0001-9292-3213

high degree of control on the generated plans. Each

of these methods has useful characteristics in IS, but

to date there is no framework allowing the use of both

systems at the same time.

IS systems are also useful for training and educa-

tion. Gupta (Gupta et al., 2008) gives several advan-

tages such as being able to more easily put learners

in a variety of situations, and to have them repeat as

needed. However, systems providing training must

meet certain criteria to ensure the effectiveness of the

experience. For example, it is important that the difﬁ-

culty is well calculated, that the scenarios are coher-

ent and explainable for the trainer, etc.

In this paper, we detail our approach for the auto-

matic planning of training scenarios within the frame-

work of this project with the use of the ANML

planning language allowing a hybrid planning, very

adapted to combine generative power and scenario

control. We also propose algorithms allowing the

generation of ofﬂine alternatives, bringing robust-

ness to the proposed training scenarios. We will ﬁrst

describe the ORCHESTRAA project, to which our

work is linked. Then we will explain the different

approaches used for scenario planning as well as the

problems related to the design of scenarios and their

robustness in an interactive environment. Then, we

will detail our different contributions.

2 THE ORCHESTRAA PROJECT

Our work is part of the ORCHESTRAA project (Or-

chestration of stessful situations for training based on

466

Goutiere, R., Lourdeaux, D. and Lagrue, S.

Automated Planning for Military Airline Controller Training Scenarios.

DOI: 10.5220/0011688700003393

In Proceedings of the 15th International Conference on Agents and Artiﬁcial Intelligence (ICAART 2023) - Volume 3, pages 466-474

ISBN: 978-989-758-623-1; ISSN: 2184-433X

 2023 by SCITEPRESS – Science and Technology Publications, Lda. Under CC license (CC BY-NC-ND 4.0)

virtual reality and adaptative agents in air operations

context) which aims at developing an interactive vir-

tual reality environment reproducing an air operations

control center. This environment must allow users

(immersed in the environment through a HMD) to

carry out training sessions that are characterized by

the management of different conﬂict situations, in-

volving air and ground troops on a ﬁctitious theater

of operations. This theater of operation is managed

by an external simulator interacting with the operators

through visualization tools available on the terminals

of the different stations and through a chat.

Our work within this project is to design a sce-

nario system allowing: the automatic generation of

training scenarios executed both in the virtual envi-

ronment and the simulator, the control of the scenario

execution and the live monitoring of the scenario by

the trainers.

3 STATE OF THE ART

3.1 Scenario Planning

Planning methods for scenario generation are widely

used today. It has several advantages: causality, gen-

erative power and narrative structure (Porteous,

2016). In the context of IS, causality refers to the re-

lation between temporal and ordered events that allow

to predict the occurrence of future events. Generative

power represents the ability of a system to generate

varied content. Finally, the narrative structure allows

to deﬁne the key stages in the progression of a story.

Classical planning is widely used in IS. A plan

is then a succession of actions and events in order

to reach a desired goal. Classical planning problems

are solved with Heuristic Search Planner (HSP) fol-

lowing a heuristic to ﬁnd the best plan. There are

then many variations aiming at handling more com-

plex cases such as the addition of temporal constraints

(Porteous et al., 2011) allowing to add duration, or

to parallelize actions. Another interesting method is

also used in IS (Aylett et al., 2006): hierarchical task

networks (HTN). The plan of an HTN problem is not

represented by a sequence of events and actions, but

as a successive decomposition of complex actions into

simple actions. The structure of a plan is then pre-

sented as a tree.

These two planning methods each offer interest-

ing features in interactive narration, but which may be

problematic depending on the context. Classical plan-

ning, using heuristics, offers great generative power.

Indeed, the planner will work by making a selection

on the set of available actions, which allows for a high

variability, good replanning capabilities and a better

resilience. However, this generative power also forces

a difﬁcult reﬂection work during the design phase in

order to avoid the generation of illogical plans from

a narrative point of view. This constraint is the more

important in a training context where it is essential to

propose scenarios having a pedagogical interest.

On the other hand, HTNs emphasize the respect

of the author’s intention. Indeed, a HTN domain is

made of a set of successive decompositions. The plan-

ner chooses the actions among the proposed decom-

positions and not among all the actions of the domain.

HTNs are also quite simple to model. It is sufﬁcient

that the problem is decomposable. For each decom-

position the alternatives are explicitly deﬁned. Thus,

whatever the choice of the planner, the ﬁnal plan is

made of elements foreseen by the author. These fea-

tures are of interest for IS and seem to be adapted to

the context of training and coaching. However, the

generative power of this method is less efﬁcient than

classical planning. Each of the alternatives has to be

imagined by the author, a bottleneck authoring prob-

lem quickly arises.

Table 1 summarizes the advantages of these plan-

ning methods in relation to interesting features in IS.

Table 1: Comparison HTN/HSP.

HTN HSP

Author intention + -

Resilience - +

Variability - +

Replaniﬁcation - +

Modelling + -

3.2 Planning Languages

To solve an IS problem by a planning method, it

is necessary to express it in a language that a plan-

ner can understand. In order to be able to compare

the performance of different planners, standardiza-

tion projects of these languages have been proposed.

The current standard planning language is the Plan-

ning Domain Description Language (PDDL) (Howe

et al., 1998), which is currently used in International

Planning Competitions (IPC). However, the domain-

s/problems studied in IS are often more complex

than those studied in classical planning because they

describe concepts of various kinds, which generally

translate into a larger number of actions, difﬁculties

in representing the state of the world without losing

too much information, or the addition of many con-

straints. This difference imposes certain choices dur-

ing the formalization of the domain/problem which

can lead to an overall decrease in terms of intelligi-

bility for the human user, and a weakening in the ex-

Automated Planning for Military Airline Controller Training Scenarios

467

pressiveness of the scenario.

The main interest of PDDL is to propose a stan-

dard that can be understood by the largest possible

number of planners. However, PDDL has limitations,

such as the impossibility of simply representing hi-

erarchical planning problems or difﬁculties in repre-

senting complex states of the world containing con-

cepts of very different natures. Moreover, the quality

criteria of plans are not the same in IS and in classical

planning, which implies that the design of PDDL is

oriented towards performances rather than other cri-

teria, more related to narration and often speciﬁc to

the studied domains.

The Action Notation Modeling Language

(ANML) (Smith et al., 2008) is an interesting op-

tion to bring better modelling to IS problems. It is a

planning language offering higher level features than

PDDL. Speciﬁcally, it allows the use of quite com-

plex temporal constraints and also allows modeling

and solving planning problems represented in both

classical and hierarchical forms. It is thus possible

to develop hybrid planning domains where it is not

necessary to adapt the modeling to one or the other

system, but where each element is represented ac-

cording to the approach that best corresponds to it.

This brings great expressiveness in a ﬁeld such as IS,

where rather different concepts must run together.

3.3 Robustness of the Scenario

In an IS system, the running of the scenario can be

problematic. Indeed, a scenario must rely on certain

actions of various actors (human users, virtual char-

acters). Thus, a scenario can potentially be blocked if

an action to be performed by a user never happens.

Many strategies have been developed to address

this problem. They are now gathered around the con-

cept of Experience Management (Riedl and Bulitko,

2013) which consists of monitoring the running of the

scenario and setting up intervention strategies to en-

able it to continue in the best possible way.

One of the main strategies, the narrative medi-

ation (Riedl et al., 2003), proposes, as long as it is

possible, to let the actors perform their actions freely.

If these actions enter into conﬂict with the planned

scenario, the scenario is replanned from the point of

divergence to take into account the problematic ac-

tions, in order to solve the same objectives as the ini-

tial scenario.

However, this solution is not perfect, because it

must be applied in interactive time during the simula-

tion. This often implies the use of fast algorithms that

do not necessarily guarantee the quality of the new

plan (not enough time available to analyze the con-

struction of the plan from a narrative point of view)

(Ramirez and Bulitko, 2014).

Finally, another approach aims at predicting po-

tential errors that could lead to scenario problems

and generating ofﬂine directly applicable alternatives.

Automated Story Director (Riedl et al., 2008) is a

good example. In this system, a scenario is made up

of events of low importance and of Island, which are

events most important for the scenario. The princi-

ple is, for each action, to foresee situations that could

be dangerous for the next Island and to generate an

alternative to reach it.

4 OUR PROPOSALS

The issues raised by the ORCHESTRAA project are

quite representative of the current challenges of using

planning methods in IS and can be extended to other

problems. We need the generative power brought by

planning to generate varied scenarios but the train-

ing context also forces us to be careful about the

emergence of potentially problematic content from

a pedagogical point of view. The interactive aspect

of the system also raises the question of the robust-

ness of our scenarios and their resilience with respect

to the pedagogical objectives of training. Moreover,

the question of formalizing a scenario content into a

planning language remains a task that requires a lot

of work to express concepts that are sometimes not

adapted to a given type of formalism.

Our proposals provide a direct response to these

problems. First, we propose a new representation

of scenarios more adapted to our needs thanks to the

use of ANML. We detail the functioning of this new

structure and its advantages for the representation of

our scenarios.

We also present an approach for automatically

generating ofﬂine alternatives, based directly on the

structure of our scenarios to increase their robustness

and resilience with respect to their pedagogical ob-

jectives.

4.1 Planning Methods and Hybrid

Planning

Choosing a planning method is not easy. Classical

planning offers great generative power. It is an inter-

esting choice for our system because it allows to eas-

ily generate different training sessions. This choice

also has the advantage of facilitating re-planning if

user interactions cause a drift from the planned sce-

nario. However, it is not easy to design a planning

domain, reaching the needs of the project, adapted to

ICAART 2023 - 15th International Conference on Agents and Artiﬁcial Intelligence

468

TIC

CAS request

open

CAS DO receive

CAS request

CAS DO choose

adapted response

CAS DO send orders

Planes perform

orders

Other tasks ?

CAS request

Yes

Figure 1: CAS process.

CAS

Manage CAS

See notif on Jchat Choose plane Send order on Jchat

Begin CAS

Choose TIC place Trigger TIC

Notif CAS request

on JCchat

List planes

(playTime)

Choose TIC position Move planes Set planes stats Display TIC on map

CAS resolution

Perform mission

Move plane Attack

Close CAS

Remove TIC on map Notif end on Jchat Move plane back

Send order on JchatChoose plane

Display TIC on map

Complexe action

Observed action

Triggered action

Figure 2: CAS representation as HTN.

this method. Indeed, the training situations proposed

to military air trafﬁc controllers are decomposed ac-

cording to a rather strict order. Figure 1 gives a proto-

col for the processing of a Close Air Support (CAS)

request, a frequent element of our training scenarios.

All the actions in the planning domain would be

constrained by a ﬁxed order, and would not beneﬁt

from the generative power of planning. These con-

straints would be all the more important as the slight-

est oversight or imprecision could lead to unforeseen

emergent situations.

The protocol presented in Figure 1 is high level.

In order for it to be instantiated in the virtual environ-

ment, it is necessary to decompose it into lower level

actions. The HTN representation therefore seems to

be a better solution. Indeed, HTNs are particularly

well adapted to frozen processes such as those we are

interested in. Figure 2 shows the beginning of the

HTN decomposition of a CAS.

These CASs do not constitute a complete training

scenario. A complete scenario is a set of CASs whose

treatment varies slightly from one to another in order

to work on a set of desired skills. These variations are

easily represented in HTN by explaining alternative

decompositions for certain actions. However, a sce-

nario containing a succession of different CAS is dif-

ﬁcult to represent in HTN. Indeed, an HTN problem is

the decomposition of a single complex task into sim-

ple tasks. In order to represent our sessions, we would

have to decompose a complex ”session” action into

several ”CAS” actions, each one linked to a particu-

lar skill. This representation poses a major problem

in case of need for replanning because it requires to

make explicit all the different possibilities of arrange-

ment of the session at the level of skill, which poses

a problem of expressiveness and does not correspond

to the logic of HTN.

On the contrary, classical planning allows a sim-

pler representation of these skills and the generative

power provided allows to generate much more simply

all the alternatives in terms of training session layout.

We thus have two planning methods, each with

advantages in some aspects and deﬁciencies in oth-

ers. The usual approach to solve this kind of prob-

lem would be to make a choice and adapt our rep-

resentation to minimize the deﬁciencies of the rep-

resentation. This method, although widely used, is

not good. We therefore propose another way, made

possible by the Action Notation Modelling Language

(ANML), to represent hybrid classical/HTN plan-

ning problems. Thanks to this language, we can han-

dle the pedagogical contents via a classical planning

method and the scenaristic aspects with HTN decom-

positions. The link between the two methods is made

through a planning operator that we have developed:

Learning Unit.

4.2 New Operator : Learning Unit

Fuel management Weapon choice In-flight refuelling

CAS

Begin CAS Manage CAS CAS resolution

Initial state Final state

Chose TIC

place

Trigger TIC

See notif on

JChat

Send order on

JChat

Close CAS

HTN

Classical

Planning

Notif CAS

request on

JChat

Choose plane

with good

weapon

Perform

mission

... ... ... ... ... ... ... ...

Figure 3: Hybrid scenario structure.

A Learning Unit (LU) is a high-level planning op-

erator used to link classical planning to a hierarchical

planning problem. A LU is characterized by one or

more target skills, a start time, a duration and an

associated hierarchical decomposition.

The ANML allows us to deﬁne a duration con-

straint expressed as an interval. Two preconditions

are also necessary. The ﬁrst one is simply to check

that the training will focus on the desired skill. The

second is more interesting:

Automated Planning for Military Airline Controller Training Scenarios

469

[ a l l ] i s V a l i d a t e d ( v1 ) ==

f a l s e :−> t r u e ;

This constraint indicates in a simple way that the

skill must not have been validated before the LU and

must be validated at an unspeciﬁed time during the

LU in order for this validation to be veriﬁed at the

end of the LU.

Finally, the last part indicates that the action is also

the root of a hierarchical decomposition. This decom-

position is only done in one element, the CAS, which

will in turn be decomposed.

This structure is not usual in planning. In normal

time, the root of a HTN problem has no conditions,

its resolution coming only from its decomposition. In

the same way, a classical planning problem does not

decompose in a hierarchical way. This structure al-

lows us to propose a strict separation between the

scenaristic and pedagogical content, and thus allows

us to use the planning method most adapted to each

content.

4.3 Global Plan

With our scenario structure, a plan can be summarized

by a succession of LUs, each of which is decomposed

into an HTN. Figure 3 shows us the global structure

of a scenario.

For the moment, we have detailed the advantages

of such a structure for the representation of differ-

ent types of narrative content. However, the advan-

tages of this structure are more extensive. First of all,

this structure brings a better expressiveness to the sce-

nario. Indeed, it is possible to give precisely the state

of the progression from a pedagogical and scenaris-

tic point of view. This information is precious for the

trainers who follow the progress of the training ses-

sions. Thanks to this structure, we can provide them

with this information in a simple way.

This structure also has advantages concerning the

robustness of the scenario. Indeed, this structure al-

lows to set up replanning strategies adapted to each

type of error and not systematically requiring a total

replanning of the plan. At the HTN level, for exam-

ple, the alternatives to each decomposition are made

explicit. If the state of the world allows them to be

planned, they constitute alternatives that can be used

directly without having to replan the entire scenario.

Such alternatives are simply represented in ANML :

a c t i o n s en d O r d e r ( L e a r n e r l1 ,

Order o1 ) {

d u r a t i o n : i n [ 3 , 8 ] ;

[ a l l ] o r d e r S e n t ( o1 ) ==

f a l s e :−> t r u e ;

: d e c o m p o s i t i o n {

[ s t a r t + 2 ] l e a r n O n C h a t ( l 1 ) ==

t r u e ;

w r i t e O r d e r ( l1 , o1 ) ,

send T oCha t ( o1 )

} ;

: d e c o m p o s i t i o n {

[ s t a r t + 2 ] l e a r n O n C h a t ( l 1 ) ==

f a l s e ;

h e l p T o L e a r n e r ( l 1 ) ,

w r i t e O r d e r ( l1 , o1 ) ,

send T oCha t ( o1 )

} ;

In this example we see two different decomposi-

tions for the sendOrder action which corresponds to

the sending of an order by the learner on the chat win-

dow. In the ﬁrst decomposition, the learner is on the

chat window after 2 units of time while he is not in the

second one. The second decomposition then adds a

helper action allowing the learner to perform the right

actions. If the state of the world allows it, these two

decompositions thus allow to plan two different sce-

narios. However, these two scenarios do not challenge

the higher level elements of the HTN, nor the organi-

zation of the LUs.

Thus, as long as there are alternative plannable

decompositions, replanning is limited to the HTN

problem and does not affect the rest of the scenario.

However, if replanning is not possible at the HTN

level (no plannable decompositions with the state of

the world), the HTN can be considered to have failed

because the decomposition can no longer be fully re-

alized. Such a problem leads to a failure of the post-

conditions of the LU (which corresponds to the root

of the HTN). From a pedagogical point of view, this

situation corresponds to the fact that the skill provided

by this LU has not been validated. It is then necessary

to have a set of alternative plans taking into account

the failures on the different LUs.

4.4 Automatic Generation of

Alternatives

Our objective is to generate ofﬂine a set of alternative

plans to the initial plan so that in case of drift during

the execution of the scenario, the system can switch to

another plan already generated, adapted to the current

situation and satisfying the same objectives as the

initial scenario. We have already seen that the gener-

ation of alternatives at the HTN level is quite trivial.

Either there are other plannable decompositions and

in this case there is an alternative plan (the system can

ICAART 2023 - 15th International Conference on Agents and Artiﬁcial Intelligence

470

Algorithm 1: Failure tree generation.

Data: LU list err, d max, err max

Result: error tree

struct (

int: d

LU: pre, potential err[], next[]

) LU

LU : root

root.d ← 0

foreach LU ∈ LU list err do

root.d ← root.d +LU .d

end

root.pre ← NULL

root.potential err ← LU list err

current err ← root

while current err 6= NULL do

if next LU ←

pop(current err.potential err) then

UA : next err

next err.d ← next LU.d

+current err.d

if next err.d <= d max &

LU list err[next LU][0] + 1 <=

err max then

LU list err[next LU][0]+ = 1

current err.next.add(next err)

next err.pre ← current err

current err ← next err

else

current err ← current err.pre

end

else

current err ← current err.pre

end

return root

choose the best adapted alternative decomposition) or

there is not, and therefore the LU during which the

drift occurs is considered as failed (associated skill

not worked).

It is therefore necessary to take this failure into

account and to have alternative plans at the LU level.

However, the problem is not as trivial as for HTN.

Indeed, two elements are likely to generate a combi-

natorial explosion problem when we want to generate

all possible alternatives. Firstly, without additional

constraints, it is theoretically possible to have an in-

deﬁnite number of failures on each LU. Second, since

LUs are relatively unconstrained actions, there is a

large number of potentially plannable permutations.

We must therefore implement a strategy to reduce

the set of possibilities. Regarding the undeﬁned num-

ber of failures, we can consider two constraints. First,

a plan represents a training session. This session is

therefore limited in time. Knowing that each LU is

also characterized by a duration, as soon as we deﬁne

a general time limit for the training session, it can only

contain a limited number of LUs (success or failure).

The second constraint consists in adding a maximum

error limit per LU. It is a heavier constraint but it is

logical in our application framework. Thanks to these

two constraints, it is possible to generate a tree con-

taining all the LU sets (success and failure).

Algorithm 1 shows how to generate such a tree.

Its operation is quite simple. As input we need an

array containing the list of LUs with an integer asso-

ciated to each representing the number of failures, the

maximum duration of the scenario and the maximum

number of errors per LU. To start we initialize the root

which represents the successful LUs. Then for each

possible failed LU we add the duration and increment

the number of errors on this LU in the table. If the

maximum values are not exceeded we add a new node

which becomes the current node. If the constraints

are not respected for all the potential failed LUs we

go back and explore other branches. The tree is thus

generated in depth ﬁrst. The tree thus generated gives

us all the possible error sets and all the arrangements.

It is then necessary to remove the duplicates to obtain

only the unordered failure sets.

The sets generated by this way contain the suc-

cessful and failed LUs but do not order them. It is

therefore necessary to generate the possible permu-

tations of LUs for each of these sets. Once the set

of permutations is generated it is then necessary to

modify our planning domain so that the planner can

determine the plannable permutations.

To simplify the work of the planner we perform

a preprocessing of the permutations to remove those

that do not meet the most trivial conditions. Two types

of permutations are thus eliminated: ﬁrst, permuta-

tions whose ﬁrst LU is different from the initial plan

(success or failure). Alternative plans are used in case

of scenario drift, which can only occur once the exe-

cution of the plan has started, so the ﬁrst LU has no

reason to be different. Then, we can simply delete the

permutations that place one or more successful LUs

before these same failed LUs. Indeed it would be il-

logical to propose again a LU that has already been

successfully processed.

This strategy allows to substantially reduce the

number of permutations that will actually be pro-

cessed by the planner. The generated plans consti-

tute a set of possible organizations for the training

session. The connection with HTNs and the avail-

able alternatives thus offer us the advantages brought

Automated Planning for Military Airline Controller Training Scenarios

471

by the generative power of classical planning and the

control brought by HTNs without being subject to the

drawbacks of these two methods.

5 ALTERNATIVE GENERATION

EXAMPLE

With our method we are able to generate a large num-

ber of potential alternative plans. We would like to

test our system with a simple planning domain to see

the generative power of our system and its limits.

5.1 Domain and Parameters

For this test we use a simple planning domain, only

composed of Learning Units characterized by a dura-

tion and linked to a skill.

With this domain, we generated a plan on which

we can rely to generate potential alternatives. The

purpose of this test is to observe the number of al-

ternative plans generated as a result of the number of

Learning Units in the plan, the maximum duration of

the training session and the maximum number of er-

rors envisaged for each LU.

We thus wish to show that, given the particular

constraints related to the planning of training scenar-

ios, it is possible to use a combinatorial method to

generate ofﬂine all the possible alternatives to the ini-

tial scenario, with a given conﬁguration of the con-

straints.

Indeed, each Learning Unit is characterized by a

duration. This duration concretely represents the time

allocated to training on a given skill. A training ses-

sion is therefore also characterized by a duration cor-

responding to the sum of the durations of the different

Learning Units to which it is necessary to add a mar-

gin to manage potential errors. This time constraint

is quite strong in the context of the generation of pos-

sible alternatives because when an error occurs that

calls into question the success of a LU, it is necessary

to reschedule it later in an alternative plan, and this

implies a new plan consisting of an additional LU.

Given the time limit for the session, the number of

errors is therefore limited. In this context it is use-

ful to also add a constraint on the maximum number

of errors per LU, in order not to favor one skill over

another (unless this is desired, in which case the max-

imum number of errors can be adapted for each LU).

Finally, the initial planning domain also reﬂects these

constraints. Thus, the smaller the margin between the

session duration limit and the optimal duration of the

initial plan, the less possibility there is to generate po-

tential alternatives. On the contrary, the larger the

150

200

250

300

350

400

500

1,000

1,500

2,000

2,500

Duration (min)

Number of Plans

MaxErr = 1

MaxErr = 2

MaxErr = 3

MaxErr = 4

MaxErr = 5

Figure 4: Ofﬂine generation with 3 LU.

200

250

300

350

400

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1.1

1.2

1.3

·10

Duration (min)

Number of Plans

MaxErr = 1

MaxErr = 2

MaxErr = 3

MaxErr = 4

MaxErr = 5

Figure 5: Ofﬂine generation with 4 LU.

margin and the larger the number of potential alter-

native plans, it is then interesting to show the limits

of this approach, when the number of generated plans

becomes too high for the system to calculate them in

an acceptable time.

We performed this test with duration constraints

representative of our application context (military air

trafﬁc controllers training session). Thus, we gener-

ated 3 plans including 3 and 4 Learning Units. These

Learning Units have respective duration ranging from

30 to 70 minutes for training sessions lasting between

100 and 400 minutes. For each plan we generated all

the potential alternatives by varying the duration be-

tween the minimum time to perform the session (ad-

dition of the LU duration) and the maximum duration

of the training session (400 minutes). We also varied

the maximum number of errors allowed for each LU

between 1 and 5.

ICAART 2023 - 15th International Conference on Agents and Artiﬁcial Intelligence

472

5.2 Results and Analysis

We generated alternatives to the 3 initial plans by

varying the duration with a step of 50 minutes. For

each of these durations we varied the maximum num-

ber of errors for each LU. What can be observed ﬁrst,

for the 2 different cases, is that the number of gener-

ated plans remains quite low for Duration < 300 with

a number of generated plans lower than 500 what-

ever the initial plan and the number of allowed er-

rors. The number of generated alternatives increases

sharply between 300 and 400 minutes, but the con-

straints still allow the plans to be generated with-

out problems. We also note that with such duration

constraints, the number of generated plans varies lit-

tle for MaxErr > 3. Although the computations run

smoothly for the duration interval [100−400], there is

a large variation in the number of generated plans de-

pending on the length of the initial plan which varies

from a few hundred for an initial plan composed of

3 LU to more than 10000 for an initial plan com-

posed of 4 LU. These results therefore show that our

method works for generating alternatives to the ini-

tial plan. The number of generated plans is high and

offers many possibilities in order to select the most

suitable alternative plan for the encountered situation.

The maximum duration used for our tests is 400 min-

utes. However, this time is only reached when several

errors are made and the initial plan, if it is carried out

without errors, is much faster. It is therefore neces-

sary to think carefully about the skills that the trainer

wants to work on and the parameters of the limits ac-

cording to the type of session that the trainer wants to

set up. Thus, thanks to our system, it is quite possi-

ble to propose a rather short session but leaving the

possibility of many attempts. This allows the learner

to take more time, or, in case of quick success, to

start a new training session. Alternatively, our system

also allows for longer sessions, with more skills be-

ing worked on, but with less opportunity for mistakes

to be made. These possibilities of adaptation make it

possible to offer training sessions to different learner

proﬁles such as beginners or experienced operators.

6 CONCLUSIONS

In this paper we have presented an original solution

to design and plan training scenarios. Hybrid plan-

ning and the use of ANML allow us to take ad-

vantage of both the generative power of planning

and the control provided by HTNs. This structure

also greatly improves the expressiveness of our plans

thanks to the separation of scenario and pedagogical

content. Finally, this method improves the robust-

ness of our scenario and its resilience with respect to

its objectives thanks to an efﬁcient generation of alter-

native plans allowing to anticipate the potential drifts

of the scenario and to repair them. Finally, these el-

ements were integrated into an interface allowing the

trainer to follow the scenario execution live and to ap-

ply the generated alternatives in case of drift. In or-

der to complete the system, we will study in our next

work the strategies of qualitative selection of the gen-

erated alternatives so that the repair of the scenario

corresponds as well as possible to the initial objec-

tives and the preferences of the trainers. This work

will be accompanied by the development of the moni-

toring system to support the execution of the scenario,

to detect drift situations and to implement the alterna-

tives.

ACKNOWLEDGEMENTS

This work is ﬁnanced by the DGA RAPID ORCHES-

TRAA project.

REFERENCES

Aylett, R., Dias, J., and Paiva, A. (2006). An affectively

driven planner for synthetic characters. In Icaps,

pages 2–10.

Gupta, S. K., Anand, D. K., Brough, J. E., Schwartz, M.,

and Kavetsky, R. (2008). Training in Virtual Envi-

ronments: A Safe, Cost-Effective, and Engaging Ap-

proach to Training. University of Maryland.

Howe, A., Knoblock, C., McDermott, I. D., Ram, A.,

Veloso, M., Weld, D., SRI, D. W., Barrett, A., Chris-

tianson, D., et al. (1998). Pddl— the planning domain

deﬁnition language. Technical Report, Tech. Rep.

Porteous, J. (2016). Planning technologies for interactive

storytelling. In Handbook of Digital Games and En-

tertainment Technologies. Springer.

Porteous, J., Teutenberg, J., Charles, F., and Cavazza, M.

(2011). Controlling narrative time in interactive story-

telling. In The 10th International Conference on Au-

tonomous Agents and Multiagent Systems-Volume 2,

pages 449–456.

Ramirez, A. and Bulitko, V. (2014). Automated planning

and player modeling for interactive storytelling. IEEE

Transactions on Computational Intelligence and AI in

Games, 7(4):375–386.

Riedl, M., Saretto, C. J., and Young, R. M. (2003). Manag-

ing interaction between users and agents in a multi-

agent storytelling environment. In Proceedings of

the 2nd international joint conference on Autonomous

agents and multiagent systems, pages 741–748.

Automated Planning for Military Airline Controller Training Scenarios

473

Riedl, M. O. and Bulitko, V. (2013). Interactive narra-

tive: An intelligent systems approach. Ai Magazine,

34(1):67–67.

Riedl, M. O., Stern, A., Dini, D., and Alderman, J. (2008).

Dynamic experience management in virtual worlds

for entertainment, education, and training. Interna-

tional Transactions on Systems Science and Applica-

tions, Special Issue on Agent Based Systems for Hu-

man Learning, 4(2):23–42.

Smith, D. E., Frank, J., and Cushing, W. (2008). The anml

language. In The ICAPS-08 Workshop on Knowledge

Engineering for Planning and Scheduling (KEPS),

volume 31.

ICAART 2023 - 15th International Conference on Agents and Artiﬁcial Intelligence

474