Towards Simulating Heterogeneous Drivers with Cognitive Agents

Arman Noroozian, Koen V. Hindriks and Catholijn M. Jonker

Interactive Intelligence, Delft University of Technology, Delft, The Netherlands

Keywords:

Simulation, Heterogeneity, Mixed Trafﬁc, Cognitive Driver Agent, Scalability.

Abstract:

Every driver behaves differently in trafﬁc. However, when it comes to micro-simulation of drivers with a high

level of detail no framework manages to model the complexities of various driving styles as well as scale

up to larger simulations. We propose a framework of micro-simulation combined with cognitive agents to

facilitate such simulation tasks. Our goal is to (i) model individual drivers, and (ii) use this framework for the

purpose of simulating realistic highway trafﬁc with heterogeneous driving styles. The challenge is therefore to

create a framework that facilitates such complex modeling and supports large scale simulations. We evaluate

the framework from two perspectives. First, the ability to represent, model and simulate dissimilar drivers in

addition to study and compare emerging behavior. Second, the scalability of the framework. We report on our

experiences with the framework, outline several challenges and identify future areas for development.

1 INTRODUCTION

Drivers behave differently as recent studies con-

ﬁrm (Ossen and Hoogendoorn, 2011; Hoogendoorn

et al., 2010). While trafﬁc micro-simulation allows us

to study driving behavior in recreated or hypothetical

scenarios in a cost effective way, currently no trafﬁc

simulation framework manages to model such vari-

ations in behavior as well as scale. The ability to

model such varying behavior is of importance because

the particular models that are used impact the expla-

nation/prediction strength of the simulation tools as

demonstrated by Ossen and Hoogendoorn.

In micro-simulation driving behavior is described

in terms of leading vehicle behavior with longitudi-

nal (car-following) models. Some established car-

following models include IDM (Treiber et al., 2000)

and Gipps (Barcel

o, 2010, Chap. 8) which respec-

tively describe driver acceleration and speed choices

as a function of attributes relating to the leading ve-

hicle (i.e. distance, relative speed) and a number of

driver related attributes (i.e. desired speed and gap).

Lane-change models extend car-following by describ-

ing behavior in the lateral direction (i.e. changing

lanes) based on the impact on vehicles affected by the

movements. Examples include LMRS (Schakel et al.,

2012) and MOBIL (Kesting et al., 2007). Such micro-

models are characterized by parameters that require

calibration against ground truth data since drivers be-

have differently in different contexts (i.e. per country,

time of day).

Calibration is a complex and time consuming pro-

cess which requires minimization of the measured er-

ror between simulated results and ground truth data

through search algorithms like gradient descent, sim-

ulated annealing, genetic algorithms or probabilistic

methods like Kalman ﬁlters (see Treiber and Kesting

(2013, Chap. 16) for an overview of the various meth-

ods). The assumption here is that calibrated parame-

ters apply to all drivers. That is, the heterogeneity

of drivers is not considered in this process. Hetero-

geneity is introduced by adding randomness a poste-

riori. This leads to limitations in explanation/predic-

tion power and the generalization of models to new

trafﬁc contexts (i.e. different highway) or may lead to

observations of unrealistic behavior. For instance in

one empirical study Ossen and Hoogendoorn (2011)

demonstrate that with calibration a model like IDM is

able to model the behavior of only 19% of drivers due

to heterogeneity.

Our goal is to model and study heterogeneous

driving styles and measure the effects of heterogene-

ity in a structured and controllable manner. In par-

ticular we are interested in studying trafﬁc scenar-

ios with a mix of autonomous vehicles and human

drivers. For this purpose we need a framework that

goes beyond the current models that allow for large

numbers of homogenous drivers and at the same time

extends on classic cognitive models (e.g. ACT-R) that

model single drivers with high detail. We require cog-

nitive modeling abilities because we want to study the

impact of cognitive phenomena (i.e. cognitive work-

147

Noroozian A., V. Hindriks K. and M. Jonker C..

Towards Simulating Heterogeneous Drivers with Cognitive Agents.

DOI: 10.5220/0004815601470155

In Proceedings of the 6th International Conference on Agents and Artiﬁcial Intelligence (ICAART-2014), pages 147-155

ISBN: 978-989-758-016-1

 2014 SCITEPRESS (Science and Technology Publications, Lda.)

load). Current simulation frameworks fall short here

because of their tight integration with speciﬁc car-

following and lane change models. This limitation not

only applies over the driver population but also over

time as driving style can also vary with time. Model-

ing such heterogeneity is not supported in current ap-

proaches and cognitive frameworks like ACT-R that

have been used so far for micro-level modeling of in-

dividual drivers do not scale enough for our purposes.

To address this issue we have built a simulation

framework that combines cognitive agent modeling

with micro-simulation of trafﬁc. Agents developed by

Agent Programming Languages (APLs) are assigned

to control simulation entities thereby removing tight

integration between model and simulator. The idea is

to provide an API similar to ones used for program-

ming AI in computer games (see Bartish and The-

vathayan (2002); Hindriks et al. (2011)). The abil-

ity to communicate and reason based on various per-

cepts such as brake lights, turn indicators, horns and

the ability to predict ahead of time are added beneﬁts

of using such a framework and desirable for modeling

complex driver behavior (Treiber and Kesting, 2013,

Chap. 12).

We study and illustrate how heterogeneity in lon-

gitudinal driving style affects macroscopic and mi-

croscopic characteristics of trafﬁc using this frame-

work. Note that we limit our models to longitudi-

nal because we are mainly interested in evaluating

our framework rather than the particular driver mod-

els that we use in doing so. This limitation will not

affect our evaluation and we plan to extend this study

with lane-change models in the future. We evaluate

our framework from two perspectives. First, the abil-

ity to model and compare dissimilar driving styles.

Our hypothesis here is that with sufﬁcient heterogene-

ity in longitudinal driving behavior, small effects on

macroscopic characteristics of trafﬁc (i.e. ﬂow and

speed) are observable; at the same time signiﬁcant ef-

fects on microscopic characteristics (i.e. gap distribu-

tion) should be observed. Second, we evaluate scala-

bility in terms of required resources for an increasing

number of agents. The expectation here is to see a

linear loss of performance with an increasing number

of agents. We require the ability to study small scale

trafﬁc scenarios such as merging at on-ramps, there-

fore we need hundreds of simulated vehicles but not

thousands.

The remainder of this paper is structured as fol-

lows: In Section 2 we present an overview of the re-

lated work. In Section 3 we present our proposed cog-

nitive framework and its evaluation in Section 4. In

Section 5 we outline some of the challenges and dis-

cuss future work. We conclude in Section 6.

2 RELATED WORK

High ﬁdelity trafﬁc simulation tools are based on

incorporating some form of agent-based simulation

with micro-simulation. Some instances are based on

ad-hoc reactive agents that are computationally light

weight but lack communication (Smith et al., 1995;

Ehlert and Rothkrantz, 2001). Because of their sim-

plicity these scale to larger simulations, however, their

driver models do not consider heterogeneity of driv-

ing style. This is likely due to a trade-off between

scalability and realism. Some instances use struc-

tured cognitive agents capable of reasoning however,

the focus has largely been on the strategic aspects

of commuting, i.e. planning activities, when/how to

travel, which routes to take and which transportation

modes to use (Rossetti et al., 2002; Rindsf

user, 2005)

rather than the detailed control of a vehicle. Other in-

stances use cognitive agents to make maneuvering de-

cisions like the decision to overtake other vehicles or

merge (Hidas, 2002; Sukthankar et al., 1998). In (Hi-

das, 2002) the focus is on developing a driver model

for lane changing behavior, however heterogeneity is

not considered. With respect to (Sukthankar et al.,

1998) the focus is on learning to drive and scalability

is not a main concern. For a survey on applications of

agent-based simulation in the trafﬁc domain we refer

the reader to (Cheng, 2010).

Multiple trafﬁc micro-simulation tools have been

developed that provide various levels of programming

control over entities in the simulation. Examples in-

clude DRACULA (Barcel

o, 2010, Chap. 8) , TRAN-

SIMS (Nagel and Rickert, 2001), SUMO (Krajzewicz

and Hertkorn, 2002) and MOTUS (Schakel et al.,

2012)

. Such tools use hard-coded car-following

models and provide application programming inter-

faces (APIs) that are geared towards studying differ-

ent aspects of trafﬁc (i.e. supply and demand or in-

frastructure control). The micro-models are an inte-

gral part of these tools and modifying them involves

serious restructuring of the code. A common factor in

these tools is that each transportation mode (i.e. pas-

senger car, truck) is assumed to behave according to

a single calibrated model of behavior. That is, a sin-

gle car-following model is used per mode of trans-

port and its calibrated parameters represent the entire

population of drivers. This tight integration makes it

difﬁcult to mix and compare the various driver mod-

els that have been proposed and studied using a single

framework.

Our approach uses BDI (Belief-Desire-Intention)

agents to model individual drivers and their driving

For a more comprehensive list of simulators refer

to (Barcel

o, 2010).

ICAART2014-InternationalConferenceonAgentsandArtificialIntelligence

148

styles. BDI agents are based on intentional stance

theory which simpliﬁes the causes for actions to de-

sires (Pokahr et al., 2005). They provide a useful

framework for modeling driving at all levels of deci-

sion making required for the task and can potentially

incorporate additional ranges of factors inﬂuencing

driver behavior (Bazzan et al., 1999). By decoupling

the driver models from the simulator we are able to

provide a ﬂexible modeling approach that supports

more complex encoding of behavior and facilitates the

use and comparison of dissimilar driving styles.

Agent-based simulations inherently involve a

trade-off between the encoded detail in the agents

and number of agents that can be simulated (Navarro

et al., 2011). One of the main challenges here is that

the scalability of the approach diminishes as the level

of detail encoded in the models increases from the

strategic to control level. To the best of our knowl-

edge, limited attention has been payed to the scalabil-

ity of BDI agents. For example Wolfe et al. (2008)

investigate the use of BDI agents in air-trafﬁc ﬂow

management and conclude that an overuse of the BDI

paradigm results in scalability limitations and draw

guidelines for the use of the BDI agents.

Other related work investigates the scalability of

the more general concept of Multi-Agent Systems

(MAS). Lee et al. (1998) provide deﬁnitions for

the performance and scalability in MAS. Important

performance indicators include throughput, response

time, number of concurrent agents and communica-

tion overhead. Scalability is deﬁned as the degree to

which performance degrades as a result of expand-

ing the number of agents. In addition the topology

of the MAS and the rate of agent arrival/departure

(churn) is also identiﬁed as an important scalability

factor (Lee et al., 1998; Turner and Jennings, 2001).

Navarro et al. (2011) propose a hybrid approach for

dynamic adjustment of the level of detail in the sim-

ulation and conclude that the impact on dissimilarity

between a full scale simulation and a simulation that

smartly reduces the level of detail depending on the

context is minimal.

Alternative cognitive modeling approaches in-

clude frameworks such as ACT-R (Salvucci, 2006)

and Soar (Langley et al., 2009). In (Salvucci, 2006)

ACT-R has been used to simulate the maneuvering of

a single driver. Soar has been used for simulating a

ﬁghter pilot in (Langley et al., 2009). One of the main

differences between BDI and cognitive frameworks is

that the latter take biological human constraints into

account to create more plausible human models (i.e.

short/long term memory, learning abilities). The inte-

gration of these constraints is motivated by modeling

the human reasoning process, its sources of knowl-

Figure 1: Framework architecture.

edge and the way they are used to reason. Our ap-

proach is focused on general reasoning that does not

necessarily reproduce the exact human reasoning pro-

cess. In terms of scalability cognitive frameworks are

not well suited for large scale simulation. In fact the

scalability of such cognitive frameworks is still an

open problem (Langley et al., 2009). Scalability is

also a concern with BDI agents however to a lesser

extent. BDI agents do not have to deal with complex

characteristics such as short/long term memory which

improves their execution time and hence the scalabil-

ity of the framework as a whole.

3 TRAFFIC SIMULATION

FRAMEWORK

Our review of the related work motivates the need for

a trafﬁc simulation framework based on a ﬂexible and

scalable formalism to model driver heterogeneity. In

this section we present our proposed framework and

its components: jSim, the GOAL agent programming

framework and the Environment Interface connecting

the former two. Figure 1 demonstrates the high level

architecture of our framework.

Our microscopic trafﬁc simulator jSim is a time-

discrete, space-continuous, open-source Java-based

simulator. It is an extended version of MOTUS

(Schakel et al., 2012) which has been modiﬁed to pro-

vide necessary percepts and actions to agents for en-

coding driver behavior.

jSim operates with a main loop where each itera-

tion represents the passage of time by one time step

(virtual time). During each time step agents that con-

trol entities within the simulation get the opportunity

to perceive changes since the previous time step and

perform actions within the current time step. The sim-

ulation loop consists of 4 phases: (i) updating the state

of the world, (ii) waiting for agents to act, (iii) exe-

cuting the agent actions and (iv) gathering statistics.

Note that the rendezvous point in phase ii is a design

decision in order to ensure fairness; that all agents get

the chance to perform an action in each cycle. This re-

TowardsSimulatingHeterogeneousDriverswithCognitiveAgents

149

--------

-----------------------

--------------------------

--------

Figure 2: jSim overview and components.

sults in a pseudo-parallel update step in which the up-

dated world state is independent of the order in which

agent actions are executed.

Some simulation entities are controlled by agents

while others may be passive. For example detectors

simply count the number of vehicles passing over a

speciﬁc point of the road network. Controllable enti-

ties include drivers controlling the vehicles, on board

units and road side units. The agents controlling them

go through a observe-decide-act cycle. Observation

involves perceiving relevant information from the en-

vironment while decision making involves executing

the agent’s code which in turn will execute some ac-

tions within the environment. Note that due to space

limitation we only give an overview of the driver spe-

ciﬁc functionality relevant to longitudinal behavior.

A driver agent can perceive time, speed, acceleration

and current lane. It can also perceive surrounding ve-

hicles within a boxed area, the gap and speed differ-

ence with a leading vehicle (if any). Relevant actions

available to a driver are adjusting the vehicle accel-

eration/deceleration. Vehicle states are updated ac-

cording to kinematic equations of motion once. Fig-

ure ?? gives and overview of the components within

jSim. We have integrated components for importing

road networks.

The second component of our framework is the

GOAL agent programming platform. GOAL agents

encode driver behavior using the GOAL programming

language (Hindriks et al., 2001). GOAL is a plat-

form for implementing Belief-Desire-Intention (BDI)

agents. They have the ability to perceive their envi-

ronment, reason about its state, adopt or drop goals

and decide on what actions to perform.

GOAL is a rule-based logic programming lan-

guage that uses Prolog for representing the beliefs and

goals of an agent. Rules consist of a condition on the

agent’s mental state and are used for deciding what to

do next. Using these rules, an agent derives its choice

of action from its beliefs and goals. There is a one-

to-one mapping between GOAL agents and driver en-

tities in the simulator.

Our motivation for using BDI agents and GOAL in

particular is threefold. First, encoding existing driver

models as BDI agents is natural as such models al-

ready use cognitive notions such as desired speed/gap

which naturally ﬁt the BDI paradigm. Second, BDI

agents support cognitive modeling and allow complex

driver models which use reasoning as part of their de-

liberation. Third, GOAL agents do not have to deal

with the complexities of short/long term memory that

other cognitive frameworks like ACT-R do and there-

fore could potentially produce more scalable agents.

As an independent motivation we also ﬁnd the traf-

ﬁc domain challenging for improving GOAL and BDI

agents in general.

Note that GOAL supports programming reactive

as well as proactive, cognitive agents. Existing driver

models that consist of a single formula can be im-

plemented as agents that simply react to changes

in the environment. Others can be implemented as

cognitive agents with rule based behavior that inte-

grates declarative goals into the deliberation process.

This type of ﬂexibility has been demonstrated in real

time (Hindriks et al., 2011) and non-real time environ-

ments (Dekker et al., 2012) and facilitates an array of

complex driving styles to be simulated.

The ﬁnal component of our framework is the En-

vironment Interface. This interface is based on the

Environment Interface Standard (EIS) (Behrens et al.,

2010). It acts as a middle-ware between the environ-

ment and the agent. This middle-ware exposes re-

quired functionality for implementing agents and is

similar to the API exposed in computer games for im-

plementing character speciﬁc AI. In addition this in-

terface also allows other APLs to be used in combina-

tion with jSim. Other platforms that provide support

for EIS include JIAC and AGENT FACTORY.

EIS is Java based and provides functionality for

encapsulating and launching virtual environments in

addition to perceiving changes and executing actions

therein. It also facilitates the coupling/decoupling of

entities and their controlling agents. This is achieved

through methods that are used to dynamically couple

and decouple agents with entities. This dynamism not

only facilitates the use of mixed agent types for con-

trolling drivers but also to change and adapt a single

driver’s behavior during a simulation.

4 FRAMEWORK EVALUATION

In this section we are interested in evaluating our

framework from two perspectives. First, the ability to

implement, mix and compare different driver models

as a method for modeling heterogeneity. Second, the

scalability of the framework in terms of the number

of agents in the simulation.

ICAART2014-InternationalConferenceonAgentsandArtificialIntelligence

150

4.1 Mixing Driver Models

Our framework provides a controllable level of het-

erogeneity with the various types of driving styles that

can be used in combination with the frequency by

which they occur in the population of our simulated

drivers. We showcase several experiments and high-

light how our framework facilitates the implementa-

tion, use and comparison of mixed driver models and

their emerging behavior.

Our experiments involve two longitudinal driver

models: (i) IDM (Treiber et al., 2000) and (ii) a sim-

ple cognitive agent (CA) implemented by ourselves.

Note that the framework facilitates the use of any

number/type of model to create heterogeneity and

is not limited to the two types of agents which we

demonstrate here.

We use the calibrated values from (Schakel et al.,

2012) for our IDM agents. The agent implementa-

tion contains a single action rule that uses the model

formula to generate acceleration/deceleration actions.

We can simply replace this single action rule with a

different formula to implement any other longitudinal

driver model without having to rebuild our tool.

Our cognitive agent (CA) is more involved than

the IDM agent. Parts of its implementation are

presented in Figure 3. In contrast to the IDM

agent, the CA agent makes use of cognitive con-

structs (i.e. goals, beliefs) to encode driving behav-

ior. Note the use of such constructs in the agent’s

action rules (Line 7-8). The modules speciﬁed on

lines 11 and 27 constitute the agent’s deliberation pro-

cess. The CA agent implements an aggressive style

of driving. That is to say that it will constantly try

to keep a time gap of 2 seconds with the leading ve-

hicle (if any) and that it uses a considerable amount

of acceleration/deceleration in doing so. This consti-

tutes the only parameter of our CA agent (2 second

headway). Figure 3 demonstrates the use of the BDI

paradigm in encoding a particular driving style. BDI

provides a useful and natural formalism for modeling

drivers (Bazzan et al., 1999). While we do not make

use of all available perceptual input to implement our

CA agent, more complex agents can be implemented

using the perceptual information (see Section 3 for

more information on the available perceptual data).

Differences between the CA and IDM agent’s longitu-

dinal driving style are demonstrated in Figure 4. The

plotted acceleration values have been gathered during

the simulation scenario depicted in Figure 5. The CA

agent aggressively ﬁlls the gaps while it avoids getting

dangerously close to its leading vehicle. A disconti-

nuity in the acceleration of the CA agent occurs at the

point where the 2 second time gap is violated and the

1 main module{

2 knowledge {

3 getting too close :− time gap( TG ), min timegap( MTG ), TG =< MTG.

4 min timegap( 2.0 ).

5 }

6 program [order=linear] {

7 if a−goal( speed( GS ) ), bel( speed( CS ), GS > CS, max acceleration(

MA ), A is MA ∗ (1 − (CS/GS)∗∗5)) then accelerate( A ).

8 if a−goal( speed( GS ) ), bel( speed( CS ), GS < CS,

max deceleration(MD), D is MD ∗ (1 − (CS/GS)∗∗5)) then

decelerate( D ).

9 if bel( true ) then skip.

10 } }

11 event module {

12 program {

13 % Update Goals based on new beliefs (Drop and Adopt).

14 %−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−

15 % Always reconsider current speed.

16 if goal( speed( Y ) ) then drop( speed( Y ) ) .

17 % Reasons for dropping speed synchronization goal:

18 % 1st: Large gap in front ; I am aggressive, close the gap.

19 if bel( time gap( TG ), min timegap( MTG ), TG >= 2 ∗ MTG) then

drop( same speed ).

20 % 2nd: no car in front anymore.

21 if bel( not( blocked(me) ) ) then drop( same speed ).

22 % Reconsider to see if we need to adopt new goals

23 % If getting too close we maintain same speed as car in front.

24 if bel( getting too close ) then adopt( same speed ).

25 if true then setSpeedGoal.

26 } }

27 module setSpeedGoal {

28 program [order=linear] {

29 % either there is a gap to ﬁll or not. set speed goal to lane speed

limit .

30 if not( a−goal( same speed ) ), bel( lane speed limit( SL ) ) then

adopt( speed( SL ) ).

31 % we’re getting too close, adjust and compute lower speed.

32 if a−goal( same speed ), bel( speed( S ), speed delta( DS ), X is S −

DS) then adopt( speed( X ) ).

33 } }

Figure 3: Cognitive driver agent code.

4 6 8 10 12 14 16 18 20

−20

−15

−10

−5

Time (S)

Acceleration CA (m/s

)

4 6 8 10 12 14 16 18 20

−20

−15

−10

−5

Acceleration IDM (m/s

)

IDM

Figure 4: Agent behavior for gap ﬁlling and speed synchro-

nization for scenario in Figure 5.

agent decelerates to stop further violation. The IDM

agent chooses its accelerations smoothly based on the

IDM formula (Schakel et al., 2012).

Given these agents we study the effects of hetero-

geneity on macroscopic and microscopic characteris-

tics of trafﬁc. The following three scenarios are sim-

ulated in our experiments: (i) All drivers use the IDM

model, (ii) all use the CA model (iii) divers use the

IDM or CA model with 50% chance.

We simulate a hypothetical 500 meter highway

section. The density of vehicles on each lane varies.

Density is controlled by the gap at which vehicles

are inserted on the lane. Vehicles enter with a speed

TowardsSimulatingHeterogeneousDriverswithCognitiveAgents

151

Figure 5: Blocked drivers scenario. Initial time gap of 4

seconds between agents and blocking cars

0 10 20 30 40 50 60 70 80

Density (Veh/Km)

Flow (Veh/ 5 sec.)

0 10 20 30 40 50 60 70 80

Speed (m/s)

IDM

50−50 Mix

Figure 6: Comparison of Fundamental Diagrams for Cog-

nitive Agent (CA), IDM and 50-50% mix.

that gives them enough time to come to a full stop if

the leading vehicle breaks at maximum deceleration.

Note that vehicles are restricted by and do not change

lane since we are comparing longitudinal driver be-

havior.

Our ﬁrst experiment demonstrates driver behav-

ior in trafﬁc ﬂow situations before and leading to a

congested trafﬁc state. For this experiment we com-

pare the emerging macro level ﬂow, density and speed

properties of the trafﬁc for the 3 scenarios outlined

above. The relations between these properties are

macroscopic features and are referred to as the fun-

damental diagrams (i.e. ﬂow and speed in relation to

density) (Treiber and Kesting, 2013, Chap. 4). Fig-

ure 6 illustrates the emerging fundamental diagrams.

A detector positioned at 10% of the segment length

was used to measure the ﬂow and speed of vehicles.

These measurements show the effects of heterogene-

ity on macroscopic features of the trafﬁc.

Due to their aggressive driving style CA agents

achieve the highest ﬂow rates (Fig. 6). Aggressive-

ness does not have the same effect on ﬂow when het-

erogeneous trafﬁc is considered. It is the result of CA

agents being restricted by IDM agents. Effectively,

CA agents have a limited maneuvering space between

the lead and following vehicle. Note however, that the

effect of heterogeneous drivers (mix) is more visible

in the speed measurements. This is due to the CA

agents using more acceleration (and higher speeds)

to ﬁll in gaps thereby lifting the average. Note that

this effect is more pronounced in mid range densities

where the differences in driving style are more likely

0 10 20 30 40 50 60 70 80

500

1,000

1,500

2,000

2,500

3,000

3,500

12,000

Gap (m)

Times Observed

IDM

50−50 Mix

Figure 7: Histogram of observed vehicle gaps for cognitive

agent (CA), IDM and 50-50% mix of CA/IDM.

to play a signiﬁcant role.

Our second experiment demonstrates driver be-

havior in congested trafﬁc. Here we repeat the previ-

ous experiment with the addition of one slow moving

vehicle per lane to block the movement of all follow-

ing vehicles and produce artiﬁcially congested traf-

ﬁc. For this experiment we demonstrate the effect of

heterogeneity on the distribution of gaps between the

vehicles in the bottom denser lanes. This is a micro-

scopic characteristic of the trafﬁc which is plotted in

Figure 7 as histograms. The ﬁgure demonstrates that

CA agents maintain shorter gaps from their leading

vehicles. For the IDM agents the gaps are generally

larger. With heterogeneous driving styles the gaps be-

come relatively smaller in comparison to IDM and the

distribution is skewed towards smaller gaps because

of the CA agent driving style.

These two experiments conﬁrm our hypothesis

with respect to the small effect of heterogeneity on

macroscopic and a larger effect on microscopic char-

acteristics of trafﬁc (see Section 1) and demonstrate

how our framework facilitates the implementation,

use and mixing of multiple driver models as a method

for modeling heterogeneity.

4.2 Scalability

In addition to the use, mixing and comparison of the

diver models we have also evaluated the scalability of

our framework. Our scalability requirements are de-

rived from our goal to study driver behavior in small

sections of highways. Such small sections may con-

tain drivers in the order of 100s. Note that all ﬁgures

and results relating to the scalability of the framework

have been obtained on an Intel i7 processor chip with

4GBs of available RAM.

Figure 8 plots the average execution time of an

iteration of the simulation loop. We measure two

quantities. The ﬁrst is the agent’s reaction window

which is the amount of time the simulation waits for

all agents to perform an action. The second is the du-

ICAART2014-InternationalConferenceonAgentsandArtificialIntelligence

152

20 40 60 80 100 120 140 160

100

200

300

400

500

600

650

Number of Agents

Time (ms)

Simulation Loop Duration

Agents Reaction Windows

f(x) = 3.22x + 56.37 (R

= 0.96)

f(x) = 3.30x + 64.35 (R

= 0.94)

Figure 8: Scalability of Framework.

ration of a single iteration of the main loop. Note that

the execution of our agent codes is the main bottle-

neck in the framework. This is not surprising since

we have executed agent codes sequentially and not

in parallel. We observe a linearly increasing trend in

the execution time and therefore a linearly decreasing

trend in the framework scalability as expected.

The step size in the measurements depicted in this

ﬁgure is equal to 25 milliseconds. Given this time

step a simulation of 10 seconds with 100 agents, re-

quires approximately 156 seconds to execute. Dou-

bling the step size to 50 ms results in a simulation

of the same duration to require 78 seconds. Us-

ing the linear approximation trend line depicted in

the ﬁgure we can draw the following relation: d

∆t

× (3.3a + 64.35) × 10

−3

, where d

is the real time

duration of the simulation in seconds, d

is the dura-

tion of simulation that we want to run in seconds, ∆t is

the simulation step size and a is the number of agents.

This property is known as the real time ratio of the

simulation. This quantity is of practical importance

because it indicates the time required to run a simula-

tion which also impacts the calibration process.

In order to put our scalability results in perspec-

tive we also report on performance results from the

literature. Bartish and Thevathayan (2002) report on

using BDI agents for controlling game characters and

arrive at comparable linear scalability results. Wolfe

et al. (2008) report on a framework for air-trafﬁc con-

trol using BDI agents. The authors identify the exe-

cution properties of the underlying BDI agent imple-

mentation as one of the important sources for the in-

efﬁciency of their framework. Salvucci (2006) report

on simulating a single driver using the ACT-R cogni-

tive framework. A comprehensive analysis of the real-

time ratio of the parallel TRANSIMS trafﬁc simulator

is also reported in (Nagel and Rickert, 2001). Real-

time ratios between 0.25 and 0.007 (faster than real-

time) have been reported for simulating urban trafﬁc

of the city of Portland.

Whether agent based simulation frameworks are

considered scalable depends on the requirements. In

light of our scalability requirement we can conclude

that the framework performance is acceptable. How-

ever, parallelism seems necessary when considering

that calibration of driver models requires multiple

rounds of simulation per parameter. We believe that

the scalability of MAS and BDI agents in particular

are important open questions. It is important to report

trends rather than qualitative results to enable detailed

analysis and comparison of frameworks.

5 CHALLENGES AND FUTURE

WORK

An important challenge that we have brieﬂy touched

upon is the calibration of heterogeneous models. Be-

cause of the larger number of parameters that need to

be calibrated with heterogeneous models, the process

is more time consuming than the methods described

in Section 1. More efﬁcient calibration methods are

required. One promising area for further research is

modeling driver behavior through learning from avail-

able data. Similar ideas have been explored in (Suk-

thankar et al., 1998) in which learning has been used

for tactical level driving. The advantage here is that

learning can be done ofﬂine to create archetypal driv-

ing styles (i.e. aggressive vs safe driver). However, it

is not yet clear how realistic such models are in com-

parison to control theoretic models and how they can

be used for studying real trafﬁc.

Furthermore, scalability of cognitive agents still

remains a challenge (Langley et al., 2009). An as-

pect of our future work is to use GOAL’s ability to

run distributed simulations to achieve better scaling.

Another important aspect with respect to BDI agents

is to improve the efﬁciency of the reasoning process

of agents. Solutions like dynamic adjustment of the

simulation ﬁdelity proposed in (Navarro et al., 2011)

also present promising research.

We are planning to extend this study with driver

behavior in both longitudinal and lateral directions.

We also plan to use the agent communication abilities

to research distributed coordination and car platoon-

ing which impact road throughput and safety.

6 CONCLUSIONS

We have outlined our initial steps towards designing

a cognitive framework for modeling heterogeneous

driving styles using BDI agents and micro-simulation

of trafﬁc. The goal of the framework is to facilitate a

TowardsSimulatingHeterogeneousDriverswithCognitiveAgents

153

heterogeneous modeling paradigm and to implement

and compare different driver models in highway traf-

ﬁc. We have evaluated our framework from two per-

spectives: (i) Its ability to meet this goal, (ii) the scala-

bility of the framework. Our evaluation demonstrates

that the our framework is quite ﬂexible in modeling

complex heterogeneous driving behavior. The main

advantage of our framework is its ability to mix mul-

tiple driver models rather than using a single model

for an entire population of drivers as is done in cur-

rent trafﬁc simulation frameworks. A second advan-

tage of our proposed framework is that in compari-

son with solutions like micro-simulation with reactive

agents, the framework’s use of BDI agents makes it

better suited for modeling the cognitive complexities

of driving behavior. While less scalable than reac-

tive agents, our BDI approach easily scales to 100s

of agents which makes them more scalable than other

cognitive frameworks such as SOAR and ACT-R. In

terms of scalability the framework shows promising

results towards our requirements. The scalability of

agents and their improvement remains an important

avenue of research.

REFERENCES

Barcel

o, J., editor (2010). Fundamentals of Trafﬁc Simula-

tion, volume 145 of Operations Research & Manage-

ment Science. Springer.

Bartish, A. and Thevathayan, C. (2002). BDI agents for

game development. In Proc. AAMAS ’02, part 2, page

668. ACM.

Bazzan, A. L. C., Wahle, J., and Kl

ugl, F. (1999). Agents in

trafﬁc modelling - from reactive to social behaviour.

KI-99: Advances in Artiﬁcial Intelligence, pages 303–

306.

Behrens, T. M., Hindriks, K. V., and Dix, J. (2010). To-

wards an environment interface standard for agent

platforms. Annals of Mathematics and Artiﬁcial In-

telligence, 61(4):261–295.

Cheng, H. H. (2010). A Review of the Applications of

Agent Technology in Trafﬁc and Transportation Sys-

tems. IEEE Transactions on Intelligent Transporta-

tion Systems, 11(2):485–497.

Dekker, M., Hameete, P., and Hegemans, M. (2012). Hac-

tarV2: an agent team strategy based on implicit coor-

dination. Programming Multi-Agent Systems , LNCS,

7217:173–184.

Ehlert, P. and Rothkrantz, L. (2001). Microscopic trafﬁc

simulation with reactive driving agents. In ITSC. 2001

IEEE Intelligent Transportation Systems., pages 860–

865. IEEE.

Hidas, P. (2002). Modelling lane changing and merging

in microscopic trafﬁc simulation. Transportation Re-

search Part C: Emerging Technologies, 10(5-6):351–

371.

Hindriks, K. V., de Boer, F. S., van der Hoek, W., and

Meyer, J.-J. C. (2001). Agent programming with

declarative goals. Intelligent Agents VII Agent The-

ories Architectures and Languages, LNCS, 1986:228–

243.

Hindriks, K. V., van Riemsdijk, B., Behrens, T., Korstanje,

R., Kraayenbrink, N., Pasman, W., and de Rijk, L.

(2011). Unreal goal bots. Agents for Games and Sim-

ulations II, LNCS, 6525:1–18.

Hoogendoorn, R., Hoogendoorn, S. P., Brookhuis, K., and

Daamen, W. (2010). Mental Workload, Longitudi-

nal Driving Behavior, and Adequacy of Car-Following

Models ... Transportation Research Record, 2188:64–

73.

Kesting, A., Treiber, M., and Helbing, D. (2007).

General Lane-Changing Model MOBIL for Car-

Following Models. Transportation Research Record,

1999(1):86–94.

Krajzewicz, D. and Hertkorn, G. (2002). Sumo (simulation

of urban mobility). Proc. of the 4th Middle East Sym-

posium on Simulation and Modelling, pages 183—-

187.

Langley, P., Laird, J. E., and Rogers, S. (2009). Cognitive

architectures: Research issues and challenges. Cogni-

tive Systems Research, 10(2):141–160.

Lee, L., Nwana, H., Ndumu, D., and Wilde, P. D. (1998).

The stability, scalability and performance of multi-

agent systems. BT Technology Journal, 16(3):94–103.

Nagel, K. and Rickert, M. (2001). Parallel implementation

of the TRANSIMS micro-simulation. Parallel Com-

puting, 27(12):1611–1639.

Navarro, L., Flacher, F., and Corruble, V. (2011). Dynamic

level of detail for large scale agent-based urban simu-

lations. Proc. AAMAS, 2:701–708.

Ossen, S. and Hoogendoorn, S. P. (2011). Heterogeneity in

car-following behavior: Theory and empirics. Trans-

portation Research Part C: Emerging Technologies,

19(2):182–195.

Pokahr, A., Braubach, L., and Lamersdorf, W. (2005).

Jadex: A BDI reasoning engine. Multi-Agent Pro-

gramming, pages 149–174.

Rindsf

user, G. (2005). Multi Agent System Simulation for

the Generation of Individual Activity Programs. Ap-

plications of Agent Technology in Trafﬁc and Trans-

portation, pages 165–180.

Rossetti, R. J., Bordini, R. H., Bazzan, A. L., Bampi, S.,

Liu, R., and Vliet, D. V. (2002). Using BDI agents

to improve driver modelling in a commuter scenario.

Transportation Research Part C: Emerging Technolo-

gies, 10(5-6):373–398.

Salvucci, D. D. (2006). Modeling Driver Behavior in a Cog-

nitive Architecture. Human Factors and Ergonomics

Society, 48(2):362–380.

Schakel, W. J., Knoop, V. L., and van Arem, B. (2012).

Integrated Lane Change Model with Relaxation and

Synchronization. Transportation Research Record,

2316:47–57.

Smith, L., Beckman, R., and Baggerly, K. (1995). TRAN-

SIMS: Transportation analysis and simulation system.

Technical report, Los Alamos National Lab (United

States).

ICAART2014-InternationalConferenceonAgentsandArtificialIntelligence

154

Sukthankar, R., Baluja, S., and Hancock, J. (1998). Mul-

tiple Adaptive Agents for Tactical Driving. Applied

Intelligence, 9(1):7–23.

Treiber, M., Hennecke, A., and Helbing, D. (2000). Con-

gested Trafﬁc States in Empirical Observations and

Microscopic Simulations. Physical Review E, pages

1805–1824.

Treiber, M. and Kesting, A. (2013). Trafﬁc Flow Dynamics

- Data, Models and Simulation. Springer.

Turner, P. and Jennings, N. (2001). Improving the scalabil-

ity of multi-agent systems. Infrastructure for Agents,

Multi-Agent Systems, and Scalable Multi-Agent Sys-

tems, LNCS, 1887:246–262.

Wolfe, S. R., Sierhuis, M., and Jarvis, P. A. (2008). To BDI,

or not to BDI: design choices in an agent-based trafﬁc

ﬂow management simulation. In Proc. SpringSim ’08,

pages 63–70.

TowardsSimulatingHeterogeneousDriverswithCognitiveAgents

155