MULTI-AGENT PLANNING FOR THE ROBOCUP

RESCUE SIMULATION

Applying Clustering into Task Allocation and Coordination

Amr Hussein, Carmen Gervet and Slim Abdennadher

Faculty of Media Engineering and Technology, German University in Cairo, Cairo, Egypt

Keywords:

Multi-agent planning, Clustering, RoboCup, Rescue, RoboCup rescue simulation.

Abstract:

The RoboCup Rescue Simulation system provides a rich environment for developing novel techniques for

multi-agent systems. The simulation provides a city map modeled as buildings and roads with civilians

amongst them. A disaster scenario is simulated causing buildings to catch ﬁre, roads to get blocked, and

civilians to get injured and/or buried. The main goal is to use the available emergency services (rescue agents)

to extinguish the ﬁres, clear the roads, and rescue the civilians. This paper describes a new multi-agent plan-

ning approach applied to the RoboCup Rescue problem. The key novelty lies in the distributed approach for

task allocation and coordination. It is done through clustering the map into several overlapping maps each

with a different group of agents assigned to it. Our results showed that not only we could compete against

the top teams in the 2011 RoboCup Rescue Agent Simulation Competition, but we ranked 3rd in this ﬁrst

participation of the GUC in the competition.

1 INTRODUCTION

The RoboCup Rescue project, established in 2001,

aims at promoting research and development in the

rescue domain at various levels including simula-

tion and multi-agent team work coordination (Kitano

and Tadokoro, 2001; Skinner and Ramchurn, 2010).

The RoboCup Rescue Simulation project generates

a model of an earthquake in an urban center. Stu-

dent teams from different universities compete to pro-

duce efﬁcient response techniques and policies for

the simulated emergency services. The earthquake

model covers building collapse, roads blocked by rub-

ble and other debris, trafﬁc movement, ﬁre, and in-

juries to civilians and emergency services workers.

Some buildings are refuges and can be used to heal

injured civilians or reﬁll ﬁre brigades. Roads include

trafﬁc movement and blocked roads. Emergency ser-

vices include ﬁre brigades, ambulance teams, and po-

lice forces.

The goal of the simulation tool is to ﬁnd the op-

timal online strategy that best utilizes the emergency

services to save the maximum number of lives, extin-

guish the maximum number of buildings, and clear

the maximum number of blockades. Most existing

systems model the problem as a decision support sys-

tem that is centralized. In this paper we show how

the RoboCup Rescue Simulation can be modeled as

a multi-agent planning problem and thus beneﬁt from

the strengths of distributed systems. In addition, our

approach tackled the different challenges and pro-

duced competitive results based on the following ad-

ditional contributions:

• Use of clustering techniques to divide the map

into overlapping regions based on some density

criteria, instead of a static even geographical di-

vide. This will then allow us to distribute the

agents over these regions and perform the rescue

operations within them.

• A multi-agent communication model allowing the

agents to exchange information and ask for help in

case of necessity.

Our RoboCup Rescue Simulation tools thus de-

ﬁnes 1) a multi-agent planning optimization prob-

lem that requires an efﬁcient strategy for distribut-

ing up-to 30 agents over the simulated map, 2) an

efﬁcient strategy for traversing the map and discov-

ering the unknown emergency events, and ﬁnally 3)

an efﬁcient communication component to share in-

formation among the agents. The German University

in Cairo team RMAS

ArtSapience participated in the

2011 RoboCup Rescue Agent Simulation Competi-

tion in Istanbul, Turkey. This work has enabled us

339

Hussein A., Gervet C. and Abdennadher S..

MULTI-AGENT PLANNING FOR THE ROBOCUP RESCUE SIMULATION - Applying Clustering into Task Allocation and Coordination.

DOI: 10.5220/0003745303390342

In Proceedings of the 4th International Conference on Agents and Artiﬁcial Intelligence (ICAART-2012), pages 339-342

ISBN: 978-989-8425-96-6

 2012 SCITEPRESS (Science and Technology Publications, Lda.)

to qualify and win the third place in the competition.

1.1 Related Work

The approaches discussed in this section are all based

on the team description papers that belong to the

teams (excluding ours) that participated in the ﬁ-

nal round of the 2011 RoboCup Rescue Competi-

tion. They share some common elements including

the even partitioning of the map and the model based

on a decision support centralized system.

SEU RedSun, Southeast University, China. Ranked

ﬁrst in the 2011 competition and participating since

2008. Their approach is based on accurate world

modeling via multi-communication. Tasks were as-

signed via a centralized decision support system. The

main strength lied in the ability to predict ﬁres early

and good utilization of agent communication. Their

main weakness lied in not having a well deﬁned strat-

egy for communication-less scenarios.

Poseidon, Farazanegan High School, Iran. Rank-

ing second in the 2011 competition and participating

since 2009, Poseidon’s approach depended on discov-

ering connected and unconnected parts of the world

via agent communication to build an enhanced world

model. The approach is based on off-line precom-

puted schedules further optimized using genetic algo-

rithms. The approach also depends on dividing the

map evenly. The main strength lies in the optimized

schedules and the enhanced world model.The weak-

ness lies in dividing the map evenly without taking

into consideration the map structure or distribution of

buildings and in relying on the availability of commu-

nication.

IAMRescue, University of Southampton, UK. Rank-

ing fourth in the 2011 competition and participat-

ing since 2008, IAMRescue’s approach a hierarchical

decision-making system supported by disaster predic-

tion via learning ﬁre spread. Their decision-making

system also depends on agent communication for co-

ordination. The accuracy of the decision-making

system marked IAMRescue’s main strength point,

but similar to the previous approaches, the lack of

communication-less strategies was a weakness.

2 OUR APPROACH

We choose to solve the rescue problem by model-

ing it as a multi-agent planning problem which dif-

fers from existing approaches in the sense that it is

distributed(Weerdt et al., 2005). The main reason is

the ease to divide the tasks, to deﬁne them and han-

dled them autonomously. The nature of the presented

rescue problem provides well deﬁned tasks that can

be easily divided among the different types of agents

with a main goal of performing and optimizing the

rescue process.The following section explain how the

different phases of multi-agent planning are created

and how they contribute to solve the rescue problem.

2.1 Deﬁning Tasks

This phase reﬁnes the global tasks and divides them

into smaller individual tasks. The rescue problem can

be divided into three main tasks: extinguishing ﬁres,

saving civilians, and clearing blocked roads. These

again can be further divided into several individual

tasks. Figure 1 shows a how the rescue tasks are di-

vided into individual tasks.

Figure 1: Rescue Tasks Tree Structure.

Finding routes to buildings and refuges will be

done by all agents individually. Extinguishing build-

ings can only be done by ﬁre brigades. Rescuing

buried civilians and moving the injured ones can only

be done by the ambulance teams. Clearing blockades

can only be done by police forces.

2.2 Task Allocation

Task allocation is required to distribute and assign

tasks to the agents. In our case, task deﬁnition al-

ready provided what each agent will be doing but did

not specify where in the map it will carry out its ac-

tion. Since it is impossible for each agent to timely

traverse the whole map, the map was divided accord-

ing to the number of agents available. Related works

generally divided the map evenly into a grid which

did not take into consideration the map structure and

the distribution of the buildings in the map. In our

approach, we choose to cluster the map on the build-

ings such that we can output regions of almost evenly

distributed buildings. Each region will be assigned an

agents that will traverse all of the buildings and roads

in the region to search for events that require rescue

actions.

ICAART 2012 - International Conference on Agents and Artificial Intelligence

340

This introduced a new challenge to the rescue

agents due to blockades. A series of blocked roads

can divide a single region into disconnected parts pro-

hibiting, or delaying in the case of police forces, other

agents from reaching some parts in the region. This

challenge was solved in our approach with the use

of fuzzy c-means (FCM) clustering (Bezdek, 1981;

Bezdek et al., 1984). Unlike many of the clustering

algorithms that produce disjoint non-empty clusters,

such as K-means, FCM clustering algorithm has the

ability to assign a data point to several clusters with a

speciﬁc membership function. This allows some de-

gree of overlap between the clusters.

Since FCM clustering produces intersecting clus-

ters, the produced regions will have common build-

ings and roads. This will allow some of the buildings

and roads within each region to be visited by at least

two agents. If it happens that the area of intersection

is disconnected in one cluster, it could be reachable

by agents from a different cluster the area is covered

by. This will increase the chance that agents will be

able to visit all buildings and roads in proper time.

2.3 Coordination before Planning

Coordination is required before planning to deﬁne the

rules and constraints that must be applied on planning

to guarantee the satisfaction of the main goal, which

is performing and optimizing the rescue process in

our case. The approach we used in task allocation in-

cluded coordination: 1) divide the map into regions,

2) assign agents to regions provided constraints on

which agents will perform the rescue process in which

region.

Another coordination challenge presented itself

when it came to optimizing the rescue process. In

many cases, it was noticed that blockades can cause

some agents to be initially stuck and some refuges

unreachable. An agent is considered to be stuck when

it is surrounded by blockades from all directions and

cannot move. A refuge is considered to be unreach-

able if all roads leading to it are blocked. The ap-

proach used in task allocation and coordination de-

pends on dividing the map and distributing the agents

on the regions. Since blockades are initially unknown,

stuck agents will be considered in this distribution,

which has a big negative effect on the rescue perfor-

mance, especially for regions with a small number of

agents. On the other hand, refuges are used for treat-

ing injured civilians and reﬁlling ﬁre brigade tanks

and they are the only buildings that do not catch ﬁre.

Blocked refuges will prevent the rescue process from

being complete and also has a big negative effect on

the rescue performance.

Since police forces are the only agents capable of

clearing blockades, they are the only ones that do not

get stuck. Normally, police forces are assigned to re-

gions as discussed earlier. The approach devised in

this paper gives the police forces ﬁrst the task of free-

ing all stuck agents and unreachable refuges, then of

performing rescue processes in their assigned regions.

2.4 Planning

The planning phase involves the creation of the indi-

vidual plans created for each agent putting in mind the

global goal. In our case, planning is carried through

considering the actions required to carry out the indi-

vidual tasks the agents should perform and the con-

straints added in the coordination step. Each agent

creates its own individual plan taking into consider-

ation its type (police, ﬁre, or ambulance agent) and

the cluster the agent is assigned to. All agents will

follow the routes that passes by all roads/buildings in

their regions. Ambulance teams Ambulance teams

are assigned the task of rescuing buried civilians and

moving injured civilians to refuges. Fire brigades are

required to scan all the buildings on its route and ex-

tinguish any building on ﬁre. When it runs out water,

the ﬁre brigade will head to the nearest refuge to reﬁll

its tank. Police forces will execute two consecutive

plans. First they will move to their assigned region

for freeing all stuck agents and blocked refuges. Then

they will move to their main region in which they will

keep searching for blocked roads and clear them.

2.5 Coordination after Planning and

Execution

Coordination after planning was carried out dynam-

ically during the execution of the agents’ plans. It

mainly consists of utilizing multi-agent communica-

tion and adding a criteria to have agents change it’s

region.

Communication. Communication is mainly used

to exchange information on the rescue events that re-

quire attention. Communication messages are divided

into informative and query messages. Informative

messages inform other agents of sensed ﬁres, block-

ades, and buried civilians.They are sent by agents that

detect events they cannot act up on. On the other

hand, query messages ask for help from agents of the

same type as the sender. They are sent when an agent

realizes that it cannot perform a rescue action on it’s

own.

MULTI-AGENT PLANNING FOR THE ROBOCUP RESCUE SIMULATION - Applying Clustering into Task Allocation

and Coordination

341

Changing Region. An agent can change its as-

signed region during execution. Changing regions is

only allowed in one of two cases: 1)changing region

temporarily in response to a communication message

informing of an event in a different region the return-

ing to the original region, 2) and when an agent does

not ﬁnd any rescue events in its region. In the second

case, the agent will change its region if there are no

more events that it can handle in its region. If some

regions are overloaded with rescue events, none of

its agents will change regions. On the other hand,

agents in other regions that are either event free or

with handled events will keep changing regions until

they reach the overloaded region. This will accumu-

late all free agents in the overloaded regions.

3 EVALUATION

The main performance measure used to judge the

efﬁciency of the implemented approach was the ﬁ-

nal simulation score of the rescue simulation. The

score is automatically calculated by the simulation

kernel. The full detailed scores of the 2011 compe-

tition can found on the RoboCup Rescue Simulation

(http://roborescue.sourceforge.net/).

3.1 Results and Ranking

Our team, RMAS_ArtSapience reached the ﬁnal

round of the 2011 competition and ranked third af-

ter tying on the 2nd place in total ranking but lost

to score difference. Our evaluation is based on two

types of maps: maps with communication enabled

and communication-less maps.

Map with Communication The maps with agent

communication enabled are designed to test the abil-

ity of the agents to detect the ﬁres early in the

simulation and coordinate with other agents through

communication to extinguish the ﬁres and clear the

blocked roads around the ﬁres. Our team ranked in

most cases in the top 4 in these maps. Table 1 shows

the results in some chosen samples from the 2011

competition. The main strength of our approach is

the efﬁcient agent distribution. Police forces where

able to spread through the map and clear most of the

blockades fast enough during the simulation.

Communication-less Maps. The communication-

less maps did not allow the use of multi-agent com-

munication. This made it hard for the agents to dis-

cover the location of ﬁres. In addition to that, ﬁres

Table 1: Top 4 scores for thee maps with communication

enabled.

Map Paris3 Istanbul3 Berlin3

Rank IAMRescue Poseidon SEU RedSun

Score 12.8098 66.1779 128.06

Rank RoboAKUT SEU RedSun RoboAKUT

Score 9.9392 60.9769 121.58

Rank Poseidon Our Team MRL

Score 9.7506 59.8821 119.50

Rank Our Team IAMRescue Our Team

Score 9.2929 49.2957 112.2160

started randomly during the simulation, which fur-

thermore increased the ﬁre discovery challenge. The

scenario is designed to mainly test how agents will

perform in the absence of communication and how it

affects their rescue actions.

Our approach was able to control the ﬁres more

efﬁciently than the other teams. This was due to efﬁ-

cient distribution of the agents over the map achieved

through the use of clustering without depending on

agent communication. The ﬁnal scores, as seen in ta-

ble 2, showed that in some cases our team achieved a

signiﬁcantly higher scores than all other teams com-

peting in the map.

Table 2: Top 4 scores for thee maps with communication

enabled.

Map Berlin4 Paris5 Kobe4

Rank Our Team Our Team Our Team

Score 140.95 15.51 90.64

Rank SEU RedSun SEU RedSun SEU RedSun

Score 132.01 12.4263 80.36

Rank RoboAKUT Poseidon Poseidon

Score 131.33 8.5938 77.76

Rank Poseidon IAMRescue IAMRescue

Score 130.46 0.9531 73.10

REFERENCES

Bezdek, J. C. (1981). Pattern Recognition with Fuzzy Ob-

jective Function Algorithms. Kluwer Academic Pub-

lishers, Norwell, MA, USA.

Bezdek, J. C., Ehrlich, R., and Full, W. (1984). Fcm: The

fuzzy c-means clustering algorithm. Computers &

Geosciences, 10(2-3):191–203.

Kitano, H. and Tadokoro, S. (2001). RoboCup Rescue: A

Grand Challenge for Multiagent and Intelligent Sys-

tems. AI Magazine, 22(1).

Skinner, C. and Ramchurn, S. (2010). The RoboCup Rescue

Simulation Platform. AAMAS ’10, pages 1647–1648.

Weerdt, M. D., Mors, A. T., and Witteveen, C. (2005).

Multi-agent planning: An introduction to planning

and coordination. In: Handouts of the European Agent

Summer.

ICAART 2012 - International Conference on Agents and Artificial Intelligence

342