ROBUST MULTI-ROBOT COOPERATION THROUGH DYNAMIC

TASK ALLOCATION AND PRECAUTION ROUTINES

Sanem Sariel

Istanbul Technical University, Computer Engineering Department, Maslak, Istanbul, Turkey

Tucker Balch

Georgia Institute of Technology, College of Computing, Atlanta, GA, USA

Nadia Erdogan

Istanbul Technical University, Computer Engineering Department, Maslak, Istanbul, Turkey

Keywords:

Distributed AI, robotics, multi-agent systems.

Abstract:

In this paper, we present the design and implementation of a multi-robot cooperation framework to collectively

execute inter-dependent tasks of an overall complex mission requiring diverse capabilities. Given a heteroge-

neous team of robots and task dependencies, the proposed framework provides a distributed mechanism for

assigning tasks to robots in an order that efﬁciently completes the mission. The approach is robust to unreli-

able communication and robot failures. It is a distributed auction-based approach, and therefore scalable. In

order to obtain optimal allocations, effective bid evaluations are needed. Additionally to maintain optimality

in noisy environments, dynamic re-allocations of tasks are needed as implemented in dynamic task selection

and coalition maintenance scheme that we propose. Real-time contingencies are handled by recovery routines,

called Plan B precautions in our framework. Here, in this paper, we present performance results of our frame-

work for robustness in simulations that include variable message loss rates and robot failures. Experiments

illustrate robustness of our approach against several contingencies.

1 INTRODUCTION

We propose Distributed and Efﬁcient Multi Robot

- Cooperation Framework (DEMiR-CF) for multi-

robot teams that must cooperate/coordinate to achieve

complex missions including tightly coupled tasks that

require diverse capabilities and collective work. It

combines auctions, coalition maintenance scheme

and recovery routines called Plan B precaution rou-

tines to provide an overall system that ﬁnds (near-)

optimal solutions in the face of noisy communication

and robot failures. The efﬁciency of our framework

was tested for the multi-robot multi-target explo-

ration problem in an earlier work (Sariel and Balch,

2006). In this paper, we evaluate the performance

of the framework for robustness on complex missions

against several noisy or faulty situations. The Plan B

precaution routines embedded in the framework en-

able the system to dynamically respond to the contin-

gencies and still complete the mission. The proposed

framework strives to provide (near-) optimal task al-

locations with effective bid evaluations while simul-

taneously and effectively responding to contingencies

and maintaining the solution quality.

Multi-robot coordination problem has been an ac-

tive area of research where both centralized and dis-

tributed approaches are considered. In most cases, the

selected domain missions include independent sub-

tasks which can be executed by a single robot. Due

to the real world requirements and issues of robust-

ness, the centralized approach has usually not been

successful in these implementations. The distributed

approach and, speciﬁcally when complemented with

the auction based methods based on Contract Net Pro-

tocol (Smith, 1980), shows great promise for multi-

robot task allocation because of its scalability. The

task allocation problem in the face of environmen-

tal changes and the optimality analysis are investi-

gated separately on earlier distributed systems. Ac-

cording to (Dias et al., 2005), existing market mech-

anisms/auction based approaches are not fully capa-

ble of re-planning task distributions, changing the de-

composition of tasks continually, rescheduling com-

mitments, and re-planning coordination during exe-

cution. Our research aims at addressing these short-

comings.

Even though task allocation issues have been stud-

ied extensively, there is not yet a complete frame-

196

Sariel S., Balch T. and Erdogan N. (2006).

ROBUST MULTI-ROBOT COOPERATION THROUGH DYNAMIC TASK ALLOCATION AND PRECAUTION ROUTINES.

In Proceedings of the Third International Conference on Informatics in Control, Automation and Robotics, pages 196-201

DOI: 10.5220/0001212901960201

 SciTePress

work which is fully capable of integrating task allo-

cation into real time execution for complex missions.

A classiﬁcation for multi-robot task allocation prob-

lem and a review of the existing approaches are given

in (Gerkey and Mataric, 2004). According to this

classiﬁcation, our framework lies in the single-task

robots, instantaneous assignment with time-extended

view of the problem and multi-robot tasks horizon.

Contingency handling is not directly integrated into

task allocation and execution in earlier frameworks.

Our work differs at this point, as it integrates con-

tingency handling into task allocation and achieves a

dynamic allocation scheme through the task execu-

tion. The framework maintains allocation (near-) op-

timality during task execution, even in the presence of

noisy and/or faulty conditions.

2 FRAMEWORK OVERVIEW

DEMiR-CF is designed to address complex missions

of tightly coupled tasks requiring diverse capabili-

ties and simultaneous and synchronous task execu-

tion. Optimal solutions are sought using effective

bid evaluation strategies. The framework also effec-

tively responds to limited or noisy communications

and robot failures during execution. These failures are

recognized by Plan B precaution routines and corre-

sponding actions are taken. Dynamic task selection

and Coalition maintenance scheme provide task (re-

)allocations to maintain the solution quality. Each

robot equipped with DEMiR-CF software executes a

part of the domain mission, achieving allocation, exe-

cution and system consistency maintenance in a fully

distributed fashion without a central coordinator. Our

system relies on a set of task dependencies that are

compiled a priori by a mission commander, or gen-

erated by a planner. The task dependencies, a speci-

ﬁcation of the mission, and a speciﬁcation of the ro-

bot teams’ capabilities are distributed (reliably) to the

robots before the mission execution begins. At that

point, each robot autonomously decides which task to

execute and negotiations are implemented in a fully

distributed fashion.

2.1 Task Representation

One of the important mechanisms of the framework is

the guaranteed validity of the generated plans for exe-

cuting tasks by robots. Although there is not a central

planner, the robots execute tasks by considering de-

pendencies among them. We use a simple represen-

tation for tasks with ordering and dependency con-

straints, similar to but simpler than some representa-

tions in TAEMS (Decker, 1996). We consider hard

(h − dep) and soft dependencies (s − dep). If task T

is hard dependent on the task T

, it cannot be executed

before the task T

is completed. In this case, there is

a strict ordering between tasks. If the task T

is soft

dependent on the task T

, it can be executed while the

task T

is being executed or before its execution be-

gins. However the task T

cannot be completed until

the task T

is completed. Therefore some portions of

the two tasks can be executed simultaneously. The re-

quired capabilities and the number of robots required

to execute the task are also encoded into the represen-

tation. Robots, which lack the necessary capabilities

for a task, are not eligible for executing and do not

consider such tasks for execution. A task can only

be executed when the required number of robots are

assigned to execute.

2.2 Roles

Some tasks may require simultaneous performance by

a group of robots. We have chosen the coalition or-

ganizational paradigm to organize the task execution.

(Horling and Lesser, 2005). These coalitions can con-

tain one or more robots according to the number of

robots required to execute tasks. Members of coali-

tions are selected by auctions. The auctioneers are

also active robots in the system. The overall objec-

tive is completing a mission (M) consisting of tasks

(0 < i ≤ ||M||) with a multi-robot team (r

∈ R,

0 < j ≤ ||R||) in a cost optimal manner. Each coali-

tion (C

) is formed to execute a task T

of the over-

all mission M . Sizes of coalitions vary according to

the required number of robots (reqno

) to execute the

task. The capabilities (cap

) of each robot r

in a

coalition should be a superset of the required capabil-

ity set for the task T

(reqcap

) to be eligible. A ro-

bot (r

) may take different roles for the task T

, such

as auctioneer, bidder (B

), coalition leader (CL

) or

coalition member (CM

) during runtime.

• An Auctioneer robot is responsible for managing

auction negotiation steps and selecting reqno

suit-

able members of a coalition.

• A Bidder robot is a candidate to become a member

of a coalition to execute a task.

• A Coalition Leader is the robot responsible for

maintaining the coalition and providing synchro-

nization while performing the coalition task.

• A Coalition Member is one of the members of the

coalition, and it executes a portion of the task.

A robot r

may be in more than one B

roles for dif-

ferent tasks, but may not be in more than one role of

auctioneer, CM

or CL

at the same time. The auc-

tioneer is responsible to select the required number of

robots (the coalition leader and members) for task ex-

ecution. The auctioneer may or may not take place in

ROBUST MULTI-ROBOT COOPERATION THROUGH DYNAMIC TASK ALLOCATION AND PRECAUTION

ROUTINES

197

the coalition for which it offers an auction. The coali-

tion leader maintains the coalition and keeps track of

the members’ conditions and their updated informa-

tion. After the execution of the task is completed, the

coalition ends. Each robot is allowed to take part in

only one coalition until it leaves the coalition. Coor-

dination between coalition members is implemented

through synchronization messages. We assume that

robots are not able to infer the state of others by obser-

vation; although such capabilities would only provide

more reliability (e.g. (Balch and Arkin, 1994)).

2.3 Precautions

To deal with the uncertainties that may arise due to

message losses, each robot keeps track of the models

of the known tasks and the other robots in its world

knowledge. The current statuses of the known tasks

are represented in ﬁnite state machines (FSM). The

state transitions are initiated either by own motiva-

tions or incoming information from the other robots.

There are eight task states in each FSM. Initially all

tasks are in free state, meaning that none of the ro-

bots is executing/auctioning and they are not achieved

yet. Then, according to the status of the task, the state

can be speciﬁed as: auctioned, selected, ready for ex-

ecution, awarded, being executed by own/others, or

achieved. When messages are received from the other

robots, the task and robot models are updated accord-

ingly. In some cases a new message may imply a con-

ﬂict with the robot’s current model of the task state.

In this case, the appropriate precaution routines are

activated to either correct the model, or initiate a re-

covery. Recovery operations may include warning the

other robots or changing the task states and dynami-

cally switching among tasks if proﬁtable. Such incon-

sistencies usually arise when robots are not accurately

informed about the tasks that are completed, under ex-

ecution, or under auction. It is assumed that robots are

trusted and benevolent.

To keep consistency, each robot executing a task

is responsible for broadcasting an execution message,

informing others that the task is under execution and

the robot is alive. Thus, robots are informed about

the task and they assume that the task is under exe-

cution for a period of time. If a robot does not re-

ceive any messages related to a task for a certain pe-

riod of time, the task is assumed to be free. Robot

failures are detected by checking the latest commu-

nication information with the robot. When parallel

auctions for the same task are detected, the robot with

the minimum cost value continues the auction negoti-

ation processes. After receiving an auction message,

a robot checks the validity of the auction by consider-

ing the task information before sending its bid value.

If the robots get an auction offer for a task that has

hard dependencies, then it assumes that these depen-

dent tasks are also achieved and the own world knowl-

edge is updated respectively. Similar update actions

are carried out when execution, cancellation messages

are received.

For synchronized coordination, the coalition leader

sends synchronization messages to all coalition mem-

bers. These messages are also used for recognizing

failures. The leader also receives the updated cost in-

formation in execution messages from the members.

If the messages cannot be received during a prede-

ﬁned time period, the existence of a problem is as-

sumed and the coalition execution is ended.

2.4 Dynamic Task Allocation

Each robot updates its world knowledge according to

the incoming messages. Before selection of an ac-

tion, each robot should perform some routine tasks

for different roles. Auctioneers perform the negotia-

tion processes. Leaders and members of the coalitions

carry out synchronization actions that ensure the si-

multaneous execution of tasks synchronously. Next,

considering the updated world knowledge and the sit-

uations of the processes for different roles, each robot

determines a rough schedule where tasks for which it

meets the capability requirements are considered and

several constraints are taken into consideration. Then,

it generates a prioritized list of tasks, from which it

selects the most suitable and precedence feasible one

and takes an action accordingly. This approach is

known as List Scheduling (Graham, 1966) and shown

to be 2−(1/m) OP T for makespan minimization ob-

jective on tasks with precedence constraints (m is the

number of machines/robots). Different priority rules

may be used. Here in this work, we use Least Flexible

Job First (LFJ) rule in a precendence feasible manner.

The selected action may be participation to a coali-

tion or becoming an auctioneer auctioning for a free

task. The dynamic action selection algorithm (Al-

gorithm 1) is performed if the robot is free or re-

leased. The releasing mechanism provides a way to

switch tasks effectively if another advantageous situa-

tion arises. Decision on releasing a robot from a coali-

tion can only be made by the coalition leader. This

restriction prevents the system from frequent switch-

ing and contributes to the achievement of tasks which

require simultaneous execution.

2.5 Cost/Bid Evaluation

The main objective to achieve a global goal (e.g. min-

imization of total energy consumed or goal achieve-

ment time) determines how robots should perform

cost evaluation. Unless effective bid evaluation strate-

gies are designed, it is not possible to observe globally

optimal solutions for NP-Hard problems, and addi-

tional adjustments are required to change allocations

ICINCO 2006 - ROBOTICS AND AUTOMATION

198

Algorithm 1 Action Selection Algorithm for r

for each known task T

if the reqcap

⊆ cap

∧ h − dep

are completed then

if the task is in one following states: under execution and cost

proﬁtable to join k awarded k free then

add the task in a priority queue ordered by LFJ

end if

end for

= queue.top()

if r

is in a coalition of T

(should be released) then

cancel executing the task, send a “leaving” message to the CL

end if

if state(T

) == free then

begin auction negotiation process

else

the current task = T

if T

is an awarded task then

send “accept” message to the auctioneer

end if

with an additional cost of communication as in com-

binatorial auctions. Difﬁculty of the problem arises

when communication is limited and robots should au-

tonomously perform task allocation at the same time

with execution. Simultaneous execution requirements

make the problem more challenging because each ro-

bot should be in its most suitable coalition in a future

formation and estimate it correctly before making a

decision in a distributed fashion and with as mini-

mum communication as possible. In our earlier work

(Sariel and Balch, 2006), we have shown that by ef-

fective bid evaluation approaches, globally near op-

timal solutions can be observed in an auction based

system. The incremental assignment approach and

dynamic task selection scheme eliminate redundant

considerations for environments in which the best so-

lution is highly probable to change, and effective bid-

ding strategies ensure the high solution quality with a

time-extended view of the problem.

2.6 Coalition Maintenance Scheme

To ensure maintaining optimality against environ-

mental changes, we propose a dynamic reconﬁgura-

tion approach. Coalition members can leave a coali-

tion when there is sufﬁcient number of members to

execute the task. The coalition leader is responsible

to broadcast the maximum cost of execution for the

task by one of the coalition members in each execu-

tion step. In the decision stage, each robot receiving

these messages evaluates the maximum cost value of

each coalition. If a robot detects that its cost is lower

than the maximum cost of the coalition and it is re-

leased from its current coalition, it sends a join re-

quest message to the coalition leader. The leader, get-

auction

bid

auction

bid

max cost1

max cost 1

Join request & bid

max cost 2

released

max cost 2

Robot with the maximum cost value

(max cost1) in the coalition

a b

Figure 1: Dynamic Coalition Reconﬁguration.

ting a join request message, directly adds the robot to

the coalition. If the coalition leader detects that the

size of the coalition is larger than required, it can re-

lease coalition members with the maximum cost value

for the current task. Getting a released message, a ro-

bot can proceed to select another suitable task. When

the coalition leader considers the size of the current

coalition, it also checks the failures. Since each robot

in the coalition broadcasts ”under execution” and up-

dated ”cost” messages, their failure can be detected

by the coalition leader. The failed robots are also

released from the coalition. If the number of mem-

bers to execute the task is below the required for a pe-

riod of time, the coalition leader cancels the coalition.

An illustrative example of coalition reconﬁguration is

given in Fig. 1. Such a situation may occur if a robot

is not reachable when the auction announcement is

made {a}. When the situation changes {b}, the robot

may take over the role of the member with the max-

imum cost value in this coalition {c}. The released

member can select another task to execute.

3 EXPERIMENTAL RESULTS

We have conducted experiments to test the perfor-

mance of the framework in terms of the total time

to complete the overall mission for collective con-

struction domain. In the ﬁrst stage of the experi-

ments, the performance is evaluated against message

losses for a complex mission that consists of tasks

with hard and soft dependencies among each other.

In the second stage of the experiments, the effective-

ness of the framework and the recovery solutions is

analyzed when the robot failures are injected into the

system. The experiments are conducted on an ab-

stract simulator simulating robots’ updated locations,

the message exchange and execution of the missions.

The results are from 100 independent runs with ran-

dom seeds. The maximum simulation step number is

ROBUST MULTI-ROBOT COOPERATION THROUGH DYNAMIC TASK ALLOCATION AND PRECAUTION

ROUTINES

199

INITIAL STATE FINAL STATE

Figure 2: Initial and Final States of the Test Domain.

Figure 3: Task dependency graph of the mission.

selected as 200. In the designed experiment, there are

three types of objects: A, B and C. The mission con-

tains tasks for locating these objects and building up

the construction while obeying the speciﬁed ordering

restrictions. In the construction, the objects should be

moved into the destination in the predetermined or-

der: A, B and C. The initial and ﬁnal stages can be

seen in Fig. 2. The requirements for the tasks and

their precedence constraints are given in Table 1. The

third and fourth columns represent the hard and soft

dependencies respectively. The required capabilities

for the task and the required number of robots to ex-

ecute are speciﬁed in the last two columns. The ca-

pabilities of having blob ﬁnder, sonar sensor, prods

for objects A and C and prods for object B are listed

as 0, 1, 2 and 3 respectively. Based on the speciﬁ-

cations of the overall mission, the task dependencies

can be represented graphically as in Fig. 3, in which

the solid and dashed lines represent the hard and soft

dependencies, respectively.

The capabilities of the robots are given in Table 2.

There is one redundant robot in each set to analyze ro-

bustness of the framework against failures. It should

be noted that, if the requirements are not met, there is

no way to execute the corresponding task.

Instead of selecting the failure step randomly,

we’ve selected the step number in which the failure

occurs as the most important time step to complete

the mission, to fairly analyze the robustness. Based

on the results of no failure case, we’ve observed that,

for most of the runs, the 25th time step is a critical

time step when the robots have already ﬁnished exe-

cuting tasks that do not require coordination, and try

to coordinate for other tasks. The failed robot is se-

lected randomly. The mission completion ratios over

all runs are illustrated in Table 3. In this table, the

main columns present results of experiments with no

Table 1: Task Speciﬁcations.

ID Description h − dep s − dep reqcap reqno

0 Locate-Obj-A - - 0, 1 1

1 Locate-Obj-B - - 0, 1 1

2 Locate-Obj-B - - 0, 1 1

3 Locate-Obj-B - - 0, 1 1

4 Locate-Obj-C - - 0, 1 1

5 Move-Obj-A-to-Dest 0 - 0, 1, 2 1

6 Move-Obj-B-to-Dest 1 5 0, 1, 3 2

7 Move-Obj-B-to-Dest 2 5 0, 1, 3 2

8 Move-Obj-B-to-Dest 3 5 0, 1, 3 2

9 Move-Obj-C-to-Dest 4 6, 7, 8 0, 1, 2 3

Table 2: Robot Set.

SetID RobotID cap

1 0, 8 0, 1

2 1, 2, 3, 4 0, 1, 2

3 5, 6, 7 0, 1, 3

failure, 1 and 2 robots failures, respectively. An addi-

tional column is added for each experiment with fail-

ures to report the number of runs in which one or two

of randomly failed robot is a coalition leader. One can

infer from these results that the completion of the mis-

sion is not guaranteed when the message loss rate is

over 50%. Since synchronization is highly required

for multi-robot coordination of tasks 6-9, message

losses cause synchronization to be lost and also the

auction negotiation steps are not completed reliably.

However it should be noted that the framework can

easily handle message loss rates smaller than 50% (a

very high message loss rate for real life experiments).

In those cases, in spite of robot failures, even if the

failed robots are coalition leaders, the mission is com-

pleted successfully as if there were no robot failures.

In the 2 failures case, the mission is not completed

in some runs, because the randomly selected robots

belong to the same set and when they fail, mission

completion is not possible. For the message loss rate

100%, the number of runs in which the failed robot

is a coalition leader is 100. This is because, in this

case, each robot is the coalition leader of its own task

with reqno = 1, and since the robots cannot com-

municate, the execution of the tasks with reqno = 1

(0-4) are completed in later steps and all robots try

to execute all these tasks by themselves. The mis-

sion completion time results can be seen in Fig. 4.

The mission completion time increases for increasing

message loss rates logarithmically. However, even

when message loss rate is 70%, the mission can be

ICINCO 2006 - ROBOTICS AND AUTOMATION

200

Table 3: Mission Completion Ratios.

Message #of runs in which the mission is completed

Loss no failure 1 robot failure 2 robots failure

Rate (% ) #runs #runs #CL

#runs #CL

0 100 100 22 83 39

10 100 100 22 79 45

20 100 100 27 81 33

30 100 100 27 78 42

40 100 100 20 77 31

50 98 95 19 72 27

60 59 48 15 30 27

70 4 3 13 2 18

80 0 0 5 0 19

90 0 0 4 0 25

100 0 0 100 0 100

−0.1 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8

100

120

140

160

180

200

220

Message loss rate

#Steps to complete the mission

No Failure

1 robot failure

2 robot failure

Figure 4: Mission Completion Time for different message

loss rates, averaged over 100 runs.

completed in some runs. As expected, no failure case

has a better success rate. However, even if there are

failures, results are still very close to the no failure

case. Therefore, we conclude that the framework can

easily handle robot failures.

The total number of messages also increases log-

arithmically for the changing message loss rates as

illustrated in Fig. 5. The robots continuously query

the free tasks and try to coordinate the simultaneous

task execution. The number of messages for no fail-

ure case is also greater when the message loss rate

increases, due to the number of robots and their ef-

forts to coordinate. When the results are evaluated, it

can be concluded that the framework handles message

losses as well as possible.

4 CONCLUSION

We evaluate our multi-robot cooperation framework

on complex missions with inter-related tasks requir-

ing heterogeneity and coordination. The recovery so-

lutions provided by precaution routines for different

−0.2 0 0.2 0.4 0.6 0.8 1 1.2

0.2

0.4

0.6

0.8

1.2

1.4

1.6

1.8

x 10

Message loss rate

Number of Messages

No Failure

1 robot failure

2 robot failure

Figure 5: Total Number of System Messages for different

message loss rates, averaged over 100 runs.

kinds of failures ensure the completeness of the ap-

proach. Experiments to validate the robustness of the

approach are conducted in the simulated object con-

struction domain with message losses and robot fail-

ures. Even for very high message loss rates, the ro-

bot team can still complete the mission using the pro-

posed framework. Our next step in this research is

the real ﬁeld experiments on real robots and extensive

cost/bid designs and analyses for different domains.

REFERENCES

Balch, T. and Arkin, R. C. (1994). Communication in reac-

tive multiagent systems. Autonomous Robots, 1(1):1–

25.

Decker, K. (1996). Foundations of Distributed Artiﬁcial

Intelligence, chapter TAEMS: A Framework for En-

vironment Centered Analysis and Design of Coordi-

nation Mechanisms, pages 429–448. John Wiley and

Sons.

Dias, M. B., Zlot, R. M., Kalra, N., and Stentz, A.

(2005). Market-based multirobot coordination: A sur-

vey and analysis. Technical Report CMU-RI-TR-05-

13, Carnegie Mellon University, Robotics Institute.

Gerkey, B. and Mataric, M. J. (2004). A formal analysis

and taxonomy of task allocation. Intl. J. of Robotics

Research, 23(9):939–954.

Graham, R. L. (1966). Bounds for certain multiprocessing

anomalies. Bell System Technical Journal, 45:1563–

1581.

Horling, B. and Lesser, V. (2005). A survey of multi-agent

organizational paradigms. The Knowledge Engineer-

ing Review, 19(4):281–316.

Sariel, S. and Balch, T. (2006). Efﬁcient bids on task allo-

cation for multi-robot exploration. In The 19th Inter-

national FLAIRS Conference.

Smith, R. G. (1980). The contract net protocol: High

level communication and control in a distributed prob-

lem solver. IEEE Transaction on Computers C-,

29(12):1104–1113.

ROBUST MULTI-ROBOT COOPERATION THROUGH DYNAMIC TASK ALLOCATION AND PRECAUTION

ROUTINES

201