World Model for Testing Urban Search and Rescue (USAR) Robots using

Petri Nets

Anneliese Andrews, Mahmoud Abdelgawad and Ahmed Gario

Department of Computer Science, University of Denver, Denver, CO 80208, U.S.A.

Keywords:

Model-based Testing, Behavioral Models, Petri Nets, USAR Robots.

Abstract:

This paper describes a model-based test generation approach for testing Urban Search and Rescue (USAR)

robots interacting with their environment (i.e., world). Unlike other approaches that assume a static world

with attributes and values, we present and test a dynamic world. We use Petri Nets to illustrate a world

model that describes behaviors of environmental entities (i.e., actors). The Abstract World Behavioral Test

Cases (AWBTCs) are generated by covering the active world model using graph coverage criteria. We also

select test-data by input-space partitioning to transform the generated AWBTCs into executable test cases.

Reachability of the active world model and efﬁciency of coverage criteria are also discussed.

1 INTRODUCTION

According to (Williamson and Carnegie, 2007),

Search and Rescue (SAR) refers to rescue activities

in high-risk emergency situations. The 2001 World

Trade Center (WTC) collapse, the 2005 Hurricanes

Katrina, Rita, and Wilma in the United States, the

2011 Tohoku earthquake and Tsunami in Japan, are

examples of emergency situations. (Murphy et al.,

2008) report that over 900,000 people were reported

killed by disasters from 1995 to 2004, with the to-

tal amount of disaster related damage estimated at

738 billion US dollars. Urban Search and Rescue

(USAR) robotics aims at minimizing human involve-

ment in these emergency situations. In a USAR mis-

sion, the USAR robots explore unknown urban dis-

aster scenes providing real-time video and other sen-

sory data about the situation while searching for vic-

tims (Liu and Nejat, 2013). Urban disaster scenes are

often inaccessible for rescue workers due to poten-

tial collapses, poisonous gases, hazardous materials,

radiation or extreme temperatures. Testing the inter-

actions between the USAR robots and urban disaster

actors- victims, rubble, debris, bricks, mobile objects,

and unknown obstacles- poses a series of challenges,

due to the complexity and uncertainty of these ac-

tors. Model-based Testing (MBT) is able to leverage

behavioral models, such as CEFSM (Li and Wong,

2002), Coloured Petri Nets (CPN) (Lill and Saglietti,

2012), and Labelled Transition Systems (LTS) (Tret-

mans, 2008), to describe the behavioral scenarios that

can occur between the USAR robots and the disaster

actors. This requires testing solutions that deal with

the large number of possible behaviors. Current MBT

approaches for testing Real-time Embedded Systems

(RTES) interacting with their worlds assume a static

world model (Iqbal et al., 2012) and (Hessel et al.,

2008) . However, for USAR robots, the world cannot

be described only statically with attributes and val-

ues, the world should also be presented and tested a

dynamically. To address these challenges, we use a

systematic MBT approach, World Model-based Test

Generation (WMBTG), that identiﬁes what, where

and how to test USAR robots interacting with their

surroundings. Test cases are generated by aggregating

test paths in the individual models. These test paths

are grouped as concurrent test paths which can be

used with simulators or test-harnesses to validate the

USAR robots. WMBTG has been introduced, in our

previous work, and was applied to Unmanned Ground

Vehicle (UGV) domain (Andrews et al., 2015). This

paper extends the applicability of WMBTG to USAR

robots. We also evaluate the efﬁciency of test path

coverage criteria used to generate Abstract World Be-

havioral Test Cases (AWBTs). We evaluate input-

space partitioning coveragecriteria (Ammann and Of-

futt, 2008) used to generate test-data. We use UML

class diagrams to construct the structural model of

disaster actors and their relationships. We also use

Petri Nets (PNs) (Murata, 1989) to represent the be-

haviors of urbandisaster actors that are involvedinthe

disaster scenes (snippets). These snippets are used to

Andrews, A., Abdelgawad, M. and Gario, A.

World Model for Testing Urban Search and Rescue (USAR) Robots using Petri Nets.

DOI: 10.5220/0005782106630670

In Proceedings of the 4th International Conference on Model-Driven Engineering and Software Development (MODELSWARD 2016), pages 663-670

ISBN: 978-989-758-168-7

663

link the behavioral models of disaster actors. This pa-

per explores the applicability of the MBT technique

for testing USAR robot behaviors in dynamic worlds

alongside behavioral testing that is systematic, scal-

able, and shows that this technique is extendable to

other domains of autonomous systems.

The remainder of this paper is organized as fol-

lows. Section 2 discuses the state of research related

to MBT, testing USAR robots, and World Model-

based Testing. Section 3 describes a case study. Sec-

tion 4 presents our approach and applies it to the case

study. We analyze and discuss reachability and efﬁ-

ciency issues in section 5. Section 6 draws conclu-

sions and future work.

2 STATE OF RESEARCH

2.1 Model-based Testing (MBT)

According to (Dias-Neto et al., 2007), MBT uses var-

ious models to automatically generate tests. MBT in-

cludes three key elements: models that describe soft-

ware behavior, criteria that guide the test-generation

algorithms, and tools that generate supporting infras-

tructure for the tests. (Zander et al., 2012) deﬁne

MBT as an algorithm that generates test cases auto-

matically from models instead of creating them man-

ually. (Utting et al., 2012) deﬁne six dimensions of

MBT approaches (a taxonomy): model scope, charac-

teristics, paradigm, test selection criteria, test genera-

tion technology and test execution. (Shirole and Ku-

mar, 2013) also present a survey on model-based test

generation. They deﬁne the process of test case gen-

eration from graphs as follows: build a graph model,

identify test requirements, select test paths to cover

those requirements, and derive test data. (Lill and

Saglietti, 2012) use Coloured Petri Nets (CPNs) for

testing a factory robot that carries a load from one

place to another. The authors deﬁne coverage criteria

for CPNs, such as color-based and event-based cov-

erage criteria. This work does not provide validation,

nor does it address dynamic worlds.

2.2 World Model-based Testing

Most approaches in the literature for modeling the

world of autonomous systems aim at improve the un-

derstandability of the decision-making module to the

surroundings, but not for testing. (Furda and Vlacic,

2010) present an object-oriented world model ap-

proach for the road trafﬁc environment of autonomous

vehicles. The approach uses UML class diagrams to

represent the structure of the world actors. The au-

thors conducted a ﬁeld trial using two autonomous

vehicles. The experiment illustrates that the world

model strongly supports the decision-making module

for making appropriate driving decisions in real-time.

This work neither intends to be a testing technique for

autonomous systems nor does it handle the dynamic

aspect of world actors. A closely related approach

for world model-based testing is presented in (Iqbal

et al., 2010). The approach generates black-box test

cases automatically based on a static world model for

real-time embedded systems. The main characteris-

tics of the approach are: 1) An extension of UML

(MARTE) is used for modeling the structural and be-

havioralworld properties, especially real-time proper-

ties. 2) Test oracles are then generated automatically

from these models. 3) To identify feasible test cases

and maximize possibilities of fault detection, heuristic

algorithms are used as test generation strategies. An

empirical study shows that Adaptive Random Testing

(ART) is more satisfactory than the others. This ap-

proach limits the world model to a static world, there-

fore it is not applicable for autonomous systems be-

cause their worlds are dynamic.

2.3 Testing USAR Robots

The literature of testing USAR robots shows that

the techniques for testing USAR robots are mostly

computer-based simulation and test ﬁelds/arenas. (Ja-

coff et al., 2003), by the National Institute of Stan-

dards and Technology (NIST), introduce a standard

for designing and evaluating test arenas (Reference

Test Arena for Autonomous Mobile Robots). The

test arenas consist of collapsed structures that are de-

signed from buildings in various stages of the col-

lapse. The authors classify the test arenas into three

categories, each labelled by a speciﬁc color (Yellow,

Orange, and Red). The Yellow arena consists of a

planar maze with isolated test sensors in the form of

obstacles, simulated victims, doors that can be closed,

blinds that can be raised or lowered, and simple col-

lapses that can block passages. The Orange arena

provides more difﬁcult challenges by adding an ele-

vated ﬂoor, holes, ramps, and stairs. The Red arena is

composed of unstable ﬂoors, harsh rubble, and piled

debris. Thirty simulated victims can be placed over

these test arenas. Each victim displays up to ﬁve signs

of life (form, motion, body heat, sound, and/or CO

emission). (Chiou and Wynn, 2009) also provide a

small test arena that is portable and can be placed in a

laboratory. Although test ﬁelds/arenas provide phys-

ical ﬁdelity, they do not fully represent the physical

conditions of actual disasters (Murphy et al., 2008).

MODELSWARD 2016 - 4th International Conference on Model-Driven Engineering and Software Development

664

(Pepper et al., 2007) illustrate a computer-based sim-

ulation technique for evaluating USAR robots us-

ing USARSim, a robot simulation tool. (Ferworn

et al., 2013) also provide virtual urban disaster mod-

els for simulating USAR robots. These models are

extracted from actual urban disaster scenes. The

computer-based simulation is ﬂexible, repeatable, and

accurate compared with physical test ﬁelds/arenas;

however, it lacks physical ﬁdelity. Both techniques,

test ﬁelds/arenas and computer-based simulation, also

limit possible behavioral scenarios that may occur in

urban disasters.

3 CASE STUDY DESCRIPTION

For modeling an urban disaster world, we use the

Reference Test Arena for Autonomous Mobile Robots

(Jacoff et al., 2003) because it provides a clear de-

scription for urban disaster actors. The urban disaster

actors include victims, constructions, rubble, debris,

and hazards. The victims include humans and ani-

mals. They can be survivors or non-survivors. These

victims can be visual, not visual, accessible, inac-

cessible, trapped, and entombed. Only the survivors

show signs of life (motion, body heat, sound, CO

emission). The survivors have different conditions,

aware, semi-conscious, and unconscious. The non-

survivors do not stimulate any signs but can be recog-

nized visually. The constructions are infrastructural

objects including posts, stairs, elevators, walls, doors,

windows, and ramps. These constructions may be in-

accessible. They also have changeable states such as

unsteady posts, loose walls, sloppy stairs, unsafe el-

evators. The posts can be placed as ﬂat, hill, and di-

agonal. The ramps can also be pitch and roll. The

Figure 1: Urban disaster, Kathmandu, Nepal, May 2015.

rubble consist of rocks, bricks, and concretes. The de-

bris are furniture, woods, ropes, wires, pipes, papers,

glasses, and plastics. These debris are usually piled

together. The rubble and debris block passages and

increase orientation complexity. These rubble and de-

bris also have changeable states as well. For example,

a brick tumbles over a collapse then stops. An urban

disaster scene may include all of these urban disaster

actors as showing in Figure 1. The hazards also can

occur when a collapse happens. Examples of the haz-

ards are high temperature, electricity, and explosion.

Regarding to the hazard severity, these hazards may

not be manipulatable. The hazard severity has ﬁve

levels (minimal, slight, moderate, serious, and sever).

4 APPROACH

Our objective is to apply a systematic model-based

test generation approach (Andrews et al., 2015) to

generate test cases from an active world model that

represents urban disaster actors of USAR robots. We

build the active world model in two steps. First, we

construct a structural model of disaster actors to rep-

resent their attributes, functions and relations. Sec-

ond, we construct the behavioral model to describe

the disaster actors’ possible states and transitions and

their interactions. Each disaster actor is presented by

Figure 2: World Behavioral Test Generation Process.

one behavioral model showing its behaviors. The in-

teractions between these disaster actors represent the

active world model. Since the disaster actors are inter-

acting simultaneously, the active world model should

cover not only the internal transitions of these disas-

ter actors, but also the interactions between them. The

active world model can then be leveraged to generate

world behavioral test cases. Once we build the active

world model, any member of the graph-based testing

criteria from (Ammann and Offutt, 2008) and (Lill

and Saglietti, 2012) can be used to generate abstract

behavioral test paths, which are AWBTCs. Finally,

we generate test-data by input-space partitioning to

transform the generated AWBTCs into executable test

cases. The test generation process is illustrated in Fig-

World Model for Testing Urban Search and Rescue (USAR) Robots using Petri Nets

665

ure 2. The World Model-based Test Generation pro-

cess follows three phases:

• Model the active world by constructing structural

and then behavioral models.

• Select proper graph-based coverage criteria for

test-path generation and proper input-space par-

titioning coverage criteria for test-data.

• Generate AWTCs which are concurrent test paths

and then generate test-data to transform these

concurrent test paths into executable test cases.

4.1 Phase 1: World Models

4.1.1 Structural Model

The structural model is constructed using a UML

class diagram, where classes represent the urban dis-

aster actors including their important characteristics,

behavioral messages, and relationships. An urban dis-

Figure 3: Structural Model for Urban Disaster Scenes.

aster scene can be represented by a single snippet

”Urban Disaster Scene”. The urban disaster actors

are of six types: survivors, non-survivors, construc-

tions, rubble, debris, and hazards. The UML class

diagram that represents the urban disaster scene is

shown in Figur 3. The urban disaster actors are ag-

gregated into a Urban Disaster Scene snippet. Similar

actors are generalized to a single class. For instance,

Survivors and Non-survivors are generalized into Vic-

tim class. The number of involved disaster actors in

the Urban Disaster Scene is determined by their mul-

tiplicity relationship. Instances of the urban disaster

actors and examples of their behavioral messages are

illustrated in Table 1. We assume that only the USAR

robots perform in a snippet; these robots are not con-

sidered world actors as they are the SUT.

4.1.2 Behavioral Model

Although a wide range of behavioral models ex-

ists, we illustrate the behavioral model using Petri

Nets (PNs). The strength of PNs is that they can

Table 1: Instances of Urban Disaster Actors.

Class Actor Instance Behavioral Messages

Survivor human (child,

adult, and el-

derly) and an-

imal (pets)

child.stimulate(){return CO

}

child.isVisual(){return true}

child.getCondition(){return

trapped}

child.isAccessible(){return

true}

Non-

survivor

human body

and part.

body.isVisual(){return true}

body.isAccessible(){return

false}

body.getDescription(){return

”entombed body”}

Construction post, wall,

stair, elevator,

door, window,

and ramp

post.getState(){return ”hill &

safe”}

post.change(”diagonal &

unsafe”)

stairs.getState(){return

”sloppy”}

door.getState(){return ”acces-

sible”}

Rubble rock, brick,

and concrete.

rock.getState(){return ”tum-

bling”}

rock.getState(){return

”steady”}

concrete.getState(){return

”sloppy”}

Debris furniture,

pipe, wood,

rope, paper,

glass, and

plastic

table.getState(){return

”steady”}

pipe.getState(){return ”un-

steady”}

wood.getState(){return ”un-

steady”}

Hazard temperature,

electricity,

ﬁre, and

explosion

temp.getSeverity(){return

”slight”}

elect.isManipulatable(){return

true}

model the functional behaviors of systems that are

characterized as being concurrent, asynchronous, dis-

tributed, parallel, nondeterministic, and/or stochastic

(Murata, 1989) and (David and Alla, 1994). For-

mally, a Petri Net is a 5-tuple, PN = (P, T, F, W,

) where P = {p

, p

, . . . , p

} is a ﬁnite set of

places, T = {t

, t

, . . . , t

} is a ﬁnite set of transitions,

F ⊆ (P× T)∪ (T × P) is a set of arcs (ﬂow relation),

W : F → {1, 2, 3, . . . } is a weight function, M

: P ⊆

{0, 1, 2, 3, . . . } is the initial marking (the initial sys-

tem state), P ∩ T = φ and P ∪ T 6= φ. A Petri Net

structure N = (P, T, F, W) without any speciﬁc initial

MODELSWARD 2016 - 4th International Conference on Model-Driven Engineering and Software Development

666

marking is denoted by N. A Petri Net with the given

initial marking is denoted by (N, M

). Modeling be-

havioral models follows two steps. First, each urban

disaster actor is modeled individually. Figure 4 shows

a set of PNs that represent a group of urban disaster

actors, so-called actors processes. It is clear that these

actors processes act independently and concurrently.

As shown in Figure 4, in the top model, the survivor

process stimulates signs of life, or visuality. The non-

Figure 4: Behavioral Models for Urban Disaster Actors.

survivor process stimulates only visuality. The con-

struction process displays its state. The construction

state changes unpredictably. It is similar to other ac-

tors’ processes. The key difference is that the behav-

ioral messages of these processes are dissimilar. All

Figure 5: Behavioral Model (High-Level).

actor processes also report either the actors are acces-

sible or inaccessible. The hazard process reports the

hazard severity. The hazard also is considered as ma-

nipulatable or not manipulatable. Secondly, these be-

havioral models that represent the disaster actors are

linked together into a higher level behavioral model

which describes the interactions among these actors.

Figure 5 illustrates the high level behavioral model.

All disaster actors can interact with each other except

the non-survivor, it can only be affected by others.

4.2 Phase 2: Coverage Criteria

An active world behavioral model is de-

ﬁned as a collection of concurrent processes

AWM = {PN

, PN

, . . . , PN

} where i is the number

of actors that share a snippet. Model-ﬂow coverage

criteria such as node coverage and edge coverage

(Ammann and Offutt, 2008) and (Lill and Saglietti,

2012), can be applied. Using any of a number of

test path generation techniques, test paths that fulﬁll

these coverage criteria can be generated. Let TP

(tp

, tp

, ..., tp

) be a set of such internal test paths

that cover a process, PN

, and k is the number of

these paths. These internal paths describe the internal

execution of the processes. We use transition-based

coverage (Lill and Saglietti, 2012) to generate inter-

nal paths that cover the processes of disaster actors.

The internal test paths that cover the disaster actors

processes are illustrated in Table 2. Transition-based

Coverage Criterion (TBCC) is deﬁned as that given

a set of input-places P

and output-places P

out

for

TBCC, a test set TP

satisﬁes TBCC on a process PN

if and only if every p

∈ P

and p

out

∈ P

out

, there

is at least one internal test path tp

∈ TP

such that

enables a sequence of transitions involved in the

execution of process PN

to be ﬁred. The internal test

path sets, (TP

, TP

), interact

concurrently with each other via the exchange of

behavioral messages (i.e., interaction messages).

These interaction messages represent the high level

of execution behavior of the active world model, as

shown in behavioral model (High-Level), in Figure

5. The interaction among the processes can be

covered by interaction test paths which represent the

possibilities of execution behavior of these processes.

To avoid cyclic paths, simple-path coverage criterion

(Ammann and Offutt, 2008), is used to generate the

interaction test paths that cover the high level of

the behavioral model. The high level of the active

world model of the disaster actors is covered by 12

interaction test paths (ITP

, ITP

, . . . , ITP

). Each

interaction test path combines the internal test paths

of the processes that are involved in the interaction

scenario. For instance, the interaction test path

ITP

= (Construction : TP

) −→ (Hazard : TP

) −→

World Model for Testing Urban Search and Rescue (USAR) Robots using Petri Nets

667

Table 2: Internal Test Path Sets.

1. survivor process, TP

= {tp

, tp

}

: unrealized

su.isVisual()

−−−−−−−→ realized

su.getCondition()

−−−−−−−−−−→

reported

su.isAccessible()

−−−−−−−−−→ realized

: unrealized

su.stimulate()

−−−−−−−−→ realized

su.getCondition()

−−−−−−−−−−→

reported

su.isAccessible()

−−−−−−−−−→ realized

2. non-survivor process, TP

= {tp

}

: unobserved

nsu.isVisual()

−−−−−−−→ observed

nsu.getDescription()

−−−−−−−−−−−→

described

nsu.isAccessible()

−−−−−−−−−−→ observed

3. construction process, TP

= {tp

}

: unspecified

cons.getState()

−−−−−−−−→ specified

cons.change(newState)

−−−−−−−−−−−−−→

specified

4. rubble process, TP

= {tp

}

: unstated

rubble.getState()

−−−−−−−−−→ stated

rubble.change(newState)

−−−−−−−−−−−−−−→

stated

5. debris process, TP

= {tp

}

: unnoticed

debris.getState()

−−−−−−−−−→ noticed

debris.change(newState)

−−−−−−−−−−−−−−→

noticed

6. hazard process, TP

= {tp

}

: unidenti fied

hazard.getSeverity()

−−−−−−−−−−−→

identified

hazard.isManipulatable()

−−−−−−−−−−−−−−→

declared

hazard.change(newSeverity)

−−−−−−−−−−−−−−−→ identified

(Survivor : TP

) covers the interaction between

the construction process, the hazard process, and

the survivor process. Thus, the internal paths

of this interaction are (tp

→ tp

) and

(tp

→ tp

). The interaction test paths

are considered concurrent paths. Therefore, the

concurrent interaction between internal test paths that

represent multiple processes produces an arbitrary

combination of internal paths. In order to cover all

possible combinations of internal paths, path combi-

nation coverage criteria should determine what com-

binations are required. Let (TP

, TP

, . . ., TP

) be

sets of internal test paths (nodes in ITP

) where TP

{tp

, tp

, . . ., tp

}, TP

= {tp

, tp

, . . ., tp

. . . , and TP

= {tp

, tp

, . . ., tp

}. Then, the

selection of tp

from TP

and tp

from TP

is called a path combination. Let len(tp) be the

number of nodes in tp, the length of tp. The

combination set for interaction test path ITP

Comb

ITP

= {(tp

, . . ., p

)|tp

∈ TP

, m =

len(ITP

), n = |TP

|, 1 ≤ j ≤ m, 1 ≤ k ≤ n}. The

number of all path combinations of ITP

will be

the product of the number of internal paths of

each TP

. Each combination introduces a set of

concurrent test paths. The path combination sets

do not show how these paths interact concurrently.

Therefor, we use the Rendezvous coverage crite-

rion (RCC), as in (Yang and Chung, 1990). The

test requirements contain a set of all paths that

have rendezvous nodes. The possible number of

rendezvous-paths of the interaction test path ITP

∏

i=1

(TP

+ 1) − 1. However, the generated concur-

rent test paths are still abstract. To transform

these concurrent test paths into executable test cases,

test-data coverage criteria, i.e. input-space parti-

tioning (Ammann and Offutt, 2008), are required.

We divide a collection of values (input-domain) into

test-data blocks that make the concurrent test paths

executable. The input-domain is the possible values

that input parameters (tokens in Petri Nets) can have.

The urban disaster snippet has ten input-domains:

survivor stimuli, survivor conditions, construction

types, construction states, rubble types, rubble states,

debris types, debris states, hazard types, and hazard

severity. The survivorstimuli block includes {motion,

body heat, sound, and CO

emission}. The survivor

conditions block consists of {aware, semi-conscious,

and unconscious}. The construction types block

consists of {posts, stairs, elevators, walls, doors,

windows, and ramps} while the rubble types block

consists of {rocks, bricks, and concretes}. The debris

types block includes {furniture, woods, ropes, wires,

pipes, papers, glasses, and plastics}. The construc-

tion states block consists of {ﬂat, hill, diagonal,

pitch, roll, steady, unsteady, loose, sloppy, safe, and

unsafe}. In addition to the construction states block

values, the rubble and debris states blocks contain

{piled, scattered, tumbled}. The hazards block

includes {exothermic-ﬁre, high-temperature, high-

voltage, and gas explosion}. The hazard severity

block consists of {minimal, slight, moderate, serious,

and sever}. We use All Combinations Coverage

(ACoC) which exercises all possible combinations

of test-data. The number of the test-data sets that

satisfy ACoC is

∏

i=1

), where B

is a block of values

for a parameter and Q is the number of parameters.

To compare with ACoC, we also use Each Choice

Coverage (ECC) that selects one value only from

each block of values. The number of the test-data

sets that satisfy ECC is MAX

i=1

) (Ammann and

Offutt, 2008).

4.3 Phase 3: Test Generation

The path combinations are represented as an ordered

references of internal test paths of the processes in-

volved in the execution. These combinations may

result in a huge number of concurrent test paths,

AWBTCs, although not all of these concurrent test

paths are feasible. We used the serialization algo-

rithm in (Yang and Chung, 1990) to generate these

concurrent test paths. The concurrent test paths

are serialized nodes of the internal paths. We ex-

MODELSWARD 2016 - 4th International Conference on Model-Driven Engineering and Software Development

668

pressed the concurrency of test paths using double-

bar ”||” as used in LOTOS for deﬁning concurrent

function (Sighireanu et al., 2000). For instance,

ITP

= (Construction : TP

) −→ (Hazard : TP

) −→

(Survivor : TP

) generates two AWBTCs. When we

impose the RCC on these AWBTCs, it produces 11

rendezvous-path. We apply ACoC and ECC coverage

criteria to select test-data that fulﬁll these AWBTCs.

For each interaction test path ITP

, a set of test-data

is selected from blocks that are only related to the

actor processes involved in this ITP

. For instance,

for ITP

, six blocks (construction types, construction

states, hazard types, hazard severity, survivor stimuli,

and survivor conditions) are used to select test-data.

The ACoC results 18480 test-data set while the ECC

produces 11 test-data set. When this 18480 test-data

set is fulﬁlled in the 11 rendezvous-paths, this trans-

formation results 203280 executable test cases. Nev-

ertheless, the number of executable test cases gener-

ated by ECC with RCC is 121 test cases.

5 REACHABILITY & CRITERIA

EFFICIENCY

To perform reachability analysis on the behavioral

models, we use Platform-Independent Petri Net Ed-

itor (PIPE) (Dingle et al., 2009). For reachability

analysis, the reachability graph module that PIPE pro-

vides is used to generate all possible states that a sys-

tem can reach. For six disaster actor processes, the

generated reachability graph consists of 216 states in-

tensively interconnected by 1161 arcs. However, the

number of reachable states grows exponentially as the

number and the size of processes increase. For in-

stance, when these six processes are replicated to 12

processes, the number of reachable states expended to

13824 states with 161280 arcs which displaying them

in PIPE may not be feasible. However, CADP (Con-

struction and Analysis of Distributed Processes) tool-

box (Garavel et al., 2013) is more practicable due to

its scalability. CADP is scalable up to 10

node. In-

stead of investigating reachability, one can also use

Markov chain techniques for probabilistic. For cov-

erage criteria efﬁciency, we exercise all interaction

test paths, (ITP

, ITP

, . . . , ITP

). The total number

of executable test cases generated by the RCC with

ACoC is 58841115 test case while the RCC with ECC

generates 1878 test case. It is clear that using RCC

with ACoC is not feasible but it is considered as an

upper bound. Nevertheless, exploiting RCC on ECC

shows feasibility and efﬁciency.

6 CONCLUSION AND FUTURE

WORK

This paper presented the applicability of a model-

based test generation approach (Andrews et al., 2015)

that allows testing of autonomous systems in their ac-

tive world. We modeled an active world of the USAR

robots. A test generation process is applied. RCC

is used to generate AWBTCs. To transform the gen-

erated AWBTCs into executable test cases, we also

exploited ACoC and ECC coverage criteria to gener-

ate test-data. The ﬁndings show that exploiting RCC

on ECC is practically feasible. The PIPE tool is used

for reachability analysis. However, a display in PIPE

is no longer easily readable. Future work will ex-

periment using the CADP (Construction and Analy-

sis of Distributed Processes) toolbox (Garavel et al.,

2013). Future work will also explore other testing

techniques that can handle the scalability of concur-

rent processes. Comparison between experiments will

also be provided.

ACKNOWLEDGEMENTS

This work was supported, in part, by NSF IUCRC

grant # 0934413, 1127947, and 1332078 to the Uni-

versity of Denver.

REFERENCES

Ammann, P. and Offutt, J. (2008). Introduction to Software

Testing. Cambridge University Press, 32 Avenue of

the Americas, New York, NY 10013, USA, ﬁrst edi-

tion.

Andrews, A., Abdelgawad, M., and Gario, A. (2015).

Towards world model-based test generation in au-

tonomous systems. In Proceedings of the 3rd In-

ternational Conference on Model-Driven Engineer-

ing and Software Development (MODELSWARD)

2015, MODELSWARD 2015, pages 165–176.

SCITEPRESS Digital Library.

Chiou, A. and Wynn, C. (2009). Urban search and res-

cue robots in test arenas: Scaled modeling of dis-

asters to test intelligent robot prototyping. In Sym-

posia and Workshops on Ubiquitous, Autonomic and

Trusted Computing, UIC-ATC, pages 200–205.

David, R. and Alla, H. (1994). Petri nets for modeling of dy-

namic systems: Survey. Automatica, 30(2):175–202.

Dias-Neto, A., Subramanyan, R., Vieira, M., and Travas-

sos, G. H. (2007). A survey on model-based test-

ing approaches: A systematic review. In Proceedings

of the 1st ACM International Workshop on Empirical

Assessment of Software Engineering Languages and

Technologies, WEASELTech ’07, pages 31–36. ACM.

World Model for Testing Urban Search and Rescue (USAR) Robots using Petri Nets

669

Dingle, N. J., Knottenbelt, W. J., and Suto, T. (2009).

PIPE2: A tool for the performance evaluation of gen-

eralised stochastic petri nets. SIGMETRICS Perform.

Eval. Rev., 36(4):34–39.

Ferworn, A., Herman, S., Tran, J., Ufkes, A., and Mcdon-

ald, R. (2013). Disaster scene reconstruction: Mod-

eling and simulating urban building collapse rubble

within a game engine. In Proceedings of the 2013

Summer Computer Simulation Conference, SCSC ’13,

pages 18:1–18:6, Vista, CA. Society for Modeling

& Simulation International.

Furda, A. and Vlacic, L. (2010). An object-oriented de-

sign of a world model for autonomous city vehicles.

In Intelligent Vehicles Symposium (IV), IEEE, pages

1054–1059.

Garavel, H., Lang, F., Mateescu, R., and Serwe, W. (2013).

CADP 2011: a toolbox for the construction and anal-

ysis of distributed processes. International Journal

on Software Tools for Technology Transfer, 15(2):89–

107.

Hessel, A., Larsen, K., Mikucionis, M., Nielsen, B., Pet-

tersson, P., and Skou, A. (2008). Formal methods and

testing. chapter Testing real-time systems using UP-

PAAL, pages 77–117. Springer-Verlag, Berlin, Hei-

delberg.

Iqbal, M., Arcuri, A., and Briand, L. (2010). Environ-

ment modeling with UML/MARTE to support black-

box system testing for real-time embedded systems:

Methodology and industrial case studies. In Model

Driven Engineering Languages and Systems, volume

6394 of Lecture Notes in Computer Science, pages

286–300. Springer Berlin Heidelberg.

Iqbal, M., Arcuri, A., and Briand, L. (2012). Empiri-

cal investigation of search algorithms for environment

model-based testing of real-time embedded software.

In Proceedings of the 2012 International Symposium

on Software Testing and Analysis, ISSTA 2012, pages

199–209, New York, NY, USA. ACM.

Jacoff, A., Messina, E., Weiss, B., Tadokoro, S., and Naka-

gawa, Y. (2003). Test arenas and performance metrics

for urban search and rescue robots. In Proceedings

of the IEEE International Conference on Intelligent

Robots and Systems (IROS), volume 4, pages 3396–

3403 vol.3.

Li, J. and Wong, W. (2002). Automatic test generation

from communicating extended ﬁnite state machine

(CEFSM)-based models. In Proceedings of 5th IEEE

International Symposium on Object-Oriented Real-

Time Distributed Computing. (ISORC 2002), pages

181–185.

Lill, R. and Saglietti, F. (2012). Model-based testing of

autonomous systems based on coloured petri nets. In

ARCS Workshops (ARCS), pages 1–5.

Liu, Y. and Nejat, G. (2013). Robotic urban search and res-

cue: A survey from the control perspective. Journal

of Intelligent and Robotic Systems, 72(2):147–165.

Murata, T. (1989). Petri nets: Properties, analysis and ap-

plications. Proceedings of the IEEE, 77(4):541–580.

Murphy, R., Tadokoro, S., Nardi, D., Jacoff, A., Fiorini,

P., Choset, H., and Erkmen, A. (2008). Search and

rescue robotics. In Siciliano, B. and Khatib, O., ed-

itors, Springer Handbook of Robotics, pages 1151–

1173. Springer Berlin Heidelberg.

Pepper, C., Balakirsky, S., and Scrapper, C. (2007). Robot

simulation physics validation. In Proceedings of the

2007 Workshop on Performance Metrics for Intelli-

gent Systems, PerMIS ’07, pages 97–104, New York,

NY, USA. ACM.

Shirole, M. and Kumar, R. (2013). Uml behavioral model

based test case generation: A survey. Softw. Eng.

Notes, SIGSOFT, 38(4):1–13.

Sighireanu, M., Chaudet, C., Garavel, H., Herbert, M., Ma-

teescu, R., and Vivien, B. (2000). Lotos NT user man-

ual.

Tretmans, J. (2008). Model based testing with labelled

transition systems. In Hierons, R., Bowen, J., and

Harman, M., editors, Formal Methods and Testing,

volume 4949 of Lecture Notes in Computer Science,

pages 1–38. Springer Berlin Heidelberg.

Utting, M., Pretschner, A., and Legeard, B. (2012). A

taxonomy of model-based testing approaches. Softw.

Test. Verif. Reliab., 22(5):297–312.

Williamson, D. and Carnegie, D. (2007). Toward hierar-

chical multi-robot urban search and rescue: Develop-

ment of a ’mother’ agent. In Mukhopadhyay, S. and

Gupta, G., editors, Autonomous Robots and Agents,

volume 76 of Studies in Computational Intelligence,

pages 1–7. Springer Berlin Heidelberg.

Yang, R. and Chung, C.-G. (1990). A path analysis ap-

proach to concurrent program testing. In Proceedings

of the 9th Annual International Phoenix Conference

on Computers and Communications, pages 425–432.

Zander, J., Schieferdecker, I., and Mosterman, P. J. (2012).

Model-based testing for embedded systems. CRC

Press, 6000 Broken Sound Parkway NW,Boca Raton,

FL 3348, USA, ﬁrst edition.

MODELSWARD 2016 - 4th International Conference on Model-Driven Engineering and Software Development

670