USING A TWO-WAY BALANCED INCOMPLETE BLOCK
DESIGN TO COMPARING AN AGENT-ORIENTED SOFTWARE
ENGINEERING METHODOLOGIES
Faezeh Parandoosh and Siavosh Kaviani
Department of Computer Science, Payamenour University, Tehran, Iran
Keywords: Incomplete block design, Agent, Agent-Oriented Software Engineering (AOSE), Agent-Oriented
methodologies, MaSE, Prometheus, Tropos and Gaia.
Abstract: There has been a surge of interest in agent-oriented software engineering in recent years. Numerous
methodologies for developing agent-based systems have been proposed in the literature and the area of
agent-oriented methodologies is maturing rapidly. Evaluating methodologies' strengths, weaknesses and
domains of applicability plays an important role in improving them and in developing the "next-generation"
of methodologies. In this paper, we present a reliable framework that adopts statistical techniques to
compare agent-oriented methodologies. Based upon this framework we performed a comparison of four
AOSE methodologies MaSE, Prometheus, Tropos and Gaia.
1 INTRODUCTION
Agent-oriented techniques represent an exciting new
means of analyzing, designing and building complex
software systems. They have the potential to
significantly improve current practice in software
engineering and to extend the range of applications
that can feasibly be tackled.
"One of the most fundamental obstacles to large-
scale take-up of agent technology is the lack of
mature software development methodologies for
agent-based systems." (Luck,
McBurney & Preist,
2003, p.11).
Even though AOSE methodologies have been
proposed, few are mature or described in sufficient
detail to be of real use. In fact, the area of agent-
oriented methodologies is maturing rapidly and that
the time has come to begin drawing together the
work of various research groups with the aim of
developing the next generation of agent-oriented
software engineering methodologies (Castro, Kolp
& Mylopoulos, 2002; Bersciani et al. n.d.).
An important step is to understand the
differences between the various key methodologies,
and to understand each methodology's strengths,
weaknesses, and domains of applicability.
In this paper we perform the comparison on
several well-known methodologies. A qualified
methodology has been selected based on the
availability of the documentation that describes it,
the familiarity of the agent community with it, and
its domain of applicability. As a result, the following
9 methodologies have been selected as treatments to
our experiment: Gaia, MaSE, Prometheus, Tropos,
MAS-CommonKADS, MESSAGE, FIPA-OS, JiVE
and CNFM (Elamy and Far, 2005). In this paper we
perform a comparison on four well-known AOSE
methodologies MaSE, Prometheus, Tropos and
Gaia.
In section 2, we briefly introduce these
methodologies. In section 3, we describe a
framework for comparing AOSE methodologies. We
then (section 5) select the Appropriate Statistical
Model techniques to compare AOSE methodologies.
In section 6 we apply the framework to compare the
methodologies.
2 THE METHODOLOGIES
2.1 MaSE
Multi agent Systems Engineering (MaSE) (Scott et
al.2001) is an agent-oriented software engineering
methodology which is an extension of the object-
oriented approach. As a software engineering
56
Parandoosh F. and Kaviani S. (2007).
USING A TWO-WAY BALANCED INCOMPLETE BLOCK DESIGN TO COMPARING AN AGENT-ORIENTED SOFTWARE ENGINEERING METHOD-
OLOGIES.
In Proceedings of the Second International Conference on Evaluation of Novel Approaches to Software Engineering , pages 56-65
DOI: 10.5220/0002585200560065
Copyright
c
ī€ SciTePress
methodology, the main goal of MaSE is to provide a
complete-lifecycle methodology to assist system
developers to design and develop a multi-agent
system. It fully describes the process which guides a
system developer from an initial system
specification to system implementation. This process
consists of seven steps, divided into two phases.
The MaSE Analysis stage includes three smaller
process steps. First, the Capturing Goals step guides
the analysts to identify goals and structure and
represent them as a goal hierarchy. This goal model
is a product of a goal decomposition process in that
goals are broken down in subgoals, subgoals to sub-
subgoals and so on. The second step involves Use
Cases, a technique which is commonly found in
object-oriented methodologies. It includes extracting
main scenarios from the initial system context or
copying them from it if they exist. The use cases
should show how a goal can be achieved during a
normal system operation as well as erroneous
conditions. The second part of this step is to apply
those use case. Firstly, an initial set of roles is
identified based on goals and use cases scenarios.
Secondly, the sequence of events that occur in the
interaction or communication between roles is
represented in a Sequence Diagram. This model is
also analogous to UML sequence diagrams except
that entities are roles rather than objects. Refining
Roles is the final step of the Analysis phase where a
Role Model and a Concurrent Task Model are
constructed. The Role Model describes the roles in
the system. It also depicts the goals which those
roles are responsible for, the tasks that each role
performs to achieve its goals and the communication
path between the roles. Tasks are then graphically
represented in fine-grained detail as a series of finite
machine automata in the Concurrent Task Model.
The first step of the Design Phase is called
"Creating Agent Classes". The output of this step is
an Agent Class Diagram which describes the entire
multi-agent system. The agent class diagram shows
agents and the roles they play. Links between agents
show conversations and are labelled with the
conversation name. The details of the conversations
are described in the second step of the design phase
("Constructing Conversations") using
communication class diagrams. These are a form of
finite state machine. The third step of the Design
stage is Assembling Agent Classes. During this step,
we need to define the agent architecture and the
components that build up the architecture. In terms
of agent architecture, MaSE does not dictate any
particular implementation platform. The fourth and
final step of the design phase is System Design. It
involves building a Deployment Diagram which
specifies the locations of agents within a system.
MaSE has extensive tool support in the form of
agent Tool. Its latest version 2.0 implements all
seven steps of MaSE.
Figure 1: Relationship between Gaia's models.
2.2 Gaia
Gaia is one of the first methodologies which is
specifically tailored to the analysis and design of
agent-based systems. Its main purpose is to provide
the designers with a modeling framework and
several associated techniques to design agent-
oriented systems.
The Gaia methodology is both general, in that it
is applicable to a wide range of multi-agent systems,
and comprehensive, in that it deals with both the
macro-level (societal) and the micro-level (agent)
aspects of systems. Gaia separates the process of
designing software into two different stages:
analysis and design (Wooldridge, Jennings & Kinny,
1999; Wooldridge, Jennings & Kinny, 2000).
Analysis involves building the conceptual
models of the target system, whereas the design
stage transforms those abstract constructs to
concrete entities which have direct mapping to
implementation code. Figure 1 depicts the main
artifacts of each stage: Role Model and Interaction
Model (Analysis), and Agent Model, Services Model,
and Acquaintance Model (Design).
2.3 Prometheus
The Prometheus methodology is a detailed AOSE
methodology that is aimed at non-experts. It has
been successfully taught to and used by
undergraduate students. Prometheus consists of three
phases: system specification, architectural design,
and detailed design (Giorgini & Henderson-Sellers,
2005; Padgham & Winikoff, 2002).
USING A TWO-WAY BALANCED INCOMPLETE BLOCK DESIGN TO COMPARING AN AGENT-ORIENTED
SOFTWARE ENGINEERING METHODOLOGIES
57
The system specification is the first phase of
Prometheus. Its main purpose is building the
system's environment model, identifying the goals
and functionalities of the system, and describing key
use case scenarios.
The architectural design is the second phase of
Prometheus. The three main activities involved in
this stage are: defining agent types, designing the
overall system structure, and defining the interaction
between agents.
The internals of each agent and how it will
accomplish its tasks within the overall system are
addressed in the detailed design phase. It focuses on
defining capabilities, internal events, plans and
detailed data structure for each agent type identified
in the previous step. (Figure 2)
Prometheus is supported by two tools. The
JACK Development Environment (JDE), developed
by Agent Oriented Software (www.agent-
software.com) includes a design tool that allows
overview diagrams to be drawn. These are linked
with the underlying model so that changes made to
diagrams, for example adding a link from a plan to
an event, are reflected in the model and in the
corresponding JACK code. The Prometheus Design
Tool (PDT) provides forms to enter design entities.
It performs cross checking to help ensure
consistency and generates a design document along
with overview diagrams. Neither PDT nor the JDE
currently support the system specification phase.
2.4 Tropos
Tropos is an agent-oriented software development
methodology created by a group of authors from
various universities in Canada and Italy (Bersciani et
al. 2004; Fatemi, NematBakhsh & TorkLadani,
2006; Giunchiglia, Mylopoulos & Perini, 2002;
Leskowsky & Anderson, 2003). Tropos is based on
two key ideas. First, the notion of agent and all
related mentalistic notions (for instance goals and
plans) are used in all phases of software
development, from early analysis down to the actual
implementation. Second, Tropos covers also the
very early phases of requirements analysis, thus
allowing for a deeper understanding of the
environment where the software must operate, and
of the kind of interactions that should occur between
software and human agents. The Tropos language
for conceptual modeling is formalized in a
metamodel described with a set of UML class
diagrams.
One of the significant differences between
Tropos and the other methodologies is its strong
focus on early requirements analysis where the
domain stake-holders and their intentions are
identified and analyzed. This analysis process allows
the reason for developing the software to be
captured. The software development process of
Tropos consists of five phases: Early Requirements,
Late Requirements, Architectural Design, Detailed
Design and Implementation.
Figure 2: Overview of 3 Phases in Prometheus.
3 A COMPARISON
FRAMEWORK
In this section, we briefly describe a methodology
evaluation framework within which the
methodology comparison is conducted.
The comparison framework covers nine major
aspects of each AOSE methodology: Concepts,
Upgradeability, Modeling language, Basic
properties, Mental attitudes, Modeling, Process,
Pragmatics and Management. This framework is
adapted from a framework proposed in (Fatemi,
NematBakhsh & TorkLadani, 2006; Giorgini &
Henderson-Sellers, 2005) for comparing Agent-
Oriented Methodologies.
3.1 Concepts
Agent-oriented concepts are of great importance for
agent-oriented methodologies in general and for
agent-oriented modeling languages in particular.
Usability- to what extend the methodology is easy to
use and implement, Modeling- to what extend
concepts can be expressed in multiple
models/diagrams; Overloading- to what extend
concepts can be overloaded; Notation- to what
extend notations are semantically and syntactically
clear and simple across models?
ENASE 2007 - International Conference on Evaluation on Novel Approaches to Software Engineering
58
3.2 Upgradeability
Mobility- how capable is the methodology in
modeling agent migration? Scalability- This
measures the methodology's support for designing
systems that are scalable. It means that the system
should allow the incorporation of additional
resources and software components with minimal
user disruption. Open Systems Support- to what
extend it can provide support for open systems to
allow dynamic integration/removal of new
agents/resources; Distribution- This criterion
measures the methodology's support for designing
distributed systems. It means the methodology
should provide mechanisms, including techniques
and models, to describe the configuration of
processing elements and the connection between
them in the running system. It shows not only the
physical of the different hardware components that
compose a system, but also the distribution of
executable programs on this hardware. More
specifically, such models need to depict the
deployment of agents over the network. Dynamic
Structure- to what extend it can provide support for
dynamic system reconfiguration when agents are
created or destroyed.
3.3 Modeling Language
Clarity and understandability- These two criteria are
closely related to each other and both of them are
fundamental requirements of a modeling language.
In fact, a methodology which provides clear
notations tends to increase the users'
understandability of the models. Consistency- to
what extend it satisfies modeling consistency (i.e. no
individual requirement is in conflict); Unambiguity-
symbols and syntax are provided to users so that
they can build a representation of a particular
concept. Thus, the semantic or meaning of a concept
is the users' interpretation of the representation
provided. However, this interpretation can be
different from observer to observer, which in turn
results in misunderstandings. Therefore, it is
important to make sure that a constructed model can
be interpreted unambiguously. Traceability- There
are relationships between models and between
models and the requirements of the target system.
Traceability requires that it has to be easy for the
designers and the audiences of the design documents
to understand and trace through the models. This
may increase the users' understanding of the system.
Tracing backwards and forwards between models
and stages also allow the users to verify that all the
requirements of the system are addressed during the
analysis and design stages. Traceability also assists
the designer produce new models by referring to the
models that have been previously constructed. A
result of doing this may be increased productivity in
the sense that information gathered from one model
can be used to construct others. Usability-It is
important for a modeling language not only to be
understandable to the users but also to be easy to
use. The first step toward using a modeling language
is to learn the notation. Hence, it is desirable that the
notation be easy to learn by both expert and novice
users. In addition, the easier the users can remember
the notation, the quicker they are able to learn to use
it. Therefore, the notation should be as simple as
possible. Furthermore, since people usually sketch
models by hand during the process of brainstorming
or reviewing designs, it is essential for the notation
be easy to draw and write by hand. Finally, as
mentioned earlier, one of the important purposes of a
modeling language is to convey information among
the users. Often this is in the form of hardcopy
documentation for reading and discussing. Hence, it
is important that the diagrams produced are easy to
read and comprehend when printed.
3.4 Basic Properties (Weak Notation)
Autonomy- Agents can operate and make their own
decision on which action they should take without
direct intervention of humans or other agents. In
other words, both agents' internal state and their own
behaviour are controlled by themselves. We want to
know does the methodology support modeling a
decision-making mechanism of agents regardless of
the environment or does the methodology support
describing an agent's self-control features? For
example, functionalities and tasks being
encapsulated within an agent may increase the
degree of autonomy. Reactivity/Proactivity- Pro-
activeness is an agent's ability to pursue goals over
time and Reactivity an agent's ability to respond in a
timely manner to changes in the environment, so the
degree of allowing reactivity/proactivity is very
important; Sociability- to what degree the
methodology provides organized relationships
among agents, represents agentsā€™ commitments and
interfaces with other entities; Adaptability- to what
degree agents are flexible to adjust their activities to
environmental changes; Concurrency- agent's ability
to deal with multiple goals and/or events at the same
time. More specifically, agents are able to perform
actions/tasks or interact with other agents
simultaneously. We want to know what extend
USING A TWO-WAY BALANCED INCOMPLETE BLOCK DESIGN TO COMPARING AN AGENT-ORIENTED
SOFTWARE ENGINEERING METHODOLOGIES
59
agents performs several tasks simultaneously;
Interactions- to what extend agents can interact with
other agents, and with environment; Human
Computer Interaction to what degree the
methodology is capable to construct models to
represent user interfaces and system-user
interaction?
3.5 Mental Attitudes (Strong Notation)
This feature relates to the strong agency definition of
agents. The three major elements of the Belief-
Desire-Intention (BDI) architecture of agents are an
example of it. The BDI architecture defines an
agent's internal architecture by its beliefs, the desires
or goals it wants to achieve and its intentions or
plans to accomplish those goals. We want to know
does the methodology support modeling mental
attitudes of agents.
3.6 Modeling
Agent-Oriented- how efficient the methodology is in
supporting modularity, hierarchical modeling,
reusability, and traceability; Abstraction- to what
degree it is efficient to produce models at various
levels of details and abstractions; Consistency-
Models should not contradict each other. This
property becomes more important as the design
evolves. More specifically, the representation of
various aspects of a system such as structure,
function and behaviour should be consistent. We
want to know does the methodology provide
guidelines and techniques for consistency checking
both within and between models, do the
methodology supported by tools that provide model
consistency checking or is data dictionary used to
avoid naming clashes between entities.
3.7 Process
An ideal methodology should cover six stages,
which are enterprise modeling, domain analysis,
requirements analysis, design, implementation and
testing. Methodologies which cover all aspects of
the system development are more likely to be chosen
because of the consistency and completeness they
provide. Development principles- This criterion
addresses the lifecycle coverage in a broad view. It
examines the development stages and their
corresponding deliverables described within the
methodology. In addition, it examines the supporting
software engineering lifecycle model and
development perspectives. Process step- Differing to
the above criterion, this one measures the lifecycle
coverage in more detail. In fact, an important aspect
in evaluating whether a methodology covers a
particular process step is the degree of detail
provided. Estimating and quality assurance
guidelines: These two criteria determine if such
guidelines are provided within the methodology
process. Estimating guidelines are important to task
planning. Quality assurance guidelines provide the
assessors with useful information in evaluating the
merit of the delivered product. Supporting
development context: This criterion identifies the
development context supported by the methodology.
A development context specifies a set of constraints
within which the software development has to take
place.
3.8 Pragmatics
Maturity: The maturity of a methodology is a factor
that can play an important role in determining the
quality of a methodology. There are several ways to
measure the maturity of a methodology, for
example: What are available resources supporting
the methodology? What is the methodology's
"experience" such as the history of the methodology
use? What are available resources supporting the
methodology? Domain applicability: This considers
whether the methodology is targeted at a specific
type of software domain such as information
systems, real time systems or component based
systems. With regard to this issue, the methodology
that is applicable to a wide range of software
domains tends to be more preferred.
3.9 Management
Management Decision- to what degree the
methodology can be accepted by management; Cost
Estimation - to what degree it is economically
feasible?
4 SELECTING PARTICIPANTS
There were 8 participants in the experiment
(evaluators), all of whom were taking a specialized
graduate course in Agent-Oriented Software
Engineering. They all had adequate knowledge and
experience in software development. The
participants were provided sufficient documentation
about the methodologies, clear instructions about the
experiment, and equal amount of time to complete
ENASE 2007 - International Conference on Evaluation on Novel Approaches to Software Engineering
60
their task using well-prepared surveying
questionnaires.
5 SELECTING THE
APPROPRIATE STATISTICAL
MODEL
In (Elamy and Far, 2005), they compared 9
methodologies and the statistical model was One-
way ANOVA model for a Complete Random Design
(CRD), they decided to have at least 4 replicas for
each of the 9 treatments. they have two limitations
there. First, they suspect that there is heterogeneity
among evaluators for many reasons, such as
technical experience; although the randomization
will tend to spread the heterogeneity around to
reduce bias, we still have a strong limitation, which
is the lack of resources; they have only 12
participants. To bypass this lack, they decided to
make use of each participant to assess more than one
methodology. This solution headed us to consider a
2-way ANOVA model with blocking.
In this model, each block of treatments will be
assigned at random to one participant. If we used a
Randomized Complete Block Design (RCBD), each
evaluator should assess a complete block, i.e. 4
methodologies, and the design will probably more
effective because we will have more replications,
3248 =Ɨ
, by assuming considerable variability
within blocks. Unfortunately, this design has also a
limitation that makes it hard to implement, because
not all the participants are familiar with the whole
methodologies.
After a thorough analysis and extended
discussions, they ended up the battle by adopting the
two-way Balanced Incomplete Block Design (BIBD)
additive model with fixed-effects,
ijjiij
Y āˆˆ+++=
Ī²Ī±Ī¼
Where
i =1 to t; t = number of treatments = 4
j=1 to b; b = number of blocks = 8
ij
Y
=the observation (effectiveness) recorded on
the treatment (methodology) i by the evaluator
assigned to the block j
Ī¼
= overall mean of experiments of this type
i
Ī±
=treatment main effects, the deviation from
the mean caused by the
th
i
treatment;
0=
āˆ‘
i
Ī±
(fixed
effects)
i
Ī²
=main effects of the column blocking variable,
the deviation from the mean caused by the
th
j
block;
0=
āˆ‘
i
Ī²
(fixed effects)
ij
āˆˆ
=independent random error
),0(~
2
Ļƒ
N
and
k = number of treatments per block = 3; r =
number of replications per treatment = 4;
Ī»
=
number of times any pair of treatments appears
together in the same block =
1)1/()1( =āˆ’āˆ’ tkr
This model is referred to as ā€œbalancedā€ because
each block will have the same number of treatments,
ā€œadditiveā€ because no interactions are considered
between factors, ā€œincompleteā€ because each
participant will not evaluate a complete set of
treatments (4 methodologies), ā€œfixedā€ because we
were limited to narrow down the selection of the
qualified methodologies upon our own interest by
looking into specific measures, and not at random
from a large number of methodologies; thus, if we
decided to repeat the experiment we would use the
same methodologies, and not reselecting new ones
randomly. In fact, this model is applicable for both
RCBD and BIBD; however, the analysis is different.
By denoting the nine methodologies with letters
from A to D as shown in Table 1, we can obtain 16
replicas; which satisfies our goal of having 4
replicas for each treatment.
Table 1: BIBD assignment.
Treatments(Methodologies),i
i
Y
M4 M3 M2 M1
1.
Y
B A E1
2.
Y
B A E2
3.
Y
D C E3
4.
Y
D C E4
5.
Y
B A E5
6.
Y
B A E6
7.
Y
D C E7
2.
Y
D C E8
..
Y
.4
Y
.3
Y
.2
Y
.1
Y
.j
Y
Block
(evaluators)
, k
M=Methodology; E=Evaluator
6 COMPARING
METHODOLOGIES
Based on the comparison framework (Section 3),
Statistical Model proposed in section 5 and based on
the users view points; we have performed a brief
comparison between selected methodologies that is
explained below.
6.1 Concepts
Usability-We attempted to measure how complex a
methodology is to users, by using UML (Unified
USING A TWO-WAY BALANCED INCOMPLETE BLOCK DESIGN TO COMPARING AN AGENT-ORIENTED
SOFTWARE ENGINEERING METHODOLOGIES
61
Modeling Language) and RUP (Rational Unified
Process) as a benchmark. There seems to be an
agreement among the respondents that the four
methodologies are about the same complexity as
UML and RUP. However, it is not clear that there
was a consensus on the perceived complexity of
UML+RUP, and so the answers to this question did
not allow any strong conclusions to be drawn.
Notation- the responders generally agreed that the
methodologies' notation were clear and the symbols
and syntax are well defined. These indicate the
notations provided by all of the four methodologies
are fairly clear and understandable.
6.2 Upgradeability
Dynamic Structure and Scalability- Regarding this
criterion, most of the respondents stood on a neutral
point of view. In our perspective, this issue is not
explicitly addressed in any of the methodologies.
More specifically, they do not tell how to deal with
the introduction of new components or modules in
an existing system. Open Systems Support- none of
the methodologies support design of open systems.
Distribution- Overall, all of the methodologies
implicitly supports distribution. This is partially due
to the nature of agent-based systems. When
developed, agents communicate with each other via
a message passing system. In other words, agents are
not coupled until an interaction needs to occur. As a
result, the agents do not necessarily populate on the
same systems. The results from the questionnaire
also agreed with that view. Responses on this
criterion on average range from Neutral to Agree.
MaSE is an exceptional case in that the system
design step of MaSE allows the developers to design
and allocate agents over the network. It is supported
by the Deployment Model, a representation of agent
types and their location on the network. Therefore,
we tend to strongly agree with all the respondents of
MaSE that the methodology provides sufficient
support for distribution.
6.3 Modeling Language
Clarity and understand ability- the responders
generally agreed that the methodologies' notation
were clear and the symbols and syntax are well
defined. These indicate the notations provided by all
of the four methodologies are fairly clear and
understandable. Consistency- In terms of
consistency checking, the level of support differs
between methodologies. MaSE and Prometheus
support it well whereas Tropos and Gaia do not
appear to support it. These responses seem to relate
the availability of tool support integrated with the
methodology. PDT (Prometheus) and agent Tool
(MaSE) provides a strong support for model and
design consistency checking. Traceability- Likewise
to consistency, MaSE and Prometheus appear to be
the leader in terms of supporting this feature. The
responders of these two methodologies, including us
agreed that there are clear links between models
provided by them. For instance, goals, roles, agents,
and tasks are all linked together. This strong
connection improves the ability to track
dependencies between different models. Such
connections, as described in one of the paper related
to MaSE, allow developers to (automatically or
manually) derive design models (e.g. an agent's
internal architecture) from analysis constructs.
Unambiguity- Semantics is also well-defined by all
the methodologies. For Gaia, the student felt that the
semantics of Gaia's modeling language are not well-
defined. However, we tend to disagree with that. The
meaning of symbols and models in Gaia is in fact
defined in detail. Tropos was another interesting
case: there was disagreement on whether the
concepts were clear, and whether the notation was
clear and easy to use; furthermore, there was
disagreement on whether the syntax was defined, but
oddly, there was consensus that the semantics were
defined. For the remaining two methodologies, there
was an agreement (strongly agree - agree) that the
modeling language is unambiguous in the sense that
the semantics of the notation is clearly defined.
Usability- Overall, most of the respondents agreed
that the notations of the four methodologies are easy
to learn and use. This also relates to the agreement
of the understand ability and clarity of the notation
as discussed above. Tropos is, however, an
exception case. One of its authors strongly agrees
and the other agrees that the modeling language of
the methodology is easy to use. In contrast, the user
and we preferred to take a neutral view on this
criterion. This is due to the fact, unlike MaSE and
Prometheus, Tropos does not have a tool support
integrated with the methodology. As a result, users
may find it difficult to draw diagrams, checking
model consistency, etc.
6.4 Basic Properties (Weak Notation)
Autonomy- According to the responses from the
survey and our assessment, all of the four agent-
oriented methodologies recognize that importance.
The level of support for autonomy in all of them is
overall "good" (ranging from medium to high). This
ENASE 2007 - International Conference on Evaluation on Novel Approaches to Software Engineering
62
is reacted by the fact that all the four methodologies
provide various supports for describing an agent's
self-control features. For instance, functionalities
and tasks are encapsulated within an agent. In
addition, plan diagrams in Tropos, concurrent task
diagrams in MaSE, or plan descriptors in
Prometheus allow the decision-making mechanism
of agents to be modelled regardless of the
environment and other entities. That mechanism is
based upon the agents' goals and their roles within
the system. Pro-activeness and reactivity- Based on
the results, it seems that these two attributes are
difficult to measure we received highly varying
responses. They seem to be fairly well supported by
some of the four methodologies (medium-high for
MaSE and Prometheus, mostly high for Tropos).
Similarly to mental attitudes, this can be explained
by the fact that in these three methodologies agents'
goals are captured and so are the execution of plans
(i.e. actions or tasks) to achieve these goals. In
addition, Prometheus has the action descriptors
which are a means for specifying agents' responses
to environment changes in terms of external events.
Concurrency- In terms of support for concurrency,
although the ratings are mostly medium high and
vary considerably, MaSE is probably best with its
concurrent task diagrams and communication class
diagrams. The former is used to specify how a single
role can concurrently execute tasks that define its
behaviour. The latter, also expressed in the form or a
finite state machine, is able to define a coordination
protocol between two agents. Prometheus was rated
as being one of the weakest although we should note
that the handling of protocols in Prometheus has
been developed since the time of the questionnaire.
Sociability- Although the methodologies all support
cooperating agents, none of them support teams of
agents in the specific sense of teamwork. All of the
methodologies provide a wide range of
communication modes. More specifically, they
support both direct/indirect and
synchronous/asynchronous communication.
6.5 Mental Attitudes (Strong Notation)
Prometheus and Tropos support well (medium to
high) the use of mental attitudes (such as beliefs,
desires, intentions) in modeling agents' internals.
The percept and action descriptors in Prometheus
and the actor diagrams in Tropos represent the agent
knowledge of the world (i.e. beliefs). Goals and
plans are also modelled in the two methodologies. In
contrast, MaSE and Gaia provide weaker support for
capturing an agent's mental attitudes. MaSE have
goal diagram but they do not have a representation
of the agent's belief.
6.6 Modeling
Agent-Oriented- To some extent, all the
methodologies supports traceability, In terms of
consistency checking, the level of support differs
between methodologies. MaSE and Prometheus
support it well whereas Tropos and Gaia do not
appear to support it. Modularity and hierarchical
modeling are generally well-supported (although
there was disagreement from the student using
Tropos) however reusability is not well handled by
any of the methodologies. Consistency- MaSE and
Prometheus support it well whereas Tropos and Gaia
do not appear to support it.
6.7 Process
Development principles- From the software
development life-cycle point of view, all four
methodologies cover requirements analysis, and
architectural design. Some of them (MaSE and
Prometheus) go further than that with description of
detailed design, implementation and
testing/debugging. Deployment is only addressed in
MaSE. Process steps: The process steps described in
the requirements analysis and design phases are also
addressed well in most of the four methodologies.
Detailed design is not well documented in Gaia.
Furthermore, a common feature in all the
methodologies is the lack of management making
decisions in performing the process steps such as
when to move to the next phase, etc. Estimating and
quality assurance guidelines: Because of the
immaturity of agent-oriented methodologies, issues
relating to cost estimating or quality assurance are
not addressed in all four methodologies. They
probably rely on the current software engineering
practice of these matters. Supporting development
context: Top down design and "Greenfield"
development is the popular approaches employed by
most of the four methodologies.
6.8 Pragmatics
Maturity- Regarding the availability of resources
supporting the methodologies, most of them are in
the form of conference papers, journal papers or
technical reports. The availability of tool support is
varies. MaSE and Prometheus are well supported
with agentTool (MaSE) and JDE and PDT
(Prometheus). According to the authors of MaSE
USING A TWO-WAY BALANCED INCOMPLETE BLOCK DESIGN TO COMPARING AN AGENT-ORIENTED
SOFTWARE ENGINEERING METHODOLOGIES
63
(based on the questionnaire's responses), agentTool
can be used as a diagram editor, a design
consistency checker, code generator and automatic
tester. They also revealed that agentTool has been
downloaded and used by many people in academia
as well as industry and government. The tools
supporting Prometheus, PDT and JDE, also provide
a similar range of functionalities. PDT supports for
drawing diagrams, checking model and design
consistency, and generating reports. JDE (Jack
Development Environment) can be used a design
tool to build the structure of an agent system in
which the concepts provided by JACK match the
artifacts constructed in Prometheus' detailed design
phase. Tropos has only weak tool support (a diagram
editor) whereas there is no tool support for Gaia that
we are aware of. Although we attempted to
determine how much "real" use (as opposed to
student projects, demonstrators etc.) had been made
of each methodology, it was not clear from the
responses to what extent each methodology had been
used, who had used the methodology, and what it
had been used for. Nevertheless, to our knowledge,
MaSE was used to design a team of autonomous,
heterogeneous search and rescue robots (Scott et al.
2002). Tropos was used to develop a web-based
broker of cultural information and services for the
government of Trentino, Italy (Bresciani et al. 2002)
and an electronic system called Single Assessment
Process to deliver an integrated assessment of health
and social care needs for older people (Mouratidis et
al. 2002 ). Domain applicability: The respondents
tended to agree that there is no limitation to the
application domains where one of the four agent-
oriented methodologies can be applied.
6.9 Management
Cost Estimation- None of the methodologies seem to
address cost estimating guidelines. Management
Decision- None of the methodologies seem to
address management decision, Although one
respondent indicated that Tropos provides some
support for decision making by management, e.g.
when to move between phases, we do not agree with
this assessment.
7 CONCLUSION AND FUTURE
WORKS
One of the most fundamental obstacles to large scale
take-up of agent technology is the lack of mature
software development methodologies for agent-
based systems. Even though many Agent Oriented
Software Engineering (AOSE) methodologies have
been proposed, few are mature or described in
sufficient detail to be of real use. An important step
towards a complete unique methodology is to
understand the differences between the various key
methodologies, and to understand each
methodology's strengths, weaknesses, and domains
of applicability.
In this paper, we use a new statistical approach,
based on adopting the incomplete block design
model to evaluate the four AOSE methodologies.
Overall, all four methodologies provide a reasonable
support for autonomy, mental attitudes, pro-
activeness, and reactive ness. The notation of the
four methodologies is generally good. Most of them
have a strong modeling language in terms of
satisfying various criteria such as clarity and
understandability, adequacy and expressiveness,
ease of use, and unambiguity. However, there are
several exceptions. Tropos was not perceived as
being easy to use whilst GAIA was both ranked
weakly on adequacy and expressiveness. In addition,
only Prometheus and MaSE provide techniques and
tools for maintaining the consistency and traceability
between models. For the other two methodologies,
there is still more room for improvement with
respect to these issues. It is also emphasized that
none of the evaluated methodologies explicitly
provide techniques, guidelines, or models to
encourage the design of reusable components or the
reuse of existing components. Regarding the
process, only Prometheus and MaSE provide
examples and heuristics to assist developers from
requirements gathering to detailed design. Gaia was
not support detailed design. Additionally, even
though all phases from early requirements to
implementation are mentioned in Tropos with
examples given, the methodology does not appear to
provide heuristics for any phase. Implementation,
testing/debugging and maintenance are not clearly
well-supported by any methodology.
Additionally, some important software
engineering issues such as quality assurance,
estimating guidelines, and supporting management
decisions are not supported by any of the
methodologies.
As agent-oriented methodologies continue to be
developed, research will keep aiming at the direction
of determining which agent-oriented methodologies
are best suited to support the development of a
particular project or system. Hence, there are various
future works that can be done in this area.
ENASE 2007 - International Conference on Evaluation on Novel Approaches to Software Engineering
64
ACKNOWLEDGEMENTS
We would like to thank Dr.Faraahi, for his valuable
comments on a draft of this paper.
REFERENCES
Bresciani, P., Giorgini, P., Giunchiglia, F., Mylopoulos, J.,
Perini, A. (2004) Troops: An agent-oriented software
development methodology. From Trento University,
Department of Information and Communication
Technology.
Fatemi, A., NematBakhsh, N.,TorkLadani, B.(2006). An
Investigation of Agent Oriented Software Engineering
Methodologies to Provide an Extended Methodology,
IEEE International conference on Information and
Communication Technologies, 0-7803-9521.
Elamy, H., Far, H. (2005) Utilizing incomplete block
designs in evaluating agent-oriented software
engineering methodologies, IEEE International
conference on Electrical and Computer Engineering,
0-7803-8886.
Giorgini, P., Henderson-Sellers, B. (2005). Agent-
Oriented Methodologies, IDEA GROUP. United
States of America, 1
th
edition.
Luck, M., McBurney, P., Preist, C. (2003). Agent
technology: Enabling next generation computing: A
roadmap for agent-based computing. AgentLink
report. Retrieved from www.agentlink.org/roadmap.
Wooldridge, M., Jennings, N.R. & Kinny,D. (1999). A
Methodology for Agent-Oriented Analysis and
Design. In Proceedings of the third international
conference on Autonomous Agents and Multi-Agent
Systems, Seattle, WA.
Wooldridge M., Jennings N.R., Kinny, D. (2000) , The
Gaia Methodology for Agent-Oriented Analysis and
Design, From Liverpool University, Department of
Computer Science.
Giunchiglia, F., Mylopoulos, J., Perini, A. (2002). The
Tropos software development methodology:
Processes, Models and Diagrams. In Third
International Workshop on Agent- Oriented Software
Engineering.
Wooldridge, M., (1997) Agent-based software engineering
In IEE Proc. on Software Engineering, 144 (1) 26-37.
Waldner, A., Fuster , P., (2002). An Agent Oriented
Methodology: High Level and Intermediate Models.
Scott A. DeLoach, Mark F.Wood, and Clint H. (2001)
Sparkman. Multiagent systems engineering.
International Journal of Software Engineering and
Knowledge Engineering, 11(3):231ā€“258.
Padgham, L., Winikoff, M., (2002). Prometheus: A
methodology for developing intelligent agents. In
Third International Workshop on Agent-Oriented
Software Engineering.
Leskowsky, Z. and D. Anderson (2003). Comparing Agent
Oriented Methodologies. American Society of System.
Castro, J., Kolp, M., Mylopoulos, J., (2002). Towards
requirements-driven information systems engineering:
The tropos project, Information Systems, Elsevier:
Amsterdam, The Netherlands.
Bresciani, P., Giorgini, P., Giunchiglia, F., Mylopoulos J.,
& Perini, A. Towards an Agent Oriented approach to
Software Engineering", In the Workshop Dagli oggetti
agli.
Bresciani, P., Giorgini, P., Giunchiglia, F., Mylopoulos, J
and Perini, A. (2002). Troops: An agent-oriented
software development methodology. Technical Report
DIT-02-0015, from Trento University, Department of
Information and Communication Technology.
Scott A. DeLoach, Eric T. Matson, and Yonghua Li.
(2002). Applying agent oriented software engineering
to cooperative robotics. In Proceedings of the The 15th
International FLAIRS Conference.
Mouratidis, H., Giorgini, P., Manson, G., & Philp, L.,
(2002). Using Tropos methodology to Model an
Integrated Health Assessment System. In Proceedings
of the 4th International Bi-Conference Workshop on
Agent-Oriented Information Systems.
USING A TWO-WAY BALANCED INCOMPLETE BLOCK DESIGN TO COMPARING AN AGENT-ORIENTED
SOFTWARE ENGINEERING METHODOLOGIES
65