Using Evolving Graphs to Evaluate Structural Openness in

Multi-Agent Systems

Sondes Hattab, Faten Ben Hmida and Wided Lejouad Chaari

COSMOS Laboratory, Ecole Nationale des Sciences de l’Informatique, Université de la Manouba, 2010, Manouba, Tunisia

Keywords: Evaluation, Evolving graph, Multi-Agent Systems, Modelling, Measure, Openness, Observation.

Abstract: The evaluation of Multi-Agent Systems (MAS) issue is invoked in the literature in a twofold manner: from

an external point of view through the assessment of design methodologies and development tools and

platforms or from an internal point of view by measuring the functional characteristics of MAS applications.

The latter kind of evaluation is not sufficiently addressed and is mostly oriented towards structural properties.

We believe behavioural characteristics may considerably affect MAS performances and have to be assessed

in order to judge correctly the quality of the MAS. Thus, our aim is to propose an approach to evaluate one

of the most important behavioural characteristics in MAS: openness. We focus especially on structural

openness and we suggest for this purpose a three-step method: observation, modelling and measure. The

modelling technique is based on an evolving graph whose properties are used to estimate metrics for the

evaluation. Then, our approach is tested and validated on a road traffic application.

1 INTRODUCTION

Thanks to their ability to solve a huge number of

complex problems, Multi-Agent Systems (MAS)

have gained increasing interest in the scientific

community. Such success has been accompanied with

several issues have aroused the curiosity of

researchers. We mention particularly the issue

concerning performance evaluation which seems to

become a more complex subject when it is related to

MAS. This is what explains, hence, the limited

number of works in the topic of performance

evaluation. Through these few works, we can

distinguish two categories of proposed approaches.

The first treats the system from an external point of

view like the evaluation of design methodologies and

development tools and platforms. The second focuses

on the evaluation of MAS through the assessment of

their functional characteristics. Unfortunately, such

kind of evaluation is not sufficiently addressed and is

mostly oriented towards structural properties such as

interaction, communication and organization.

Behavioural and interface ones were rarely assessed.

This paper contributes in the enrichment of MAS

evaluation field by focusing on their functional

characteristics. We believe behavioural ones may

considerably affect MAS performances and have to

be estimated in order to well-judge the quality of the

MAS. Thus, our long-term goal is to study and

evaluate behavioural characteristics in MAS. At this

stage, we focus on openness. We propose to put the

spotlight on the structure of open MAS and its

evolution in time, and we will propose an approach to

evaluate structural openness.

The paper is organized as follows: in section 2, a

literature review of multi-agent systems evaluation is

presented, section 3 deals with openness in MAS and

the related existing works. Our proposed evaluation

approach is described in section 4. The application on

which Experimentations were carried out is described

in section 5. We finish by presenting our conclusions

and perspectives in section 6.

2 EVALUATION OF

MULTI-AGENT SYSTEMS

The existing MAS evaluation approaches and our

work position are presented and criticized in Table1.

Hattab, S., Hmida, F. and Chaari, W.

Using Evolving Graphs to Evaluate Structural Openness in Multi-Agent Systems.

DOI: 10.5220/0006334505910598

In Proceedings of the 19th International Conference on Enterprise Information Systems (ICEIS 2017) - Volume 1, pages 591-598

ISBN: 978-989-758-247-9

591

Table 1: MAS evaluation works.

3 OPENNESS IN MULTI-AGENT

SYSTEMS

Openness in MAS has been identified in the literature

regarding two aspects:

 Structural aspect: It considers openness as the

ability to deal with inconstant entities that leave

and enter to the system causing a change in its

structure (Valckenaers, 2007). In this case,

openness is called structural openness.

 Functional aspect: openness here is based on

the internal evolution of agents and interactions

and considers the modification of their content,

goals, attributes and functionalities (Boissier,

2004). In this case, openness is called

functional openness.

The evaluation of such property is not well

explored in the previous researches. The existing

related works do not evaluate openness as a set of

additions, removals and internal evolution of entities.

Related works Issue Solution

Black-Box-

Based

approaches

They are studied

independently of internal

properties and functionalities:

the evaluation of Agent-

Oriented technology (Tveit,

2001), the evaluation of design

methodologies (Cernuzzi and

Rossi, 2003), the evaluation of

development platforms and

tools (Occello, 2002).

Such evaluation is

done from an external

point of view and does

not address the

running of the

application, its

internal properties or

even the relationship

between its

components and their

evolution in time.

The solution proposed in the literature is to

take into account the internal evolution of

MAS by evaluating implemented multi-agent

applications. It aims to assess MAS

performances regardless of the used design

methodology and development tool.

The

evaluation

of multi-

agent

applications

The literature has revealed two

categories of evaluations in

this context: the evaluation of

the functional adequacy

(Kaddoum, 2009) and the

evaluation of global

performances. We mention as

example the work of (Joumaa,

2008) that interests in

assessing interactions in a

robots’ society.

Such evaluation is

considered in (Ben

Hmida, 2013) as

system dependent and

specific to a given

topic.

The solution proposed by (Ben Hmida, 2013)

is to evaluate MAS functional properties from

a generic point of view. It takes into account

three categories of characteristics: the

structural properties describing how agents

are organized in the system, namely:

communication, interaction, organization,

distribution and decentralization, the

behavioural properties describing the way the

system evolves in time and behaves towards

itself, its components, its environment and

other external systems : autonomy, openness,

adaptation and emergence, the third is the

interface properties focusing on the

relationship between the MAS and the outside

whether it is its environment, a different

system or a human actor: personalization,

delegation, intelligibility and the situation in

the environment.

Generic

Fuctional-

Characterist

ics-Based

approaches

We interest especially in Ben

Hmida works in which

structural properties are

evaluated, more particularly

communication (Ben Hmida,

2008) and organization (Ben

Hmida, 2012). In this context,

a graph-theory based approach

is proposed. It follows a three-

step process: observation,

modelling and measuring.

The existing works are

essentially oriented

towards structural

characteristics and

neglect behavioural

and interface ones.

Behavioural and interface characteristics are

not yet evaluated from a generic point of view.

We argue that behavioural properties may

affect considerably MAS performances so

that measuring only the structural

characteristics seems to be not enough. Thus,

our challenge is to propose an approach to

evaluate behavioural properties, and then test

it on real world applications. At this stage of

our research work, we are interested in the

evaluation of openness and its impact on the

system.

ICEIS 2017 - 19th International Conference on Enterprise Information Systems

592

But, they take into account other associated aspects.

For example, (Vercouter, 2001) evaluated the

approaches dedicated to managing openness in MAS

namely the centralized and the distributed

approaches. Both of them were compared to identify

the more advantageous one by using few criteria,

namely the number of agents to communicate with,

the relevance of other agents representation and the

robustness of the system. In (Berreur, 2005),

openness is measured by referring to three aspects:

openness on the environment, openness to the user

and openness to other agents. To this end, an

evaluation criterion was proposed. It consists in

quantifying each aspect depending on the number of

exchanges between the agent and one of the three

facets. Other efforts are devoted to assessing trust

level in open MAS. Trust is, in fact defined as a

relationship between two agents where a trustor agent

performs for estimating the credibility of the trustee.

In this context, (Khosravifar, 2009) provides a trust

assessment process to evaluate the trustworthiness of

the participating agents. In the present work, we

interest in structural openness. We aim to evaluate the

latter property as defined in the literature and bring,

as a consequence, added value to the field of MAS

evaluation.

4 PROPOSED APPROACH FOR

STRUCTURAL OPENNESS

EVALUATION

The addition and removal of entities in MAS make its

structure difficult to analyse. Thus, it is necessary to

reduce this complexity by using an easy to handle

mean. Modelling represents an appropriate technique

allowing the explanation of the structure and the

determination of some important aspects. So, in order

to evaluate structural openness in MAS, we propose

to adopt the process of (Ben Hmida, 2013):

observation, modelling and measure.

4.1 Observation

In our evaluation approach, observation consists in

generating traces each time a significant event occurs

in the MAS. That’s to say, when an addition or/and a

removal of agents or/and interactions is detected, a

specific model is produced. Such operation is

performed through software probes based on Aspect

Oriented Programming (AOP). As for the generated

model, it will be discussed and justified in the

following paragraphs.

4.2 Modelling

Due to its dynamicity and strong evolution, structural

openness in MAS requires a changing-nature model

to be represented. In other words, the proposed model

must be dynamic and have to allow highlighting the

different events causing the observed modifications.

Besides, whenever a system consists of many single

components interacting together, it becomes natural

to represent it as a graph where each node stands for

one component and interactions are symbolized by

edges.

Thus, a dynamic graph seems to be appropriate to

represent the structure of open MAS. It is indeed a

powerful mean that has interested many researchers

i.e (Afrasiabi Rad, 2016), (Zaki, 2016) and (Beck,

2016). They all give a consensual definition of a

dynamic graph and define a set of related metrics.: it

is a sequence ᴦ := (G1,G2,…,Gn) where each static

graph Gi := (Vi, Ei) models a set of objects Vi, called

vertices or nodes and their relationships

⊆

ViVicalled edges or arcs. The indices refer

to a sequence of time steps:  1, 2, … , .The

graph G

i+1

is obtained from G

by simple

modifications: additions and removals of vertices and

edges.

Many dynamic graphs were studied and analyzed

in the literature. We mention mainly: Complex

Networks (Boccaletti, 2006), evolving graphs

(Ferreira, 2003) and (Bui-Xuan, 2003), Re-

optimization graphs and finally cumulative graphs

and Space-Time Networks. These graphs are

classified in (Pigné, 2009) following three criteria:

 Graph dynamicity: it refers to the ability of the

dynamic graph to undergo modifications in its

components. In this context, (Zaki, 2016)

distinguishes between two types of dynamic

graphs. The first is the fully-dynamic graphs

allowing modifications in nodes, arcs and their

associated attributes namely Complex

Networks and evolving graphs. The second is

the partially-dynamic graphs allowing changes

only at some components namely Re-

optimization graphs, cumulative graphs and

Space-Time Networks.

 The knowing level of events evolution:

According to this criterion, dynamic graphs are

classified into two categories: those whom

evolution is known in advance such as complex

networks, Re-optimization graphs and

cumulative graphs, and those whom changes

are not initially known but progressively-

revealed in time like evolving graphs and

Space-Time Networks.

Using Evolving Graphs to Evaluate Structural Openness in Multi-Agent Systems

593

 The evolution process of the graph: this

criterion distinguishes between dynamic

graphs depending on how the events of the

evolution are generated. In fact, the process to

generate the events may be described in the

model such as in complex networks or are

simply the result of random applications, which

is the case for other dynamic graphs

Actually, an open MAS is a strongly-dynamic

system which permit any operation of addition and

removal of agents and interactions. Its environment is

so uncertain that it is impossible to know in advance

which are the events responsible for the structural

change or how are they generated. Therefore, the

dynamic graph that we will base on should have the

following properties: a full dynamicity, a

progressively-revealed events evolution and an

unknown evolution process. So, we say that an

evolving graph is more appropriate to model

structural openness in MAS.

Evolving graphs are studied in several works. We

mention mainly (Ferreira, 2002), (Monteiro, 2006)

and (Jarry, 2008). Its definition is given below:

Let be given a graph G (V, E) and an ordered

sequence of its sub-graphs SG = G

, . . . ,G

such

that

⋃

Gi  G





. Let S

= t

, t

. . . t

be a sequence

of time instants. Then, the system GE = (G, S

, S

where each Gi is the sub-graph during [t

i−1

, t

[, is

called an evolving graph. The graph G is called

underlying graph. Figure1 shows an illustrating

example of an evolving graph built from three sub-

graphs G

, G

and G

Figure 1: Illustrating example of an evolving graph.

Our evolving graph consists of a set of vertices V

that represent the agents having existed in the system

and a set of arcs E symbolizing the interactions

having appeared over the time. Each edge stands for

a transmitted message between two agents. Thus, our

graph is directed for the simple reason that any

message has a sender and a receiver agent. We

propose to label each node of the underlying graph G

with a vector < id, P

, app

, disp

> where id is the

identifier of the corresponding agent, P

is the

presence vector of the node, app

is the number of

appearances of the corresponding agent in the system

and disp

is its number of disappearances. Similarly,

each arc of the underlying graph G is labelled with a

vector < SrcId, DesId, P

, app

, disp

> where SrcId is

the identifier of the sender agent, DesId is the

identifier of the receiver agent, P

is the presence

vector of the arc, app

is the number of appearances

of the corresponding interaction and disp

is its

number of disappearances. The generated graph will

be exploited in order to propose measures to evaluate

the structural openness in MAS. This is what we will

detail in the next paragraph.

4.3 Measuring

In this section, we will analyse our model and

estimate some appropriate measures.

4.3.1 Alpha Index α

Alpha index is the measure used to estimate the

variation of the number of agents. It is applied on the

underlying graph G and expressed as the difference

between the total number of nodes’ appearances and

their total number of disappearances:

α = ∑

app

- ∑

disp

(1)

α reflects the nature of the evaluated open MAS.

In fact:

 If α>0 then MAS is said to be increasing.

 If α<0 then MAS is said to be decreasing.

 If α=0 then MAS is said to be conservative.

4.3.2 Degree of Structural Extensibility χ

The degree of structural extensibility is proposed to

evaluate how much the system is able to manage the

free entering and leaving of agents. It is estimated

through the renewal rate of the graph and applied on

the sequence of the sub-graphs S

This latter is

defined in (Pigné, 2009) as follows:

Let be a structure of a dynamic graph observed on two

instants S

and S

. The renewal rate tr (S

, S

) is the

number of changes M between the two dates

compared to the number of elements |S1| in the

starting structure.

We consider that the sub-graph G

is the structure

on which we conduct our calculation and we interest

only in the additions and removals of nodes. Thus,

the renewal rate of G

is defined as follows:

tr (G

i-1

) =

)G(Ord

1i 

(2)

ICEIS 2017 - 19th International Conference on Enterprise Information Systems

594

M is the total number of appearances and

disappearances of nodes and Ord(G

i-1

) is the number

of nodes in the sub-graph G

i-1

.Then, we define χ as

the average renewal rate of G

χ =

)G,G(tr

i1i









(3)

N stands for the number of sub-graphs in the sequence

χ may reflect the degree of scalability of the open

MAS. This latter is defined as the ability of the system

to adapt itself from a dimensional point of view as to

larger sizes than to smaller ones while maintaining its

efficiency. Therefore, when χ 0.5 i.e. when the

number of entering and leaving agents between G

i-1

and G

exceeds on average the half of the existing

agents in G

i-1

, the MAS is qualified as highly scalable.

On the opposite case, the system is low scalable.

4.3.3 Structural Impact π

Structural impact π

is the proportion of time during

which the agent j is functional in the MAS. Based on

the sequence of sub-graphs S

we define it as the ratio

between the cumulative age of the corresponding

node S

and the observation duration D. The

cumulative age of an element in a dynamic graph is

defined in (Pigné, 2009) as the sum of the durations

of the time intervals during which it is present in the

dynamic graph:

)Sj(CA

,VS

jj 

(4)

In open MAS where agents can be removed at any

time, an agent that persists for a long time is said to

be stable. It may acquire a significant amount of

information and behave as a leader. Therefore, the

structural impact π

may reflect the degree of stability

of the agent j in the evaluated MAS. The more π

elevated, the more the corresponding agent is stable

and its convergence to the leadership is important.

4.3.4 Dependency Rate τ

We call the dependency rate τj the relationship

between the number of added interactions by an agent

j and the possible number of interactions could be

added by the same agent. Indeed, the number of the

established interactions by an agent j, symbolized by

a node Sj, is the sum of the outgoing edges’ activities.

Actually, the activity of an edge e is denoted δ

(e) and

defined in (Ferreira, 2003) as the number of its

presence intervals. The formula of τ

is as follows:













ivv

kjE

vkj

)G,S(P

))S,S((

j,VS,S,S

(5)

In open MAS, an agent has a partial view of its

environment. It must cooperate and interact with

other agents in order to acquire what it needs to

achieve its goal. Thus, the dependency rate τj reflects

the deficiency degree in terms of capabilities and

resources that leads the corresponding agent to

establish new interactions.

4.3.5 Lambda Index λ

Lambda index λ estimates the variation of the number

of interactions. It is applied on the underlying graph

G and expressed as the difference between the total

number of edges’ appearances and their total number

of disappearances:

λ = ∑

app

- ∑

disp

(6)

Lambda index λ reflects the effect of the structural

openness on the abundance of communication

between the agents. In fact:

 If λ ≥0 then we say that the structural openness

promotes the abundance of communication in

the system.

 If λ <0 then we say that the structural openness

demotes the abundance of communication in

the system.

4.3.6 Transitivity T

We define transitivity T

as the ability of an agent j to

acquire data from the other agents. We express it as

the relationship between the number of existing

journeys between the corresponding nodes S

and S

denoted N

and the number of all the existing journeys

in the graph N

. In (Ferreira, 2003), a journey in an

evolving graph is defined as follows:

Let R be a path R = e

, e

... e

with ∈ . Let

σ =

,...,σ

be a time schedule indicating that edge

is to be traversed at time σ

A journey J = (R, R

)

is defined if and only if R

is in accordance with R

and GE i.e., J allows for a traversal over time from u

to v in GE.

The formula we propose in order to calculate T

is:

Tj,S 

(7)

Using Evolving Graphs to Evaluate Structural Openness in Multi-Agent Systems

595

In this case, an interaction is seen as a trust

relationship. It is progressively acquired by agents

thanks to the MAS openness. Thus, the more Tj is

elevated, the more agent j is trusted. The agent having

the greatest transitivity is said to be the more stressed

to the communication over the time.

4.3.7 Structural Distribution ρ

In open MAS where agents and interactions can

freely enter and leave, coalition structures can be

progressively created. In other words, agents having

compatible goals form a group and cooperate together

in order to perform a common task (Hasan, 2013).

Thus, we define ρ as the number of maximal time-

connected components. This latter is defined in

(Ferreira, 2003) as follows:

A maximum time-connected component (MTCC) in

an evolving graph is the maximum set of vertices

U ⊆ such that for any pair of u, v ∈, there exists

a journey from u to v and a journey from v to u using

only vertices in U.

According to (Gruszczyk, 2008), When agents

cannot complete their tasks individually they may

exchange information and form coalitions which

gives them best efficiency in terms of solved problem.

Thus, structural distribution ρ allows determining the

effect of the structural openness on the system’s

performance. We say, the more the number of

obtained MTCC is elevated, the faster agents are able

to achieve their local goals.

5 EXPERIMENTATION AND

RESULTS

Our approach is tested on a multi-agent road traffic

application described as follows.

5.1 Agent-Based Test Application

The tested multi-agent application is a road traffic

simulator. It allows, at any time, the addition and

removal of agents and interactions. We believe that

testing such application is much more useful for MAS

community when it is used to compare coordination

strategies in dynamic scenarios. But, it is also

beneficial for us in the sense that we can take

advantages of its dynamicity and openness to conduct

our calculation and validate our evaluation approach.

Actually, our road traffic simulator is developed upon

the JADE framework and made of several agent

types:

 Driver: this agent is able to stop, to forward, to

choose a path, to change its position and to

respect the lights at intersections. It can

communicate with its predecessor, its pursuer,

the road on which it is moving and the

intersection met in its movement. Its identifier

is prefixed by the word “Cond:”.

 Road: this agent interacts with drivers

providing them with information concerning

their predecessors and pursuers. Its identifier is

prefixed by the word “Route”.



Intersection: this agent conveys information

concerning lights’ states and drivers at the

same intersection. Its identifier is prefixed by

the word “Carrefour”.



Light: this agent has for role to inform drivers

and pedestrians about its current state. It has an

identifier prefixed by the word “Feu”.



Pedestrian: it aims to cross a road while taking

into account the lights state. It has an identifier

prefixed by the word “Pieton”.

5.2 Results and Interpretations

Due to its strong dynamicity, our simulator generates

evolving datasets: the user can’t predict neither the

number of appearing and disappearing agents nor the

way they coordinate. That’s why, it seems judicious

to present a test case and analyse its results which are

with no doubt real datasets. As explained in section

4.2, we base on a directed evolving graph where each

edge is a transmitted message between two agents.

Figure 2 shows the sub-graphs S

representing the

evolution of the system during 4 seconds. The

different sub-graphs are generated using

Graphstream, a java library that focuses on the

dynamic aspects of graphs.

Figure 2: The set of sub-graphs S

generated during 4

seconds.

ICEIS 2017 - 19th International Conference on Enterprise Information Systems

596

Figure 3: The underlying graph G.

Figure 3 shows the underlying graph G, the

aggregation of the previous sub-graphs.

Now, we can calculate our measures. The global

ones are presented in Table2.

Table 2: Global performance metrics.

Property Measure Value

Nature of the open

MAS

Alpha Index α 7

Degree of

scalability

Degree of structural

extensibility χ

0.89

The effect on the

communication

abundance

Lambda Index λ 5

Open MAS’

performance

Structural Distribution

As for the local measures, they are illustrated in

Figure 4.

Figure 4: Local performance metrics.

According to the obtained results, we have an

increasing trafic flow. Having a degree of structural

extensibility greater than 0.5, our system is said to be

high scalable. Its ability to manage the free entering

and leaving of drivers and pedestrians, is elevated

which reflects a high congestion level supported by

the simulated roads and helps to think about new

policies to make better the transport infrastructure.

Besides, the communication between

the agents is abundant and promoted in time. Thus,

our road trafic simulator is an interactive application

that puts together a set of social agents. The structural

openness of the application caused 4 coalitions. We

say that vehicles and infrastructure have a great

ability to communicate in order to get an optimal

transport network with efficient movement of traffic.

CondC1C408 have the highest value of π

It is the

more stable agent in the system and the elected one to

be leader in the future. We can say also that

CondC1C408 is the more cost-effective in terms of

travel time and delay. RouteC1C4 have the greatest

dependency rate τ

which means that it has to interact

with other agents in order to overcome the issue of

capabilities insufficiency and avoid as a consequence

serious problems like accidents . Having the more

elevated transitivity T

, CondC1C406 and

CondC1C408 are the more trusted and the more

stressed to the communication over the time. Besides,

Route C1C4 is the road having the greatest

transitivity. This latter is then usually used by drivers

and pedestrians and that can be subject to traffic jam.

6 CONCLUSION

In this paper, we presented an evaluation approach of

structural openness in MAS. The proposed method

follows three steps: observation, modelling and

measure. The modelling technique is based on an

evolving graph which is used to estimate metrics to

get a clear idea about the quality of the open MAS.

The tests and experimentations were carried out on a

road traffic simulator, an open multi-agent

application which puts together a set of volatile and

persistent agents. The obtained results allowed us to

validate the proposed evaluation model and to give

some interpretations related to the underlying road

traffic state. As a perspective of this work, we aim to

focus on two points. The first consists in working on

the functional openness interesting in the internal

evolution of agents and interactions. The second is to

evaluate the behavioural characteristics in MAS:

adaptation, emergence and autonomy.

Using Evolving Graphs to Evaluate Structural Openness in Multi-Agent Systems

597

REFERENCES

Afrasiabi Rad, A., 2016. Social Network Analysis and Time

Varying Graphs. Ph.D dissertation, University of

Ottawa,Canada.

Beck, F., Burch, M., Diehl, S., Weiskopf, D., 2016. A

Taxonomy and Survey of Dynamic Graph

Visualization. In Computer Graphics Forum.

Ben Hmida, F. B., Séguy, A., and Dupas, R. 2012.

MultiAgent Systems for Production Planning and

Control in Supply Chains. In Proceedings of the 9th

International Conference on Distributed Computing

and Artificial Intelligence (pp. 205- 212). Springer

Berlin Heidelberg.

Ben Hmida, F., Lejouad Chaari, W., and Tagina, M. 2008.

Performance evaluation of multiagent systems:

communication criterion. In Proceedings of the 2nd

KES International Conference on Agent and Multi-

Agent Systems: Technologies and Applications (pp.

773-782). Springer Berlin Heidelberg.

Ben Hmida, F., 2013. Evaluation des performances des

systèmes multi-agents. Ph.D dissertation, University of

Manouba,Tunisia.

Berreur, P., 2005. Evaluer les systèmes multi-agents.

Technical report. Pellucid 5FP IST-200134519,

University of Joseph fourier, France.

Boissier, O., Gitton, S., and Glize, P. 2004.

Caractéristiques des Systèmes et des Applications.

Systèmes multi-agents / Observatoire français des

techniques avancées, ARAGO 29.

Boccaletti, S., Latora , V., Moreno, Y., Chavez, M., Hwang,

D-U., 2006. Complex Networks: Structure and

Dynamics. Physics reports., 424(4), 175-308.

Bui-Xuan, B., Ferreira,A., Jarry,A., 2003. Computing

Shortest, fastest and foremost journeys in dynamic

networks. In International Journal of Foundations of

Computer Sciences, April,2003, 14(02), 267-285.

Cernuzzi, L., Rossi, G., & Plata, L. 2002. On the evaluation

of agent oriented modeling methods. In Proceedings of

the 1st Workshop on Agent Oriented Methodology,

Seattle (pp. 21-30).

Demetrescu, C., Italiano, G., 2006. Fully dynamic all pairs

shortest paths with real edge weights. In Journal of

Computer and system Sciences. February 2, 2006,72(5),

813-837.

Ferreira, A., 2002. On Models and Algorithms for Dynamic

Networks: The case for evolving Graphs. In 4e

Rencontres Francophones sur les Aspects

Algorithmiques des Télécommunications (ALGOTEL’

2002), Mèze, France.

Ferreira, A., 2003. Building a Reference Combinatorial

Model For Dynamic Networks : Initial results in

Evolving Graphs. Ph.d dissertation, INRIA.

Gruszczyk, W., & Kwasnicka, H. 2008. Coalition

formation in multi-agent systems—an evolutionary

approach. In Computer Science and Information

Technology, 2008. IMCSIT 2008. International

Multiconference on (pp. 125-130). IEEE.

Hasan, M-R., and Raja. A., 2013. Emergence of Multiagent

Coalition by Leveraging Complex Network. In

Proceedings of AAMAS 2013 Fifth International

Workshop on Emergent Intelligence on Networked

Agents (WEIN'13), (pp 1-15).

Jarry, A., 2008. Connexité dans les réseaux de

télécommunication. Ph.D dissertation, University of

Nice, Sophia Antipolis, France.

Joumaa, H., Demazeau, Y., and Vincent, J. M. 2008.

Evaluation of multi-agent systems: The case of

interaction. In Proceedings of the 3rd International

Conference on Information and Communication

Technologies: From Theory to Applications, (pp. 1-6).

IEEE.

Kaddoum, E., Gleizes, M. P., George, J. P., Glize, P., and

Picard, G. 2009. Analyse des critères d'évaluation de

systèmes multi-agents adaptatifs. In Journées

Francophones sur les Systèmes Multi-Agents, (pp. 123-

132).

Khosravifar, B.,Gomrokchi,M., benTaher,J., Thiran, P.,

2009. Maintenance-Based Trust of Multi-Agent

Systems. In Proceedings of the 8

International

Conference on Autonomous Agents and multi-Agent

Systems-Volume 2 (pp.1017-1024). International

Foundation for Autonomous Agents and Multi-Agent

Systems.

Monteiro, J., Goldman, A., & Ferreira, A. 2006.

Performance evaluation of dynamic networks using an

evolving graph combinatorial model. In Wireless and

Mobile Computing, Networking and Communications,

2006.(WiMob'2006). IEEE International Conference

on (pp. 173-180). IEEE.

Occello, M., Guessoum, Z., and Boissier, O. 2002. Un

essai de définition de critères pour l étude comparative

de plates-formes multi-agents. Technique et Science

Informatiques (TSI), 21(4), 549-552.

Pigné, Y., 2009. Modélisation et traitement décentralisé

des graphes dynamiques : application aux réseaux

mobiles ad hoc (pp.1-41). Ph.D dissertation, University

of Havre, France.

Tveit, A. 2001. A survey of agent-oriented software

engineering. In NTNU Computer Science Graduate

Student Conference, Norwegian University of Science

and Technology. Vercouter, L., 2001. Conception et

mise en œuvre de systèmes multi-agents ouverts et

distribués. Ph.D dissertation, University of Jean

Monnet and School of Mines de Saint-Etienne, France.

Valckenaers, P., Sauter, J.A., Sierra, C., Rodrıguez

Aguilar, J.A. 2007. Applications and environments for

multi-agent systems. In Autonomous Agents and Multi-

Agent Systems, 14(1), 61–85

Vercouter, L., 2001. Conception et mise en œuvre de

systèmes multi-agents ouverts et distribués. Ph.D

dissertation, University of Jean Monnet and School of

Mines de Saint-Etienne, France.

Zaki, A., Attia, M., Hegazy, D., Amin, S., 2016.

Comprehensive Survey on Dynamic Graph models. In

International Journal of Advanced Computer Science

and Applications, 7(2), 573-582.

ICEIS 2017 - 19th International Conference on Enterprise Information Systems

598