Adaptive Agents for Cyber-Physical Systems

Ichiro Satoh

National Institute of Informatics, 2-1-2 Hitotsubashi Chiyoda-ku, Tokyo, 101-8430, Japan

Keywords:

Self-adaptation, Bio-inspired Approach, Cyber-Physical Systems, Agent.

Abstract:

This paper proposes a bio-inspired approach to adapting software components in CPSs. It introduces the

notions of differentiation and dedifferentiation in cellular slime molds. When a software component delegates

a function to another component coordinating with it, if the former has thefunction, thisfunction becomes less-

developed and the latter’s function becomes well-developed like that in cellular differentiation. The approach

enables software components on CPSs to be naturally adapted to changes in the cyber and physical world in a

self-organizing manner. It is constructed as a middleware system to execute general purpose applications on a

CPS. We present several evaluations of the approach in CPSs.

1 INTRODUCTION

Cyber-physical systems (CPSs) are complicated and

dynamic by nature. They must adapt themselves to

changes in the physical world. For example, the

sensing systems for catastrophes, e.g., earthquakes,

deluges, radiation, and forest ﬁres, have been de-

signed extensivelyagainst disasters. CPSs must be re-

silient. In fact, even when most sensor nodes are dam-

aged, remaining nodes should organize themselves

and achieve their goals as much as possible. CPSs

should also adapt themselves to the requirements of

multiple applications. Nodes in CPSs may be het-

erogenous and do not need to provide the same tasks

unlike those in peer-to-peer computing systems and

sensor networks. They should delegate some tasks to

other nodes to save their resources.

The complexity and dynamism of CPSs are be-

yond our ability to build and manage systems through

traditional approaches, such as those that are central-

ized and top-down. Instead, we propose an evolution-

ary approach to manage CPSs with small and sim-

ple rules for adaptation. It introduces the notion of

(de)differentiation into CPSs. When similar functions

are running on different nodes connected through a

network, one of them should automatically be placed

by the other. Remaining nodes should provide those

functions that were provided by disconnected nodes

as much as possible, when the network is partitioned.

Differentiation is the mechanism by which cells in

a multicellular organism become specialized to per-

form speciﬁc functions in a variety of tissues and or-

gans. Different kinds of cell behaviors can be ob-

served during embryogenesis: cells double, change

in shape, and attach at and migrate to various sites

within the embryo without any obvious signs of dif-

ferentiation. A CPS is a system featuring a tight com-

bination of computational and physical components.

The former consists of hardware and software compo-

nents. Our approach offers a mechanism for adapting

software components, which may be running on com-

puters connected through a network. The approach

involves treating the undertaking/delegation of func-

tions in software components from/to other compo-

nents, including physical and hardware components,

as their differentiation factors. When a software com-

ponent delegates a function to another component,

if the former has the function, its function becomes

less-developed. The latter’s function becomes well-

developed if the latter is a software component. The

proposed system also should be reasonable and avail-

able in real systems instead of in simulation-based

systems.

2 RELATED WORK

The notion of self-organization is rapidly gaining im-

portance in the area of CPSs. We discuss several re-

lated studies on software adaptation in distributed and

ubiquitous computing systems in addition to CPSs.

Several researchers have explored adaptive ap-

proaches to managing CPSs, but most of them have

aimed at managing networks rather than functions

in CPSs. In fact, they were constructed based on

257

Satoh I..

Adaptive Agents for Cyber-Physical Systems.

DOI: 10.5220/0004244402570262

In Proceedings of the 5th International Conference on Agents and Artiﬁcial Intelligence (ICAART-2013), pages 257-262

ISBN: 978-989-8565-39-6

 2013 SCITEPRESS (Science and Technology Publications, Lda.)

wireless sensor networks. One of the most typical

self-organization approaches to distributed systems is

swarm intelligence (Bonabeau et al., 1999; Dorigo

and Stutzle, 2004). Although there is no central-

ized control structure dictating how individual agents

should behave, interactions between simple agents

with static rules often lead to the emergence of in-

telligent global behavior. Most existing approaches

have only focused on their target problems or ap-

plications but they are not general purpose, whereas

CPSs are general purpose. Our software adaptation

approach should be independent of applications. Fur-

thermore, most existing self-organization approaches

explicitly or implicitly assume a large population of

agents or boids. However, real CPSs have no room

to execute such large numbers of agents. In fact, a

CPS consists of embedded computers, whose com-

putational resources are limited. Consequently, our

software adaptation mechanism for CPSs must spend

little computational resources of the systems for soft-

ware adaptation as possible.

Georgiadis et al. (Georgiadis et al., 2003)

presented connection-based architecture for self-

organizing software components on a distributed sys-

tem. Like other software component architectures,

they intended to customize their systems by chang-

ing the connections between components instead of

the internal behaviors inside them. Like ours, Cheng

at al. (Cheng et al., 2006) presented an adaptive se-

lection mechanism for servers by enabling selection

policies, but they did not customize the servers them-

selves. They also needed to execute different servers

simultaneously. Herrman et al. proposed the bio-

inspired deployment of services on sensor networks

(Herrman, 2008). Unlike ours, their work focused on

the deployment and coordination of services, instead

of the adaptation of software itself to provide services.

We proposed a nature-inspired approach to dy-

namically deploying agents at computers in our previ-

ous paper (Satoh, 2007). The approach enabled each

agent to describe its own deployment as a relation-

ship between its location and another agent’s location.

However, the approach had no mechanism for dif-

ferentiating or adapting agents themselves. We also

presented an early prototype of this approach (Satoh,

2011) and its extension for sensor networks (Satoh,

2012). The former aimed at an adaptive mechanism

in application-speciﬁc services and the latter was an

early implementation of the approach.

3 APPROACH

Traditional management approaches may not be able

to support complications or dynamism in CPSs. In-

stead, we introduce a bio-inspired approach to CPSs.

The basic inspiration for our approach lies in cellular

differentiation, which is the process by which a less

specialized cell develops or matures to possess a more

distinct form and function in developmental biology.

When a parent cell divides into two or more daugh-

ter cells, the latter may be differentiated from the for-

mer. For example, cellular slime molds, e.g., dic-

tyostelium discoideum, are eukaryotic microorgan-

isms in the soil. Once food becomes scarce, dic-

tyostelium discoideum cells start to aggregate and dif-

ferentiate themselves. They can also be differentiated

into two types: prespore cells and prestalk cells. Each

cell tries to become a prespore cell and periodically

secretes cyclic-adenosine-monophosphate (cAMP) to

other cells. If a cell can receive more than a speciﬁed

amount of cAMP from other cells, it can become a

prespore cell. There are three rules. 1) cAMP chemo-

tactically leads other cells to prestalk cells. 2) A cell

that is becoming a prespore cell can secrete a large

amount of cAMP to other cells. 3) When a cell re-

ceives more cAMP from other cells, it can secrete less

cAMP to other cells.

Computer 1

Behavior A

Initial phase

Differentiation

phase

Behavior B Behavior A Behavior B

Computer 2

Computer 1

Computer 2

Network

Agent 1

Agent 2

Network

Computer 1

Dedifferentiation

phase

Computer 2

Network

partioning

Behavior B

(progression)

Behavior A

(regression)

Behavior A

(progression)

Behavior B

(regression)

Behavior A Behavior B Behavior A Behavior B

Figure 1: Differentiation mechanism for software adapta-

tion.

Our approach introduces the notion of

(de)differentiation into software components,

called agents that is like a cell, where each agent is

deﬁned as autonomous software consisting of one or

more functions. We provide hardware or physical

components in CPSs with proxies that can interact

with agents. The approach involves treating the

undertaking/delegation of functions in agents from/to

other agents as their differentiation factors. Each

function is a programmable entity and has its own

weight corresponding to the amount of cAMP and

this indicates its superiority. When an agent delegates

a function to another agent, if the former has the

function, its function becomes less-developed and

the latter’s function becomes well-developed. Agents

ICAART2013-InternationalConferenceonAgentsandArtificialIntelligence

258

may lose their functions due to differentiation as well

as being busy, partitioned, or having failed. The ap-

proach also offers a mechanism for recovering from

such problems based on dedifferentiation, which is a

mechanism for regressing specialized cells to simpler,

more embryonic, unspecialized forms. As happens

in the dedifferentiation process, if there are no other

agents that are sending restraining messages to an

agent, the agent can perform its dedifferentiation

process and strengthen its less-developed or inactive

functions again.

4 SELF-ADAPTATION

MECHANISM

Our approach is maintained through two parts: run-

time systems and agents. The former is a middleware

system for running on computers in CPSs and the lat-

ter is a self-contained and autonomous software en-

tity. It has three protocols for (de)differentiation and

delegation.

4.1 Software Component

Software components in CPSs are implemented

as agents, where each agent consists of one or

more functions, called the behavior parts, and its

state, called the body part, with information for

(de)differentiation, called the attribute part. The body

part maintains program variables shared by its behav-

iors parts like instance variables in object orientation.

The behavior part deﬁnes more than one application-

speciﬁc behavior. It corresponds to a method in ob-

ject orientation. The attribute part maintains descrip-

tive information with regard to the agent, including its

own identiﬁer.

The agent has behaviors b

, . . . , b

and w

is the

weight of behavior b

. Each agent (k-th) assigns its

own maximum to the total of the weights of all its

behaviors. The W

is the maximum of the weight of

behavior b

. The maximum total of the weights of

its behaviors in the k-th agent must be less than W

≥

∑

i=1

), where w

− 1 is 0 if w

is 0. The W

may depend on agents. In fact, W

corresponds to the

upper limit of the ability of each agent and may de-

pend on the performance of the underlyingsystem, in-

cluding the processor. Note that we never expect that

the latter will be complete, since agents periodically

exchange their information with neighboring agents.

Furthermore, when agents receive no retraining mes-

sages from others for longer than a certain duration,

they remove information about them.

4.2 Protocol for Function Invocation

When an agent wants to execute a behavior, it needs

to select one of the available behaviors (b

, . . . , b

even if it has the behavior, according to the values of

their weights. This involves three steps.

i) When an agent (k-th agent) wants to execute be-

havior b

, it looks up the weight (w

) of the

same or a compatible behavior from its database

and the weights (w

, . . . , w

) of such behaviors

, . . . , b

) from the database (Figure 2 (a))

ii) If multiple agents, includingitself, can provide the

wanted behavior, the k-th agent selects one of the

agents according to selection function φ

, which

maps from w

and w

, . . . , w

to b

, where l is k or

j, . . . , m.

iii) The k-th agent delegates the selected agent to ex-

ecute the behavior b

and waits for the result from

the l-th agent (Figure 2 (b)).

Body

part

Attribute

part

Agent B

Agent A

(b) Behavior selection according to weights

Behavior 1

Behavior 2

Behavior 3

Body

part

Attribute

part

Body

part

Attribute

part

Agent B

Agent A

Agent B

(a) Invocation request

Behavior 1

Behavior 2

Behavior 3

Body

part

Attribute

part

Figure 2: Selective invocation.

There is no universal selection function, φ, for map-

ping from the weights of behaviors to at most one ap-

propriate behavior like that in a variety of creatures.

Instead, the approach permits agents to use their own

evaluation functions, because the selection of behav-

iors often depends on their applications. For exam-

ple, one of the simplest evaluation functions makes

the agent that wants to execute a behavior select one

whose weight has the highest value and whose signa-

ture matches the wanted behavior if its database rec-

ognizes one or more agents that provide the same be-

havior, including itself.

4.3 Protocol for Adaptation

The approach introduces the undertaking/delegation

of behaviors in agents from other agents as a differen-

tiation factor. Behaviors in an agent, which are del-

egated from other agents more frequently, are well

developed, whereas other behaviors, which are dele-

gated from other agents less frequently, in the cell are

AdaptiveAgentsforCyber-PhysicalSystems

259

less developed. Finally, the agent only provides the

former behaviors and delegates the latter behaviors to

other agents. Our differentiation mechanism consists

of two phases. The ﬁrst involves the progression of

behaviors in three steps.

i) When an agent (k-th agent) receives a request

message from another agent, it selects the behav-

ior (b

) speciﬁed in the message from its behavior

part and dispatches the message to the selected be-

havior (Figure 3 (a)). It executes the b

behavior

and returns the result.

ii) The k-th agent increases weight w

of the b

be-

havior (Figure 3 (b)).

iii) The k-th agent multicasts a restraining message

with the signature of the behavior, its identiﬁer

(k), and the behavior’s weight (w

) to other agents

(Figure 3 (c)).

Body

part

Attribute

part

Agent B

Restraining

message

Agent A

Request message

Behavior 1

Behavior 2

Behavior 3

Body

part

Attribute

part

Body

part

Attribute

part

Agent B

Agent A

(b) Progression

Behavior 1

Behavior 2

Behavior 3

Body

part

Attribute

part

Body

part

Attribute

part

Agent B

Agent A

Agent B

(a) Behavior execution

Behavior 1

Behavior 2

Behavior 3

Body

part

Attribute

part

Incremental of weight

Well-developed

Figure 3: Differentiation-based adaptation on called side.

When behaviors are internally invoked by their

agents, their weights are not increased. If the total

weights of the agent’s behaviors,

∑

, is equal to

their maximal total weight W

, it decreases one of

the minimal (and positive) weights (w

is replaced

by w

− 1 where w

= min(w

, . . . , w

) and w

≥ 0).

Although restraining messages correspond to the dif-

fusion of cAMP in differentiation, they can explic-

itly carry the weights of the agents that send them to

reduce the number of restraining messages, because

they can be substituted for more than one retaining

message without weights. The second phase supports

the retrogression of behaviors in three steps.

i) When an agent (k-th agent) receives a restraining

message with regard to b

from another agent (j-

Figures 2, 3, and 4 assume W

and W

to be ten.

th), it looks for the behaviors (b

, . . . b

) that can

have the signature speciﬁed in the received mes-

sage (Figure 3 (c)).

ii) If it has such behaviors, it decreases their weights

, . . . w

) in its database and updates the weight

) in its database (Figure 4 (a)).

iii) If the weights (w

, . . . , w

) are under a speciﬁed

value, e.g., 0, the behaviors (b

, . . . b

) are inacti-

vated (Figure 4 (b)).

Body

part

Attribute

part

Agent B

Agent A

Behavior 1

Behavior 2

Behavior 3

Body

part

Attribute

part

(b) Regression

(a) Regression

Body

part

Attribute

part

Agent B

Agent A

Behavior 1

Behavior 2

Behavior 3

Body

part

Attribute

part

Less-developed

Dismission

Figure 4: Differentiation-based adaptation on other sides.

CPSs may be damaged or stop due to disasters and

problems. We need a mechanism for detecting and

remedying failures in networking, computers, agents,

remote computers, and other agents. To do this, each

agent (j-th) periodically multicasts messages, called

heartbeat messages, for behavior (b

), which is still

activated with its identiﬁer (j). This involves two

cases.

i) When an agent (k-th) receivesa heartbeat message

with regard to behavior (b

) from another agent

(j-th), it retains the weight (w

) of the behavior

) in its second database.

ii) When an agent (k-th) does not receive any heart-

beat messages with regard to behavior (b

) from

another agent (j-th) for a speciﬁed time, it au-

tomatically decreases the weight (w

) of the be-

havior (b

) in its second database, and resets the

weight (w

) of the behavior (b

) to the initial value

or increases the weight (w

) in its ﬁrst database

(Figure 5).

Note that behavior b

is provided by the k-th agent

and behavior b

is provided by the j-th agent. The

weights of behaviors provided by other agents au-

tomatically decrease without any heartbeat messages

from the agents. Therefore, when an agent terminates

or fails, other agents decrease the weights of the be-

haviors provided by the agent. If they have the same

or compatible behaviors, they can then activate the be-

haviors, which may be inactivated. After a request

ICAART2013-InternationalConferenceonAgentsandArtificialIntelligence

260

Body

part

Attribute

part

Agent B

Agent A

Behavior 1

Behavior 2

Behavior 3

Body

part

Attribute

part

(a) Isolation

(b) Dedifferentiated phase

Body

part

Attribute

part

Agent BAgent A

Behavior 1

Behavior 2

Behavior 3

Body

part

Attribute

part

Partitioning

Figure 5: Differentiation-based adaptation on other sides.

message is sent to another agent, if the agent waits for

the result to arrive for longer than a speciﬁed time,

it selects one of the agents that can handle the mes-

sage from its database and requests the selected agent.

If there are no agents that can provide the behavior

that can handle the behavior quickly, it promotes other

agents that have the behavior in less-developed form.

5 EVALUATION

In the current implementation, each runtime system

is constructed as a middleware system with Java. It

is responsible for executing agents and for exchang-

ing messages in runtime systems on other comput-

ers through a network. When a runtime system is

(re)connected to a network, it multicasts heartbeat

messages to other runtime systems to advertise it-

self, including its network address in a plug-and-play

protocol manner. Adaptation messages, i.e., restrain-

ing and heartbeat messages, are transmitted as multi-

cast UDP packets, which are unreliable. Application-

speciﬁc messages, i.e., request and reply, are imple-

mented through TCP sessions as reliable communica-

tions.

Let us suppose that a sensor-network system con-

sisting of 15 × 15 nodes is connected through a

grid network, as shown in Figure 6. Each node

can communicate with its four neighboring nodes

and the diameter of a circle in each node represents

the weight of a behavior. The system consists of

225 runtime systems running on twenty-ﬁve PCs (In-

tel Core 2 Duo 1.8 GHz, MacOS X 10.6, and Java

Runtime Environment ver.6) connected via a 1-GbE

switch, where each PC executes nine runtime sys-

tems.

Each runtime provided nine runtime systems

In this experiment, there were no serious differences

between this system and 225 servers, because our approach

was not processor- or memory- intensive. Since the loss

rate of UDP packets was less than 5%, we could ignore col-

running on different Java VMs. These runtime sys-

tems, called nodes in this section, were connected

according to the topology of the target grid network

and could multicast to four neighboring runtime sys-

tems through the grid network (Figure 6 (i)). We

also assumed that each node had two photosensors

for two different wavelengths, called lights A and B.

Each agent had two behaviors, called behaviors A

and B, where the ﬁrst responded to light in wave-

length A and the second to light in wavelength B.

Agents were distributed at all nodes. To emulate pho-

tosensors, we also deployed two (non-differentiated)

agents as pseudo-sensors agents at each node, where

such agents periodically notiﬁed their runtime sys-

tems of events every second. Some of the following

experiments explicitly limited the reachability of mes-

sages for differentiation-based adaptation, e.g., re-

straining and heartbeat messages within UDP multi-

cast domains.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

behavior in agent

15×15 grid network

i) 4-neighboring nodes

ii) 8-neighboring nodes

Message reachability from

center agent:

Figure 6: Visualization of Behavior weights in agents on

grid network.

We assumed light was irradiated in one wave-

length to all nodes in the ﬁrst experiment and each

node could only receive messages from at most four-

neighboring nodes (Figure 6 (i)). Two behaviors of

every agent were initially inactive because there was

no light. When we started pseudo-photosensor agents

for light A to emulate the irradiation of light A, all

agents activated behavior A (Figure 7 (i)). After ﬁve

seconds, some agents were more developed and the

others were less developed (Figure 7 (ii)). As a re-

sult, the weights of behavior-A in agents were speck-

led and unstable at nodes. Ten seconds after this, the

weights of the behaviors converged and stabilized in

a grill shape (Figure 7 (iii)). When we stopped irra-

diating light A, agents diverged and then inactivated.

This experiment proved that our approach enabled

software-level adaptations to converge on their proper

differentiated states.

We changed the reachability of messages to eight

lisions on the grid network in our performance evaluation in

the experiment.

AdaptiveAgentsforCyber-PhysicalSystems

261

ii) Weights (5 second later)

i) Initial weights

iii) Weights (10 second later)

Diameter is propostional to weight

Weight

Figure 7: Behavior weights in agents with the whole stimu-

lus on grid network.

neighboring nodes (Figure 6 (ii)). Like the ﬁrst ex-

periment, we irradiated light in one wavelength to

all nodes. Figure 8 (i) and (ii) show the weights

of behaviors in agents ten and twenty seconds later.

Even though differentiated behaviors were uneven,

they could be placed within certain intervals, i.s., two

edges on the grid network. This proved that our ap-

proach was useful in developing particular functions

of software components at nodes.

ii) Weights (10 second later)i) Initial weights iii) Weights (20 second later)

Diameter is propostional to weight

Weight

Figure 8: Behavior weights in agents with different message

reachability.

We still assumed the reachability of messages

eight neighboring nodes (Figure 6 (ii)). Figure 9 (i)

was the initial weights of agents on the network. We

explicitly made a ﬂawed part in the network (Figure 9

(ii)). Some agents dedifferentiate themselves in nodes

when a ﬂawed part made in the network. In the exper-

iment agents around the hole started to activate them-

selves through dedifferentiation. The weights of their

behaviors converged according to the weights of their

behaviors to the behaviors of other newly activated

agents in addition to existing agents. Finally, some

agents around the hole could support the behaviors

on behalf of the dismissed agents with the ﬂawed part.

This result provethat our approach could remedy such

a damage appropriatively in a self-organized manner.

This is useful managing CPSs for sensing catastro-

phes, e.g., earthquakes and deluges.

6 CONCLUSIONS

This paper proposed an approach to adapting soft-

ware components on CPSs. It is unique to other

ii) Partial destructioni) Initial weights iii) 20 second later

Figure 9: Adaption to ﬂawed part in network.

existing software adaptations in introducing the no-

tions of (de)differentiation and cellular division in

cellular slime molds, e.g., dictyostelium discoideum,

into software components. When a software com-

ponent delegates a function to another component,

if the former has the function, its function becomes

less-developedand the latter’s function becomes well-

developed. The approach was constructed as a mid-

dleware system on CPSs instead of any simulation-

based systems. Components can be composed from

Java objects written in JavaBean. We described sev-

eral evaluations of the approach with a practical ap-

plication.

REFERENCES

Bonabeau, E., Dorigo, M., and Theraulaz, G. (1999).

Swarm Intelligence: From Natural to Artiﬁcial Sys-

tems. Oxford University Press.

Cheng, S., Garlan, D., and Schmerl, B. (2006).

Architecture-based self-adaptation in the presence of

multiple objectives. In Proceedings of International

Workshop on Self-adaptation and Self-managing Sys-

tems (SEAMS’2006), pages 2–8. ACM Press.

Dorigo, M. and Stutzle, T. (2004). Ant Colony Optimiza-

tion. MIT Press.

Georgiadis, I., Magee, J., and Kramer, J. (2003). Self-

organising software architectures for distributed sys-

tems. In Proceedings of 1st Workshop on Self-healing

systems (WOSS’2002), pages 33–38. ACM Press.

Herrman, K. (2008). Self-organizing Ambient Intelligence.

VDM.

Satoh, I. (2007). Self-organizing software components in

distributed systems. In Proceedings of 20th Interna-

tional Conference on Architecture of Computing Sys-

tems (ARCS’2007), volume 4415 of Lecture Notes in

Computer Science (LNCS), pages 185–198. Springer.

Satoh, I. (2011). Cellular differentiation-based service

adaptation. In Proceedings of International Confer-

ence on Service Oriented Computing. Springer.

Satoh, I. (2012). Resilient architecture for sensor networks.

In Proceedings of 9th International Conference on

Networked Sensing Systems (INSS’2012). IEEE Com-

puter Society.

ICAART2013-InternationalConferenceonAgentsandArtificialIntelligence

262