SMART SEMANTIC MIDDLEWARE FOR THE INTERNET OF

THINGS

Artem Katasonov, Olena Kaykova, Oleksiy Khriyenko, Sergiy Nikitin and Vagan Terziyan

Agora Center, University of Jyv¨askyl¨a, P.O.Box 35, 40014, Finland

Keywords:

Internet of Things, Middleware, Interoperability, Heterogeneous Resources, Software Agents, Semantic Tech-

nologies, Ontologies.

Abstract:

As ubiquitous systems become increasingly complex, traditional solutions to manage and control them reach

their limits and pose a need for self-manageability. Also, heterogeneity of the ubiquitous components, stan-

dards, data formats, etc, creates signiﬁcant obstacles for interoperability in such complex systems. The promis-

ing technologies to tackle these problems are the Semantic technologies, for interoperability, and the Agent

technologies for management of complex systems. This paper describes our vision of a middleware for the

Internet of Things, which will allow creation of self-managed complex systems, in particular industrial ones,

consisting of distributed and heterogeneous components of different nature. We also present an analysis of

issues to be resolved to realize such a middleware.

1 INTRODUCTION

Recent advances in networking, sensor and RFID

technologies allow connecting various physical world

objects to the IT infrastructure, which could, ulti-

mately, enable realization of the Internet of Things

and the Ubiquitous Computing visions. Also, this

opens new horizons for industrial automation, i.e. au-

tomated monitoring, control, maintenance planning,

etc, of industrial resources and processes. A much

larger than in present number of resources (machines,

infrastructure elements, materials, products) can get

connected to the IT systems, thus be monitored and

potentially controlled. Such development will also

necessarily create demand for a much wider integra-

tion with various external resources, such as data stor-

ages, information services, and algorithms, which can

be found in other units of the same organization, in

other organizations, or on the Internet.

The interconnectivity of computing and physical

systems could, however, become ”the nightmare of

ubiquitous computing” (Kephart and Chess, 2003)

in which human operators will be unable to manage

the complexity of interactions in the system, neither

even architects will be able to anticipate that com-

plexity, and thus to design the system. The IBM

vision of autonomic computing (Kephart and Chess,

2003) proclaims the need for computing systems ca-

pable of ”running themselves” with minimal human

management which is mainly limited to deﬁnition of

some higher-level policies rather than direct admin-

istration. The computing systems will therefore be

self-managed, which, according to the IBM vision,

includes self-conﬁguration, self-optimization, self-

protection, and self-healing. The IBM vision empha-

sizes that the run-time self-manageability of a com-

plex system requires its components to be to a cer-

tain degree autonomous themselves. Following this,

we envision that the software agent technologies will

play an important part in building such complex sys-

tems. Agent-based approach to software engineering

is considered to be facilitating the design of complex

systems (Jennings, 2001). A signiﬁcant attention is

paid in the ﬁeld of multi-agent systems to the task of

building decentralized systems capable of supporting

spontaneous conﬁguration, tolerating partial failures,

or arranging adaptive reorganization of the whole sys-

tem (Mamei and Zambonelli, 2006).

A major problem is inherent heterogeneity in

ubiquitous computing systems, with respect to the na-

169

Katasonov A., Kaykova O., Khriyenko O., Nikitin S. and Terziyan V. (2008).

SMART SEMANTIC MIDDLEWARE FOR THE INTERNET OF THINGS.

In Proceedings of the Fifth International Conference on Informatics in Control, Automation and Robotics - ICSO, pages 169-178

DOI: 10.5220/0001489001690178

 SciTePress

ture of components, standards, data formats, proto-

cols, etc, which creates signiﬁcant obstacles for in-

teroperability among the components. Semantic tech-

nologies are viewed today as a key technology to re-

solve the problems of interoperability and integration

within heterogeneous world of ubiquitously intercon-

nected objects and systems. Semantic technologies

are claimed to be a qualitatively stronger approach to

interoperability than contemporary standards-based

approaches (Lassila, 2005). The Internet of Things

should become in fact the Semantic Web of Things

(Brock and Schuster, 2006). We subscribe to this

view. Moreover, we apply semantic technologies not

only to facilitate the discovery of heterogeneous com-

ponents and data integration, but also for the behav-

ioral control and coordination of those components

(i.e. prescriptive speciﬁcation of the expected be-

haviour, declarative semantic programming).

It seems to be generally recognized that achiev-

ing the interoperability by imposing some rigid stan-

dards and making everyone comply could not be a

case in ubiquitous environments. Therefore, the in-

teroperability requires existence of some middleware

to act as the glue joining heterogeneous components

together. A consistent set of middleware, offering

application programming interfaces, communications

and other services to applications, will simplify the

creation of applications and help to move from static

programming approaches towards a conﬁgurable and

dynamic composition capability (Buckley, 2006).

There are a couple of EU FP6 research projects

that have as one of their goals the develop-

ment of some middleware for embedded systems.

They are RUNES (Reconﬁgurable Ubiquitous Net-

worked Embedded Systems, 2004-2007) and ongo-

ing SOCRADES (Service-Oriented Cross-Layer In-

frastructurefor Distributed Smart Embedded Devices,

2006-2009). We believe, however, that the middle-

ware needs of the Internet of Things domain go well

beyond interconnectivity of embedded systems them-

selves. There is a more general need for middle-

ware to enable something we refer to as Global Enter-

prise Resource Integration(GERI), where all different

types of resources get seamlessly integrated: physi-

cal devices with embedded electronics, web services,

software applications, humans along with their inter-

faces, and other. In the concept of GERI, we also

stress the need for true global interoperability, not

just interconnectivity. The components of ubiquitous

computing systems should be able not only to com-

municate and exchange data, but also to ﬂexibly co-

ordinate with each other, discover and use each other,

and jointly engage in different business processes.

Such more general middleware needs are em-

phasized in the Strategic Research Agenda (SRA)

of the ARTEMIS European Technology Platform.

ARTEMIS’ SRA has ”Seamless Connectivity and

Middleware” as one of its three parts. Some of

the relevant research priorities listed are the middle-

ware as the key enabler for declarative programming

paradigm, where the components and their interac-

tion are deﬁned and conﬁgured declaratively rather

than programmatically (and we believe that the se-

mantic technologies are a natural candidate here), ef-

ﬁciently bridging information between global, enter-

prise, and embedded systems, use of ontologies for

cross-domain systems’ organization and for interop-

erability in heterogeneous environments, dynamic re-

conﬁguration capabilities, adaptive resource manage-

ment, and appropriate security infrastructures.

In this paper, we describe our vision of such a

middleware for the Internet of Things, which has also

formed the basis for our research project UBIWARE.

The project aims at a new generation middleware plat-

form which will allow creation of self-managed com-

plex systems, in particular industrial ones, consist-

ing of distributed, heterogeneous, shared and reusable

components of different nature, e.g. smart machines

and devices, sensors, RFIDs, web-services, software

applications, humans along with their interfaces, and

others. Such middleware will enable various compo-

nents to automatically discover each other and to con-

ﬁgure a system with complex functionality based on

the atomic functionalities of the components.

The rest of the paper is structured as follows. Sec-

tion 2 describes our general vision. Section 3 presents

an analysis of how the goals of UBIWARE can be

achieved. The result is a set of sub-problems, which

we address in corresponding work-packages of the

project. Section 4 describes several industrial cases

(application areas) that we consider in the project;

those are proposed by the project’s industrial partners.

Finally, Section 5 concludes the paper.

2 THE GENERAL VISION

This section describes our general vision of the smart

semantic middleware for the Internet of Things. We

believe that tasks of automatic integration, orches-

tration and composition of complex systems on the

Internet of Things will be impossible in a central-

ized manner due to the scalability and other issues.

Therefore, the components of such systems should

be to a certain degree autonomous and proactive. In

other words, utilization of the agent technologies is

needed to enable ﬂexible communication and coordi-

nation of the components. Agent technologies also al-

low mobility of service components between various

platforms, decentralized service discovery, utilization

ICINCO 2008 - International Conference on Informatics in Control, Automation and Robotics

170

Figure 1: The conceptual architecture: resources, adapters, agents and ontology.

of FIPA communication protocols, and negotiation-

based integration/composition of services. Interoper-

ability among the components requires use of meta-

data and ontologies. As the amount of components

can grow dramatically, without their ontological clas-

siﬁcation and semantic annotation processes, the au-

tomatic discovery will be impossible.

Figure 1 depicts our vision. In this vision, each

resource to be connected has a representative – an au-

tonomous software agent (a proactive ”player” within

certain integration scenario). This agent is assumed to

be able to monitor the state of the resource, make de-

cisions on the behalf on the resource, and to discover,

request and utilize external help if needed. The con-

nection between the resource itself and its agent is or-

ganized through the semantic mediation of an adapter

(or interface). Such an adapter may include (if nec-

essary) sensors with digital output, data structuring

(e.g. XML), semantic adapter components (convert-

ing to a semantic representation), and actuators. Ac-

cording to this vision, each relevant component of a

ubiquitous system is to become a ”smart resource”

- proactive and self-managing. The part-of hierarchy

of resources may result in corresponding hierarchy of

the agents. This can also be recursive. For exam-

ple, an adapter of a system component can become a

smart resource itself, i.e. it can have its own respon-

sible agent, semantically adapted sensors and actua-

tors, history, commitments with other resources, and

self-monitoring and self-maintenance activities. This

could be done to guarantee high level of dynamism

and ﬂexibility of the adapter.

The semantic technologies have in our vision a

two-fold value. First, they are the basis for the discov-

ery of heterogeneous resources and data integration

across multiple domains (a well-known advantage).

Second, they are used for behavioural control and co-

ordination of the agents representing those resources

(a novel use). Therefore, semantic technologies are

used both for descriptive speciﬁcation of the services

delivered by the resources and for prescriptive speci-

ﬁcation of the expected behaviour of the resources as

well as the integrated system. Such two-fold applica-

tion of semantics can, in ideal case, lead to a ”global

understanding” between the resources. This means

that a resource A can understand all of (1) the prop-

erties and the state of a resource B, (2) the potential

and actual behaviors of B, and (3) the business pro-

cesses in which A and B, and maybe other resources,

are jointly involved.

The UBIWARE is to be, roughly speaking, a set

of tools providing means for effective development

of agents and adapters, and a run-time environment

for the operation of those. An important central com-

ponent of UBIWARE is the agent core (see Section

3.1). Each agent is assumed to be created based on

this core and specialized through semantic declara-

tive programs and a set of Reusable Atomic Behav-

iors (RAB). Combination of the run-time environ-

ment, the agent core and a set of standard semantic

programs and RABs will make UBIWARE a plat-

form providing both (semantic) communication ser-

vices and collaboration-support (scenario-driven inte-

gration) services for heterogeneous resources.

In a sense, our intention is to apply the con-

cepts of automatic discovery, composition, orches-

tration, integration, execution monitoring, communi-

cation, negotiation, context awareness, etc. (which

were, so far, mostly related only to the Semantic Web-

Services domain) to a more general ”Semantic Web

SMART SEMANTIC MIDDLEWARE FOR THE INTERNET OF THINGS

171

of Things” domain. Also we want to expand this

list by adding automatic self-management including

(self-*)organization, diagnostics, forecasting, conﬁg-

uration, tuning, and maintenance.

3 ANALYSIS

This section presents our analysis of how the goals of

UBIWARE can be achieved. The result is a set of sub-

problems, which we address in corresponding work-

packages of the project. UBIWARE project aims at

a relatively complete and self-sufﬁcient middleware

platform. Therefore, it elaborates on our central ideas

(Section 2), and also works towards solutions in sup-

porting but mandatory-to-treat areas such as security,

human interfaces and other.

3.1 Core Agent Platform

Although the ﬂexibility of agent interactions has

many advantages when it comes to engineering com-

plex systems, the downside is that it leads to unpre-

dictability in the run time system; as agents are au-

tonomous, the patterns and the effects of their inter-

actions are uncertain (Jennings, 2000). It is common

in speciﬁc systems to circumvent these difﬁculties by

using interaction protocols whose properties can be

formally analyzed, by adopting rigid and preset or-

ganizational structures, and/or by limiting the nature

and the scope of the agent interplay. However, these

restrictions also limit the power of the agent-based ap-

proach; thus, in order to realize its full potential some

longer term solutions are required (Jennings, 2000).

Realization of the UBIWARE vision requires a

reliable core platform that would provide means for

building systems that are ﬂexible and consist of au-

tonomous components, yet predictable in operation.

Two important research directions, acknowledged in

the literature, are: social level characterization of

agent-based systems, and ontological approaches to

coordination. The former direction presents the need

for a better understanding of the impact of sociality

and organizational context on an individuals behavior

and of the symbiotic link between the behavior of the

individual agents and that of the overall system (Jen-

nings, 2000). In particular, it requires modeling be-

havior of an agent as being deﬁned or restricted by the

roles the agent plays in one or several organizations

(Vazquez-Salceda et al., 2005). Role-based modelling

also provides the advantage of separation of concerns

and in so design of complex systems (Cabri et al.,

2004). The latter direction presents the need to enable

agents to communicate their intentions with respect to

future activities and resource utilization and to reason

about the actions, plans, and knowledge of each other,

in real time (Tamma et al., 2005).

Our previous work resulted in a platform (Kata-

sonov and Terziyan, 2007) (Figure 2) that has done

some steps into both these directions. It can be seen as

consisting of three layers: reusable atomic behaviors

(RAB), behavior models corresponding to different

roles the agent plays, and the behavior engine (or the

agent core). A RAB is a Java component implement-

ing a reasonably atomic function (sensing, acting or

data processing). A behavior model is an RDF-based

document specifying a certain organizational role. A

behavior model consists of a set of beliefs represent-

ing the knowledge needed for playing the role and a

set of behavior rules. Roughly speaking, a behavior

rule speciﬁes conditions of (and parameters for) exe-

cution of a RAB. The behavior engine is responsible

for parsing RDF-based scripts, and it implements the

run-time loop of an agent. In the platform, the agents

access the behavior models from an external reposi-

tory, which is assumed to be managed by the organi-

zation which ”hires” the agents to enact those roles.

As can be seen from the picture, the platform allows

also on-demand access of RABs.

Such a 3-layer agent architecture with external-

ization of behavior models and on-demand access of

atomic code components provides a basis for devel-

opment of the UBIWARE core platform. At present,

we are working on the following important research

issues: (1) Development of the language for roles’

scripts towards a full-scale Semantic Agent Program-

ming Language. (2) Implementation of the separa-

tion between a role’s capabilities (individual function-

ality), and the business processes in which this role

can be involved (complex functionality). (3) Devel-

opment of mechanisms for ﬂexible treating the poten-

tial (and likely) conﬂicts among the roles played by

one agent. (4) Development of mechanisms to en-

able agents to ﬂexibly discover each other, based both

on the roles played and on particular capabilities pos-

sessed. (5) Analysis of concrete beneﬁts and mecha-

nisms for accessing and using a role’s script by agents

who are not playing that role but wish to coordinate or

interact with an agent that does. (6) Making an agent’s

roles to be higher-level commitments of the agent that

restrict its behavior, still leaving freedom for learning

and adaptation on lower-levels, instead of totally and

rigidly prescribing the behavior.

3.2 Distributed Resource Histories

In the UBIWARE vision, every resource is repre-

sented by an agent. Among the responsibilities of the

agent is monitoring the condition of the resource and

ICINCO 2008 - International Conference on Informatics in Control, Automation and Robotics

172

Figure 2: The current core platform.

the resource’s interactions with other components of

the system and humans. The beliefs storage of the

agent will, therefore, naturally include the history of

the resource, in a sense ”blogged” by the agent. Obvi-

ously, the value of such a history is not limited to that

particular resource. A resource may beneﬁt from the

information collected with respect to other resources

of the same (or similar) type, e.g. in a situation which

it faces for the ﬁrst time while other may have faced

that situation before. Also, mining the data collected

and integrated from many resources may result in dis-

covery of some knowledge important at the level of

the whole ubiquitous computing system.

A straightforward approach would be maintaining

a central repository integrating the histories of all the

resources of the system. However, such an approach

is often impractical in realistic dynamic applications

because even just keeping one agent informed of all

the events and actions in the system would swamp

the available bandwidth, and also such agent would

become a severe bottleneck and might render the re-

maining components unusable if it failed (Jennings,

1993). A scalable solution requires mechanisms for

inter-agent information sharing and data mining on

integrated information which would allow keeping

the resource histories distributed without need to copy

those histories to a central repository.

At present, we are working on the following im-

portant research issues: (1) Development of mech-

anisms for representing the history of a resource in

a system making it reusable for other resources and

at the system-level. (2) Development of mechanisms

for effective and efﬁcient sharing of information be-

tween the agents representing different resources. (3)

Development of mechanisms for querying and inte-

gration of responses from distributed, autonomous,

and, hence, inevitably semantically heterogeneous re-

source histories. (4) Enabling mining (utilizing intel-

ligent data mining and machine learning techniques)

of distributed histories.

3.3 Peer-to-Peer Discovery

Following the IEEE FIPA agent system model, the

UBIWARE platform is to include a system agent

called Directory Facilitator (DF). In UBIWARE, the

Directory Facilitator maintains a mapping between

agents and the roles they play. If the behavior model

of an agent X (see Section 3.1) prescribes the need of

interaction with another agent Y, the agent Y is always

speciﬁed by its role, not the name or another unique

identiﬁer of a particular agent. Therefore, the agent X

must contact the Directory Facilitator in order to dis-

cover the unique ID (needed for communication) of

the agent or agents playing the role needed. Also, be-

cause every agent plays at least one role, DF naturally

has a list of all the agents on the platform. This can be

used, e.g., when an agent needs some information and

wants to broadcast the request for that information to

all the agents on the platform (see Section 3.2).

Existence of such a Directory Facilitator is an ef-

fective and efﬁcient solution. However, DF obviously

presents a severe bottleneck in the system, and can

render the whole system unusable if it failed. To im-

prove the survivability of UBIWARE-based systems,

there has to be a complementary mechanism which

can be utilized in an exception situation where DF be-

came for some reason unavailable.

After the system is deployed and operational for

some time, a signiﬁcant part of the DF knowledge,

piece by piece, ends up in local knowledge storages

of different platform agents. The combination of the

local storages presents thus a kind of distributed direc-

tory. Therefore, a Peer-to-Peer (P2P) mechanism im-

SMART SEMANTIC MIDDLEWARE FOR THE INTERNET OF THINGS

173

plemented on such a distributed directory can work as

the needed mechanism complementary to the central

DF. Of course, replicating DF is another and proba-

bly simpler option. It would not, however, provide the

same level of survivability as P2P. Also, in some busi-

ness scenarios, it is possible that some of the services

would prefer not to advertise themselves through the

central DF altogether, for security or other reasons.

Therefore, P2P discovery of such services would not

be an exception path but the only viable solution.

The objective here is the design of mechanisms

which will extend the scale of semantic resource

discovery in UBIWARE with P2P discovery. Such

mechanisms have to enable an agent (1) to discover

agents playing a certain organizational role, (2) to dis-

cover an agent (or agents) possessing certain needed

information, and (3) to discover resources (through

its agents) of certain type or possessing certain prop-

erties (e.g. a device in state X, a web-resource provid-

ing some information searched for, or a human with

some speciﬁc skills). In all cases, existence of a cen-

tral DF is not assumed. Rather, the request is sent to

all/some of the agents on the contact list of the agent

in question (ones with who it has a history of commu-

nication, or at least about whom it heard from others).

Those agents can forward the request to all/some of

the agents on their lists, and so on.

3.4 Conﬁgurability

UBIWARE aims to be a platform that can be applied

in different application areas. Therefore, the elements

of the platform have to be adjustable, could be tuned

or conﬁgured allowing the platform to run different

business scenarios in different environments. Such

ﬂexibility calls for existence of a special conﬁgura-

tion layer of the platform. All building blocks of the

platform, i.e. software agents, agent behaviors, re-

source adapters, etc, become subject to conﬁguration.

Also, a ﬂexible system should have a long lifespan.

Hence, the platform should allow extensions, compo-

nent replacements, and component adjustments dur-

ing the operation time. In addition, every agent in

UBIWARE is self-aware and self-manageable entity.

Therefore, it may evolve with time and modify some

of its functional or non-functional properties. At the

same time, we need a stable and predictable environ-

ment for running business scenarios. Thus, the adjust-

ment made to a componentmust also be propagatedto

higher platform levels such as business processes and

contracting. Also vice versa, a change in a business

process may require some adjustments to be made to

several participating components.

On the level of software design, we aim at deﬁning

patterns for development of conﬁgurable elements.

On the level of inter-agent communication, we aim at

establishing some protocols for negotiation in differ-

ent conﬁgurability cases. For example, when an agent

changes its behavior, a check must be performed for

the consequences of this change for other compo-

nents, and for who should be informed about this

change. Also, there should be a mechanism for re-

stricting the range of possible adjustments to certain

functionality of certain components in order to main-

tain the consistency in critical business processes.

An important issue is conﬁgurability of resource

adapters. There is a need for a methodology both

for creating such adapters and for their run-time re-

conﬁguration. It is reasonable to expect, e.g., that

when a resource modiﬁes the format of its output, the

existing adapter can be adjusted to this new format

without a need to build a new adapter from scratch.

Conﬁguration of business processes poses a sepa-

rate problem. A business process is an abstract entity

which mainly consists of a ﬂow of messages between

agents. There is nothing like a business process exe-

cution engine. There could be a dedicated responsible

agent, which would coordinate the involved agents,

resolve conﬂicts and exceptions. However, similarly

to the case of a centralized history storage (see Sec-

tion 3.2), such a controller agent would become a se-

vere bottleneck and might render the remaining com-

ponents unusable if it failed. In a most scalable solu-

tion, a business process would be just a set of agents

with some commitments, i.e. signed Service Level

Agreements. In such autonomic medium, we will

need a (re-)conﬁguration mechanism for stable error-

tolerant operation of components leading to success-

fully achieving the business goals.

3.5 Security in UBIWARE

The security is often seen as an add-on feature of a

system. However, in many systems (and UBIWARE

is one of them), the system remains nothing more but

a research prototype, without a real potential of prac-

tical use, until an adequate security infrastructure is

embedded into it. UBIWARE will advance existing

technologies to a qualitatively new level and bring

to life new complex ubiquitous environments, where

traditional approaches to manage security fall short.

Also, existing security measures for the technologies

on which UBIWARE relies, e.g. multi-agent, are not

in a mature stage and still require signiﬁcant elabora-

tion to mitigate associated risks.

Traditional security goals like conﬁdentiality,

availability, reliability, integrity, manageability, ac-

countability, responsibility etc, together with conven-

tional measures and mechanisms that support secu-

rity, do not cover all the needs and threats of new

ICINCO 2008 - International Conference on Informatics in Control, Automation and Robotics

174

emerging computing environments. Ambient intelli-

gence and ubiquity of information technologies have

tightened the digital and physical worlds to the ex-

tent when security becomes the ultimate issue. The

major implication of penetrating ICTs on security is

that the risks and negative consequences of security

threats become higher than ever. On the other hand,

the security infrastructure itself has to become per-

vasive, interoperable and intelligent enough to natu-

rally ﬁt UBIWARE. The security cannot be added to

the UBIWARE platform later but the design decisions

regarding security have to be thoroughly correlated

with the requirements, characteristics and design of

the platform, due to mutual impact on resulting fea-

tures of UBIWARE.

The main objective here is the design of an in-

frastructure for policy-based optimal collecting, com-

posing, conﬁguring and provisioning of security mea-

sures in multi-agent systems like UBIWARE (Nau-

menko et al., 2007). This work is to follow the gen-

eral UBIWARE vision – conﬁguring and adding new

functionality on-the-ﬂy by changing high leveldeclar-

ative descriptions. Regarding security, this means that

UBIWARE will be able of smoothly including new,

and reconﬁguring existing, security policies (simi-

lar to behavior models) and mechanisms (similar to

RABs) in response to the dynamically changing en-

vironment. The optimal state is always a tradeoff be-

tween security and other qualities like performance,

functionality, usability, applicability and other.

3.6 Smart Interfaces

In UBIWARE, humans are important resources,

which can play several distinct roles: (1) User – one

getting some information or services from other re-

sources. (2) Resource under care – one under online

care of the integrated system (e.g. monitoring the

health of an employee). (3) Service provider – one

providing services to other resources (e.g. a main-

tenance expert). (4) UBIWARE administrator – one

monitoring and conﬁguring the integrated system.

Obviously, humans need graphical interfaces to

interact with the rest of the system. The same person

can play several roles, switch between them depend-

ing on the context, and, in result, require different in-

terfaces at different times. In addition, a UBIWARE-

based system presents a large integration environment

with potentially huge amounts of heterogeneous data.

Therefore, there is a need for tools facilitating infor-

mation access and manipulation by humans.

From the UBIWARE point of view, a human inter-

face is just a special case of a resource adapter. We be-

lieve, however, that it is unreasonable to embed all the

data acquisition, ﬁltering and visualization logic into

such an adapter. Instead, external services and appli-

cation should be effectively utilized. Therefore, the

intelligence of a smart interface will be a result of col-

laboration of multiple agents: the human’s agent, the

agents representing resources of interest (those to be

monitored or/and controlled), and the agents of vari-

ous visualization services. This approach makes hu-

man interfaces different from other resource adapters

and indicates a need for devoted research. There is

a need to enable creation of such smart human inter-

faces through ﬂexible collaboration of an Intelligent

GUI Shell, various visualization modules, which we

refer to as MetaProvider-services, and the resources

of interest (Khriyenko, 2007).

A MetaProvider is responsible for acting as a por-

tal so that various relevant resources can register on

it, for data integration, for context-dependent ﬁlter-

ing (inclusion of only relevant resources and relevant

properties of those resources), and for creating visu-

alizations of data. The GUI shell is, in turn, responsi-

ble for context-dependentselection of MetaProviders,

communication with them, and cross-MetaProvider

browsing and integration.

Based on such an approach, an infrastructure will

be embedded into UBIWARE enabling effective real-

ization of the following system functions: (1) Visu-

alization of data provided by a service in response to

a request. (2) Search, retrieving and visualization of

data required by a human expert. (3) Providing access

to contextual information, and visualization of it. (4)

Visualization of resource registration, conﬁguration,

and security policy establishment processes. (5) Re-

source discovery via MetaProviders (because they act

as thematic portals).

4 INDUSTRIAL CASES

This section describes several industrial cases (appli-

cation areas) that we consider in our project. These

cases are proposed by UBIWARE industrial part-

ners and are being analyzed, designed and prototyped

based on the UBIWARE platform. These cases pro-

vide examples of beneﬁts that a smart semantic mid-

dleware can bring into various industries.

4.1 Power Network Maintenance

One of the project’s partners is a vendor of hardware

and software for power networks. With respect to

UBIWARE, this partner’s main area of interest is in

extending, mainly through integration with external

resources, the functionality of its existing software

systems. These systems provide a graphical view over

SMART SEMANTIC MIDDLEWARE FOR THE INTERNET OF THINGS

175

the power network, data acquisition from the substa-

tions and remote control over the relays, switches, etc.

They also include implementations of various algo-

rithms: for fault localization, for calculation of opti-

mal reconﬁguration of the network and other.

The UBIWARE technology could allow connect-

ing to the existing products some new system intel-

ligence tools, for example, statistical and data min-

ing tools. One case that is considered is about possi-

bility of using data mining techniques for automated

interpretation of the situation in the power network.

The motivating issue is that a certain condition, e.g.

a fault, in the power network causes a series of alert

messages to be sent to the operation center. It remains

a responsibility of a human operator to understand

the reason underlying such a sequence of alert mes-

sages. Based on UBIWARE, a system could be cre-

ated allowing that: (1) alert data is automatically col-

lected, (2) the experts are able to annotate sequences

of events with their reasons, (3) data mining algo-

rithms are applied to discover some patterns, and (4)

the operator is provided with an interface giving a

more structured view of events (e.g. ﬁltered based on

the source of event) and including the system’s inter-

pretation (from the models built) of what is the mean-

ing of the situation and its reason.

In collaboration with this partner, we analysed

further the potential add-value that the company and

their customers, i.e. electrical companies, could re-

ceive from introducingUBIWARE into their business.

Below, we sketch several scenarios when UBIWARE

can enable new features, or help otherwise.

One scenario is related to extending the user inter-

faces, so that various groups of users could get ﬂexi-

ble access to data and functionality present in the ex-

isting products. Traditionally, those software products

are only used inside the walls of operation centers by

the network operators, through proprietary user inter-

faces. The data and functionality of those systems,

however, has a value beyond that use. A UBIWARE-

based solution will enable a more ubiquitous and ﬂex-

ible information access to data and algorithms and

will extend the user base to, e.g., maintenance work-

ers, management, etc.

Another scenario arises from the fact that the

medium-voltagesub-networks of the integral network

are usually owned, controlled and maintained then by

some local companies. It is noticeable that the oper-

ation centers of different companies have no connec-

tion to each other, so information exchange among

them is nearly impossible. In the case of a fault af-

fecting two different sub-networks, such information

exchange, though, may be very important, for all of

fault localization, network reconﬁguration, and net-

work restoration. Introducing an inter-organizational

system based on UBIWARE could solve this issue.

The information ﬂow will go through the agents rep-

resenting the sub-networks on the UBIWARE plat-

form. Utilization of semantic technologies will al-

low such interoperability even if the sub-networks

use software systems from different vendors, and thus

maybe different data formats and protocols.

One more scenario is related to a new business

model. At present, all expertise of the company gets

embedded into hardware or software systems and sold

to the customers as it is. A new business model would

be to start own Web-service providing implementa-

tion of certain algorithms, so the customers will uti-

lize those algorithms online when needed. The com-

pany will be always able to update algorithms, add

new, and so on. UBIWARE will ensure interoperabil-

ity and coordination between such Web-service and

customers’ software systems, and also a relative ease

of implementation of such a solution – because it will

not require changes in existing software systems, only

extension with the UBIWARE platform. Noticeable

that, if semantically deﬁned, such Web-service can

potentially be utilized across the globe even by the

customers who never purchased any of the company’s

hardware or software.

The next scenario is related to the possibility of in-

tegrating data, which is currently utilized in the power

network management (network structure and conﬁg-

uration, feeder relay readings), with contextual in-

formation from the external sources. Such integra-

tion can be used in at least three tasks. The ﬁrst

is risk analysis. Information about weather condi-

tions, ongoing forest works, or forest ﬁres can be

used for evaluating existing threats for the power net-

work. This may be used to trigger an alert state for

the maintenance team, or even to do a precaution-

ary reconﬁguration of the network to minimize pos-

sible damage. The second is facilitation of fault lo-

calization. The output of fault localization algorithms

is not always certain. The information about threats

for the power network that existed at the time when

the fault occurred (which thus may have caused the

fault) may greatly facilitate the localization. In some

situations, contextual information alone may even be

sufﬁcient for localization. The third is operator in-

terface enhancement. Contextual information may

be used also just to extend the operators’ view of

the power network. For example, satellite imagery

can be used for geographic view (instead of locally

stored bitmaps as it is in the current systems); also,

dynamically-changing information can be accessed

and represented on the interface.

The last scenario is about the possibility of trans-

ferring the knowledge of human experts to automated

systems, by means of various data mining tools. In

ICINCO 2008 - International Conference on Informatics in Control, Automation and Robotics

176

the power network management case, one scenario

that seems to be highly appropriate for such knowl-

edge transfer is the following. In present, it is always

a decision of a human expert which of the existing

fault localization algorithms will perform the best in

the context of the current conﬁguration of the power

network and the nature of the fault. Such decisions

made by an expert along with the input data could, be

forwarded to a learning Web-service. After a sufﬁ-

cient learning sample, this Web-service could start to

be used in some situations instead of the human ex-

pert, e.g. in situations when a faster decision is needed

or when the expert is unavailable.

4.2 Maintenance of Paper Machines

Another of the project’s partners is a supplier of ma-

chinery and automation systems for a set of industries

including the paper industry. With respect to UBI-

WARE, this partner’s main areas of interest are in

information integration and analytical data process-

ing. To support customers with additional informa-

tion along the product (a paper machine) lifecycle,

the company foresees the need for intelligent prod-

uct history management. Such a history is supposed

to integrate alert reports, experts’ diagnoses, mainte-

nance work performed, maintenance costs, and other

types of information.

Every paper machine, installed at a customer’s

site, is equipped with a set of sensors and embedded

intelligence for observing the state of the machine and

alarming when an exceptional situation occurs. The

alarm information is forwarded to the central hub for

diagnostics and decision making.

First of all, UBIWARE can provide means for

proactive integration of alarm data with different cor-

porate information systems, such as the engineering

database, the ﬁnancial database. This will give a

common basis for product history management and

further intelligent data processing. UBIWARE can

also enable integration with external web services to

provide more powerful analytical processing of data.

Also, the ﬂexibility and ease of extension via incor-

poration of external functionality will enable evolu-

tion of the service infrastructure to meet the customer

needs in a sustainable fashion. The UBIWARE plat-

form will also enable a high level of security in the

collaborative work of the company and its customers.

Among the central problems to be solved by UBI-

WARE, there are those related to the evolution of the

service infrastructure. During the paper machine op-

eration lifecycle, formats of messages or algorithms

for issuing alarms may change. Furthermore, some

changes to the supporting ERP or other automation

systems may take place. In other words, data for-

mats and data structures, which were bound to on-

tology, may require re-adaptation (i.e. adapter re-

conﬁguration) to become consistent with the inte-

grated storage and applications. From a life-time

perspective, the complexity of business processes be-

tween the company and the customer may become an-

other obstacle for updates and renovation of the ser-

vice infrastructure. It is because such changes are

hard to trace and, therefore, it might be difﬁcult to

analyze their impact on the overall system.

4.3 Telecom Operator’s Service Desk

One more project’s partner is a provider of telecom-

munication services, both wireless and ﬁxed-line.

With respect to UBIWARE, this partner’s main area

of interest is in possibility of constructing, mainly

through integration of existing components, systems

that would (partially) automate some traditionally

manual processes. Such a manual process to be con-

sidered is a Service Desk. Among other things, the

company’s service desk is a point of contact for the

customers seeking to report unavailability of service

(mobile or ﬁxed-line phone connection, IDSL Inter-

net connection, and other) or another problem. If the

problem is at the customer side, the service desk ex-

perts are supposed to provide instructions for trou-

bleshooting it. The whole interaction is performed

over the phone, i.e. the customer calls, explains the

problem to a service desk worker, probably gets trans-

ferred to a relevant expert, tries what the expert rec-

ommends, reports the results, and so on. In result,

the waiting times for customers to get through to the

service desk are long, while the effectiveness of the

troubleshooting process is quite low.

Equipping the service desk with an UBIWARE-

based solution could enable the following new func-

tionality: (1) Customers get possibility to report their

problems through a web-service interface. (2) The

system could collect, integrate and present to the ex-

pert all the available information on the customer

(type of connection, address, etc), removing a need to

spend time asking that information during the phone

conversation. (3) Automated analysis of related prob-

lem reports could enable fast responses like: ”it is

a network problem, we have received other similar

reports from your area, we are already working on

this problem”. In the case of a non-working Internet

connection, some relevant data can be acquired, i.e.

sensed, from the network side. Also, some modems

can be conﬁgured remotely by uploading a conﬁgura-

tion ﬁle from the network. In such a case, a fully au-

tomatic troubleshooting could even be implemented.

SMART SEMANTIC MIDDLEWARE FOR THE INTERNET OF THINGS

177

5 CONCLUSIONS

In this paper, we described our vision of a solution

to meet the middleware needs of the domain of the

Internet of Things. We aim at a new generation mid-

dleware platform which will allow creation of self-

managed complex systems, in particular industrial

ones, consisting of distributed, heterogeneous, shared

and reusable components of different nature.

Self-management of systems is one of the central

themes in the EU 7-th Framework ICT Programme

(2007-2013). The Objective ”Service and Software

Architectures” of the Challenge 1 ”Network and Ser-

vice Infrastructures” includes the need for strate-

gies and technologies enabling mastery of complex-

ity, dependability and behavioral stability, and also

the need for integrated solutions supporting the net-

worked enterprise. Also, the Objective ”The Net-

work of the Future” of this Challenge includes the

need for re-conﬁgurability, self-organization and self-

management for optimized control, management and

ﬂexibility of the future network infrastructure. In

addition, the whole Challenge 2 ”Cognition, Inter-

action, Robotics” has as its motivation the need for

creating ”artiﬁcial systems that can achieve general

goals in a largely unsupervised way, and persevere

under adverse or uncertain conditions; adapt, within

reasonable constraints, to changing service and per-

formance requirements, without the need for exter-

nal re-programming, re-conﬁguring, or re-adjusting”.

It is noticeable that the systems (stand-alone or net-

worked) monitoring and controlling material or infor-

mational processes is one of the three focus areas of

this Challenge.

According to a more global view to the Internet of

Things technology, UBIWARE will classify and reg-

ister various ubiquitous devices and link them with

web resources, services, software and humansas busi-

ness processes’ components. UBIWARE will also

consider sensors, sensor networks, embedded sys-

tems, alarm detectors, actuators, communication in-

frastructure, etc. as ”smart objects” and will provide

similar care to them as to other resources.

The innovative nature of UBIWARE is demon-

strated well by the very idea of the case in Section

4.3, the smart service desk. An operator’s service

desk is an important service element which, however,

still remains largely non-automated and thus has a

low effectiveness. Any automation of such a process

would require integration and complex interoperabil-

ity of highly heterogeneous components, including

hardware (e.g. customer’s equipment), software and

database systems, and humans (the customer, the ser-

vice desk operator, and experts). In result, the possi-

bility of automation of the service desk was not really

considered before. However, the introduction of the

UBIWARE concept has led to realizing that some au-

tomation can be possible after all.

REFERENCES

Brock, D. and Schuster, E. (2006). On the semantic web of

things. In Semantic Days 2006, Stavanger, Norway,

April 26-27.

Buckley, J. (2006). From RFID to the Internet of Things:

Pervasive networked systems. In Report on the Con-

ference organized by DG Information Society and Me-

dia, Networks and Communication Technologies Di-

rectorate, Brussels, March 6-7.

Cabri, G., Ferrari, L., and Zambonelli, F. (2004). Role-

based approaches for engineering interactions in

large-scale multi-agent systems. In Software Engi-

neering for Multi-Agent Systems II, LNCS v.2940,

pages 19–25. Springer.

Jennings, N. (1993). Commitments and conventions: The

foundation of coordination in multiagent systems.

Knowledge Engineering Review, 8(3):223–250.

Jennings, N. (2000). On agent-based software engineering.

Artiﬁcial Intelligence, 117(2):277–296.

Jennings, N. (2001). An agent-based approach for building

complex software systems. Communications of the

ACM, 44(4):35–41.

Katasonov, A. and Terziyan, V. (2007). SmartResource plat-

form and Semantic Agent Programming Language (S-

APL). In Proc. 5th Conf. Multi-Agent Technologies,

LNAI v.4687, pages 25–36. Springer.

Kephart, J. O. and Chess, D. M. (2003). The vision of auto-

nomic computing. IEEE Computer, 36(1):41–50.

Khriyenko, O. (2007). 4i (FOR EYE) technology: Intel-

ligent interface for integrated information. In Proc.

9th Conf. Enterprise Information Systems, Volume 5,

pages 278–281.

Lassila, O. (2005). Using the Semantic Web in mobile and

ubiquitous computing. In Proc. IFIP WG12.5 Conf.

Industrial Applications of Semantic Web, pages 19–

25. Springer.

Mamei, M. and Zambonelli, F. (2006). Field-Based Coor-

dination for Pervasive Multiagent Systems. Springer.

Naumenko, A., Katasonov, A., and Terziyan, V. (2007). A

security framework for smart ubiquitous industrial re-

sources. In Proc. 3rd Conf. Interoperability of En-

terprise Software and Applications, pages 183–194.

Springer.

Tamma, V., Aart, C., Moyaux, T., Paurobally, S., Lithgow-

Smith, B., and Wooldridge, M. (2005). An ontological

framework for dynamic coordination. In Proc. 4th In-

ternational Semantic Web Conference’05, LNCS vol.

3729, pages 638–652. Springer.

Vazquez-Salceda, J., Dignum, V., and Dignum, F. (2005).

Organizing multiagent systems. Autonomous Agents

and Multi-Agent Systems, 11(3):307–360.

ICINCO 2008 - International Conference on Informatics in Control, Automation and Robotics

178