A UBIQUITOUS SYSTEM USING SEMANTIC KNOWLEDGE

IN DYNAMIC SERVICE INTEGRATION

Tomasz Dziubich, Paweł Kaczmarek and Sławomir Nasiadka

Department of Computer Systems Architecture, Faculty of Electronics, Telecommunications and Informatics,

Gda´nsk University of Technology, Narutowicza 11/12 Str., 80-233 Gda´nsk, Poland

Keywords:

Ubiquitous systems, Service integration, Semantic information, Workﬂow systems.

Abstract:

The paper presents a ubiquitous system that uses semantic information to manage dynamic changes in system

elements. We design the system using the Context-Controller-Action architecture that covers the main parts

of ubiquitous environments, including: environment devices (such as sensors and actuators), contexts and ac-

tions, all of which may be dynamically conﬁgured during system operation. A graphical user interface for the

system is supplied, which enables non-expert users to perform changes in system conﬁguration. Internally,

semantic information is used throughout the system to deﬁne a common language between devices, contexts

and services. A Web services interface is used in services that handle environment devices to support their

integration with the system regardless of internally used communication protocols. The architecture was im-

plemented as a framework for developing and running applications in ubiquitous environments. We integrated

a number of devices in the framework, including, RFID sensors, digital displays, cameras, electronic door

locks and others. We also deﬁned exemplary contexts and actions in the environment.

1 INTRODUCTION

Ubiquitous systems signiﬁcantly extend everyday life

environments with an ability to dynamically adapt to

changing environment conditions and user behavior.

The conditions and behavior are detected by dedi-

cated sensors (Poslad, 2009) that supply relevant in-

formation for the ubiquitous system. The informa-

tion is used to determine the current context (Loke,

2006) (Salber et al., 1999) of the environment and

to perform various real-world actions using actuators.

Examples of performed actions include door unlock-

ing depending on an identiﬁed user, modiﬁcations of

digital display contents, and temperature adjustment.

The emerge of ubiquitous systems opened new

challenges and requirements as compared to tradi-

tional desktop computing. Systems must be able to

dynamically manage changes that occur both in the

environment and in a system structure. The nature

and the result of environment actions should change

depending on user preferences or availability of ser-

vices. For example, a user might expect a system

to change its behavior because of the holiday season,

which has not been anticipated earlier. Additionally,

the dynamic changes of system behavior should be

managed by users who usually do not have advanced

knowledge about software development.

Considering the requirements, we propose a sys-

tem that supports both dynamic management of re-

sources and a user-friendly approach to system pro-

gramming. Dynamic management of resources is re-

alized by the Context-Controller-Action architecture

that integrates different system elements, such as sen-

sors, context deﬁnitions, available actuators and ac-

tions. Context and action deﬁnitions are managed

by a user-friendly interface adequate for non-expert

users. We shift from traditional coding-based appli-

cation development to declarative-based development

supported by a graphical user interface. Internally,

we use semantic information and Web services to im-

proveseamless integration and interoperability of sys-

tem elements. Semantic information is used through-

out the system to deﬁne a common language between

devices, contexts and actions.

2 SYSTEM ARCHITECTURE

We propose to use the Context Controller Action

(CCA) architecture, as shown in Fig. 1. In the ar-

chitecture, a system is composed from the following

main modules:

Dziubich T., Kaczmarek P. and Nasiadka S..

A UBIQUITOUS SYSTEM USING SEMANTIC KNOWLEDGE IN DYNAMIC SERVICE INTEGRATION.

DOI: 10.5220/0003801100450050

In Proceedings of the 2nd International Conference on Pervasive Embedded Computing and Communication Systems (PECCS-2012), pages 45-50

ISBN: 978-989-8565-00-6

 2012 SCITEPRESS (Science and Technology Publications, Lda.)

Figure 1: Main components of the Context Controller Ac-

tion (CCA) architecture.

Basic Services Environment. A basic service repre-

sents a single component available in the smart

space such as a sensor, an actuator, or a wrapper

for an external data source. The basic services en-

vironment hosts the services.

Integration Module. The module supplies a user in-

terface enabling the management of system ele-

ments such as basic services, contexts and actions.

A user can conﬁgure which actions are taken in

response to a speciﬁc context occurrence.

Context Processing Module. In our approach, a

context is a set of data that describes the environ-

ment and that can be interpreted by the applica-

tion (Loke, 2006). The module is responsible for

runtime processing of contexts. It collects infor-

mation from basic services and processes them to

determine if a speciﬁc context occurred.

Scenario (Action) Invocation Module. The module

executes scenarios in the environment in response

to requests sent from Context Processing Module

through Integration Module. We use the concept

of scenario to represent a set of basic actions exe-

cuted.

Semantic Knowledge Module. The module stores

semantic concepts concerning basic services. The

information is available for other system modules.

2.1 Basic Services Environment

Basic Services Environment hosts basic services that

manage sensors, actuators and, potentially, other ap-

plications. Basic services supply a Web services in-

terface that is used to integrate with the system. Ser-

vices typically communicate with sensors and actua-

tors (called transducers) using a device controller that

manages low level operations.

Apart from hosting basic services, the Basic ser-

vices module contains two components: a Service

Registry and a Service Wrapper. Service registry

stores information about known basic services to-

gether with their WSDL deﬁnitions, semantic de-

scription and internal information. The registry sup-

plies the information for other system modules. Ser-

vice wrapper manages concrete registrations and in-

vocations of services, which uniﬁes the invocations.

2.2 Integration Module

The integration module is a central part of the system

and supplies two main functionalities: communica-

tion between other system modules and a convenient

user interface. System modules register in Integration

Module to enable data interchange and references be-

tween the modules.

The module supplies also a user interface that en-

ables deﬁnition of contexts, scenarios and associa-

tions between them. We used the declarative oriented

approach to programming to simplify the develop-

ment process. Using a graphical tool, a user declares

context deﬁnition that should be processed by the sys-

tem. Then the user may register the deﬁnition using

Integration Module that in turn passes the deﬁnition

on to Context Processing Module. Additionally, a

user may register scenario deﬁnitions in BPEL and

specify which scenarios should be invoked on occur-

rence of a context.

2.3 Context and Scenario Processing

Modules

Context Processing Module listens for notiﬁcations

from basic services and processes them to determine

if a context has occurred. On notiﬁcation arrival, it

veriﬁes conditions for context occurrence. If a context

is identiﬁed, the module sends an event to Integration

Module that in turn searches for a corresponding sce-

nario and sends a request for scenario execution to

Scenario Invocation Module.

We utilize an enhanced Petri net (Jensen et al.,

2007) to process conditions required for a context to

occur. Initial places in Petri net deﬁnition represent

input data from basic services or data sources, tokens

represent concrete data that occurred, while transi-

tions represent conditions that must be met for a con-

text to occur (AND, OR). Each token has an assigned

living time which enables us to identify time depen-

dencies between them.

As an example, consider a context that represents

a lecturer starting a lecture in a room. The context

can identiﬁed by two conditions: the lecturer is in the

room, and there is the time for the lecture in the room.

We must ensure that the system processes the start of

PECCS 2012 - International Conference on Pervasive and Embedded Computing and Communication Systems

the lecture only once. The lecturer can be identiﬁed

either by an image recognition system or RFID. A no-

tiﬁcation from any of the services places a token in an

initial state and ﬁres the ﬁrst transition. Lecturer’s

timetable is read from the Timetable service. A con-

text is detected if transitions are ﬁred and they reach

the ﬁnal state of the context deﬁnition.

The Scenario (Action) invocation module is re-

sponsible for executing scenarios (sequences of ac-

tions) in the ubiquitous environment. Scenarios are

deﬁned as workﬂows in the BPEL language and ac-

cept data from Context Processing Module as input.

Using Integration Module, the scenario is registered

in the system by giving a link to its location. Sce-

nario invocation is triggered by a context occurrence

identiﬁed by Context Processing Module. The mod-

ule sends a request through the Integration Module.

3 SYSTEM OPERATION

The Context-Controller-Action architecture supplies

a coherent pattern for development, deployment and

operation of context-aware applications. It is focused

on both ﬂexibility and a user-friendly interface. Flexi-

bility is achieved by semantic description of concepts

and dynamic registration of services, context and sce-

narios. A user-friendly interface enables non-expert

users to deﬁne their applications and conﬁgure the

system.

3.1 Component Integration using

Semantic Knowledge

Semantic Knowledge Module is the central point of

semantic processing as it stores the ontology that de-

scribe concepts occurring in the ubiquitous system.

The concepts concern two main aspects: descrip-

tion of environment entities and description of ser-

vices. Environment entities cover information simi-

lar to other context ontologies (such as SOUPA (Chen

et al., 2004), CONON (Wang et al., 2004a)). The con-

cepts are used to deﬁne the environment and to iden-

tify contexts. Selected concepts used in our ontology

include: Person, Name, Location, TimeOfDay, GP-

SPosition, RoadNumber, Weather. Semantic descrip-

tion of services improves service registration and pro-

cessing of service data. Fig 2shows main components

of the semantic infrastructure.

Basic services are described using concepts from

environment and service ontologies. On registration,

each service supplies its semantic description to the

Semantic Knowledge Module. The description spec-

iﬁes a mapping between raw data of service interface

Figure 2: Semantic information structure and processing.

and concepts from the ontology. The mapping is used

in Basic Services Environmentthat receives data from

services and translates it into semantic information.

The information is then passed to context identiﬁca-

tion. Context deﬁnitions use semantic concepts from

the environmentontology without concrete references

to services or service data. In this solution, it is possi-

ble to exchange a service without modifying context

deﬁnitions and processing.

The ontology is stored in the OWL format as the

most popular format for ontology description. We

used the Jena engine to process the ontology and rea-

son about concepts.

3.2 Processing of Information

Information processing covers two main areas: sys-

tem conﬁguration and system operation. Consider-

ing the dynamic nature of a ubiquitous environment

and dynamic adaptation of system operation, the ar-

eas may interchange. Main steps of making the sys-

tem operational are as follows:

1. Registration of Basic Services. The phase re-

quires installation of adequate devices, deploying

sensor services and registering the services.

2. Conﬁguration of Contexts and Scenarios. The

phase covers deﬁnition of contexts using a sup-

plied graphical tool, deﬁnition of BPEL scenar-

ios and registering them using Integration Mod-

ule. Integration Module passes contexts to Con-

text Processing Module and scenarios to Scenario

Invocation Module.

3. Basic Services Information Processing. The

processing is performed in a loop in two alter-

native modes. Basic Services Environment either

A UBIQUITOUS SYSTEM USING SEMANTIC KNOWLEDGE IN DYNAMIC SERVICE INTEGRATION

explicitly invokes methods of basic services to get

required data or listens for asynchronous notiﬁ-

cations from basic services. After receiving in-

formation, Basic Services Environment translates

raw sensor data to semantic concepts and sends

the information to Integration Module that in turn

passes it on to Context Processing Module.

4. Context Processing Module identiﬁes Context.

The module retrieves contextual information and

maps it to context deﬁnitions using the deﬁned on-

tology. If a context is identiﬁed, the module sends

information to Integration Module.

5. Scenario Invocation. Integration Module maps

an identiﬁed context with an assigned scenario

and passes a scenario invocation request to Ser-

vice Invocation Module. The latter invokes the

BPEL scenario that may invoke services.

4 PROTOTYPICAL

IMPLEMENTATION

As a part of the research, we implemented a frame-

work in the Context Controller Action architecture

and used the framework to deploy exemplary services,

contexts and scenarios. The implementation covers

all modules of the CCA architecture. A Web services

interface is supplied for each of the modules to enable

inter-module cooperation.

The majority of the system is implemented in the

.NET 3.5 Platform. Knowledge Base and Scenario In-

vocation Modules are implemented in Java consider-

ing better support for ontology processing and BPEL

processing in this language.

The .NET part contains a dedicated library for

processing of the Petri net structure that processes

the net considering elements speciﬁc for this system;

such as token lifetime, different transition conditions

and token data. We used the Jena framework for

management of ontological structures and reasoning

about concepts. Scenario Invocation Module uses the

Apache Ode for invocation of BPEL workﬂows.

The system supplies both a regular user interface

and an administration application interface. The reg-

ular user interface enables deﬁnition of contexts by

composing block elements and associations between

the elements. After deﬁnition of the models, a user

may register the context and scenario, and associate

contexts with scenarios.

The administration application enables monitor-

ing and veriﬁcation of system behavior such as: ses-

sion establishment, registration of a new context or

scenario together with its XML contents, connection

and disconnection between modules. Fig. 3 shows an

exemplarydialog box with monitoring data displayed.

4.1 Exemplary Services, Contexts and

Scenarios

Context Processing Environment was used to imple-

ment exemplary ubiquitous applications that cover

basic services, context deﬁnitions and scenario deﬁni-

tions. We used the following hardware devices in pro-

totypical use of the system: BERA Electronic Door

Lock, CAENRFID 949EU IP65 UHF RFiD sensor

with HUBER+SUHNER AG 806-960MHz antenna,

video cameras: Logitech QuickCam Pro5000, DLink

SecuriCam DCS-5300G, PDA device based on Intel

PXA270, a stepping motor, a multimedia projector

and a PDA device. Using the devices we implemented

basic services that covered different areas of function-

ality. Examples of basic services include: context

recognition (timetable service, camera service, user

authentication based on RFID, projector service), ac-

cess veriﬁcation (door lock manipulation, MMS no-

tiﬁcation), environment actions (PowerPoint presen-

tation management, control of a motor, ﬁle transfer

to/from PDA). Basic services were used to implement

concrete contexts and invoke scenarios:

Management of Lecture Room Access. The func-

tionality uses the RFID identiﬁcation, Door lock

service and Timetable service. Context deﬁnition

covers place and person identiﬁcation using the

RFID service and lecture time veriﬁcation using

the Timetable service. Door is unlocked if a right

person is close to the door, and the Timetable con-

ﬁrms that the person has a lecture in the room.

Illumination Control. The functionality uses the

Camera service, Electric stepping motor service

and Projector service. Context deﬁnition covers

illumination calculation using the Camera service

and projector state checking using the Projector

service. Electric motor is activated (scenario) if

illumination is higher that a speciﬁed limit and

the Projector is turned on. The scenario simu-

lates Roller-blinds closing in case the illumination

is too high during a presentation.

4.2 Sensor Service Integration Process

Sensor services were implemented by independent

groups of students and then deployed in the system by

a single team that was familiar with system operation.

The process supplied interesting data for development

and runtime interoperability (Ullberg et al., 2008)

analysis. All services in the system supply a Web ser-

PECCS 2012 - International Conference on Pervasive and Embedded Computing and Communication Systems

Figure 3: A dialog box that enables monitoring of internal system behavior.

vices interface for system communication, which sig-

niﬁcantly improved the interoperability. The services,

however, have to communicate with the sensors us-

ing lower-layer protocols (RS232, 802.11, WEP, FTP,

USB, Ethernet), which requires custom implementa-

tions (Kaczmarek and Downar, 2010).

Sensor communication was implemented in each

service by an individual team, which enabled us to

split the work between a large number of developers,

but raised difﬁculties during the deployment phase.

The deployment was performed by a single team that

had to familiarize with the service. File manipula-

tion, RFID and Gesture recognition services required

the highest work to deploy, 24 man hours, 14 man

hours and 12 man hours respectively. Other services

were analyzed, deployed and integrated in less than

10 hours for each.

As a part of system testing, we performed ba-

sic performance analysis (Krawczyk and Nasiadka,

2011). Gathering of context related data has been

identiﬁed as a potential performance bottleneck. Ac-

tion invocation was identiﬁed as less signiﬁcant as ac-

tions are invoked externally to the environment. We

measured the dependence between context processing

time and the number of parameters passed for pro-

cessing (context size). Experiments shown that pro-

cessing time depends almost linearly on context size

and takes approximately 100 ms for contexts with 10

inputs up to 700 ms for contexts with 200 input.

5 RELATED WORK

Context-aware application development is an active

research ﬁeld addressed both by industry and aca-

demic teams. Our work differs from existing solution

in the following main areas:

• We propose the Context-Controller-Action archi-

tecture that extends the standard architecture with

Integration and Knowledge Base Modules.

• We enable dynamic management of environment

resources and performingenvironmentactions de-

pending on available devices.

• We present a complete solution that uses seman-

tic description in all system elements, which cov-

ers basic services management, context process-

ing and action invocation.

• We supply a user-friendly interface that enables

non-expert users to dynamically conﬁgure a ubiq-

uitous environment.

Context-aware systems are usually composed of

an instrumentation layer, a middleware layer and

an applications layer (Meyer and Rakotonirainy,

2003) (Hsu et al., 2010). The instrumentation cov-

ers smart space sensors and physical devices that sup-

ply information to the middleware layer that iden-

tiﬁes high level context information. The informa-

tion is used by the application layer to gather con-

textual knowledge and deliver the expected service.

The work (Rehman et al., 2007) proposes a differ-

ent architecture for context aware applications that

adapts the Model-View-Controller to ubiquitous envi-

ronments. The work (Sui and Wang, 2006) proposes a

solution for automated generation of service ﬂow pat-

terns, which intends to simplify user interaction with

pervasive systems.

The use of ontologies and automated service dis-

covery in context-aware applications is an active re-

search direction. Unfortunately, there are “far too

many standards” for service description and discov-

ery (Mcgrath et al., 2007) without a consensus about

interoperability formats and common semantics. The

work (Chen et al., 2004) presents the SOUPA on-

tology that is an extensive and mature ontology of

ubiquitous applications. The core of the system cov-

ers information about events, time, people, space, ac-

tions and other. Another ontology of context-aware

systems is presented in (Wang et al., 2004a). Both

ontologies may be extended with concepts speciﬁc

for application domain. In (Wang et al., 2004b), au-

thors designed and used an upper-level context ontol-

A UBIQUITOUS SYSTEM USING SEMANTIC KNOWLEDGE IN DYNAMIC SERVICE INTEGRATION

ogy that covers typical concepts occurring in context-

aware applications.

Transformation of raw data from basic sensors to

high level information is required to identify con-

text effectively. Context awareness may be imple-

mented either on the device level or on the infras-

tructure level (Loke, 2006). The ﬁrst requires more

intelligent devices, while the second requires more

advanced context processing on the application level.

A Context Widget (Salber et al., 1999) supplies ab-

stract context information to application hiding the

complexity of the actual sensors. The work (Dimakis

et al., 2008) presents the SitCom and CHILix sys-

tems that enable development of context-aware appli-

cations and incorporation of smart space devices.

6 CONCLUSIONS AND FUTURE

WORK

The designed and implemented solution proved that

the Context-Controller-Action architecture can be

used to develop a dynamic ubiquitous system. Sen-

sor services, context and scenarios, can be conﬁgured

at runtime by a non-expert user who is equipped with

a graphical user interface. The internal use of seman-

tic description enables interchange of service infor-

mation and universal deﬁnition of contexts. The Web

services interface was used throughout the system to

integrate services with the system, and system mod-

ules between themselves.

Incorporation of semantic information in scenario

deﬁnitions is an interesting area of future work. BPEL

for Semantic Web Services is an important initiative

to include semantics into the workﬂow technology,

but the language has not been implemented in a pro-

cessing engine or design tool yet. Deployment of

further sensors and deﬁnition of contexts/scenarios is

another important ﬁeld of future work. The current

implementation demonstrated the usability of the sys-

tem, but it will be advisable to developmore advanced

behavior scenarios.

ACKNOWLEDGEMENTS

This work was supported in part by the Polish Min-

istry of Science and Higher Education under research

project N N519 172337. Students from the Faculty

of ETI brought signiﬁcant help in implementation of

basic services.

REFERENCES

Chen, H., Perich, F., Finin, T., and Joshi, A. (2004). Soupa:

Standard ontology for ubiquitous and pervasive appli-

cations. In International Conference on Mobile and

Ubiquitous Systems: Networking and Services.

Dimakis, N., Soldatos, J., Polymenakos, L., Fleury, P.,

Curin, J., and Kleindienst, J. (2008). Integrated devel-

opment of context-aware applications in smart spaces.

Pervasive Computing.

Hsu, H.-J., Wu, S.-Y., and Wang, F.-J. (2010). A methodol-

ogy to developing situation-aware pervasive applica-

tions with service oriented architecture. In 6th IEEE

World Congress on Services.

Jensen, K., Kristensen, L. M., and Wells, L. (2007).

Coloured petri nets and cpn tools for modelling and

validation of concurrent systems. International Jour-

nal on Software Tools for Technology Transfer (STTT).

Kaczmarek, P. L. and Downar, M. (2010). Interoperabil-

ity analysis of sensor interface in ubiquitous environ-

ments. In 31th International Conference on Informa-

tion Systems Architecture and Technology.

Krawczyk, H. and Nasiadka, S. (2011). A new model for

context-aware applications analysis and design. In

UBICOMM: The Fifth International Conference on

Mobile Ubiquitous Computing, Systems, Services and

Technologies.

Loke, S. W. (2006). Context-aware artifacts: Two develop-

ment approaches. IEEE Pervasive Computing.

Mcgrath, R. E., Mickunas, M. D., and Campbell, R. H.

(2007). Semantic discovery for ubiquitous comput-

ing. CiteSeerX - Scientiﬁc Literature Digital Library

and Search Engine.

Meyer, S. and Rakotonirainy, A. (2003). A survey of re-

search on context-aware homes. In Australasian In-

formation Security Workshop Conference on ACSW.

Poslad, S. (2009). Ubiquitous Computing: Smart Devices,

Environments and Interactions. Wiley Publishing.

Rehman, K., Stajano, F., and Coulouris, G. (2007). An ar-

chitecture for interactive context-aware applications.

IEEE Pervasive Computing.

Salber, D., Dey, A. K., and Abowd, G. D. (1999). The

context toolkit: Aiding the development of context-

enabled applications. In CHI, Pittsburgh, PA.

Sui, Q. and Wang, H.-y. (2006). A service oriented ﬂow pat-

tern in pervasive computing environments. In 1st In-

ternational Symposium on Pervasive Computing and

Applications.

Ullberg, J., Lagerstr¨om, R., and Johnson, P. (2008). A

framework for service interoperability analysis using

enterprise architecture models. In IEEE SCC (2),

pages 99–107.

Wang, C., Zhang, D., Hung, T., and Pung, K. (2004a). On-

tology based context modeling and reasoning using

owl. In Second IEEE Annual Conference on Perva-

sive Computing and Communications Workshops.

Wang, X., Dong, J. S., Chin, C., Hettiarachchi, S. R., and

Zhang, D. (2004b). Semantic space: An infrastructure

for smart spaces. IEEE Pervasive Computing.

PECCS 2012 - International Conference on Pervasive and Embedded Computing and Communication Systems