THE VOCABULARY ONTOLOGY ENGINEERING FOR THE

SEMANTIC MODELLING OF HOME SERVICES

Jarmo Kalaoja, Julia Kantorovitch

VTT Technical Research Centre of Finlands,Kaitoväylä , P.O Box 1100 FI-90571, Oulu, Finland

Sara Carro, José María Miranda, Álvaro Ramos

Telefónica I+D, S.A. Parque Tecnológico Walqa, Huesca, Spain

Jorge Parra

Ikerlan Technological Research Centre, P.O.Box 146, 20500 Arrasate, Gipuzkoa, Spain

Keywords: Home networked services, service discovery, semantic modelling, requirements analysis, ontology

engineering.

Abstract: With great advance in information technology and broadband networks, the interconnected networked home

devices are becoming increasingly popular. Number of heterogeneous networked devices and services

which belong to the traditionally separated functional islands such as PC (i.e. Internet), mobile, CE

broadcasting, and home automation, not working together can be found in our today’s home. Merging of

these devices and services would offer home individual residents user-friendly, intelligent, and meaningful

interfaces to handle home information and services. The semantic ontology based modelling of home

services can enable interoperability of heterogeneous services. The ontology may facilitate clear description

on how far each device is suitable for different kinds of information and different interaction demands. This

paper is presenting an analysis on the kind of vocabulary ontologies necessary in different functional

domains to cope with heterogeneity of service descriptions. The ontology based rich representation of

services will facilitate an efficient service discovery, integration and composition.

1 INTRODUCTION

Merging of devices and services from the

traditionally separated domains presented in the

home would offer home individual residents user-

friendly, intelligent, and meaningful interfaces to

handle home information and services. To achieve

this, the heterogeneous networked services in the

home, where a wide variety of heterogeneous

networks, devices and software infrastructures,

coexist, should be discovered, integrated and

composed in a ubiquitous and seamless manner to

address the user requirements.

In Amigo project (Georgantas, 2005) we adapt

the semantic modelling approach for modelling of

the services to enable efficient automated execution

of the aforementioned tasks. In service oriented

software architectures (TINA, 1997) (WS, 2004),

service modelling is usually performed by the

operations that a service provides and requires from

other services in the architecture. Thus the Amigo

services are modelled based on the number of

required and provided capabilities. Each capability

specifies a number of inputs and outputs. The

capabilities along with their inputs and outputs can

be used for the service discovery process (i.e.

capabilities/requirements matching). Some other

properties of the service, often referred as non-

functional properties, such as quality of service

(QoS), context in which a service is executed, and

preferences specified in user profiles may assist to

dynamically select the services that best meet the

user needs.

In order to allow a rich representation of services

and thus facilitate efficient service discovery and

composition, functional capabilities, inputs, outputs

461

Kalaoja J., Kantorovitch J., Carro S., María Miranda J., Ramos Á. and Parra J. (2006).

THE VOCABULARY ONTOLOGY ENGINEERING FOR THE SEMANTIC MODELLING OF HOME SERVICES.

In Proceedings of the Eighth International Conference on Enterprise Information Systems - ISAS, pages 461-466

DOI: 10.5220/0002487004610466

 SciTePress

and non-functional attributes of the services can be

further semantically annotated using external

vocabulary ontologies. The use of ontologies enables

computational entities and services to have a

common set of concepts and vocabularies for

representing knowledge about a domain of interest,

while being able to interact with each other. By

using such ontologies, the relationships between

entities can be more clearly expressed and these

allow for better reasoning on their properties.

Ontologies are also beneficial for the re-usage of

knowledge, as several ontologies from various

sources can be integrated to describe the specific

domain. Finally, the deployment and customization

of such systems as Amigo in any home can be

considerably facilitated by using a common set of

concepts and vocabularies developed for the

networked home environment. The integration of

third-party applications and services can be much

eased using clear ontology classifications.

This paper elaborates on the vocabulary

ontologies for the semantic modelling of home

services. In the following, one possible realization

of home system is visualized in Section 2. Here the

approach and steps have been used to identify the

required vocabularies and classifications are

outlined. The initial analysis and results for different

functional domains (mobile, PC, domotic, CE) are

presented in Section 3. Section 4 concludes the

paper providing an overview of the current

contribution and the future work. As the work on

domains analysis and definition of different

ontologies is in progress, the objective of this paper

is to share and validate the approaches used, to

discuss the initial results obtained, and to verify the

directions for the future work.

2 SEMANTIC MODELLING OF

AMIGO SERVICES

The intensive user research (Rocker, 2005)

performed in Amigo project gives some ideas on

how the home system could interact with people in a

service-rich home environment. Based on this

research, a home system (we call it Amigo system)

can be visualized as handling a number of separate

aspects or application domains of the intelligent

home, demonstrating various properties and

functionalities that support the persons living in the

home.

The Amigo system (in role of supporting gaming

and entertainment domain) can start a user's day

with playing music from preferred play list, showing

the personalized news or general summaries of hot

news topics. The content (music, film, TV/radio

program or game) can follow the user everywhere in

the house or even outside the home area using a

mobile or a portable device. It helps to select games

to play and supports the usage of various displays in

the home environment during the game session. It

downloads personal profiles and integrates game

devices when friends are coming over to the user’s

house for interactive multi-player game sessions.

The Amigo system (in role of supporting

household task assistance domain) can select the

correct settings for home appliances. It may even

know how to detect the presence of inappropriate

objects inside or near the appliances. It downloads

recipes and cooking programs to the kitchen and

displays them to facilitate the food preparation

procedure.

The Amigo system (in role of supporting

ambience assistance domain) can configure the

home environment during game sessions, or while

the user is watching TV by adapting the light, sound

and video features/levels throughout the room. It

can adapt the home environment and create an all-

surround audio, light and video experience.

Amigo system cannot depend on the availability

of any specific set or types of devices in handling

these aspects. In order to support this, we need to

introduce an engineer role into the Amigo system

that supports higher level Amigo applications and

hides the necessary underlying technologies.

Examples of the support for this role can range from

controlling specific appliances to downloading

content to a PDA device. This "Engineer" role is

handled by the Amigo Intelligent services, and a

number of other services that are dynamically

registered using the service discovery functionality

of various appliances located in the Amigo house

and composed by the Amigo system when

necessary.

The aforementioned scenes have been analysed

to gather the requirements for various domain

ontologies and vocabularies to cope with the

heterogeneity of services descriptions (i.e.

capabilities, inputs, outputs) and address the

requirement specifications. As an example: "John

and Robert start to play an interactive multi-player

adventure game in John's living room. The Amigo

system adjusts the ambience for the game experience

by controlling the lights and sound. Robert notices

that it is late so he has to leave, but he continues to

play the game under way..." Some examples of the

ICEIS 2006 - INFORMATION SYSTEMS ANALYSIS AND SPECIFICATION

462

vocabularies related to the various Amigo domains

in this scene are:

From positional information and/or gesture

recognition output Amigo User Interfaces detect that

John and Robert want to play. Upon activation of

the game, the system queries the capabilities for

PositionControlInput User Interface Services provided

by the Amigo User interface. When controlling the

ambience in a room the Amigo system queries

services providing the domotic domain

ControlLight

and LightDetectors capabilities available in the

context of that room. TV monitor, when plugged in,

provides CE domain capability of

DigitalMediaRenderer for AVStream.

To be able to support the continuation of the

gaming session, John's Amigo system checks the

DeviceContext of Roberts PDA to match a most

suitable mobile client technology provided by game

console and uses the PDA’s

ContentDownload

capability to load a game into it.

Other supporting services that are not that visible

in the example above are seamlessly involved. For

example, context management service will be used

to provide user location. The Amigo system may

ask the User modelling and profiling service for

relevant user context and preferences about favorite

adventure game to be automatically downloaded.

Different levels of ontologies (see Figure 1) can

be envisaged to model the entire Amigo home

environment appropriately in order to address the

service/application developer’s and user’s

requirements for efficient service discovery,

composition and invocation.

Middleware

Applications

Platforms

QoS

- Performance

- Security

- Privacy

Context

-User

- Physical

-Device

Non-Functional

Capabilities

CE / PC / Mobile Domotic Devices

Functional

Data /

Content

Mobile

Capabilities

Domotic

Capabilities

HW/ SW/ Communication Platforms

Abstract service Description

Context

Management

User Modeling

and Profiling

…

Figure 1: Different levels of services and ontologies.

To verify and validate this approach, the

following iterative steps have been used in order to

identify the required vocabularies and classifications

on which the ontologies will be based (Noy, 2001):

Step 1:

We gathered Amigo requirements for the

various domain ontologies. Some domain-specific

examples, based on user scenarios that can help to

determine what types of information are required

and what types of query support the ontology should

provide have been identified.

As an example, the mobile domain information

and services are useful in such cases as

communication (call, SMS), notification (alerts

about some events in the home), and entertainment

on the way. Examples of mobile platforms used in

home are a mobile phone, PDA, laptop, etc.

The PC domain is related to the “classical” view

that we have about computers. Personal computers,

Web cameras, peripherals such as printers, scanners,

etc, are included in this domain. The role of PC

domain is mainly related to storing, accessing and

processing information.

Analysis of the CE domain concerns audio,

video and other entertainment devices presented in

the home. The interaction of home individuals with

this domain mainly takes place during their leisure

time, thus making the CE domain a very attractive,

actively utilized and potentially profitable one.

Plenty of CE devices such as HiFi systems, TV

screens, video consoles, sound speakers, etc, can be

found in our homes. Especially the entertainment

content that is displayed and exchanged between

those CE devices is quite attractive and important

from the user’s point of view. This content includes

songs, films, still photos, etc, and can be seen as

indispensable attribute of the CE domain. In

addition, a many of the data linked to this content

(such as size, bit rate, and parental permissions) can

affect the user’s experience directly or indirectly.

The domotic domain is related to identifying the

requirements of home automation devices to Amigo

system. Some examples of such devices could be a

lighting system, a washing machine, a gas sensor,

etc. Energy cost savings, security of persons and

goods, comfort improvement are some of the

benefits of home automation. The role of domotic

domain is quite critical as it is also associated with

home care and safety. Safety and security of persons

and goods is increased with functions such as

detection of fire or gas leakage, tele-transmission of

alarm, or detection of intrusion.

Step 2

: The numerous available sources of

information that may provide classifications and

ontologies have been considered (FIPA, 2001,

FIPAAV, 2001, OMA, 2005, CCPP, 2004,

TvAnytime, MPEG7, 2004, MPEG21, 2002,

Shimizu, 2005, UPnP, 2003, DLNA, 2005). The

above enumerated sources provide suitable

information to model device capabilities to be useful

in adaptation of content information to particular

THE VOCABULARY ONTOLOGY ENGINEERING FOR THE SEMANTIC MODELLING OF HOME SERVICES

463

device mode, however they lack in providing service

functional descriptions that would allow the

automatic selection of particular devices based on

the services available, through the service capability

matching. Moreover, most of information is mobile

domain oriented and might give only some ideas for

service modelling in other domains. As regarding to

domotic domain, due to the great heterogeneity of

devices that can be found in the domain, there is no

available classification covering the entire home

automation world. Therefore the Amigo

classifications can be a starting point in modelling of

domotic information.

Step 3:

Here some draft lists of information (i.e.

vocabularies) for different domains based on various

sources enumerated have been created

Step 4:

Analysis and classifications. In this step

we analyse and filter the lists obtained from step 3

against the requirements from step 1 to identify

information needed to support the Amigo scenarios.

We are in the process of identifying the required

ontologies and relations between them for describing

the information of a particular domain. The

preliminary taxonomies and class hierarchies for the

ontology are initially defined and presented in the

following.

3 ANALYSIS AND

CLASSIFICATIONS

The following set of taxonomies for different

functional domains (mobile, PC, domotic, CE) can

be identified for further analysis:

• Taxonomy for the typical functional

capabilities of devices, provided as

software services

• Taxonomy of functional capabilities of

software services which provide the

intelligence support in the home

• Taxonomy for devices and platforms that

can be used to define device context and

facilitate the adaptive service discovery

• Taxonomy of content including multimedia

content

When classifying the functional capabilities, we

focus on those capabilities of devices and services

that can be used for service discovery and further

service composition in Amigo. The various aspects

identified in the example are reflected in the

classification. Most of the analysed capabilities are

closely associated with the control of devices

discoverable in Amigo home (the engineering

aspect), but classifications that help modelling of

services associated with higher level aspects

supported by Amigo home (e.g. gaming and

entertainment domain support) are also analysed.

Figure 2 gives an idea of functional capabilities

class hierarchies in Amigo. The high level

FunctionalCapability of services found in Amigo

home can be described using three application

domains presented in Amigo home. These are

ExtendedHomeSupport, HomeCareAndSafetySupport, and

HomeInformationAndEntertainment. Naturally this level

of hierarchy can be easily extended if services

supporting other application domains functionality

will be developed in the future. Accordingly lower

level class hierarchies present the capabilities of the

services from four functional domains (mobile, PC,

domotic, CE) and intelligent user services which

support at least one of the above enumerated

application domains. In essence, the communication

with users (

UserComminication) by calling and

messaging is an inherent capability of the mobile

domain. In the PC domain which is much

overlapped with mobile domain, this may

additionally involve means for user

AudioVideoCommunication through the

Videoconference. MultimediaApplicationSupport may

be described by capabilities providing control to AV

devices in home, and

GameApplicationSupport by

support for game downloading and user

communities (not presented in the picture).

IntelligentHomeSupport can be viewed as the glue,

expanding through all application domains and

described by a number of services, which

capabilities support

UserIntefaceServices with devices

(GUI, speech, gesture services), provide

IntelligentSoftwareSupport through e.g.

ContextManagement, UserModellingAndProfilling, and

other intelligent services and control home devices

(

HomeDeviceControl). The Functional Capability

class hierarchies can be easily extended by service

developers with respect to application domains

possibly present in any future homes and also with

respect to particular domain of interest. For example,

HomeInformationAndEntertainment can be further

extended by

VideoApplicationSupport,

MusicNewsApplicationSuppport, etc. Accordingly

HomeCareAndSafetySupport will be further extended

in our future work with services capabilities specific

to devices from domotic domain such as e.g.

ControlLight, ControlBlinds, ControlTemperature, etc.

ICEIS 2006 - INFORMATION SYSTEMS ANALYSIS AND SPECIFICATION

464

Figure 2: Partial class hierarchies of functional capabilities.

As previously mentioned, the device capability

profiles provide background for the devices and

platform taxonomy. A common problem with the

various standards considered is that they mostly

describe only the variable set of capabilities of a

device leaving out the standard set of capabilities for

the underlying platform technology. We think that

such assumptions should be explicit. As an

example, a partial draft of low level ontology based

on mobile platform taxonomy is illustrated in Figure

Figure 3: A mobile platform Taxonomy.

The proposed taxonomy is based on various

device property descriptions but is more generic in

order to cover also implicit information left out of

those classifications. Again, each class in this

taxonomy is a specialisation of its parent classes. For

clarity, the taxonomy is restricted to three levels in

which the first level defines a role of the platform,

the second is a sub-classification of platforms

provided in this role and the third level is usually a

specific technology with possibly several instances.

The vocabulary can describe for example the

communication properties of a mobile platform, or

classes of mobile devices with typical platform

properties. A mobile phone model can be specified

as an individual of class

MobilePhone with specific

properties. Similar ontologies can be created for the

devices and platforms from other functional domain

(PC, domotic, CE). The integration of those is part

of our future work.

The taxonomy for content is related to input and

output information provided by services and can be

used for the purpose of service selection/matching.

The types of content in mobile domain are related to

CE domain. Typical to these domains is that many

applications are fixed in device but others, like Java

games, are downloaded as content to device. Typical

application data e.g. in mobile and PC domains is

related to contact and calendar information:

MobileContent

Application

Game / Calendar / Phonebook / CameraImaging

EmailClient

ApplicationData

Bookmark / CalendarEntry / ContactInformation

Media

Audio / Image / Video / Text

Stream / Still

QoS aspects in the service discovery may rise

some questions related to the format of used

multimedia content in particular situation/context,

i.e., which format for this content is the best suitable

for the current state of the network. Furthermore,

some capabilities of content-related services may

only apply to certain content formats or media. Thus

some explicit relation to these ontologies is

necessary. Device capabilities in CE are essentially

described by the way in which they behave in

relation to content. It is not our purpose to define a

content description language but rather to provide a

bounded taxonomy that will enable the modelling of

capabilities under the

MultimediaApplicationSupport

class presented above. Therefore, after analysing the

THE VOCABULARY ONTOLOGY ENGINEERING FOR THE SEMANTIC MODELLING OF HOME SERVICES

465

aforementioned classifications and scenario

requirements, an essential classification may be:

MultimediaContentType

ContentSize / Bitrate / NumberOfChannels

Resolution

SpatialResolution / TemporalResolution

MediaFormat

AudioFormat

ADPCM / CS-ACELP / G.711 / MP3 / OGG / PCM /

WAV / Mono-Stereo

ImageFormat

BMP / GIF / JPEG / TIFF

TextFormat

Plain / RTF

VideoFormat

AVI / DivX / MPEG2 / MPEG4 / WMV / XVid /

A separate taxonomy for QoS characteristics is

also considered, but not yet elaborated, for mobile

and PC domains. This taxonomy is related mostly to

service usability because of restrictions of mobile

devices, selection of most suitable communication

channels, and determination whether a specific

content can be presented on a particular device.

4 CONCLUSION

This paper has presented the requirements analysis

and the initial results on ontology vocabularies and

classifications modelling for the services and

devices capabilities from four functional domains

(mobile, PC, domotic, CE) presented in the home

environment.

In our future work we are planning to iteratively

refine and extend the existing classifications with

generic services and devices capabilities found from

four functional domains and also with services

capabilities developed in duration of Amigo project

towards the support of the demonstrations for three

application domains of Home Information and

Entertainment, Home Care and Safety, and Extended

Home. Special attention will be paid to the

description of domotic devices and services with

their possible integration with domotic information

on devices states and events as most critical ones in

developing of services for the Home Care and Safety

application domain.

It is also planned to elaborate the ontology on

QoS related aspects that will contain the devices

related capabilities and the content related

information (the initial results are presented in this

research), and also network related QoS parameters

which would allow to tackle the interoperability

issues between different QoS schemas (e.g. UPnP

and RSVP) presented around the home.

We would like to thank IST, the European

research program. Amigo (IST project 004182) is a

European project in the Ambient Intelligence and

Pervasive Computing vision.

REFERENCES

CCPP, 2004. Composite Capabilities / Preference Profiles

http://www.w3.org/Mobile/CCPP/

Deliverable, 2005. Amigo Consortium. Deliverable D1.1:

Report on user research

DLNA, 2005. DLNA information site.

http://www.dlna.org/home.

FIPA, 2001. FIPA Device Ontology Specification. 2001.

http://www.fipa.org/specs/fipa00091/PC00091A.html

FIPAAV, 2001. FIPA Audio/Visual Entertainment and

Broadcasting Specification

http://www.fipa.org/specs/fipa00081/XC00081B.html#

_Toc505481810

Georgantas, 2005. Georgantas, N., Ben Mokhtar, S.,

Bromberg Y.-D., Issarny, V., Kalaoja, J.,

Kantorovitch, J., Gérodolle, A., Mevissen, R. The

Amigo Service Architecture for the Open Networked

Home Environment. Short paper. In Proceedings of

5th Working IEEE/IFIP Conference on Software

Architecture (WICSA).

Noy, N., McGuinness, 2001. D.L. Ontology development

101: A Guide to Creating Your First Ontology.

Technical report, SMI-2001-0880

MPEG7, 2004. MPEG-7 Overview.

http://www.chiariglione.org/mpeg/standards/mpeg-

7/mpeg-7.htm

MPEG21, 2002. MPEG-21 Overview.

http://www.chiariglione.org/mpeg/standards/mpeg-

21/mpeg-21.htm

OMA, 2005. OMA User Agent

http://www.openmobilealliance.org/release_program/

uap_v20.html

Rocker, C., Streitz, N., Jance., M, Portolan., N., 2005.

User Requirements for intelligent Home

Environments: A Scenario-Driven Approach and

Empirical Cross-Cultural Study. In SOC-EUSAI 2005

Smart Objects & Ambient Intelligence

Shimizu, N., Kitagawa, K., Wan, H., Ishikawa, N., Hjelm,

J., 2005. A Device and Service Description

Framework for Discovering and Reasoning in

Autonomous P2P Environment. MobEA III - Customer

Focused Mobile Services

TvAnytime. TV-Anytime Forum. http://www.tv-

anytime.org/

TINA, 1997. Service Architecture Specification.

http://www.tinac.com.

UPnP, 2003. The standardized device and service

descriptions

http://www.upnp.org/standardizeddcps/lighting.asp

WS, 2004. Web Services Architecture, W3C Working

Group Note 11 February 2004.

http://www.w3.org/TR/ws-arch/

ICEIS 2006 - INFORMATION SYSTEMS ANALYSIS AND SPECIFICATION

466