BROWSING REVERSIBLE NEIGHBORHOOD RELATIONS

IN LINKED DATA ON MOBILE DEVICES

Johannes Pfeffer, Markus Graube and Leon Urbas

Institute of Automation, Dresden University of Technology, Dresden, Germany

Keywords:

Semantic Web, Linked Data, Mobile Interaction, Mobile Computing, Human-computer Interaction, Human-

computer Interfaces.

Abstract:

Within the manufacturing and process industries pervasive computing is still less prominent than in other areas.

This is mainly due to the lack of mobile solutions that are adapted to the special requirements for industrial

tasks. This paper presents a novel mobile application for navigation in Linked Data. It follows the principle of

limited purpose applications: support a single task and be good at it. First, we introduce a data model for rep-

resenting reversible neighborhood relations in Linked Data. Second, we provide a human computer-interface

for mobile devices that hides the complexity of the Linked Data Cloud. It allows browsing of reversible neigh-

borhood relations such as industrial Piping & Instrumentation Diagrams and can be generalized to support

arbitrary predecessor-successor networks. Third, we discuss our concept in respect to a real life example of a

maintenance task in a large industrial plant.

1 INTRODUCTION

In recent years, the boundaries between ofﬁce and

ﬁeld work have been dissolving and fewer people

spend their whole workday at a single desk. Con-

sequently, mobile devices are becoming increasingly

important and increasingly pervasive (Pousttchi and

Thurnher, 2007). At the same time, another develop-

ment takes place: Innovation cycles are rapidly short-

ening while the pressure to create new products con-

tinues to increase. Enterprises try to face these chal-

lenges by concentrating on their core competencies

and entering short-term cooperations with comple-

menting companies (Schenker, 2010; AT&T Knowl-

edge Ventures, 2008). Through their agility, these vir-

tual companies have substantial competitive advan-

tages over monolithic enterprises (Heck and Vervest,

2007).

Within these new organizational entities arises the

need to share business and engineering information.

One solution to this problem of data interoperability

is Linked Data (Graube et al., 2011). Information to

be shared is exported to the Linked Data Cloud and

supplemented by unifying ontologies. In our paper

we focus on mobile interfaces to this new information

space in industrial working environments.

To fully exploit the potential of mobile devices in

business and industrial applications, custom solutions

with a high degree of usability are needed. When

good mobile tools are available, e.g. on-site work-

ing can lead to higher precision and quality of manu-

facturing (Viehland and Yang, 2007). However, engi-

neering processes, shop ﬂoor maintenance and other

industrial core tasks have rarely been integrated into

previously suggested approaches to the aforemen-

tioned challenges (Urbas, 2010).

The goal of this paper is to present a mobile ap-

plication that is suitable for engineering & mainte-

nance tasks based on integrating technologies such as

Linked Data, OWL & SPARQL. The Semantic Neigh-

borhood Browser is meant to be part of a larger set

of limited purpose applications supporting engineers

and maintenance personnel (Urbas et al., 2011). It

is optimized for usage in industrial environments that

require adapted input patterns (Krausman and Nuss-

baum, 2007) and can be adapted to make arbitrary re-

versible Linked Data relations accessible to the user.

All used data is available from planning and control

systems and does not need additional manual cura-

tion.

The rest of the paper is structured as follows.

First we provide a brief overview of the related work

followed by an example scenario for the Semantic

Neighborhood Browser. Then we describe our tech-

nical and theoretical approach. In the next section we

describe the implementation and elaborate on possi-

150

Pfeffer J., Graube M. and Urbas L..

BROWSING REVERSIBLE NEIGHBORHOOD RELATIONS IN LINKED DATA ON MOBILE DEVICES.

DOI: 10.5220/0003823901500155

In Proceedings of the 2nd International Conference on Pervasive Embedded Computing and Communication Systems (PECCS-2012), pages 150-155

ISBN: 978-989-8565-00-6

 2012 SCITEPRESS (Science and Technology Publications, Lda.)

ble use cases. The paper is completed by a discussion

of features and open issues. Eventually, we give some

concluding remarks and an outlook.

2 RELATED WORK

There are several established Linked Data browsers

such as RelFinder (Heim et al., 2010), Disco (Bizer

and Gauß, 2007) or Tabulator (Berners-Lee et al.,

2008), among others (W3C, 2011). They mainly ad-

dress end users who know how to interpret Linked

Data and/or RDF. This knowledge is necessary be-

cause the browsers act directly on the RDF represen-

tation of the data.

Recognizing the general need for presenting RDF

content to non-technical users and wanting to pro-

mote the exchange of presentation knowledge Pietriga

et al., 2006 have designed Fresnel as a browser-

independent vocabulary of core RDF display con-

cepts. Fresnel is the basis for server-side applications

such as Marbles (Bizer, 2009) or browsers such as

LENA (Franz et al., 2010). Fresnel is not yet used in

the proposed application but is a promising technol-

ogy that may be incorporated in a later version.

The mentioned approaches call for large desktop

displays and are designed with the WIMP-Pattern

mind. Thus, these concepts may not be transferred

directly to mobile devices.

DBpedia Mobile (Becker and Bizer, 2008), is a

location-centric DBpedia client application for mo-

bile devices consisting of a map view, the Marbles

Linked Data Browser and a GPS-enabled launcher ap-

plication. Sonntag and Heim, 2007, try to map RDF

graphs directly towards appropriate multimedia visu-

alizations to allow for semantic navigation within the

football domain. An optimal graph layout is com-

puted on the server-side, the mobile device works on

an XML representation. While the implementation is

a promising approach, the authors concentrate on nat-

ural language processing and consumer needs, rather

than on industrial requirements. Further, Ontowiki

Mobile (Ermilov et al., 2011) is a general browser for

hierarchical and faceted browsing with conﬂict reso-

lution for RDF content. On Android, Sparql Droid

(Read, 2011) can be used for exploring RDF data.

We argue that not another general purpose

browser for semantic data is needed. Instead, we fol-

low the paradigm of single purpose applications and

apply it to Linked Data. By limiting the scope to a sin-

gle purpose, we enable users to leverage the power of

semantic data sources by hiding their complexity. An

Window, Icon, Menu, Pointer.

implementation like ours which is is based on a real

scenario that illustrates the potential of Linked Data in

industrial environments has not been available before.

3 EXAMPLE SCENARIO

To illustrate a typical application for the solution pre-

sented in this paper we provide a short scenario in this

section. We introduce a persona called Wolfgang and

follow him throughout an exemplary workday:

Wolfgang is 30 years old and works as a shift su-

pervisor in a chemical plant. He has wide experience

in his ﬁeld because he has worked at this plant for

more than ﬁve years. The plant consists of a multi-

tude of basins, reactors, valves, pipes, etc. Protective

garment such has heavy gloves, helmet and boots are

obligatory in such a hazardous working environment.

Just as every other morning, Wolfgang heads out to

carry out his daily maintenance course through the

facility. He will have to verify the operational reli-

ability of various devices, calibrate equipment, take

samples and complete similar tasks. He leaves the

heavy briefcase containing the P&IDs

in his ofﬁce.

His maintenance routine is prescribed by the corpo-

rate regulations and helps to prevent failures or devi-

ation of quality, as well as to conform with accepted

standards.

When Wolfgang reaches a reactor in which sev-

eral liquids are mixed by a stirrer, he realizes that it

does not have the required ﬁll level. Before ﬁling a re-

port, he decides to check the connected valves which

control the inﬂow. He takes out his ruggedized An-

droid tablet and scans the RFID tag embedded in the

reactor basin. On the screen he now sees the Seman-

tic Neighborhood Browser centered on the equipment

under observation. In a large plant, devices can be

far away from each other and hard to locate. Without

ever taking off his gloves, Wolfgang navigates to the

connected valves, displays their ID-number, veriﬁes

their location and reviews their speciﬁcation. Having

this information at hand, it is no problem for him to

locate and check all connected valves for failure.

4 APPROACH

In our approach, we abstract from the real life chal-

lenge of navigation in a neighborhood environment to

a graph-theoretical problem. The information ﬂow for

our approach can be seen in Figure 1.

The Linked Data principles (Berners-Lee, 2006)

Piping & Instrumentation Diagram.

BROWSING REVERSIBLE NEIGHBORHOOD RELATIONS IN LINKED DATA ON MOBILE DEVICES

151

allow an easy adaption and extension of the data as

well as connections to other Linked Data informa-

tion clouds without much effort. By following these

principles the infrastructure of the Semantic Neigh-

borhood Browser has been kept as generic as possi-

ble: All required data was aggregated as Linked Data

in an external information cloud. There, the informa-

tion was modeled with the help of RDF

as a semantic

network consisting of triples. Each triple connects a

subject node via a predicate arc to an object node.

Most of the data in industrial companies is stored

as structured information in form of SQL databases,

spreadsheets, XML ﬁles or proprietary databases.

This data was automatically transformed to RDF us-

ing a converter with domain speciﬁc knowledge ex-

pressed by an ontology. The RDF model is stored

in a triple store, an optimized database for seman-

tic triples. The server hosting the data provides a

SPARQL

endpoint. This allows powerful queries on

graph patterns as well as inserting new triples in the

triple store.

Figure 1: Flow of information.

5 IMPLEMENTATION

5.1 Hardware Platform

A 7

HTC Flyer tablet with Android 2.3.4 was used

to evaluate our application. The Semantic Neighbor-

hood Browser is adaptive to screen size and is also

well usable on 5.3

or 10

displays.

Resource Description Framework:

http://www.w3.org/TR/rdf-primer/

SPARQL Query Language for RDF: http://www.w3.

org/TR/rdf-sparql-query/

5.2 Development Framework

The Semantic Neighborhood Browser was imple-

mented using a combination of the cross-platform de-

velopment framework Qt

and Semantic Web tech-

nologies.

The mobile application itself was developed using

Qt Quick

, a declarative language for rapid UI devel-

opment. To query the SPARQL endpoint we made

use of QSparql

, a SPARQL library for Qt, that is

currently in development. Since Android is not an

ofﬁcially supported platform for Qt, we took advan-

tage of the Necessitas project

, which is a community

supported effort to port the Qt framework to Android

and is based on Qt Lighthouse

On the server side, we use an instance of the Vir-

tuoso Linked Data Server

to hold the RDF data, to

provide the SPARQL endpoint and to allow automatic

creation of reverse links.

5.3 Use Case Example: Flow of Matter

5.3.1 Data Model

To illustrate the way the Semantic Neighborhood

Browser works, we will give a short example showing

how to navigate along a product stream in a chemical

plant. In the example given in this paper, the data

comes from a small batch plant used for educational

purposes. The plant consists of some tanks, reactors,

pipes, pumps and valves. These equipments allow to

mix, heat and stir liquids from three reactant tanks in

two reactors. The products can then be used for an-

other reaction in the other tanks or ﬁlled into product

tanks.

The whole plant was engineered in Comos

, a

Computer-Aided-Engineering (CAE) system. It pro-

vides a model of the plant equipment with all at-

tributes and connections. Ab excerpt of the P&ID can

be seen in ﬁgure 2. The relevant parts for the given

scenario have been printed in bold. The reactor R001

is located in the center of the ﬁgure, it can be ﬁlled

Nokia Qt: http://qt.nokia.com

Qt UI Creation Kit: http://qt.nokia.com/qtquick

QSparql project: http://maemo.gitorious.org/maemo-

af/qsparql

Necessitas: http://http://sourceforge.net/p/necessitas

Qt Lighthouse: http://labs.qt.nokia.com/2011/05/31/lig

hthouse-has-grown-up-now/

OpenLink Software Virtuoso: http://virtuoso.openlinks

w.com/linked-data/

Comos is a plant engineering software by SIEMENS:

http://www.automation.siemens.com/mcms/plant-engineer

ing-software/en/Pages/

PECCS 2012 - International Conference on Pervasive and Embedded Computing and Communication Systems

152

A1T2S001

SO+

A1T2S001

A1T2X002

GO+-

A1T2X002

A1T2X003

GO+-

A1T2X003

A1T2S003

SO+

A1T2S003

TIC

A1T2T001

TSA+

A1T2T001

LISA+

A1T2L001

A1T3X001

GO+-

A1T3X001

A1T4X003

GO+-

A1T4X003

P001

Reactor

R001

V001

V003

V008

V007

V006

V002

V004

V005

Figure 2: Detail of the piping & instrumentation diagram

used in the example.

from three reactant tanks via the valves V001, V002

and V003. The output of the reactor is connected to

pump P001 which delivers the product to the valves

V006, V007, from where it ﬂows to the product tanks

and to valve V008. Reactor R001 can also be ﬁlled

from a purging tank (V005) or another source (V004).

The engineering data is exported from Comos to

an XML format. This data is transformed to RDF us-

ing a Maintenance Structure Ontology (labeled mso)

which describes the structure of the plant regard-

ing the product ﬂow. The namespace sce is used

to describe the instances of the actual devices in

the plant. The concepts for the different types of

equipments are expressed as mso:Valve, mso:Reactor

and mso:Pump which hold common attributes like

mso:rﬁd and mso:plantID as well as type speciﬁc at-

tributes (such as mso:capacity for tanks). The pred-

icate for a directed ﬂow connection from one equip-

ment to another is mso:connectedTo.

The predicate mso:connectedTo is the one that de-

ﬁnes neighborhood in our case.

The parts of the RDF model which belong to the

described excerpt of the plant are shown in ﬁgure 3

in a serialized Turtle

format. This serialization is

already human-readable, however, a graph visualiza-

tion is provided in ﬁgure 4. It allows a better overview

and a direct comparison to the P&ID in ﬁgure 2.

5.3.2 Example Navigation

After scanning the RFID tag attached to the device of

interest, the Semantic Neighborhood Browser starts at

Turtle - Terse RDF Triple Language:

http://www.w3.org/TeamSubmission/turtle/

sce:V001 rdf:type mso:Valve .

sce:V002 rdf:type mso:Valve .

...

sce:V008 rdf:type mso:Valve .

sce:R001 rdf:type mso:Reactor .

sce:P001 rdf:type mso:Pump .

sce:V001 mso:connectedTo sce:R001 .

sce:V002 mso:connectedTo sce:R001 .

sce:V003 mso:connectedTo sce:R001 .

sce:V004 mso:connectedTo sce:R001 .

sce:V005 mso:connectedTo sce:R001 .

sce:R001 mso:connectedTo sce:P001 .

sce:P001 mso:connectedTo sce:V006 .

sce:P001 mso:connectedTo sce:V007 .

sce:P001 mso:connectedTo sce:V008 .

Figure 3: Excerpt from the RDF representation of the plant

structure.

Figure 4: Graph visualization of the RDF model.

the pump R001 in the plant structure. All its relevant

attributes are shown in the center of the UI, as can

be seen in ﬁgure 5. They comprise the equipment’s

Name, plant ID, description, RFID tag and others. On

the left side of the screen the list of equipments that

can deliver liquids into the reactor, the valves V001

to V005, can be seen. The right side contains the el-

ements that are located downstream from the reactor.

In this case, it is only the pump P001. Each element

upstream and downstream from the current “center of

the universe” has its title and its description shown to

give the user a brief overview of the connected equip-

ment.

Pressing the Make Center button of the pump

P001 brings P001 to the middle of the UI and re-

trieves all of its attributes. There now is only one in-

put item, namely R001. The new output equipments

BROWSING REVERSIBLE NEIGHBORHOOD RELATIONS IN LINKED DATA ON MOBILE DEVICES

153

Figure 5: Screenshot of the Semantic Neighborhood

Browser accessing the ﬂow of matter.

V006, V007 and V008 are listed on the right side.

5.4 Further Applications

The Semantic Neighborhood Browser was primarily

developed to support device diagnosis tasks. How-

ever, it can be easily adapted to other navigation and

information processing tasks with similar informa-

tional needs. It allows browsing of any reversible

Linked Data relation and has therefore a broad range

of possible applications. As a simple example one

could follow physical or logical inputs and outputs

of devices. This tracking covers different types of

ﬂows, for example matter ﬂow, energy ﬂow or in-

formation ﬂow. However, because of its generic ap-

proach, it is suitable for all meshed networks. Also

for non meshed connections the Semantic Neighbor-

hood Browser can help to get a better overview of the

connection structure. It is a good solution for nav-

igating in predecessor and successor connections, in

cause and effects connections or series of implica-

tions. Even the navigation through tree structures can

be achieved.

6 DISCUSSION

6.1 Features

The developed Semantic Neighborhood Browser pro-

totype provides the following features:

Generalization. The coupling of the UI to the data

is very loose. The example data set can be eas-

ily replaced by other data accessible through a

SPARQL endpoint.

Displayed Attributes. The item in the middle (“cen-

ter of the universe”) displays the attributes of an

entity depending on the ontology of the displayed

data.

Ontology-based Neighborhood. The predicates us-

ed for the predecessor- and successor-relation are

chosen at runtime depending on the ontology

which is used for modeling the desired data. Thus,

it is possible to switch to other predicates on the

ﬂy.

Back/Forward Navigation. Users can navigate by

following the desired path through the reversible

predicates and by using the forward/backward

button.

Filtering. The items in the neighborhood can be ﬁl-

tered according to the displayed attributes with the

power of regular expressions.

Sorting. By default, the predecessors and successors

are sorted by title. But there is also the possibility

to sort by other existing predicates in the ontology

at runtime.

6.2 Challenges

When implementing the Semantic Neighborhood

Browser we faced several issues concerning data in-

tegration, data availability and usability.

One bottleneck for integration of Linked Data is

the transformation from other legacy systems to RDF.

It requires additional effort, but since most data can

be extracted in structured formats such as XML this

conversion is not too difﬁcult.

At the moment, our application needs a permanent

connection to the remote SPARQL endpoint. This

connectivity cannot always be ensured in industrial

environments. A caching mechanism could resolve

this problem in most cases. However, the evolving

expansion of wireless technology (Industrial WLAN,

UMTS) is probably going to mitigate this limitation

in the future.

7 CONCLUSIONS & OUTLOOK

In this paper we presented a simple app that may be

considered a ﬁrst building block for an app orchestra-

tion approach on top of Linked Data. This app which

is called Semantic Neighborhood Browser is good at

presenting neighboring abstract or physical things in

a Linked Data Cloud. The only precondition for its

usage is that the objects in the cloud are reliably con-

nected by a predicate that carries the semantics of be-

ing close.

A ﬁrst expert evaluation of the demonstrator has

revealed that the current implementation still needs

PECCS 2012 - International Conference on Pervasive and Embedded Computing and Communication Systems

154

some ﬁne tuning to be candidate for a fair usability

study. First the app is missing graphical features like

valve symbols which are essential in the domain of

application, second the ﬁlter and sort functions which

are essential in more complex settings have to be re-

ﬁned. These features are currently under develop-

ment. For the next iteration a formal evaluation of

usability and effectiveness is planned and will be con-

ducted with engineering students on the one hand and

engineers from partner companies on the other.

ACKNOWLEDGEMENTS

The research leading to these results was par-

tially funded by the European Community’s Seventh

Framework Programme under grant agreement no.

FP7-284928 ComVantage.

REFERENCES

AT&T Knowledge Ventures (2008). Collaboration

across borders. http://www.corp.att.com/emea/docs

/s5 collaboration eng.pdf.

Becker, C. and Bizer, C. (2008). Dbpedia mobile: A

location-enabled linked data browser. In Proceedings

of the Linked Data on the Web Workshop, volume 369.

Berners-Lee, T. (2006). Design Issues - Linked Data.

http://www.w3.org/DesignIssues/LinkedData.html.

Berners-Lee, T., Hollenbach, J., Lu, K., Presbrey, J.,

Prud’hommeaux, E., and Schraefel, M. (2008). Tab-

ulator redux: Browsing and writing linked data. In

Proceedings of the Linked Data on the Web Workshop,

volume 369.

Bizer, C. (2009). The emerging web of linked data. Intelli-

gent Systems, IEEE, 24(5):87–92.

Bizer, C. and Gauß, T. (2007). Disco hyperdata browser.

http://www4.wiwiss.fu-berlin.de/bizer/ng4j/disco/.

Ermilov, T., Heino, N., Tramp, S., and Auer, S. (2011).

Ontowiki mobile–knowledge management in your

pocket. The Semantic Web: Research and Applica-

tions, pages 185–199.

Franz, T., Koch, J., Dividino, R., and Staab, S. (2010).

Lena-tr: Browsing linked open data along knowledge-

aspects. In AAAI Spring Symposium Series.

Graube, M., Pfeffer, J., Ziegler, J., and Urbas, L. (2011).

Linked data as integrating technology for industrial

data. In Proceedings of the 14th International Confer-

ence on Network-Based Information Systems, pages

162–167.

Heck, E. V. and Vervest, P. (2007). Smart business net-

works: how the network wins. Communications of

the ACM, 50:28–37.

Heim, P., Lohmann, S., and Stegemann, T. (2010). Inter-

active relationship discovery via the semantic web. In

Proceedings of the 7th Extended Semantic Web Con-

ference (ESWC 2010), volume 6088 of LNCS, pages

303–317, Berlin/Heidelberg. Springer.

Krausman, A. S. and Nussbaum, M. A. (2007). Effects

of wearing chemical protective clothing on text en-

try when using wearable input devices. International

Journal of Industrial Ergonomics, 37(6):525–530.

Pietriga, E., Bizer, C., Karger, D., and Lee, R. (2006).

Fresnel - a browser-independent presentation vocab-

ulary for RDF. In Proceedings of the Second Inter-

national Workshop on Interaction Design and the Se-

mantic Web, pages 158–171. Springer.

Pousttchi, K. and Thurnher, B. (2007). Adoption and im-

pact of Mobile-Integrated business processes - com-

parison of existing frameworks and analysis of their

generalization potential. 8. Internationale Tagung

Wirtschaftsinformatik 2007 - Band 1, pages 273–290.

Read, D. (2011). Dave’s reﬂections blog archive sparql

droid a semantic technology application for the an-

droid platform. http://monead.com/blog/?p=1028.

Schenker, J. L. (2010). Cloud services hasten the rise of the

virtual company.

Sonntag, D. and Heim, P. (2007). Semantic graph vi-

sualisation for mobile semantic web interfaces. In

Proceeedings KI 2007: Advances in Artiﬁcial Intel-

ligence, pages 506–509.

Urbas, L. (2010). Hard-, soft-, net- & socialware: Chances

and challenges for useware engineering. In VDI-

Bericht 2010, pages 3–13.

Urbas, L., Pfeffer, J., and Ziegler, J. (2011). iLD-Apps: Us-

able mobile access to linked data clouds at the shop

ﬂoor. In Proceedings of the Workshop on Visual Inter-

faces to the Social and Semantic Web, volume 694.

Viehland, D. and Yang, C. (2007). Bringing the mobile

workforce to business: a case study in a ﬁeld ser-

vice organization. In Management of Mobile Business,

2007. ICMB 2007. International Conference on the,

page 3939.

W3C (2011). LinkedData - W3C wiki. http://www.w3.org/

wiki/LinkedData.

BROWSING REVERSIBLE NEIGHBORHOOD RELATIONS IN LINKED DATA ON MOBILE DEVICES

155