Integration of Augmented Reality Technologies in Process Modeling

The Augmentation of Real World Scenarios With the KMDL

Marcus Grum and Norbert Gronau

Department of Business Informatics, esp. Processes and Systems, University of Potsdam,

August-Bebel-Strasse 89, 14482 Potsdam, Germany

mgrum@lswi.de, ngronau@lswi.de

Keywords:

Augmented Reality, Process Modeling, Knowledge Modeling Description Language, KMDL, Tacit Knowl-

edge Transfer Visualization, Cyber-Physical Market Visualization, Process Simulation Visualization,

Simulation Process Building, Industry 4.0, CPS, CPPS, Internet of Things.

Abstract:

The integration of powerful technologies in traditional domains realizes promising potentials but mostly rises

complexity and shrinks comprehensibility of underlying processes. With the aim to rise comprehensibility

of non-transparent processes, a process modeling language has been transfered in 3-D and prepared to aug-

ment the real world. Deﬁnitions for a methodological proceeding have been created and were mapped to a

software tool. Three scenarios then have been realized as demonstration and proof the working of drawn AR

integrations. Focusing non-transparent processes, scenarios visualize tacit knowledge transfers (1), complex

coordination mechanisms (2) and process simulations (3) in the domain of cyber-physical production systems.

1 INTRODUCTION

Once, new technologies are integrated in existing

processes, new potentials can be established. As

one takes the integration of Internet of Things tech-

nologies in traditional production systems, there can

be realized customized productions and ﬂexible, fast

changing production processes based on further feed-

back loops between cyber-physical systems (Gronau

et al., 2016a). With this, new coordination efforts

among them require much more complex processes,

with time-dependent system states and numerous data

transfers. Since, all of them are hard to comprehend

for non-experts and the current processing is hard

to categorize correctly, those further are referred to

as non-transparent processes. Although not limited

to the domain of cyber-physical production systems,

this domain is very suitable for the selection of non-

transparent processes.

Since 2-D modeling approaches can be extended

easily with a third dimension, new AR technologies

bring in potential to increase the comprehensibility of

those non-transparent processes in using available di-

mensions and located existing 2-D models within the

real world. Hence, the following research question

will be focused within this paper: ”How can non-

transparent processes be visualized with help of AR

technologies?” This includes the process of modeling.

Because of the interplay of the real world, augmented

world and the more or less paper based 2-D world of

common process models, lots of potentials can be re-

alized during all process modeling phases. This paper

intends not to collect an all-embracing collection of

potentials rather than drawing a ﬁrst way of structur-

ing. Although several options can be realized to use

three dimensions of the AR technology, the following

focuses only on a spacial placement.

The research approach is intended to be design-

oriented as Peffers proposes (Peffers et al., 2006) and

(Peffers et al., 2007), such that the paper is structured

as follows: A second section presents underlying con-

cepts, the third sections derives objectives for an in-

tegration of augmented reality in process modeling.

The fourth section provides the design, followed by

its demonstration and evaluation. A ﬁnal section con-

cludes the paper.

2 UNDERLYING CONCEPTS

Starting with the selection of a modeling approach in

the ﬁrst subsection, the application center for industry

4.0 is identiﬁed as a promising environment for non-

transparent processes since participating systems pro-

vide separate knowledge bases. Further, approaches

such as the cyber-physical market require numerous

206

non-transparent coordination efforts and provide a

fruitful environment for process simulations and co-

ordination approach benchmarks. In a last subsection,

available AR technologies are discussed.

2.1 Process Modeling Areas

Since non-transparent processes shall be focused on

the base of separate knowledge bases and complex

conversations within Internet of Things similar struc-

tures, the following concentrates on knowledge mod-

eling methods.

An overview of existing modeling methods and

a comparison of their ability to represent knowl-

edge can be found by (Remus, 2002, p. 216f.).

Here, ARIS, INCOME, PROMOTE, WORKWARE,

EULE2 and FORWISS are only some representa-

tives. Hereunder, (Gronau and Maasdorp, 2016, )

identify the Knowledge Modeling Description Lan-

guage (short: KMDL) as only representative to over-

come lacks in visualizations and analyses through the

combination of several views (process view, activity

view and communication view). Focusing on the even

broader context of organizational, behavior-oriented,

informational and knowledge-oriented perspectives,

(Sultanow et al., 2012, ) identify the KMDL to be in-

ferior in the comparison of twelve common modeling

approaches as well.

Being developed iteratively and being applied in

numerous projects, the KMDL has been developed

and optimized over more then ten years. An evolu-

tion of the KMDL can be found in (Gronau, 2012)

and currently, the development of the version 3.0 is

in progress (Gronau et al., 2016b). The KMDL has

proven its beneﬁts in numerous application areas such

as software engineering, product development, qual-

ity assurance and investment good sales. It provides a

fully developed research method which can be found

in Figure 1 and is described by (Gronau, 2009, p. 386)

in detail.

With its strengths in visualization, the KMDL

seems attractive for augmenting the reality. To the

best of our knowledge, so far an augmentation of

the real world with spacial correct positioned process

models has not been realized yet. A prototype of an

AR collaborative process modeling tool augments the

real world with BPMN process models, which are

augmented only w.r.t. inner model relations (Poppe

et al., 2011). Eichhorn et al. presented geometric 3-D

Figure models in a virtual space and created statistical

insights from those (Eichhorn et al., 2009).

Hence, the current paper builds on the wide spread

KMDL version 2.2 (Gronau and Maasdorp, 2016, ).

With its intention to focus on the generation of

knowledge following (Nonaka and Takeuchi, 1995),

the KMDL enables the modeling of tacit knowledge

bases, single or numerous knowledge transfers, the

socializing of several conversion partners in complex

control ﬂows and their time-dependent development.

Since all can be identiﬁed as non-transparent pro-

cesses, the KMDL seems attractive for the scenario

design in section 4.3.

2.2 Application Center for Industry 4.0

Since the physical meaning of classical production

components can be enhanced by a virtual represen-

tation, these can be considered as cyber-physical sys-

tems (short: CPS), providing more or less distinctive

characteristics in abilities to perceive its environment

via sensors, to interact with its environment via actu-

ators, process data via processors and communicate

via communicators (Gronau et al., 2016a). Equipped

with memory, each CPS can build individual knowl-

edge bases and can hold time-dependent states.

A cyber-physical production system (short:

CPPS) integrates several CPS with the purpose to

realize productions. For this, huge communication

efforts are necessary and complex coordination

mechanisms are required. As one of many, a

cyber-physical market can realize this coordination

analogous to real market mechanisms (Grum et al.,

2016) such as each CPS is considered as market

participant and has to negotiate with its environment

before tasks are realized.

The Application Center for Industry 4.0 (short:

ACI4.0) is build as CPPS, containing several types

of CPS. Machines are surrounded by computer dis-

plays and can visualize different kind of production

steps. Conveyors connect machines and transport

workpieces. A workpiece is a small box surrounded

by displays such that its current production state can

be visualized. Next to the conveyors, robots or hu-

mans are placed, that are part of the production pro-

cess. All of them are considered to be a CPS pro-

viding more or less distinctive characteristics within a

cyber-physical market. Hence, the ACI4.0 is a fruit-

ful environment for tacit knowledge transfers, pro-

cess simulations and coordination efforts, which are

all non-transparent processes. In section 4.3, the sce-

nario design will therefore be based on the ACI4.0.

2.3 Available AR Technologies

As augmented reality (short: AR), the paper follows

the deﬁnition of (Azuma, 1997), who identiﬁes AR

as a variation of virtual environments, which allows

users to perceive the real world, superimposed and

Integration of Augmented Reality Technologies in Process Modeling - The Augmentation of Real World Scenarios With the

KMDL

207

Figure 1: KMDL procedural model.

composited virtual objects such that users have im-

pression those worlds would coexist. Therefore, AR

systems have the following characteristics:

• a real world and a virtual world is combined;

• an interaction with both worlds is possible in real

time;

• objects are registered in 3-D.

Following (Schart and Tschanz, 2015), the visual-

ization can be realized with help of screen displays,

mobile devices (handhelds), head-up displays, head-

mounted displays and contact lenses. Here, head-

mounted displays (HMD) were focused because of

the intention to quickly hand over the visualization

technique from person to person and do not limit their

interaction via hands during the production process.

HDM gadgets are available as optical see-through

HMD and closed-view HMD. While the ﬁrst variant

lets the user see the real world directly, the second

variant does not allow any direct contact with the real

world. Each brings individual advantages and (Jan-

nick et al., 1994) discusses tradeoffs. The follow-

ing focused optical see-through HMD with the in-

tention to choose the most realistic system. Here,

Google Glass, the Epson Moverio BT-200 and the Mi-

crosoft HoloLens were compared with respect to the

following criteria: price, processor performance, bat-

tery runtime, RAM, ﬁeld of vision size, display solu-

tion, usability and availability. Here, the AR glasses

from Epson were selected since other products did not

provide a handheld control unit.

The Epson Moverio BT-200 is available for about

EUR 700 and provides two miniature projectors

which are placed on each glasses side piece. The pro-

jection surface is positioned within the ﬁeld of view

and transparent, such that it is possible to perceive

both, the real world and projected world directly. Be-

ing equipped with a dual core processor (1300Mhz)

and one gigabyte RAM, the AR component is sufﬁ-

cient for ﬁrst augmentation purposes. A GPS mod-

ule beside software computer vision components can

be used for the placement of the AR glasses within

space.

3 OBJECTIVES OF AN

AUGMENTED REALITY

INTEGRATION IN PROCESS

MODELING

Since a modeling language shall augment the real

world, objectives of three domains have been iden-

tiﬁed: The modeling language itself, the context for

the modeling language as well as the augmentation

technique.

Aiming to prepare the KMDL for the purpose to

augment the real world, the following set of require-

ments has to be considered:

• the augmentation has to build up on an existing

version of the KMDL;

• existing shapes have to be mapped to 3-D;

• the augmentation has to be included within the

methodological approach of the KMDL;

• the augmentation of the KMDL has to go along

with the extension of the corresponding modeling

software, which is Modelangelo

;

http://www.kmdl.de/en/node/46

Seventh International Symposium on Business Modeling and Software Design

208

• time-dependent visualizations have to be consid-

ered, since process models can change on an ab-

stract level and the content of modeled items can

change as well on a concrete level;

• fast time-dependent visualizations have to be

slowed down so that the human perception is able

to deal with.

With respect to the scenario creation, the follow-

ing objectives have been identiﬁed:

• a real world tacit knowledge transfer has to be vi-

sualized;

• the communication of heavily complex processes

has to be visualized;

• the simulation of processes has to be visualized;

• all three, the activity view, process view and com-

munication view shall be visualizable;

• process modeling phases shall be supported.

Focusing on the hardware selection, the following

criteria were relevant additionally to AR technique in-

herent requirements such as the positioning within an

area, performance issues, etc.:

• AR glasses shall be used within closed rooms;

• AR glasses shall be used within the outdoor area;

• AR glasses shall cost less than Euro 1.000;

• AR glasses shall bring programming libraries for

free and ideally open source;

• AR glasses shall realize free movements;

• AR glasses shall consider real physics so that ob-

stacles cover shapes in the background;

• AR techniques shall be used on base of common

cameras, so that persons who currently do not

wear AR glasses can see the augmented world on

a projector;

• AR glasses shall realize interactions with the aug-

mented world.

Each identiﬁed objective of those three domains

has been relevant for the augmentation of non-

transparent processes and serves as input for the fol-

lowing sections.

4 DESIGN OF AN AUGMENTED

REALITY INTEGRATION IN

PROCESS MODELING

The design of AR integration in process modeling

is presented with help of four subsections. For the

ﬁrst, the KMDL is augmented, then the method is ex-

panded. Afterwards, three scenarios are built and ﬁ-

nally, software tooling issues are designed.

4.1 Augmenting the KMDL

On base of existing shapes of the KMDL as it was

selected in section 2.1, items were mapped to three

dimensions, which were required by the scenarios and

can be seen in the modelings of subsection 4.3. Those

items were constructed with Autodesk Fusion360 and

can be found in Figure 2.

Figure 2: Mapping of existing KMDL shapes to three di-

mensions.

Here, mostly simple extrusions and colored glass

materials were used because of their transparent char-

acteristics in the augmentation.

Since those shapes shall be located within the real

world, the following new attributes have to be brought

in the KMDL:

A set of origin coordinates in x-, y- and z-axis de-

ﬁne the global point of origin within space. A set of

coordinates in x-, y- and z-axis with respect to a point

of origin locates the center of any body within space.

Initially, the global point of origin is selected but those

coordinates can consider the position of other bodies

as a relative point of origin as well, such that easy

spacial movements are possible. A set of size in x-,

y- and z-axis manages the spacial requirements and

is ideally automatically adjusted in relation to other

bodies’ sizes. A set of rotation in x-, y- and z-axis

can assure the optimal angle with respect to a person

wearing AR glasses, such that bodies and texts can

be identiﬁed easily. Since each item can hold a 3-D

model, a ﬁlepath to this model and a checkbox for the

standard KMDL shape visualization and a checkbox

for the 3-D model visualization shall switch them on

or off.

For the modeling, only a single shape has been in-

troduced, which looks similar to the shape of the con-

version method but holds an ”AR” on it. This shape

indicates a proper prepared AR perspective and saves

previously mentioned 3-D information. Since the

same knowledge conversion can be visualized in dif-

ferent views, the modeling can hold several of them.

Integration of Augmented Reality Technologies in Process Modeling - The Augmentation of Real World Scenarios With the

KMDL

209

All together, those extensions are the basis for the

augmentation of the real world with process modeling

languages.

4.2 Expansion of the Methodological

Appraoch

Faced with a well described procedural model of the

KMDL as can be seen in Figure 1, the following de-

scribes the integration of AR technology. For this, the

numbers within the Figure serve as orientation.

Phases from the project initiation (phase 0) until

phase 2.1 can be realized as usual.

The modeling (phase 2.2) can be enriched by 3-

D information as was required in section 4.1, can be

simpliﬁed with help of a ground plan and sketch plan

as is described in section 4.4, and is visualized as can

be seen in Figures 10 and 11.

The model validation (phase 2.3) can be enriched

with the spacial observation within those plans as well

as with a look on the augmented reality as it is de-

scribed in subsection 5.3.

Phases from the identiﬁcation of knowledge inten-

sive tasks (phase 3) until phase 4.1 can be realized as

usual.

Analogous to modeling and validation steps be-

fore, 3-D information can enrich the activity model-

ing with help of a ground plan and sketch plan as is

described in section 4.4, and is visualized as can be

seen in Figures 6 and 7 as well as in Figures 8 and 9.

The model validation (phase 4.3) can be enriched

with the spacial observation within those plans as well

as with a look on the augmented reality as it is de-

scribed in subsection 5.1 and 5.2.

The work of analysts can be enriched by the im-

pression of the collection of created plans and real

world augmentations (phase 5).

Phases 6-8 can be enriched in AR visualizations

of a static to-be concept and non-static simulation vi-

sualizations as they can be seen in all three scenar-

ios. Hence, is- and planned-to-be comparisons can

be realized, the working of an implementation can

be tested. Further, comparisons of planned-to-be and

realized-to-be can be realized.

All together, those methodological expansions

serve as guideline and show how to integrate AR

technology in the process modeling. The following

was realized considering those extensions and demon-

strates its working.

4.3 Integration of AR Technology

The integration of AR technology in process model-

ing shall be visualized with help of three scenarios.

All can be located within the ACI4.0, which was de-

scribed in section 2.2.

A laser manipulation scenario shall visualize the

tacit knowledge transfer in complex, multi-interaction

partner settings. As can be seen in Figure 3, a sit-

uation has been designed within the CPM, wherein a

robot manipulates a workpiece with a laser cutter. The

workpiece stands on top of a conveyor and is observed

by a human worker. The worker’s task is to control

the production progress and interfere when necessary.

For this, basic knowledge in laser manipulation and

the experience gained from former production is con-

sidered as tacit knowledge.

Figure 3: Current activity view of the laser manipulation

task (laser manipulation scenario).

A negotiation scenario shall visualize the com-

plex interplay of numerous CPS during a negotiation

within the CPM following (Grum et al., 2016). Here,

four CPS socialize and update themselves with price

value, amount and duration information.

Figure 4: Current activity view of the negotiation task from

CPS3 (negotiation scenario).

A process variation scenario shall focus process

variations during process simulations within CPPS.

As becomes clear in Figure 5, a selected workpiece

can be produced following alternative process op-

tions. Either, a closely situated robot is used for hu-

man controlled laser manipulation (see scenario 1),

or robots are used that are located in greater distances

so that the workpiece requires further transport steps

to reach the robot. Place holder tasks showing ”...”

visualize the idea that neither the entire production

process of the workpiece nor all process options are

Seventh International Symposium on Business Modeling and Software Design

210

visualized in this Figure.

Figure 5: Simpliﬁed current process model of a workpiece

within a CPPS simulation (process variation scenario).

Since more than one workpiece is produced within

CPPS simultaneously, this Figure only shows a very

limited insight in the current process simulation

model.

All together, those three scenarios serve as fruitful

context to integrate AR technology with the purpose

to visualize non-transparent processes.

4.4 Extending the Modelangelo

Modeling Tool

With help of a modeling tool called Modelangelo, the

following software design supports the integration of

AR in process modeling, as it was designed in sec-

tion 4.2.

In section 4.1 identiﬁed attributes are introduced

in the properties space of Modelangelo.

Beside the normal modeling environment, two

further modeling surfaces are introduced, that sim-

plify the model enrichment with required 3-D infor-

mation. Those take existing model information and

transfer them in a sketch plan and a ground plan.

Here, the modeling is not realized w.r.t. the reading

direction from left to right (see Figure 3-5), but lo-

cates existing shapes within space per drag and drop.

Figure 6 and Figure 7 visualize this difference w.r.t.

the laser manipulation scenario. Before the position-

ing of items within space, white rectangles have been

used to model realistic dimensions and distances eas-

ily. Then, items were placed and resized.

Figure 6: Ground plan of the laser manipulation scenario.

Having a detailed look on Figure 7, shape overlaps

can be identiﬁed because of the spatial arrangement.

Figure 7: Sketch plan of the laser manipulation scenario.

The creation of the ground plan and sketch plan of

the negotiation scenario was realized similarly to the

positioning of the laser manipulation scenario and can

be found in Figure 8 and Figure 9.

Figure 8: Ground plan of the negotiation scenario.

Figure 9: Sketch plan of the negotiation scenario.

Further, a KMDL Augmentor is introduced, that

translates existing KMDL models to the selected AR

Integration of Augmented Reality Technologies in Process Modeling - The Augmentation of Real World Scenarios With the

KMDL

211

glasses (Epson Moverio BT-200), such that models

can be visualized easily. For this, the Vuforia frame-

work has been chosen. KMDL models that lay on the

department servers serve as interface and can be aug-

mented easily.

Further, the KMDL Augmentor can be connected

via interfaces, such that models can visualize time dy-

namics. Then, real world sensory data e.g. coming

from the ACI4.0 or connected simulation frameworks

can be displayed. Since tasks can hold repetitive ma-

chine components, three-dimensional visualizations

can be attached directly to modeling items and option-

ally displayed by activating the 3-D model visualiza-

tion. Hence, quickly huge production settings can be

build. Figure 10 and Figure 11 visualize this by indi-

cating task elements displaying its three-dimensional

components in blue.

Figure 10: Current ground plan of the process variation sce-

nario.

Similar to Figure 7 and 9, task overlaps can be

found in Figure 11 because of the spatial arrangement.

Figure 11: Current sketch plan of the process variation sce-

nario.

A zoomed version of those Figures considering

correct denominations can be found in the appendix.

All together, those software extensions help to lo-

cate model items easily per drag and drop in real spa-

cial dimensions, transfer existing 2-D process models

on AR glasses and help to connect further simulation

frameworks.

5 DEMONSTRATION

The following subsections show the realization of se-

lected scenarios with help of the Vuforia Developer

Libary and the Moverio BT-200. Full videos are avail-

able at following links and complete the here pre-

sented screenshots. Videos have been cut with Pow-

erDirector of CyberLink.

5.1 Scenario 1

The positioning of knowledge modeling entities, such

as the KMDL proposes, can be seen in Figure 12.

As the robot manipulates the workpiece, the conveyor

holds the workpiece and pauses the movement of its

rolls. Observed by a human worker currently wearing

the AR glasses, the worker’s experience is considered

in the externalization as well. So, a controlled laser

manipulation can be realized.

Figure 12: A potential look through AR glasses on the aug-

mented KMDL modeling (using Epson Moverio BT-200).

The prepared ”AR Manipulation Template” can be

seen in Figure 13. As the human worker observes

transparently arranged knowledge objects, a more de-

tailed view can be realized because of a manual acti-

vation of a virtual button by a cyber-physical contact

with the worker’s real hand. This is the way, a well-

grounded decision to interrupt the observed laser ma-

nipulation can be found. Since this decision has to be

realized continuously, relevant knowledge modeling

entities update in the video, of course.

Figure 13: A look through AR glasses on the AR laser ma-

nipulation template (using Epson Moverio BT-200).

The corresponding video is available at

https://mediaup.uni-potsdam.de/Play/7230.

5.2 Scenario 2

The positioning of knowledge modeling entities of the

negotiation scenario, can be seen in Figure 14. Since

available CPS are communicating via the CPM and

are exchanging value, amount and duration informa-

tion, green arrows indicate a socialization.

As relevant knowledge modeling entities are not

required within the ”AR CPM Template of CPS3”,

Figure 15 visualizes the conversion on a minimal

base. The CPM is visualized by the red planet. Since

Seventh International Symposium on Business Modeling and Software Design

212

Figure 14: A potential look through AR glasses on the aug-

mented KMDL modeling (using Epson Moverio BT-200).

the communication direction changes over time, the

video shows bidirectional and changing communica-

tion partners, of course.

The corresponding video is available at

https://mediaup.uni-potsdam.de/Play/7231.

Figure 15: A look through AR glasses on the AR CPM tem-

plate of CPS3 (using Epson Moverio BT-200).

5.3 Scenario 3

Figure 16 shows the positioning of knowledge mod-

eling entities of the process variation scenario. Here,

a process view is realized considering tasks, logical

operators and current control ﬂows. Since not only

real world elements can be augmented, here, the focus

lays on the virtual extension of the physically avail-

able production setting within the ACI4.0.

Figure 16: A potential look through AR glasses on the cur-

rent KMDL simulation setting (using a common camera).

The simulation can be nicely regarded as knowl-

edge modeling entities are not visualized (Figure 17).

Hence, the video shows the simulated production of

numerous workpieces on the left next to real, physi-

cal productions on the right.

The corresponding video is available at

https://mediaup.uni-potsdam.de/Play/7232.

Figure 17: A look through AR glasses on the current simu-

lation setting (using a common camera).

6 EVALUATION

Considering the presented demonstration, the objec-

tives identiﬁed in section 3 could be met with respect

to three objective groups.

Objectives of the KMDL augmentation have been

met as follows: Time-dependent visualizations have

been considered in all three scenarios. Since a trig-

ger such as an information exchange started a visual-

ization, the speed of that visualizations was adjusted

on base of the human perception. Since the concrete

content of the scenarios (scenario 1 and 2) and the ab-

stract process models (scenario 3) changed, all time-

dependent objectives were met. Building up on the

model items of KMDL (version 2.2) and extending its

shapes with a third dimension, the ﬁrst two objectives

were considered as well. The extension of Modelan-

gelo has been met with the integration of the ground

and sketch plan, the identiﬁed set of properties as well

as the integration with the selected AR framework.

Objectives of the scenario selection have been met

as follows: An activity view has been realized in the

laser manipulation scenario, a communication view

has been realized in the negotiation scenario. A pro-

cess view has been realized in scenario three. Hence,

all objectives have been met.

Objectives of the hardware selection have been

met as follows: All three scenarios have been real-

ized within a closed room with help of the computer

vision technique of the Vuforia framework. For this,

three QR codes have been placed within the produc-

tion setting, so that each scenario has been identiﬁed

easily. The objective to move freely was met but has

to be limited: The identiﬁcation was only possible

when the QR code was detected by the camera sys-

tem. Hence, the degree of freedom was limited and

dependent on the size, position and viewing angle of

the QR code relative to the AR glasses. Hence, the

design of each scenario was optimized w.r.t. the po-

sition of the intended QR code relation. Although an

outdoor arena has not been part of any scenario, one

can proceed on the assumption that all three scenarios

would have worked there with the aid of the QR code

as well. Further, a GPS signal could have been used

additionally so that greater movements were possi-

Integration of Augmented Reality Technologies in Process Modeling - The Augmentation of Real World Scenarios With the

KMDL

213

ble. With about Euro 700, the price objective was

met by the selection of the Epson Moverio BT-200.

The performance was acceptable although the hard-

ware was not powerful enough to carry all three sce-

narios within one common application. Each scenario

has had to be realized as separate application. Since

the room was not scanned and 3-D modeled by the

AR system, augmentations did not consider physical

obstacles so that the objective to consider real world

physics has to be limited. Augmented parts, which

should not be visible because of real world elements,

were visualized although. Hence, the design of each

scenario was optimized w.r.t. the position of the in-

tended QR code. Interactions with the augmented

world were possible on behalf of the camera system

of the AR glasses. The results of a look through a

common camera (scenario 3) were very good so that

further parties will be able to enjoy a persons perspec-

tive on a display.

7 CONCLUSIONS

In this paper, an integration of AR technologies in

process modeling has been drawn and realized on be-

half of the KMDL. Main contributions and scientiﬁc

novelties are the following: A modeling language has

been prepared for augmenting the real world. This in-

cludes the building of three-dimensional shapes of the

KMDL, the identiﬁcation of required shape proper-

ties and the deﬁnition of AR required modeling tech-

niques such as the ground and sketch plan. An ex-

pansion of an methodological approach for augmen-

tations has been drawn. On that base, three non-

transparent process scenarios have been designed on

behalf of the KMDL and brought to a time dynamic

realization. With this, the drawn integration could

have been applied and proven. Hence, the research

question was answered and the following potentials

are suitable next steps:

The realization of an outdoor scenario was attrac-

tive in order to get insights about the precision of aug-

mentations. Further, the comparison of AR glasses of

the same price level was attractive as well as the com-

parison with more powerful AR glasses. Still promis-

ing is the deepening of the AR integration in pro-

cess modeling phases such as the bidirectional inter-

play of modeling within the augmented world and the

two dimensional process model world. For example

a process model could be created while standing on a

real world position and dropping model items. Fur-

ther, process optimizations could be realized within

the augmented world in grabbing and moving certain

process steps. Here, a systematic research consider-

ing all modeling phases as shown in Figure 1 was

attractive. In presented approaches, given three di-

mensions were interpreted as spacial dimensions but

the use of further meanings can rise comprehensibil-

ity as well. Considering further dimensions in created

scenarios, those can be tested quantitatively through

surveys that shall identify a rise in comprehension.

REFERENCES

Azuma, R. T. (1997). A survey of augmented reality. Tele-

operators and Virtual Environments, 4(6):1–48.

Eichhorn, D., Koschmider, A., Li, Y., Strzel, P., Oberweis,

A., and Trunko, R. (2009). 3d support for business

process simulation. Computer Software and Appli-

cations Conference COMPSAC09. 33rd Annual IEEE

International, 1.

Gronau, N. (2009). Process Oriented Management of

Knowledge: Methods and Tools for the Employment

of Knowledge as a Competitive Factor in Organiza-

tions (Wissen prozessorientiert managen: Methode

und Werkzeuge f

ur die Nutzung des Wettbewerbsfak-

tors Wissen in Unternehmen). Oldenbourg Verlag

unchen.

Gronau, N. (2012). Modeling and Analyzing knowledge in-

tensive business processes with KMDL - Comprehen-

sive insights into theory and practice. GITO mbH Ver-

lag Berlin.

Gronau, N., Grum, M., and Bender, B. (2016a). Determin-

ing the optimal level of autonomy in cyber-physical

production systems. Proceedings of the 14th Interna-

tional Conference on Industrial Informatics (INDIN).

Gronau, N. and Maasdorp, C. (2016). Modeling of or-

ganizational knowledge and information : analyzing

knowledge-intensive business processes with KMDL.

GITO mbH Verlag Berlin.

Gronau, N., Thiem, C., Ullrich, A., Vladova, G., and Weber,

E. (2016b). Ein Vorschlag zur Modellierung von Wis-

sen in wissensintensiven Gesch

aftsprozessen. Tech-

nical report, University of Potsdam, Department of

Business Informatics, esp. Processes and Systems.

Grum, M., Dehnert, M., Vollmer, F., and Zhao, M. (2016).

The conception of a cyber-physical market model

as coordination instrument for production systems.

Technical report, University of Potsdam, Department

of Business Informatics, esp. Processes and Systems.

Jannick, R., Holloway, R., and Fuchs, H. (1994). A compar-

ison of optical and video see-through head-mounted

displays. SPIE Proceedings, Telemanipulator and

Telepresence Technologies, 2351:293–307.

Nonaka, I. and Takeuchi, H. (1995). The knowledge-

creating company: How Japanese companies create

the dynamics of innovation. Oxford university press.

Peffers, K., Tuunanen, T., Gengler, C. E., Rossi, M., Hui,

W., Virtanen, V., and Bragge, J. (2006). The design

science research process: A model for producing and

presenting information systems reseach. 1st Interna-

Seventh International Symposium on Business Modeling and Software Design

214

tional Conference on Design Science in Information

Systems and Technology (DESRIST), 24(3):83–106.

Peffers, K., Tuunanen, T., Rothenberger, M. A., and Chat-

terjee, S. (2007). A design science research method-

ology for information systems research. Management

Informations Systems, 24(3):45–78.

Poppe, E., Brown, R., Johnson, D., and Recker, J. (2011).

A prototype augmented reality collaborative process

modelling tool. 9th International Conference on Busi-

ness Process Management.

Remus, U. (2002). Process-oriented knowledge manage-

ment. Design and modelling. PhD thesis, University

of Regensburg.

Schart, D. and Tschanz, N. (2015). Augmented Reality.

UVK Verlagsgesellschaft Konstanz, M

unchen.

Sultanow, E., Zhou, X., Gronau, N., and Cox, S. (2012).

Modeling of processes, systems and knowledge: a

multi-dimensional comparison of 13 chosen meth-

ods. International Review on Computers and Software

(I.RE.CO.S.),, 7(6):3309–3320.

APPENDIX

Figure 18: Zoomed current ground plan of the process vari-

ation scenario.

Figure 19: Zoomed current sketch plan of the process vari-

ation scenario.

Integration of Augmented Reality Technologies in Process Modeling - The Augmentation of Real World Scenarios With the

KMDL

215