An Ontology-Driven Framework to Support Scenario Representation

in a 3D Operator Training Simulator

Flávio Torres Filho

and Maria de Fátima Queiroz Vieira

Post-Graduate Program in Electrical Engineering - PPgEE – COPELE, Campina Grande, Brazil

Department of Electrical Engineering, Federal University of Campina Grande, Campina Grande, Brazil

Keywords: Operator Training Systems, Ontology, Modelling and Simulation, 3D Representation, Coloured Petri Net.

Abstract: This paper presents a framework to support the development of three-dimensional virtual scenarios for an

operator training simulator for electrical systems. Given the need to represent a variety of scenarios of

interest, this can be an effort and time consuming task. The proposed framework promotes a systematic

approach when building scenarios and is supported by tools from the ontology-based domain. An editing

tool is also under construction. As it will be discussed in the paper, this approach resulted in the

simplification of the scenario building process which is achieved by interchanging descriptions at different

levels of abstraction. The descriptions concern the situation to be represented; plant objects’ representation,

both visual and behavioural, the latter represented as Coloured Petri Nets (CPN) models.

1 INTRODUCTION

Operators in electrical systems’ substations must be

prepared to manage a large volume of information

during routine tasks and to solve critical problems in

strict deadlines. Even when performing frequent and

scheduled tasks, cognitive pressures are present to

keep very high performances. Human errors in this

context could result in serious consequences for

operators and the system, leading companies to

intensifying operator training.

Different approaches are employed when

training; updating and certifying electrical systems

operators. These include: reading technical materials

and operating standards, offering theoretical and

practical courses, promoting technical visits and

company forums to discuss good working practices.

A broader training practice consists in simulating

critical situations during which, a dramatization of

the real operating environment takes place. This

method, also adopted when training other

professional such as fire-fighters and lifeguards,

consists in simulating an event during which

operators are immersed in a close to real situation.

During those drills, the operators are expected to

identify problems and their cause, as well as to

demonstrate their specific problem solving skills.

This kind of training aims to prepare operators to

deal with very specific situations, and allows

identifying weak points in the operator skills that

need to be addressed.

Effectiveness of training can be better achieved

with the aid of simulation tools which replicate

situations of interest for the operating environment.

On the other hand, training effectiveness depends on

the degree of realism provided by the simulator and

on the relevance of proposed scenarios.

Scenarios can represent routine situations, with

the system operating under normal conditions,

during which operators perform simple and well

known tasks. They can also portray critical and

unusual situations, during which operators must

perform complex tasks, exercising their skills. With

a variety of scenarios, simulators can be employed in

different levels of training; from preparing novice

operators for the routine, to preparing experienced

operators to handle new equipment or new operating

tasks.

The focus of this paper is to address the effort

and complexity of scenario building. The authors

propose the adoption of a systematic approach to

developing scenarios, to be performed in three-

dimensional operator training simulators. This

approach relies on instantiating domain ontologies in

the context of operator training. The aim is to reduce

the effort required in creating the simulated virtual

environment, to facilitate the understanding of the

298

Torres Filho F. and Queiroz Vieira M..

An Ontology-Driven Framework to Support Scenario Representation in a 3D Operator Training Simulator.

DOI: 10.5220/0005111302980303

In Proceedings of the 4th International Conference on Simulation and Modeling Methodologies, Technologies and Applications (SIMULTECH-2014),

pages 298-303

ISBN: 978-989-758-038-3

 2014 SCITEPRESS (Science and Technology Publications, Lda.)

training requirements and to promote the reuse and

refinement of the 3D objects; animation components

and simulation models.

This paper is organized as follows. Section 2

presents a brief review of related work. Section 3

introduces the simulator SimuLIHM. Section 4

presents the scenario generating approach and an

application. Finally, section 5, presents the next

steps, and considerations on the preliminary results.

2 RELATED WORK

As reported in the literature, ontology has been

employed in modelling industrial plants and in

process simulation. An example of ontology

application is the development platform Simantic,

presented in (Luukkainen, Karhela, 2008). This

platform enables the representation of a process

plant from a library of components available in the

3D environment. It provides ontologies for

describing the graphical representation of plant

components used to describe the components’ visual

behaviour (activities); and the configuration of

simulation models which represent the physical

behaviour of each component. From a user described

plant, Simantic automatically generates the code of a

simulator, mapping between ontologies.

Similarly, Parisi et al (2007) propose a

methodology for the automatic generation of 3D

animations to support the training of industrial

operators. Its method consists in using ontology to

capture and filter the generic training requirements,

expressed in natural language. The result is an

adaptive animation, which can be refined by a non-

expert designer in the field, before being presented

to the trainees.

Kalogerakis et. al. (2006) presents a method for

integrating domain ontologies with 3D virtual reality

scenes. Domain knowledge application results in the

enrichment of the virtual environment. This method

is supported by the development platform I3DVP.

Rocha et al (2009) proposed an architecture to

support the modelling of fire-fighters training

simulations. This architecture is based on a set of

ontologies in the domain of fire fighting.

From this review one can conclude that the use

of ontologies seems appropriate to support the

development of operator training simulators and the

modelling of scenarios, for a variety of contexts and

domains. Thus, in this work the focus is on the

development of 3D virtual environments and

scenarios based on ontologies.

3 SimuLIHM

SimuLIHM is an operator training simulator for

electric systems operation. It was developed at the

Human Machine Interface Laboratory (LIHM), at

the Federal University of Campina Grande (Brazil).

This simulator supports the training of electric

substations operators in identifying faults in a virtual

3D environment. It was conceived originally as a

research tool, to support the study of human errors

during critical situations when operating industrial

systems (2007).

This simulator’s architecture is distributed and is

organized in three modules: trainee module, tutor

module, and a server module (Figure 1). The tutor

and operator environments can be accessed via

Internet or intranet, allowing for distance training

and for the simultaneous training of groups of

operators, who can interact between themselves and

with the tutor.

Figure 1: SimuLIHM architecture.

The operator module implements the virtual

environment during training. It reproduces, in virtual

reality, a typical control room of an electric system

substation. In this three-dimensional virtual

environment, operators, can move in the control

room, interacting with its panels and with a real

supervisory system (accessed through the virtual

world. This can be achieved using a mouse or

keyboard. There they can perform tasks in a similar

way as they would in the real environment.

The tutor module enables the communication

with the server and with clients during the training

sessions. This environment offers tools to support

the tutor in the designing and editing of training

scenarios; monitoring trainees’ activities during the

training session and generating reports based on the

sessions’ logs.

AnOntology-DrivenFrameworktoSupportScenarioRepresentationina3DOperatorTrainingSimulator

299

Training scenarios should reproduce situations

from the electric system operation routine or

contingency situations. The choice of the situation

can be based on the analysis of human error reports

from this system’s operation. In this project, error

reports are a source of knowledge represented by

ontology.

The server module contains the simulation

engine and a database built with SQL Server, which

stores the training log. The communication between

modules uses the protocol TCP/IP.

During a training session, trainees interact with

3D objects, which are represented in the simulator in

three layers or levels of abstraction, as illustrated in

Figure 2. Each layer consists of an executable model

that will be described below.

Figure 2: Model layers that constitute an interaction object

in SimuLIHM.

3.1 3D Model

The models that constitute the layer of visual-

geometric representation are described using the

X3D (eXtensible 3D) language. These models are

run by the viewer Xj3D (Brutzman and Daly, 2007),

allowing navigation in the virtual space and the

interaction with the elements that compose it.

A library of 3D models has been built to enable

the representation of different scenarios. In Figure 3

it is shown a subset of these interaction objects in

the context of this work. These models were

designed using the 3D modelling tool X3D-Edit, and

constitute a library of domain objects, which are

configured to compose the virtual environment of a

substation control room.

The models in the object library can be reused

when composing different scenarios for a specific

substation and for any other installation that shares

the same kind of objects.

Figure 3: Examples of objects from 3D object library.

3.2 Animation Model

Model animation is performed in the viewer Xj3D.

The API - Scene Access Interface (SAI) was

developed in Java according to ISO / IEC 19775-

2:2004 standard. It provides a set of methods for

accessing the 3D scene and for running the

behaviour of its objects. Object behaviour is trigged

by user actions on them, such as turning a key in the

control panel, or pressing a pushbutton.

This layer communicates with the 3D models

described in section 3.1 and, with the simulation

models described in the next section.

3.3 Simulation Model

In order to describe the behaviour of real world

objects, in the virtual world, it was built a library

containing the simulation models. These models,

which are described in the formalism Coloured Petri

Nets (Jensen, 1997), represent the behaviour of the

interaction objects in the control room which are

used during the operation of a substation. As a

consequence of an operator command in the virtual

world, the objects’ statuses are updated.

In SimuLIHM, the plant behaviour is described

using Coloured Petri Nets. This formalism’s graphic

and mathematical notations allow representing and

verifying the plant behaviour according to a set of

properties. The simulation model has a modular

structure and its building process is based on an

object library which facilitates the construction of a

variety of scenarios.

The library contains two CPN model classes:

models that represent the behaviour of the control

room objects (switches; pushbuttons; dials; panels

and others); and models that represent the behaviour

of the plant equipment in the substation:

transmission lines; protective equipment (circuit

breakers, switch breakers); transformers, etc.

SIMULTECH2014-4thInternationalConferenceonSimulationandModelingMethodologies,Technologiesand

Applications

300

4 BUILDING SCENARIOS

Defining a scenario consists in specifying the

following items: the initial state of the simulation

environment; the sequence of events that must occur

during the training session and the required

resources (human and material). Since the operator’s

task is ruled by a set of documents with the

operating procedures, the trainee must perform the

task according to these rules and regulations.

It follows a description of the proposed approach

for developing the training scenarios, which is based

on ontologies in the domain. In this work, scenario

building relies on the reuse of software components

(3D models, animations and simulation models).

These are combined and configured according to the

ontology information, in a sequence of five steps, as

represented in Figure 4.

Figure 4: Ontological approach to building 3D scenarios.

4.1 Describing the Scenario

The starting point for the proposed 3D scenario

building process consists in describing it by

instantiating the domain ontologies. The terms,

relationships and rules defined in these ontologies

describe the elements in a scenario.

The scenario building process for the simulator

SimuLIHM is driven by five ontologies: Training,

ScenarioTraining, 3dModel, Plant and HMI. The

ontologies were built using the ontology editor

Protégé (2014), version 3.5, which is associated with

the plug-in Protege-OWL. This plug-in allows

building ontologies in OWL (Web Ontology

Language) - the World Wide Web Consortium

(W3C) standard.

Each of these ontologies is a subset of the

domain, as illustrated in Figure 5. Concepts of the

ontologies: 3dModel; Plant and HMI, are

incorporated into the ontology ScenarioTraining.

Furthermore, some concepts of the ontology

ScenarioTraining are incorporated into the ontology

Training. It follows a description of each ontology.

4.1.1 Ontology Training

The ontology Training defines a set of concepts and

properties for describing electrical systems operators

training. The vocabulary includes types of training

methods; training objectives; training themes and the

required resources (human, material and financial).

It also defines constraints to be adopted in the

description of training, such as: (a) a training session

consists of a set of scenarios; (b) participants play

specific roles during a scenario session.

Figure 5: Ontological representation for the semantic

description of the training domain.

4.1.2 Ontology ScenarioTraining

This ontology defines the terms used in the general

description of a training scenario and its elements;

which are structured as shown in Table 1.

Table 1: Generic structure of a training scenario.

Training Scenario

General Description

Scenario

theme

Scenario Title

Scenario

description

Plant’s

Initial state

Plant’s

Final state

Pre requisites

Objective

General objective

Specific objectives

Task

Task description

Task type

Level of difficulty

Urgency of the problem

Problem frequency

Anticipated duration

Supporting Documents

Scenario Elements

Environment

(Id of the 3D

model)

HMI in the scene and their

respective statuses

Plant Configuration

Participants’ roles (operator, engineer, etc.)

Events

Scheduled

event

Time_driven

Conditional

Human

driven

event

Expected

action

Performed

action

AnOntology-DrivenFrameworktoSupportScenarioRepresentationina3DOperatorTrainingSimulator

301

4.1.3 Ontology Plant

In the context of this work the term plant refers to an

electric system installation, such as a substation. So

this ontology defines the electrical system

components and their relationship. This ontology is

based on the semantic data model defined in the

standards IEC 61970-301 and IEC 61968-11,

collectively known as CIM - Common Information

Model. This abstract model represents all the major

objects in an electric system.

By providing a standard way of representing

electrical system resources as objects; classes and

attributes; along with their relationships, the CIM

facilitates the interoperability and compatibility

between applications and systems, independently of

any particular implementation (IEC 61970:2011).

4.1.4 Ontology IHM

The IHM ontology defines the concepts necessary to

describe the human-machine interface in a

substation of an electrical system. This interface

consists of objects to interac with the control panels:

switches, push buttons, displays, alarms and

message panels. It also defines the relationship

between these components and their statuses.

4.2 Generating the 3D Scenario

Instantiating the described ontologies creates a

knowledge base with the elements of a scenario

description and the models that represent them.

The scenario is generated with the aid of a

software module that interprets the information

stored in the knowledge base and automatically

generates the 3D virtual environment.

There are several APIs that support the

development of software applications based on

ontologies represented in OWL. In this work, the

framework Jena (2009) was used to implement the

software module. Jena is a Java framework, open

source and free, which provides features for editing

and consulting ontologies based on inference

engines.

As already mentioned, 3D models were

described in X3D language, which follows the XML

standard. Thus, an API for editing documents in

XML format instantiates 3D models from the

ontological model. The JDOM Java API (2014),

enables changing, creating and navigating the X3D

document structure.

Figure 6 illustrates the generation of a 3D object,

represented in the virtual world. It illustrates the

representation of a switch on a panel. The SW14C1

switch’s attributes are parameters from the

geometric model of switches, as indicated by the

arrows in the figure. The resulting visual

representation is also presented in Figure 6.

Figure 6: Instantiation of a 3D object using ontology.

The knowledge base is structured hierarchically.

3D objects, such as switches, buttons and displays

compose more complex objects such as control and

message panels. In turn, compound objects are

placed in the 3D virtual environment of a control

panel (Figure 7).

Figure 7: Components of a control panel and their

association with components of the 3D object library.

Therefore, the process of consulting,

instantiating models and associating objects can be

recursive, and is completed when all the objects

have been instantiated, automatically generating the

3D virtual environment.

SIMULTECH2014-4thInternationalConferenceonSimulationandModelingMethodologies,Technologiesand

Applications

302

4.3 Configuring Simulation Models

The simulation (behaviour) models of the objects in

the virtual plant, act as the simulation engine. These

were built and run on the CPN Tools environment

(2014). These models, represented in XML format,

can be configured to represent a specific scenario

using the JDOM API (2014). This API enables to

edit the .cpn file, according to the information

extracted from the knowledge base (ontologies).

Models representing object behaviour in the virtual

environment can be found in (Turnell et al, 2010).

4.4 Saving the Scenario

The 3D virtual environment, animation models and

simulation models, configured according to the

content of the knowledge base, represent the training

scenario to be run by the simulator.

Once completed, this scenario must be stored in

a database that supports XML file format. The

devices in the training scenario must also be

configured and stored with their statuses.

4.5 Running the Scenario

Both the trainee and the tutor interact through a

graphical interface when selecting the scenario in the

simulator database. Once selected, the scenario is

presented to the trainee in the 3D simulator

environment. From within the virtual environment it

is possible to interact with a real supervisory system

there represented on the trainee’s desktop. The

supervisory software must be previously configured

to represent the plant and must be initialized in the

same status as the virtual world representation.

During the scenario simulation, the knowledge

base is queried and updated, recording the

simulation log. The log is later used to analyze the

trainee’s performance.

5 FINAL CONSIDERATIONS

This paper presented a framework for the

construction of three-dimensional virtual reality

training environment, based on its ontological

description. This approach:

 provides a scenario description which can be

processed and interpreted by simulation

environments;

 promotes the rapid development of scenarios

by domain experts, without demanding the

knowledge of modelling in 3D, Petri nets, or

any specific programming language;

 promotes the reuse of components from a

library, which have been tested and validated.

 promotes the interdependence of simulation

models, 3D models and animations,

simplifying the maintenance of each

individual component and its replacement.

The current step in this research consists in

developing an integrated environment with a tool to

support the approach application and that integrates

a scenario editor for the simulator SimuLIHM.

REFERENCES

Brutzman, D., Daly, L., 2007. X3D: Extensible 3D

Graphics for Web Authors. Morgan Kaufmann

Publications. California, 2nd edition.

CPN Tools Project, 2014. Available at: http://cpntools.org/

International Electrotechnical Commission. IEC 61970-

301 Energy management system application program

interface – part 301. IEC 61970:2011.

JDOM, 2014. JDOM Project. Available at:

http://www.jdom.org/index.html.

Jena, 2009. Jena2 ontology API. Available at:

http://jena.sourceforge.net/ontology/index.html

Jensen, K., 1997. Coloured Petri Nets. Basic Concepts,

Analysis Methods and Practical Use. Springer. Berlin;

Heidelberg; New York; Barcelona; Budapest; Hong

Kong; London; Milan; Paris; Santa Clara; Singapore;

Tokyo; 2

edition.

Kalogerakis, E., Christodoulakis, S., Moumoutzis, N.,

2006. Coupling Ontologies with Graphics Content for

Knowledge Driven Visualization. Proceedings of the

IEEE Virtual Reality Conference (VR’06). 43-50.

Luukkainen, M., Karhela, T., 2008. Ontology approach for

co-use of 3D plant modelling and large scale process

simulation. The 48th Scandinavian Conference on

Simulation and Modeling (SIMS 2007). Linköping

University Electronic Press, 166–172.

Parisi, S., Bauch, J., Berssenbr, J., Radkowski, R., 2007.

Ontology-driven Generation of 3D Animations for

Training and Maintenance. International Conference

on Multimedia and Ubiquitous Engineering. 608-614.

Protégé, 2014. The Protégé Ontology Editor and

Knowledge Acquisition System. Available at:

http://protege.stanford.edu/.

Rocha, R.V., Araujo, R.B., Campos, M.R., Boukerche, A.,

2009. HLA Compliant Training Simulations Creation

Tool, Distributed Simulation and Real Time

Application. 13th IEEE/ACM International

Symposium. Doi: 10.1109/DS-RT.2009.38.

Turnell, M. F. Q. V., Neto, J. A. N., Santoni, C., Scaico,

A., Mercantini, J-M., 2010. A real-time interface

simulator for operator's training: a proposed

architecture. In Simulation vol. 86 No.1; pp 53-63;

ISSN: 1741-3133.

AnOntology-DrivenFrameworktoSupportScenarioRepresentationina3DOperatorTrainingSimulator

303