An Experimental Analysis of Tools for Ontology Evolution Management

essica S. Santos

, Viviane T. Silva, Leonardo G. Azevedo, Elton F. S. Soares

and Raphael M. Thiago

IBM Research, Pasteur Ave, 146, Rio de Janeiro, Brazil

Keywords:

Ontology Evolution, Ontology Engineering, Change Management, Change Detection, Complex Changes.

Abstract:

The use of ontology is widely spread among software engineering groups as a way to represent, structure,

share and reuse knowledge. As projects progress, the ontological understanding of the domain may change,

evolve. New domain concepts may emerge and existing ones may disappear or be updated, causing changes

in the ontology. Taking this into account, the management of the ontology evolution/life cycle should be

addressed by the ones that adopt them as a way of representing knowledge. This paper provides an analysis of

tools related to ontology evolution management focusing on the ones able to identify elementary and complex

changes (by grouping elementary changes) that help on externalizing the intention of the user. The main

contribution of this analysis is to present the state of the art of tools related to ontology evolution by identifying

their strengths and limitations. The results are particularly useful to ontology designers who need to choose a

tool to help them to inspect, understand and manage ontology evolution. Besides, we point out several research

issues to be addressed in different kinds of research initiatives.

1 INTRODUCTION

In Computer Science, ontologies are deﬁned as “ex-

plicit speciﬁcations of conceptualizations” (Gruber,

1993) or, similarly, as “formal speciﬁcations of a

shared conceptualization” (Borst and Borst, 1997).

Basically, they aim at providing a uniﬁed understand-

ing of the world, formally representing the descrip-

tion of the concepts that exist in a given domain of

interest, their relationships and relationships between

instances that belong to such concepts.

Ontology usage presents several advantages, as

for example: (i) interoperability – ontologies enable

different systems or persons to communicate and in-

teroperate by using a shared vocabulary of concepts

(Maedche and Staab, 2001) that can be machine-

processable (Fensel, 2001) (semantic exchange); (ii)

reuse – when adopting ontologies, one can import

and extend existing concepts, reusing knowledge

(Uschold and Gruninger, 1996); and (iii) reasoning

– the formally structured nature of ontologies allows

logical reasoning and inference mechanisms to be ex-

ploited (Wang et al., 2004). Those advantages make

ontologies a popular topic among the knowledge en-

gineering community, which can adopt them in order

to build/describe the structure of their knowledge base

https://orcid.org/0000-0001-5082-4583

(Studer et al., 1998) and to construct Semantic Web

based applications (Berners-Lee et al., 2001).

During the life cycle of an ontology, changes in its

structure can be needed. This can occur, for example,

when experts achieve a deeper understanding of the

domain, when there are changes in the domain mod-

elled by the ontology, when new requirements arise in

the applications that the ontologies support (Zablith

et al., 2015), or even when design errors are detected

in earlier ontology versions (Hartung et al., 2012).

Therefore, ontology-based applications are also sub-

jected to a continual change (Abcckcr and Stojanovic,

2005).

In addition to elementary changes

, there are oth-

ers called composite or complex changes that group

a set of elementary changes to provide more mean-

ingful semantics. A merge operation is an example

of complex change where a list of existing concepts

, .., x

} are merged into one concept x

that ac-

cumulates all attributes/relationships of the involved

concepts. This change can be decomposed in several

isolated changes without meaning: the creation of a

new concept x

, several move operations involving at-

tributes/relations (from classes {x

, .., x

} to x

), and

Elementary changes are the ones that cannot be decom-

posed in simpler changes (e.g.: additions or deletions of

concepts).

Santos, J., Silva, V., Azevedo, L., Soares, E. and Thiago, R.

An Experimental Analysis of Tools for Ontology Evolution Management.

DOI: 10.5220/0009395601110121

In Proceedings of the 22nd International Conference on Enterprise Information Systems (ICEIS 2020) - Volume 2, pages 111-121

ISBN: 978-989-758-423-7

 2020 by SCITEPRESS – Science and Technology Publications, Lda. All rights reser ved

111

the sequence of deletions of classes {x

, .., x

}. The

renaming is another example of complex change that

can be decomposed into a sequence of changes with-

out meaning: the deletion of a concept and the ad-

dition of a new one. However, these actions, when

seen in isolation, provide the wrong semantics to the

change (more examples of complex changes are pre-

sented in Table 2, marked by an asterisk (*)).

This research concerns with the ontology evolu-

tion management, the process that deals with support-

ing changes on the ontology, identifying and tracking

them, and managing the propagation of changes to

related artifacts (documentation, instances, code and

dependent ontologies) (Stojanovic, 2004). This pro-

cess is very challenging since when an ontology is

modiﬁed, (i) ontology instances need to be changed to

preserve consistency; (ii) dependent ontologies must

be revised and immediately synchronized with the

modiﬁed ontology; (iii) application code has to ac-

commodate the changes on the ontology; and (iv) re-

lated documentations must be updated in order to doc-

ument the evolution.

Inconsistencies between an ontology and its ar-

tifacts may cause severe problems, such as: (i) loss

of application instances, i.e., loss of knowledge gen-

erated by users when using the application; (ii) ap-

plications may behave unexpectedly; (iii) errors be-

ing propagated to other applications whose ontolo-

gies depend on the evolved one; and (iv) difﬁculty to

maintain the system since the documentation is not

up-to-date, i.e., does not document the evolution. In

this way, managing ontology evolution becomes more

complex as the ontology grows in size (Klein and

Fensel, 2001)(Stojanovic et al., 2002).

Some tools have been proposed in the literature to

support or facilitate the ontology evolution manage-

ment. In this research, we present an overview of ex-

isting tools and concentrate our experimental analysis

on the ones that support features related to the detec-

tion/inspection of changes in a given ontology over

time (diff tools

). One of the main features investi-

gated in this research is the set of kinds of changes

that can be captured and the level of explainability

returned to the user as a conceptual interpretation

of the meaning of a list of elementary changes (the

so called complex change detection). We created a

case study using the well known Pizza Ontology tu-

torial (Section 4), simulating several kinds of elemen-

tary and complex changes. These changes were se-

lected based on our experience in designing ontolo-

gies, the changes presented in the tutorial, and com-

plex changes presented by (Stojanovic, 2004), (Sto-

janovic et al., 2002) and (Klein, 2004).

Tools able to track or detect ontology changes.

The rest of this paper is organized as follows. Sec-

tion 2 presents related work. Section 3 presents an

overview of tools related to ontology evolution. Sec-

tion 4 details the evaluation criteria and the case study

adopted to evaluate the tools. Section 5 presents a

comparison between the tools and a discussion con-

sidering the supported features and the criteria estab-

lished in Section 4. Lastly, Section 6 points out our

conclusions and open issues identiﬁed.

2 RELATED WORK

The topic “ontology evolution” is the subject

of several lines of research in the literature.

Gonc¸alves (Gonc¸alves, 2014) argues that strategies

for computing differences between ontologies can be

classiﬁed as syntactic or semantic. Although his re-

search is not focused on tools developed to cope with

the ontology evolution management, some tools such

as ontology editors that track changes are mentioned.

Different from our work, Gonc¸alves (Gonc¸alves,

2014) does not explore the detailed list of speciﬁc

kinds of changes that can be detected. Our research

in this topic is much broader since we establish a set

of criteria that are relevant to the ontology evolution

problem and in addition to focus on evaluating diff

tools, we present an overview of tools related to other

parts of the ontology evolution process, as the ones

that propose means to propagate changes.

The work presented in (Lambrix et al., 2016) fo-

cuses on the ontology visualization problem and dis-

cusses strategies that can be adopted to facilitate the

visualization of ontology evolution. The authors ar-

gue that visualization tools must be able to represent

the richness of ontologies in a human comprehensive

way and present visualizations with different levels of

granularity. Additionally, it emphasizes that existing

techniques for the visualization of software evolution

could be applied to address ontology evolution visu-

alization. The strategy used to present differences be-

tween ontologies is only one of the features that we

use to categorize tools in the present study.

The work in (Stojanovic, 2004) presents a brief

discussion about ontology designing tools comparing

them according to their ability to evolve an ontology.

Different ontology editors were compared according

to dimensions that include: functionality – the kinds

of changes the ontology editor allows the user to ap-

ply on an ontology (e.g., add, delete, copy concept);

reﬁnement – the ability to make recommendations to

improve the ontology; reversibility – the ability to

undo changes; usability – the ability to edit ontolo-

gies easily. In contrast, we do not concentrate our

ICEIS 2020 - 22nd International Conference on Enterprise Information Systems

112

research in ontology editors neither in their ability to

implement changes, but in tools that detect changes

between ontology versions (what may be done inde-

pendently of a speciﬁc ontology editor).

3 ONTOLOGY EVOLUTION

MANAGEMENT TOOLS

This section presents the tools that provide features

for ontology evolution management. We present

the most popular tools (based on citation num-

bers), namely, PromptDiff (Noy et al., 2002) and

WebProt

e (Tudorache et al., 2013) and the ones that

were designed to cope with speciﬁc features that are

needed to understand ontology evolution (see Section

4 for more detail), namely, complex change detec-

tion (CODEX (Hartung et al., 2012) and KAON (Sto-

janovic et al., 2002)), change explanation (OWL Diff

(Redmond and Noy, 2011)) and visualization (OntoD-

iffGraph (Lara et al., 2017)). In addition to diff tools,

we also search for tools related to change propagation,

however only one tool was found (OIM tool (Davi-

dovsky et al., 2011)). A brief overview of these tools

and their features will be presented as follows.

3.1 PromptDiff

The PromptDiff (Noy et al., 2002) is a tool that adopts

a set of heuristic matchers for ﬁnding structural differ-

ences between two versions of an ontology. PromptD-

iff compares ontologies as follows (Noy et al., 2002):

• A matching between elements of the two ontolo-

gies is performed considering the elements’ iden-

tiﬁer (IRI) and type;

• When there is a matching between a class of the

ontology original version and a class of the on-

tology new version and both classes have a single

unmatched subclass, these subclasses are consid-

ered the same element (an entity renaming is iden-

tiﬁed);

• Unmatched siblings whose preﬁxes or sufﬁxes

changed in the same way in the new version of

the ontology are considered the same (an entity

renaming is identiﬁed);

• If there is a matching between a class of the origi-

nal version of the ontology and a class of the new

version of the ontology and both classes have a

single unmatched slot

whose range is the same,

they are considered the same element (an entity

renaming is identiﬁed);

Slots describe properties of classes or instances.

• If there is a matching between a slot of the on-

tology original version and a slot of the ontology

new version and these slots have inverse slots that

are unmatched, the unmatched inverse slots are

considered the same elements (a slot renaming is

identiﬁed);

• It can identify classes that were splitted or merged

based on analysis of their instances;

In addition to the limited support for complex detec-

tion, another drawback of this tool is that it still uses

an API for accessing OWL ontologies that is not able

to support OWL 2

ontologies. Additionally, Prompt-

Diff is only available as a plugin for Prot

old ver-

sions (3.x and earlier series) and its source code is not

available.

3.2 OWL Diff

The OWL Diff

(Redmond and Noy, 2011) is a tool

that is able to compare different versions of the same

ontology. It is available as a standalone applica-

tion, and as a plugin for the Prot

e (version 4.2 and

higher). OWL Diff was inspired by PromptDiff and

was developed to support OWL 2. The OWL Diff re-

ceives as input two OWL ﬁles and display the changes

identiﬁed in a text log. Before identifying ontology

structural changes, the following alignment strategies

are considered (Redmond and Noy, 2011):

• Align by the same IRI and type;

• Align by rendering, i.e. concepts that have the

same rendering (rdfs:label annotation property)

are matched even if their IRI has changed;

• Align entities that differ only by their namespace;

• Siblings of a matched class are aligned when they

have very similar rendering (it is interpreted as a

refactoring to correct a typo or misspelling). It can

also detect cases when all classes changed from

plural to singular;

• Entities with the same parent and child are

aligned;

• Alignment between alone unmatched classes

when all the remainder classes are matched.

The support provided by this tool to complex detec-

tion is very limited since it only covers entity rename

and subclass changes. Additionally, the heuristics

presented are very simple and their alignments can

point out wrong matches, as in the case of alignment

of alone unmatched classes.

https://www.w3.org/TR/owl2-overview/

An open-source ontology editor

https://protegewiki.stanford.edu/wiki/Protege4OWL- Diff

An Experimental Analysis of Tools for Ontology Evolution Management

113

3.3 OntoDiffGraph

The OntoDiffGraph

(Lara et al., 2017) is a tool that

was developed to exhibit differences between on-

tologies in a graphical manner, by using graphs.

The graph notation adopted is based on the VOWL

(Lohmann et al., 2016) – a notation that maps each

ontology element to a graphical element – but ex-

tends it in order to highlight differences. In sum-

mary, the graph notation is as follows. Classes are

represented as blue circles, object properties are rep-

resented as blue rectangles, data properties are repre-

sented as green rectangles, annotation properties are

represented as yellow rectangles, data types are repre-

sented as yellow rectangles with black borders and el-

ement characteristics (functional, inverse functional,

transitive, symmetric, asymmetric, reﬂexive and ir-

reﬂexive) are represented as white rectangles. The

differences identiﬁed are displayed in different col-

ors, according to the kind of change, by changing the

color of the borders/edges of the elements. Additions,

deletions and updates are displayed in blue, red and

green colors, respectively. An example is illustrated

in Figure 1. The strategy adopted to detect changes is

based on the analysis of the IRI, which is the unique

identiﬁer of each element of the ontology. In this way,

an addition is detected when there is a new IRI in the

new version. Similarly, removals are detected when

IRIs of the original version do not appear anymore

in the new version. Updates are identiﬁed when the

content of an existing element has changed in the new

version of the ontology. This tool is not able to de-

tect complex changes and the graphical approach to

present the detected changes is not enough when the

list of changes is too large.

Figure 1: OntoDiffGraph Visualization.

http://www4.di.uminho.pt/ gepl/OntoDiffGraph/

3.4 CODEX

The CODEX (Hartung et al., 2012) is a tool that was

designed to ﬁnd differences between life science on-

tologies. It supports as input ontologies described in

the OBO

format, a representation language com-

monly used to describe life science ontologies. A

limited support to OWL ontologies is also presented.

First of all, the algorithm tries to ﬁnd a matching be-

tween the elements of the different versions of the

ontology. After that, it tries to identify elementary

changes such as additions and deletions of concepts,

relationships or attributes. Finally, a rule-based strat-

egy is applied over the detected changes in order to

ﬁnd some kinds of complex changes, providing a

compact result. The main algorithm to compute dif-

ferences used by CODEX is called COntoDiff and is

detailed in (Hartung et al., 2010). The CODEX mech-

anism to detect complex changes is able to detect: the

addition or deletion of entire subgraphs, the addition

or deletion of leaf concepts, the move of concepts,

the split (tries to ﬁnd a map between one concept of

the old version and more than one concept of the new

version) or merge of concepts (tries to ﬁnd a map be-

tween two or more concepts of the old version and

one concept of the new version), or even speciﬁc op-

erations of life science ontologies (set the status of an

active concept to be obsolete or the status of an obso-

lete concept to be active). Such an approach does not

consider changes involving instances of the ontology

since, according to the authors, life science ontologies

typically do not have instances. The main drawback

of this approach is that it cannot detect some kinds

of simple changes (as the ones involving properties)

due to its limited support to OWL. Additionally, the

support to detect merge/split of concepts is also lim-

ited since it can detect such operations only in speciﬁc

cases.

3.5 WebProt

The WebProt

e (Tudorache et al., 2013) is an ap-

plication to support collaborative ontology manage-

ment. This tool is available online and provides func-

tionalities for editing, tracking changes and merging

ontologies. Given the current version of the ontol-

ogy loaded in WebProt

e V1 and a version V2 rep-

resenting a set of changes externally applied to V1,

one can merge V1 and V2 using WebProt

e in order

to apply those changes described in V2 to V1. Con-

cepts that are in V1 but are not in V2 are interpreted

as concepts removed. Since this tool records a change

http://www.cs.man.ac.uk/

horrocks/obo/

ICEIS 2020 - 22nd International Conference on Enterprise Information Systems

114

history, the author of each change made inside the ap-

plication is recorded. Before merging ontology ver-

sions, the changes detected between the versions are

informed to the user and a conﬁrmation dialog is ex-

hibited. Additionally, it presents a functionality to re-

vert/undo changes. The change history is presented in

a text log and there is an option to ﬁlter changes ac-

cording a given entity, facilitating change inspection.

As a drawback, complex change detection is not well

supported, covering few cases such as the detection

of subclass changes. In addition, ontology elements

cannot be renamed by using this tool, and renaming

actions performed outside this editor are not detected

too.

3.6 KAON

KAON (Stojanovic et al., 2002) is a framework that

was developed to partially support the ontology evo-

lution management. KAON does not support RDF

but it adopts a proprietary ontology language that ex-

tends RDFS changing the semantics of domain, range

and cardinality presented in the RDFS and OWL. This

is because they are interpreted as constraints in the

KAON language, and not as inference rules/axioms

like in the OWL, RDFS. KAON provides some func-

tionalities that allows the user to control and cus-

tomize the process of ontology evolution, by setting

some evolution strategies. When the ontology is mod-

iﬁed, changes that will be propagated according to the

predeﬁned evolution strategies are also exhibited to

the user. Examples of evolution strategies are as fol-

lows (Gabel et al., 2004):

• When a concept is deleted, the user can choose

to delete orphaned concepts, reconnect them to

the ontology root, or reconnect them to supercon-

cepts;

• When a property is deleted, the user can choose

to delete orphaned properties, reconnect them to

superproperties, or leave them as they are;

• When concept’s parent is removed, the user can

choose to do not propagate properties or to add all

inherit properties to the child concept.

Although the goal of this tool is to help the user

while she is changing the ontology by indicating some

changes propagation in the same ontology, it also

records the changes made and, therefore, was also

analyzed. Therefore, KAON provides a support to

the implementation of changes in a given ontology.

Among the limitations identiﬁed, we can cite that

KAON uses a speciﬁc ontology language and does

not present strategies for change detection or change

propagation to other artifacts, relying on the fact that

ontologies are only designed inside KAON.

3.7 OIM

The work in (Davidovsky et al., 2011) proposes a

method for semi-automated updating ontology in-

stances (ABox) accordingly to the TBox new version,

process that the authors call ontology instance migra-

tion. The transformation rules needed to transfer in-

stances of a given version to another one, are as fol-

lows: (i) concept addition – nothing happens to in-

stances; (ii) concept rename – instances of the old

name become instances of the new name; (iii) ob-

ject property deletion – instances of the given rela-

tionship are discarded; (iv) object property addition

– nothing happens to instances and is not possible to

migrate instances automatically because the values of

the respective object properties must be added to the

instances but we do know the exact set of individu-

als that have to be related; (v) object property rename

– instances of the relationship old name become the

instances of the new name; (vi) object property car-

dinality change – some instances of the old name be-

come the instances of the new name but is not pos-

sible to migrate instances automatically because we

do not know the instances that should be related to

the individual having the property whose cardinality

changed; (vii) object property range change – noth-

ing happens to instances. Similarly to cardinality

change, instances cannot be automatically migrated;

(viii) datatype property deletion – all instances of the

relationship are discarded; (ix) datatype property ad-

dition – nothing happens to instances; (x) datatype

property rename – all instances of the datatype prop-

erty old name become instances of the datatype prop-

erty new name; (xi) datatype property type change –

all instances of the relationship old name are trans-

formed to instances of the relationship new name hav-

ing a different data type. The transformation rules of

such an approach only exploits a small set of changes.

Additionally, heuristics for change detection were not

presented by the authors. It is important to note that

OIM is the only tool analyzed that propagate changes

to ontology instances. To the best of our knowledge

no other tool is able to do such task neither is able to

propagate ontology changes to code.

4 EVALUATION

This section presents the criteria used to evaluate

the tools that provide features for ontology evolu-

tion management. Additionally, we describe the case

An Experimental Analysis of Tools for Ontology Evolution Management

115

study used in the evaluation process.

4.1 Evaluation Criteria

The evaluation criteria were selected based on their

importance to provide support to several parts of the

process of ontology evolution, whose steps include

the detection of ontology structural changes, the inter-

pretation of their meaning, the analysis of their side-

effects, and the propagation of changes to related ar-

tifacts. Since only one tool related to propagation of

changes to ontology changes was found in the litera-

ture, we chose not evaluate this topic. The criteria are

detailed as follows:

1. Change Detection: The elementary changes the

tool is able to detect (listed in Table 1) and the

way it is done. We identiﬁed two different ways in

the analyzed tools: (i) store the ontology change

history when the user uses the tool, providing the

traceability between ontology versions; and, (ii)

provide a mechanism to identify differences be-

tween two ontologies (e.g., original ontology and

its new version).

2. Complex Change Detection: Ability to identify

complex changes, i.e., to consider a composition

of elementary changes as a unique change, e.g.,

identify a concept renaming from a deletion and

addition of concepts.

3. Change Inspection: How the detected changes

are presented to the user. There are two main pos-

sibilities: (i) a list containing the changes is re-

turned to the user as a log in text format; and, (ii)

the list of changes is exhibited to the user graphi-

cally, e.g., as a diagram.

4. Extensibility: Whether the tool can be extended,

i.e., if it is provided as a plugin or even it its source

code is available to be used.

The predeﬁned criteria (1 – 4) are very relevant for the

ones that adopt ontologies as a way of representing

knowledge for the reasons explained as follows.

The existence of a list reﬂecting the modiﬁcations

that an ontology underwent during its evolution – Cri-

teria 1 and 2 – allows those that are not the authors

of the changes to follow and understand how the on-

tology has evolved over time (Hartung et al., 2010),

(Noy and Klein, 2003). Furthermore, the detailed

information about ontology evolution is essential to

investigate the real impact of the changes on related

artifacts, and the manual detection of differences be-

tween ontology versions is a time-consuming task that

can be very challenging for large ontologies (Kremen

et al., 2011). The mapping between ontology versions

is also very important for the ontology’s migration.

It consists in ﬁnding changes to transform an ontol-

ogy V1 in an ontology V2 (Hartung et al., 2010). In

this context, we categorize strategies for highlighting

differences between versions as change history or de-

tecting differences strategies. Note that, the need for

speciﬁc tools for storing ontology change history oc-

curs due to the fact that functionalities provided by

traditional versioning systems (used to track source

code changes like Git

) are not enough to track ontol-

ogy changes. This is because they are based on text

comparison (checking if two ﬁles have the same char-

acters in the same order) and equivalent ontologies –

ontologies that have the same axioms – can have dif-

ferent text representations, requiring a structural com-

parison (Noy et al., 2002).

The importance of detecting complex changes

– Criterion 2 – instead of simply list elementary

changes, occurs due to the fact that complex changes

have a conceptual meaning and detecting them can

help one to understand the user intention behind the

performance of a list of elementary changes. Those

changes represent composite/coarse-grained actions

which can involve the combination of elementary

changes such as a change in the concept hierar-

chy (Stojanovic, 2004). The ontology designer does

not need to know, for example, the list of elemen-

tary changes indicating that a given concept A was de-

tached from concept B and attached to concept C. He

only needs to know that the hierarchy of concepts has

changed. The explainability of a list of elementary

changes (after an automatic interpretation) would be

helpful to the ontology designers, leading to a proper

propagation of changes and adjustments, making the

evolution process faster, and facilitating the under-

standing of the ontology evolution. In some cases, to

analyze each change of a list is not enough to capture

the change semantics, as is the case of the rename ac-

tion, explained as follows. A rename action changing

the identiﬁer of an element from A to B erroneously

detected as the action of removing A and adding B can

cause undesirable side-effects that would reﬂect in an

erroneous evolution. This is because, when a class is

removed, its instances also would be removed, what

would lead to an instance loss for the renamed class B

in the ontology newest version.

Criterion 3 was deﬁned since a proper presenta-

tion of the set of detected changes can facilitate hu-

man comprehension (Lambrix et al., 2016).

Finally, Criterion 4 was deﬁned since a partial fea-

ture support of a tool (or new feature needs) could

be solved by extending the tool instead of building a

new tool from scratch. We evaluated this criterion by

checking tools documentation.

https://git-scm.com/

ICEIS 2020 - 22nd International Conference on Enterprise Information Systems

116

4.2 Case Study

In order to evaluate the Criteria 1, 2, and 3 we simu-

late the ontology evolution process by applying a list

of changes to a given ontology. We take as base a well

known use case published in the literature, the Pizza

Ontology

described in Prot

e’s tutorial

, which

describes in step-by-step exercises what the reader

must do to create the Pizza Ontology. In summary,

the Pizza Ontology describes the concepts of pizza,

the country of origin of the pizza, toppings that can

be added to the pizza, and so on. The ﬁrst version

of the ontology V1 is an OWL ﬁle with no concepts.

The next versions of the ontology are built by apply-

ing the changes described in the exercises (1 – 60)

presented in the Pizza tutorial. Therefore, the ontol-

ogy version 20, for instance, is the one containing all

changes made from exercise 1 to 20 and so on. Table

1 summarizes the complete list of changes considered

in this research.

Some of the tutorial’s exercises are not mentioned

in Table 1 because they do not require the ontology

to be changed (e.g., “Exercise 30: Use the reasoner

to automatically compute the subclasses of CheesyP-

izza”). The column Prot

e Tutorial Exercise maps

each kind of change to the exercise that implements it.

Prot

e’s tutorial does not cover all possible changes.

However, some changes can be derived from pre-

existing ones by using a simple method: inverting

the version order (e.g., if the second version of the

ontology is created by adding a class to its ﬁrst ver-

sion, the delete operation can be simulated taking the

second ontology version as the original one and the

ﬁrst ontology version as the evolved ontology). Those

cases are indicated with (-) in the column Prot

e Tu-

torial exercise. Some kinds of changes were manu-

ally elaborated and included in our analysis and are

represented with (x) in the column Prot

e Tutorial

exercise. The terms object properties and data prop-

erties refer to relationships between two individuals

and relationships between individuals and data val-

ues, respectively. The term restriction refers to the

class of individuals that contains a given property re-

striction (e.g., class of individuals that only have veg-

etable toppings). The term abstraction refers to the

inclusion or deletion of a class between parent/child

classes. The property characteristics can describe ob-

ject properties as being functional, inverse, transitive,

reﬂexive, etc. Similarly, a data property can be char-

acterized as functional. More information about prop-

erty characteristics can be found in the Pizza tutorial.

https://protege.stanford.edu/ontologies/pizza/pizza.owl

http://mowl-power.cs.man.ac.uk/protegeowltutorial/res-

ources/ProtegeOWLTutorial P4 v1 3.pdf

Complex changes are signalled by an asterisk (*). The

list of complex changes was built based on the combi-

nation of complex changes discussed in different ap-

proaches (Stojanovic, 2004), (Stojanovic et al., 2002)

and (Klein, 2004).

5 COMPARATIVE ANALYSIS

In this section we present the result of the exper-

imental analysis of different tools. In such analy-

sis we evaluated the level of functionalities that each

tool supports based on the criteria presented in Sec-

tion 4. Among the tools surveyed, WebProt

e and

KAON adopt the approach of ontology change his-

tory. Therefore, they do not need to discover what

changes occurred in the ontology during its evolu-

tion because it is explicitly stored. Change history

approaches have also the differential of keeping the

exact order in which the sequence of changes were

performed on the ontology, what it is not possible to

know in approaches that detect differences without

saving the trace. An advantage of using WebProt

is that it also stores the author of each change per-

formed inside it, i.e., changes that were not incorpo-

rated by merging an external ﬁle. This factor is ex-

tremely important when the ontologies are managed

collaboratively. The ability to revert changes also

is another point in favour for both WebProt

e and

KAON. A disadvantage of the KAON is that the man-

agement of the ontology is coupled with it, i.e., the

user will be able to inspect the detailed sequence of

changes when the ontology is only edited using this

tool. In contrast to change history approaches, tools

that do not track changes but can detect differences

between ontologies have the advantage that the on-

tologies to be compared can be edited using differ-

ent ontology editors. This feature is specially desir-

able when the ontology designer reuses ontologies de-

veloped by independent organizations (Zablith et al.,

2015).

Among the tools surveyed, the OntoDiffGraph

was the only one designed to address the ontology

evolution visualization problem, exhibiting the on-

tology as a graph, where different kinds of changes

are displayed using different colors. Although this

functionality can facilitate the visualization for the

ontology engineer when the number of changes is

not so numerous, for our case study it was difﬁ-

cult to visualize the ontology evolution since multiple

changes occurred between versions. In order to deal

with this problems, future tools for ontology evolution

management could present means to collapse/expand

parts of the ontology when adopting a graphical visu-

An Experimental Analysis of Tools for Ontology Evolution Management

117

Table 1: Ontology changes.

Kind of Change Prot

e Tutorial Exercise

1. Class

1.1. Include Class 7, 21, 22, 28, 31, 41, 43, 44

1.2. Delete Class -

1.3. Change IRI

1.4. Annotation

1.4.1. Include Label x

1.4.2. Delete Label -

1.4.3. Change Label from A to B x

1.4.4. Include Comment 18

1.4.5. Delete Comment -

1.4.6. Change Comment x

1.5. Description

1.5.1. Equivalent To

1.5.1.1. Include 60

1.5.1.2. Delete -

1.5.1.3. Change x

1.5.2. SubClass Of

1.5.2.1. Include 6, 7, 18, 21, 22, 28, 31, 41, 43, 44

1.5.2.2. Delete -

1.5.2.3. Change

1.5.3. Instances

1.5.3.1. Include 47, 58

1.5.3.2. Delete -

1.5.3.3. Change x

1.5.4. Disjoint With

1.5.4.1. Include 5, 6, 23, 52

1.5.4.2. Delete -

1.5.4.3. Change

26, 27

1.6. Convert to Deﬁned

29, 32, 41, 43, 54

1.7. Convert to Primitive

2. Object

Property

2.1. Include property 8, 10

2.2. Delete property -

2.3. Change IRI

2.4. Annotation

2.4.1. Include Label x

2.4.2. Delete Label -

2.4.3. Change Label from A to B x

2.5. Domain

2.5.1. Include Domain 14, 15

2.5.2. Delete Domain -

2.5.3. Change Domain from A to B

2.6. Range

2.6.1. Include Range 13, 15

2.6.2. Delete Range -

2.6.3. Change Range from A to B

2.7. Characteristics

2.7.1. Include 11, 12

2.7.2. Delete -

2.7.3. Change x

2.8. Description

2.8.1. Inverse Property 10

2.8.2. Disjoint Property x

2.8.3. SubProperty Of

3. Data

Property

3.1. Include property 46

3.2. Delete property -

3.3. Change IRI

3.4. Annotation

3.4.1. Include Label x

3.4.2. Delete Label -

3.4.3. Change Label from A to B x

3.4.4. Include comment x

3.4.5. Delete comment -

3.4.6. Change comment from A to B x

3.5. Domain

3.5.1. Include Domain 47, 48

3.5.2. Delete Domain -

3.5.3. Change Domain from A to B

3.6. Range

3.6.1. Include Range 47, 48

3.6.2. Delete Range -

3.6.3. Change Range from A to B

3.7. Characteristics

3.7.1. Include 51

3.7.2. Delete -

3.7.3. Change x

4. Restric-

tions

4.1. Quantiﬁer

4.1.1. existential restrictions (some) 16, 17, 19, 20, 21, 22, 28, 36, 37, 40, 41, 48, 49

4.1.2. universal restrictions (only) 31, 34, 35

4.2. Cardinality

43, 45

4.3. hasValue 59

5. Others

(changes

involving

more than

one class

or property)

5.1. SubTree

5.1.1. Include SubTree 7, 18

5.1.2. Delete SubTree -

5.1.3. Move SubTree x

5.2. Abstraction

5.2.1. Include x

5.2.2. Delete -

5.3. Siblings

5.3.1. Include 4, 6, 22

5.3.2. Delete -

5.3.3. Move x

5.3.4. Make all siblings disjoint 6, 7

5.4. Merge Classes x

5.5. Split Class x

Complex Change.

Kinds of changes that are not in the Prot

e tutorial and were manually included.

Kinds of changes that are not in the Prot

e tutorial but can be derived from the existing ones.

alization.

In general, the support to complex detection is

very limited. Only two approaches, namely, OWL

Diff and PromptDiff address the entity renaming

problem by presenting some simple heuristics that try

to ﬁnd renaming operations based on, for example,

the analysis of the labels associated with the entities

or based on the analysis on the parent and child of

ICEIS 2020 - 22nd International Conference on Enterprise Information Systems

118

the elements to be compared. Although WebProt

stores the change history, it cannot detect IRIs renam-

ing when ontologies are merged (loaded from an ex-

ternal ﬁle, compared and combined). We noticed that

although subclass changes can be already well de-

tected by some approaches such as CODEX, OWL

Diff and PromptDiff, other complex changes such

as splits and merges of concepts are not satisfactory

identiﬁed since the approaches that deal with them

rely on instance analysis and in many times instances

of the ontology new version are not available. Fur-

thermore, complex changes such as the conversion of

a class to primitive or deﬁned, changes involving sib-

lings classes or changes creating/deleting abstractions

are not addressed by any of the tools. In addition,

domain/range changes are not properly interpreted by

the tools (e.g.: they do not inform to the user that the

range of a property has been enlarged when it is asso-

ciated with a broader class). Although CODEX was

designed to identify some cases of complex changes,

it was not able to detect many cases of simple changes

because it does not support all OWL constructs.

In the ontology evolution problem we want to

identify differences between an old and a new ver-

sion and, after that, automatically propagate changes

to ontology instances based on the changes that were

detected. A problem identiﬁed in relation to this is-

sue is the lack of approaches supporting the automatic

propagation of ontology structural changes to ontol-

ogy instances (when it is possible), only the OIM tool

proposes some rules to propagate changes to ontol-

ogy instances. However, the rules considered by such

an approach do not contemplate the complete list of

changes presented in this paper. We also do not agree

with some instance migration rules presented by the

authors, such as the case in which the type of a data

property changes. We argue that this case cannot be

automatically solved since some conversions between

types are not possible to be performed. KAON was

the only tool that proposes a set of strategies that de-

termine patterns for updating ontology elements when

a change occurs.

Table 2 presents the changes that cannot be de-

tected by the tools or that can be partially detected.

All changes considered by OIM tool are described in

section 3.7 by the transformation rules and are related

to the following changes listed in Table 1: 1.1, 1.3,

2.1, 2.2, 2.3, 2.6.3, 3.1, 3.2, 3.3, and 3.6.3. Table 3

summarizes the characteristics of the surveyed tools.

KAON and OIM tools are not mentioned in Table 2

because they do not propose own mechanisms to de-

tect differences between ontology versions. The OIM

tool is not mentioned in Table 3 because it does not

support any of the criteria listed in the table. On the

Table 2: Change Detection Problems.

Change

No Support

Limited Support (*)

Class

1.3. Change IRI

OWL Diff*

OntoDiffGraph

PromptDiff*

WebProt

CODEX

1.5.1. Equivalent To CODEX

1.5.3. Instances CODEX

1.5.2.3. Change SubClassOf OntoDiffGraph

1.5.4. Disjoint With CODEX

1.5.4.3. Change Disjoint With

OntoDiffGraph,

WebProt

1.6. Convert to Deﬁned

OWL Diff

OntoDiffGraph

PromptDiff

WebProt

CODEX

1.7. Convert to Primitive

OWL Diff

OntoDiffGraph

PromptDiff

WebProt

CODEX

Object Property 2.3. Change IRI

OWL Diff*

OntoDiffGraph

PromptDiff

WebProt

Data Property 3.3. Change IRI

OntoDiffGraph,

PromptDiff,

WebProt

Restrictions

4.2. Cardinality CODEX

4.3. hasValue CODEX

Others

(changes

involving

more than

one class

or property)

5.1. SubTree

OWL Diff

OntoDiffGraph

PromptDiff*

WebProt

5.2. Abstraction

OWL Diff

OntoDiffGraph

PromptDiff

WebProt

5.3. Siblings

OWL Diff

OntoDiffGraph

WebProt

5.4. Merge Classes

OWL Diff

OntoDiffGraph

PromptDiff*

WebProt

5.5. Split Class

OWL Diff

OntoDiffGraph

PromptDiff*

WebProt

CODEX

Limited support to detect this change.

Table 3: Tools characteristics.

Change

Detection

Complex

Changes

Change

Inspection

Extensi-

bility

Tracking

changes

Detecting

differences

Text

log

Graphic.

visualiz.

OWL Diff X X

X X

OntoDiff-

Graph

X X X

PromptDiff X X

WebProt

e X X X X

KAON X X

CODEX X X

Functionality partially supported.

other hand, it is the only tool that has proposed au-

tomatic rules for automatic ontology instance migra-

tion, which is a factor that can be useful to the ontol-

ogy evolution management.

An Experimental Analysis of Tools for Ontology Evolution Management

119

6 CONCLUSIONS

In this paper we compared and evaluated different

tools that present features related to the process of

management of ontology evolution. We established

a set of criteria to analyze the tools and employed a

case study based on the Pizza Ontology to simulate

the ontology evolution process. We applied different

relevant kinds of changes in Pizza Ontology and ana-

lyzed how the tools deal with them.

By exploiting the features of existing diff tools we

have concluded that none of the surveyed tools com-

pletely supports ontology evolution needs. A set of

limitations have been identiﬁed, as follows:

1. Change Detection: The majority of tools are able

to identify and return to the user/ontology de-

signer a list containing the sequences of several

elementary changes that occurred during the on-

tology life cycle. However, they deal with a dif-

ferent set of changes, and some of them (e.g.:

CODEX) are not able to detect the complete list

of elementary changes evaluated in this research

(Table 2). In addition, most of tools analyzed

(namely, OWLDiff, OntoDiffGraph, PromptDiff,

WebProt

e and Codex) provide a mechanism to

identify differences between two ontologies in-

stead of tracking user changes.

2. Complex Change Detection: Although many

diff tools have been found in the literature, they do

not can deal with complex change detection prop-

erly since the identiﬁcation of complex changes

is still highly limited and partially addressed, as

discussed in Section 5.

3. Change Inspection: Even tools designed to cope

with visualization issues cannot present extensive

list of changes in a comprehensive way.

4. Extensibility: Some tools tackle speciﬁc prob-

lems, but are not extensible (KAON, OntoDiff-

Graph, CODEX, PromptDiff) since they are not

provided as plugins and their source code is not

freely available.

In addition, we also notice that tools that propagate

changes to impacted queries, dependent ontologies

or source code have not been found. Only one tool

(namely, OIM tool) proposes limited heuristics re-

lated to changes in instances.

The open problems found in the literature high-

light the need for future work in the following direc-

tions:

• Improvement of existing techniques for complex

change detection by performing, for instance, a

deeper structural analysis of relationships and at-

tributes of the ontology elements. This entails the

improvement of change presentation since com-

plex change detection includes grouping and in-

terpret related changes that can be exhibited to the

user in a compact way;

• Creation of a broader set of transformation

rules to migrate instances considering the list of

changes presented in this paper;

• Development of tools to propagate changes to

other related artifacts (and not only ontology in-

stances), such as documentation, related code and

queries;

• Development of extensible tools that address both

the difference detection task and the propagation

of changes to related artifacts;

• The integration of existing approaches can also be

investigated aiming at building a tool that covers

different tasks of ontology evolution management

or ontology life cycle.

Our lines for further research include: (i) to extend

this work by conducting experiments with ontologies

used in real world scenarios and conducting a study

about other evaluation criteria that would be relevant

for ontology evolution; (ii) to create a uniﬁed list (tax-

onomy) of complex changes found in the literature,

and relating them by hierarchical relationships when

it is possible. In addition, in that sense we want to

analyze the kinds of complex changes that can be for-

malized and the ones that depend on human interpre-

tation; and (iii) development of a tool focused on de-

tecting complex changes, returning to the user high

level messages about the ontology evolution instead

of a long list of simple/elementary changes.

ACKNOWLEDGEMENT

This project was executed under the Brazilian Na-

tional Petroleum Agency (ANP) R&D incentive reg-

ulatory framework.

REFERENCES

Abcckcr, A. and Stojanovic, L. (2005). Ontology evolution:

Medline case study. In Wirtschaftsinformatik 2005,

pages 1291–1308. Springer.

Berners-Lee, T., Hendler, J., and Lassila, O. (2001). The

semantic web. Scientiﬁc american, 284(5):34–43.

Borst, W. N. and Borst, W. (1997). Construction of engi-

neering ontologies for knowledge sharing and reuse.

Davidovsky, M., Ermolayev, V., and Tolok, V. (2011). In-

stance migration between ontologies having structural

differences. International Journal on Artiﬁcial Intel-

ligence Tools, 20(06):1127–1156.

ICEIS 2020 - 22nd International Conference on Enterprise Information Systems

120

Fensel, D. (2001). Ontologies: Dynamic networks of for-

mally represented meaning. Vrije University: Amster-

dam.

Gabel, T., Sure, Y., and Voelker, J. (2004). D3. 1.1. a:

Kaon–ontology management infrastructure. SEKT in-

formal deliverable.

Gonc¸alves, J. R. (2014). Impact analysis in descrip-

tion logic ontologies. PhD thesis, The University of

Manchester (United Kingdom).

Gruber, T. R. (1993). A translation approach to portable

ontology speciﬁcations. Knowledge acquisition,

5(2):199–220.

Hartung, M., Groß, A., and Rahm, E. (2010). Rule-based

generation of diff evolution mappings between ontol-

ogy versions. arXiv preprint arXiv:1010.0122.

Hartung, M., Gross, A., and Rahm, E. (2012). Codex: ex-

ploration of semantic changes between ontology ver-

sions. Bioinformatics, 28(6):895–896.

Klein, M. C. and Fensel, D. (2001). Ontology versioning

on the semantic web. In SWWS, pages 75–91.

Klein, M. C. A. (2004). Change management for distributed

ontologies. PhD thesis, Vrije Universiteit Amsterdam.

Kremen, P., Smid, M., and Kouba, Z. (2011). Owldiff:

A practical tool for comparison and merge of owl

ontologies. In 2011 22nd International Workshop

on Database and Expert Systems Applications, pages

229–233. IEEE.

Lambrix, P., Dragisic, Z., Ivanova, V., and Anslow, C.

(2016). Visualization for ontology evolution. In

2nd International Workshop on Visualization and In-

teraction for Ontologies and Linked Data, Kobe,

Japan, October 17, 2016, pages 54–67. Rheinisch-

Westfaelische Technische Hochschule Aachen Uni-

versity.

Lara, A., Henriques, P. R., and Ganc¸arski, A. L. (2017).

Visualization of ontology evolution using ontodiff-

graph. In 6th Symposium on Languages, Applications

and Technologies (SLATE 2017). Schloss Dagstuhl-

Leibniz-Zentrum fuer Informatik.

Lohmann, S., Negru, S., Haag, F., and Ertl, T. (2016).

Visualizing ontologies with vowl. Semantic Web,

7(4):399–419.

Maedche, A. and Staab, S. (2001). Ontology learning for

the semantic web. IEEE Intelligent systems, 16(2):72–

79.

Noy, N. F. and Klein, M. (2003). Tracking complex changes

during ontology evolution. ISWC-2003 Poster Pro-

ceedings, Sanibel Island, Florida.

Noy, N. F., Musen, M. A., et al. (2002). PromptDiff:

A ﬁxed-point algorithm for comparing ontology ver-

sions. AAAI/IAAI, 2002:744–750.

Redmond, T. and Noy, N. (2011). Computing the changes

between ontologies. In Joint Workshop on Knowledge

Evolution and Ontology Dynamics, pages 1–14.

Stojanovic, L. (2004). Methods and tools for ontology evo-

lution.

Stojanovic, L., Maedche, A., Motik, B., and Stojanovic,

N. (2002). User-driven ontology evolution manage-

ment. In International Conference on Knowledge En-

gineering and Knowledge Management, pages 285–

300. Springer.

Studer, R., Benjamins, V. R., Fensel, D., et al. (1998).

Knowledge engineering: principles and methods.

Data and knowledge engineering, 25(1):161–198.

Tudorache, T., Nyulas, C., Noy, N. F., and Musen, M. A.

(2013). Webprot

e: A collaborative ontology editor

and knowledge acquisition tool for the web. Semantic

web, 4(1):89–99.

Uschold, M. and Gruninger, M. (1996). Ontologies: Princi-

ples, methods and applications. The knowledge engi-

neering review, 11(2):93–136.

Wang, X., Zhang, D., Gu, T., Pung, H. K., et al. (2004).

Ontology based context modeling and reasoning us-

ing owl. In Percom workshops, volume 18, page 22.

Citeseer.

Zablith, F., Antoniou, G., d’Aquin, M., Flouris, G., Kondy-

lakis, H., Motta, E., Plexousakis, D., and Sabou, M.

(2015). Ontology evolution: a process-centric survey.

The knowledge engineering review, 30(1):45–75.

An Experimental Analysis of Tools for Ontology Evolution Management

121