experiment includes three steps. In Step 1, we utilize
the Terminological approach to generate a mapping
baseline. Step 2 introduces SUMO as the reference
ontology, showing the mapping result of this generic
reference ontology. In Step 3, STAR is used to
facilitate the attempt to capture correspondence in
the automotive retail industry.
4.2 Measurement
Basically, a threshold score is a lower limit for the
similarity score of two concepts that belong to the
respective source and target ontologies and that will
be treated as mapping pairs. For instance, if pairs of
mapping results produced have a threshold score of
0.60, indicating that the two concepts are considered
as mapping pairs if the similarity score between
them is greater than or equal to 0.60.
The effectiveness of the mapping approaches can
be measured by the precision of the mapping results.
We define the mapping accuracy as the ratio of
correct mappings N to the number of discovered
mappings M. The formula in Equation 1 shows the
mapping precision as a percentage value.
Equation 1: Mapping Precision.
4.3 Generic Experiment Analysis
The generic experiments consist of Experiments 1,
2, 3, and 4. Accordingly, Figure 3(a) demonstrates
the mapping precision arranged by the threshold
value of similarity, where the left side of the graph
displays the lowest value; the right side shows the
highest value; and the vertical line illustrates the
precision. The Terminological approach in
Experiment 1 is presented as the baseline experiment
in order to evaluate the other mapping results.
Specifically, we are attempting to directly discover
mapping pairs between the local ontologies.
The results of the four generic experiments are
presented in Figure 3(b). In comparison to the
Terminological approach used in Experiment 1, the
WordNet Distance approach in the second
experiment generates more mapping pairs.
For example, when the threshold is 0.8, the
WordNet Distance method (Experiment 2) returns
68 mapping pairs and among them, 16 are correct as
the precision is only 24%, whereas in the
Terminological approach (Experiment 1), the result
is 29 mapping pairs with 16 correct mappings, and
the mapping precision increases as 55%. Overall, the
results indicate that the mapping precision of the
WordNet Distance approach is not necessarily more
effective than the Terminological approach.
Furthermore, the federated mapping approach of
Terminological and WordNet in Experiment 3
demonstrates that the mapping results are more
effective than they are in the two approaches used in
Experiments 1 and 2 respectively, and moreover, a
lot of incorrect mapping results are eliminated in
Experiment 3. For example, when the threshold level
is 0.6, the mapping precision of the federated
approach is 67%; in comparison to the 16% of the
Terminological approach.
We also obtain a high mapping precision when
using the reference ontology based approach in
Experiment 4. However, we also observe while the
similarity threshold is increased, the mapping pairs
are reduced. For example, the number of mapping
pairs is 14 at threshold 0.2, while this number is
reduced to 1 at a threshold of 0.7. This drastic
decrease occurs because the terms used in
experimental ontologies do not have corresponding
definitions in SUMO, and therefore, most terms in
local ontologies cannot be bridged by the reference
ontology. Therefore, in order to overcome this
disadvantage, the appropriate reference ontology
needs to be selected prior to mapping in a specific
domain. Moreover, in order to obtain a more
effective mapping result, the pre-selected reference
ontology should include as many terms as possible.
4.4 Specified Experiment Analysis
In comparison to the traditional Terminological
approach, the domain specific reference ontology
results in improved mapping precision at the same
threshold level. For example, Figure 4(a) shows that
at threshold level 0.2, the mapping precision of
using STAR is 90%. Compared to the 12% achieved
in the Terminological approach, the precision with
STAR is increased by 78%.
The experiment also proves that domain-specific
information can help to improve the mapping results
for the reference ontology based approach. For
example, when using either SUMO or STAR, the
mapping precision curves are very similar; however,
when the threshold level is increased from 0.2 to 0.8,
STAR is more effective than SUMO. In particular,
as shown in Figure 4(b), using SUMO reduces the
mapping pairs from 41 to 3, whereas using STAR
only decreases the pairs from 59 to 34. This
discrepancy occurs since the increased inclusion of
terms in the STAR, has a significant effect on the
mappings. In particular, the resulting similarity
A REFERENCE ONTOLOGY BASED APPROACH FOR SERVICE ORIENTED ONTOLOGY MANAGEMENT
73