Lightweight Ontologies in Context

Relationship between Ontology Characteristics and Context Parameters

Ilya M. Waldstein

, Rosina O. Weber

and Neal Handly

School of Information Science and Technology, Drexel University, 3141 Chestnut St., Philadelphia, PA, 19104, U.S.A.

Drexel University College of Medicine, 245 North 15th St., Philadelphia, PA, 19102, U.S.A.

Keywords: Ontology Engineering, Sharing, Reuse, Ontology Characteristics, Context, Context Parameters.

Abstract: Ontologies, mainly lightweight ontologies, are ubiquitous throughout the Internet and are succeeding in

replacing human expertise. We conducted a study with physicians and nurses performing a search task in

the medical domain that demonstrates that lightweight ontologies perform well as a substitute for expertise.

The extent of success of the substitution depends upon context of use. Our study investigates lightweight

ontologies with respect to the context of use in which they are applied. The better we understand the context

of use, the better we can inform ontology design and evaluation. We describe ontologies through

characteristics and context through parameters. By varying ontology characteristics and testing the effect on

the performance of an ontology-supported task for a context parameter, such as the level of user expertise,

we increase our understanding of ontology design and evaluation. Our study shows that changing ontologies

by varying some of its characteristics has a direct and significant impact on the performance of the

ontology-supported task for different levels of user expertise.

1 INTRODUCTION

Ontologies are formalisms for knowledge

representation commonly used in support of

application tasks such as natural language

processing and search. Lightweight ontologies are

simpler forms that trade expressiveness for usability

and ease of implementation in real-world

applications (Brewster and O’Hara, 2007). The

extensive use of hierarchical information on the

Internet brought lightweight ontologies into use.

The context of use for ontologies is expressed by

context parameters such as the user, the application

task the ontology supports, and the domain. Context

plays a crucial role in designing and evaluating

ontologies. Most current ontology engineering and

evaluation approaches are focused on constructing

an ontology that correctly represents only the

domain. However, when the ontology is used to

support a user performing a task within that domain,

the combination of the ontology and context may

fail to produce acceptable performance results. The

poor result by itself cannot be used to indicate the

specific changes to be made in the ontology to

improve the combined performance. The reason for

this disconnect is that, with respect to context, most

ontology evaluation methods are akin to black-box

software testing that examine the functionality of an

ontology without peering into its internal structure.

Moreover, the surrounding context is not described

and linked to the characteristics of the ontology.

Ontology research literature describes context as

being important and suggests its consideration when

designing ontologies (Noy and McGuinness, 2001;

Tatir et al., 2010). However there is no current

scientific guidance to determine what kind of

characteristics an ontology should have for a specific

type of user and domain to be adequate to support a

specific application task.

Our study investigates the connection between

lightweight ontologies, their use within a context,

and the effect of expertise within that context to

increase understanding of ontology design. For this

purpose, we describe our methodology to investigate

the performance of ontology-supported application

tasks for different contexts of use. By varying

characteristics and testing the effect on the

performance of an ontology-supported task for a

context parameter, we increase our understanding of

ontology design and evaluation.

In the next section we provide the background to

our research. We then introduce our methodology.

308

M. Waldstein I., O. Weber R. and Handly N..

Lightweight Ontologies in Context - Relationship between Ontology Characteristics and Context Parameters.

DOI: 10.5220/0004550303080315

In Proceedings of the International Conference on Knowledge Engineering and Ontology Development (KEOD-2013), pages 308-315

ISBN: 978-989-8565-81-5

 2013 SCITEPRESS (Science and Technology Publications, Lda.)

Next, we describe our experimental study that

utilizes our methodology using a lightweight

ontology in a healthcare context. Finally, we present

and discuss the results and conclude.

2 BACKGROUND AND RELATED

WORK

Ontologies represent knowledge through explicit

conceptualizations, which are consensual views of a

domain represented for some purpose (Gruber

1995). In this sense, what is included in the ontology

and how it is represented indicate commitments in

that the concepts selected and their inter-

relationships provide a particular perspective about

the world (Brewster and O’Hara 2007). This may

take the form of expert knowledge representing a

particular view of objects and interrelationships in a

domain. Ontologies are eminently suitable for

representing taxonomic information (Brewster and

O’Hara 2007), and there are numerous applications

utilizing this strength to support diverse tasks.

Ontologies range in their specification from

lightweight to heavyweight. Lightweight ontologies

consist of terms that have only minimal specification

of the term meaning whereas heavyweight

ontologies consist of rigorously formalized logical

theories (Uschold and Grunninger, 2004). Typically,

lightweight ontologies consist of a simple taxonomy

(Uschold and Grunninger, 2004) where less

expressivity is traded for usability (Brewster and

O’Hara 2007).

An important aspect to our research is the idea of

context. Context can be defined by location,

identities of people and objects, changes to those

objects (Schilit and Theimer, 1994); environment,

(Ryan et al., 1997); orientation, and date (Dey

1998). These list who, where, what, and when –

specifically the identity, location, activity, and time

(Dey and Abowd, 1999). User (identity), application

task (activity), and domain (location) are context

parameters that represent the context. They are used

in our methodology descriptively because they can

link context to ontology characteristics. Context

variables vary the context parameters and are used in

our methodology to distinguish different

configurations of context. For example, Expertise

can be used as a variable to vary the User context

parameter with the values of “novice” and “expert”.

The way to describe, manipulate and evaluate

ontology is through its characteristics. Ontology

characteristics can be structural, conceptual, or user

defined (Yu et al., 2009). Structural characteristics

are physical dimensions of the ontology schema.

Examples of structural characteristics used in design

and evaluation approaches include depth, breadth,

tangleness, fanoutness (Gangemi et al., 2006),

circularity, and partition (Gomez-Perez, 2001).

Past research has tried to utilize context in design

and evaluation approaches. A democratic ranking

Web system was proposed that separates the

reviewers into groups of domain experts, ontology

researchers, and common users that subjectively

evaluate ontologies uploaded by other users for

some context (Xu and Ma, 2008). A recent task-

based research direction for evaluation has looked at

equating characteristics to measures based on user

requirements that can predict performance in a task

(Yu et al., 2009). A three-level evaluation

framework looking at concept tagging was used to

evaluate how well an ontology performs (Porzel and

Malaka, 2005). Precision and recall were used in an

evaluation framework to evaluate an ontology in the

Web search task (Strasunskas and Tomassen, 2008).

Nevertheless, none of the evaluation approaches

above test or incorporate the connection between

ontology characteristics and context parameters that

can provide an actionable guideline for ontology

engineers. They do not provide a methodology to do

so, nor test the performance of an ontology at that

level of specificity.

3 METHODOLOGY

TO INVESTIGATE

ONTOLOGIES IN CONTEXT

OF USE

The methodology we propose associates ontologies

with context parameters to assess how changes in

ontologies influence overall performance. It is

implemented using the performance of an

application task that uses an ontology in a context.

An objective quality metric measures the

performance of the application task; indirectly we

are measuring the overall performance. The

methodology contains two phases; before changes

and after changes. In Phase 1 the context,

characteristics, and metrics are defined and

performance measured with the unchanged

ontology. In Phase 2 the ontology characteristics are

changed and performance is measured. Based on the

variation of performance, we analyze the impact of

the changes on overall performance.

LightweightOntologiesinContext-RelationshipbetweenOntologyCharacteristicsandContextParameters

309

3.1 Elements of the Methodology

The elements of the methodology, shown in Figure

1, are a context c in a set of contexts C, multiple

selected characteristics Ch in a set of characteristics

CH, and an application task with performance p. The

methodology asks:

For a context c, does a variation in the value of

one or more characteristics Ch lead to a variation in

performance from p

to p

Figure 1: Change in ontology reflecting a change in

characteristics and performance for a given context of use.

The context remains the same for both Phase 1 and

Phase 2. Only ontology characteristics are changed.

3.2 Methodology Steps

Phase 1

Select (populates and documents the elements)

 First, define the selected context parameter to the

variable level and document the remaining

parameters that define context.

 Second, select the ontology characteristic(s) for

performance measurement.

 Third, select a change to the characteristic(s).

 Forth, select quality metric(s) to use for

performance validation.

Measure Initial Performance

 The initial performance (p

) of the application

task supported by the ontology before the change

is measured.

Phase 2

Change Values

 The ontology characteristic(s) values are changed.

Measure Resulting Performance

 The resulting performance (p

) of the application

task supported by the ontology after the change is

measured.

Analyze

 The difference in results for performance p

between Phase 1 and 2 is analyzed. Test the

difference for statistical significance. A difference

in performance indicates that the change to the

ontology characteristics impacted perfromance.

Save

 If the results are statistically significant, the

specific combination of context c and selected

characteristics Ch, and change is documented for

future reference and used as a guideline for

ontology engineering and evaluation.

4 EXPERIMENTAL STUDY

In this section, we describe a study where we

implement the methodology we introduced in

Section 3.

4.1 Study Design

Figure 1 illustrates our two-phase methodology

employed in the study. In this study, the ontology

design is changed (as further detailed in section 4.5)

using the selected structural characteristics of

breadth, depth, and fanoutness as shown in Table 1.

These structural characteristics were selected for

four reasons. First, breadth, depth, and fanoutness

are well defined. Second, the characteristics can be

easily calculated using their defined formulae. Third,

the characteristics are straightforward to understand

and thus likely to be applied and documented in an

ontology engineering approach. Finally, results from

this research can be immediately applied because all

ontologies – lightweight and heavyweight – have

these structural characteristics.

Table 1: Calculation of characteristics used in the study.

Characteristic Calculated as

Depth

The average of the sum of all is-a paths

starting at the top node and terminating

in a leaf node.

Breadth

The sum of all is-a edges per level

divided by the number of levels.

Fanoutness

The total sum of the average of is-a

edges per node divided by the number of

levels. Nodes with zero child nodes are

not counted in the calculation.

The parameters that describe and operationalize the

context of use are the participating expert and novice

KEOD2013-InternationalConferenceonKnowledgeEngineeringandOntologyDevelopment

310

healthcare professionals (users), the search task

(application task), and the medical guidelines

(domain). The lightweight ontology is a pruned

MeSH ontology focusing on Internal Medicine. Two

metrics are used to determine performance: the

selection of query terms and the final selection of

relevant documents. The hypotheses we test are:

 H1: A change in the depth, breadth, and

fanoutness significantly impacts the performance

of the search for medical guidelines for novices in

the medical domain.

 H2: A change in the depth, breadth, and

fanoutness does not significantly impact the

performance of the search for medical guidelines

for experts in the medical domain.

As hypothesized in H1, novices, who are not as

intimately familiar with the domain and have less

procedural knowledge, will not be able to overcome

the effect of the detrimental change. For the novice

user the ontology replaces human expertise thereby

helping the novice to complete tasks. As

hypothesized in H2, experts should be able to use

their knowledge to overcome the effect of a

detrimental change to ontology from a change in its

characteristics.

4.2 User Parameter: Participating

Healthcare Professionals

We conducted the study with participants in the

healthcare field at a local university and university

affiliated hospital. We selected novice and expert

resident physicians and nurses as participants. Table

2 contains the number of participants for each group.

For the resident physician participant group, a

novice is a resident physician within the first or

second year of residency; an expert is a resident in

their last year of residency. For the nursing

participant group, a novice is a nursing student in the

last year of the B.S.N. program, and an expert is a

nurse who is also a student in the M.S.N. program

and has work experience.

Table 2: Study participants.

Participant Group

Number of

Participants

Resident Physicians

Expert resident physicians 16

Novice resident physicians 12

Nursing School Students

Expert nursing school students

Novice nursing school students

4.2.1 Expertise Variable

The user context parameter is defined by the

expertise variable. Experts have knowledge of the

domain, and have the experience to apply that

knowledge to the application task. It is exactly this

expertise that is lacking in novices, which we expect

good quality ontologies to enhance and poor quality

ontologies to diminish.

We reviewed literature to determine the

definition for experts and novices in the healthcare

context. Research looking at physician

understanding of updated guidelines for the

diagnosis and management of asthma found that

resident physician test scores improved with

duration of training (Doerschug et al., 1999). As

residents move through the program, their domain

expertise, and especially their diagnostic and

procedural knowledge increases. Research looking

at the assessment of nursing competence and

expertise in nursing utilizing the expert-performance

approach described that diagnostic expertise

improved with deliberate practice, i.e. extended

supervised training with feedback; and that graduate

training and specialized training successfully

differentiated expertise (Ericsson et al., 2007).

Literature confirmed that duration of on the job

experience and training can distinguish expertise.

4.3 Application Task Parameter:

Each phase, described in Section 4.1, consists of

users reading a medical scenario that presents a

medical disease or condition, which in turn creates a

need for medical guidelines to be searched. Medical

scenarios are randomly chosen and assigned for each

user out of three scenarios we prepared for the study.

Figure 2 is a sample medical scenario.

Participants are given a domain ontology that

helps them find and select keywords to be utilized to

search for guidelines based on a question in the

A type of hospital-acquired pneumonia (HAP) can occur in

people who are on a breathing machine through an

endotracheal or tracheostomy tube for at least 48 hours. The

pneumonia primarily occurs because the tube allows free

passage of bacteria into the lower segments of the lung in a

person who often has underlying lung or immune problems.

Your patient shows the following signs: alternating fever and

low body temperature, purulent sputum, and hypoxia.

What intervention is most likely to decrease the incidence of

this type of pneumonia in the intensive care unit?

Figure 2: Sample medical scenario.

LightweightOntologiesinContext-RelationshipbetweenOntologyCharacteristicsandContextParameters

311

medical scenario. Both novice and expert

participants in the study completed Phase 1 and

Phase 2.

4.4 Domain Parameter: Medical

Guidelines

The medical domain consists of guidelines from the

National Guideline Clearinghouse (NGC). The NGC

is a public resource for evidence-based clinical

practice guidelines. The NGC is maintained by the

Agency for Healthcare Research and Quality, an

operating division of the Department of Health and

Human Services.

The domain is specified by a lightweight

ontology based on the Medical Subject Headings

(MeSH) hierarchy in the NGC. Due to the size of the

medical guideline text a sample cannot be provided,

but can be viewed at http://www.guideline.gov/. A

sample of the MeSH ontology is shown in the next

section (Figures 3 and 4) to demonstrate the changes

applied to the selected structural characteristics.

4.5 Ontology Design Changes

4.5.1 Phase 1 – Before Changes

Ontology categories are organized in such a way as

to have the basic level in the middle of the hierarchy

with generalization moving upward and

specialization moving downward (Uschold and

King, 1995). The basic level contains the basic

categories. Basic categories are easy to perceive and

quick to identify, have the most attributes, have high

within-category similarity, and have high between-

category dissimilarity (Markman and Wisniewski

1997). Basic categories can be used with the

hierarchy to draw inferences based on their location

(Markman and Wisniewski 1997) – in our case

about the correct intervention for a disease or

condition by the novice and expert healthcare

L1. Bacterial Infections and Mycoses

L2. Bacterial Infections

L3. Gram-Negative Bacterial Infections

L4. Bordetella Infections

L5. Whooping Cough

L4. Enterobacteriaceae Infections

L5. Escherichia coli Infections

L5. Granuloma Inguinale

L5. Salmonella Infections

L6. Salmonella Food Poisoning

L4. Tick-Borne Diseases

L5. Tularemia

Figure 3: Excerpt of unchanged MeSH ontology.

professionals for the presented medical scenario.

Figure 3 displays an excerpt section of the

unchanged MeSH ontology used in the study with

the hierarchy levels (L*) shown. The starting

ontology contained 10 hierarchical levels with level

5 being the basic level.

4.5.2 Phase 2 – After Changes

The modification in the ontology design that

changed characteristics values for breadth, depth,

and fanoutness was performed at the basic level to

impact the selected context user parameter through

the expertise variable. The adjustment to the

ontology was made by eliminating level 4 entirely

(see Figures 3 and 4, L* codes were left to show the

change and were not visible in actual study). Level 4

contained superordinate concepts. Figure 4 shows

the result where all subordinate categories were

moved one level up in the hierarchy. By removing

the concepts at Level 4 we eliminated the link

between the superordinate and subordinate concepts.

This made it necessary for the participants in the

study to know specific diseases with a disease

etiology that was made less clear by the ontology

change. The change was made because it directly

impacts the expertise variable that we selected to

define the user context parameter.

L1. Bacterial Infections and Mycoses

L2. Bacterial Infections

L3. Gram-Negative Bacterial Infections

L 5. Whooping Cough

L 5. Escherichia coli Infections

L 5. Granuloma Inguinale

L 5. Salmonella Infections

L 6. Salmonella Food Poisoning

L 5. Tularemia

Figure 4: Excerpt of changed MeSH ontology with Level

4 removed and Level 5 promoted.

4.6 Measuring Overall Performance

Before performing the application task, study

participants were asked to read a medical scenario. It

is only after understanding the medical scenario that

motivates the need for searching for guidelines, that

they traversed the ontology to select keywords to

create a query. The path selected through the

hierarchy leading to query terms is the first result of

using the ontology to perform the application task.

We adopted this as the first metric and called it the

Path Selected quality metric.

When the query is submitted, a set of guidelines

is retrieved for a participant to select. The specific

KEOD2013-InternationalConferenceonKnowledgeEngineeringandOntologyDevelopment

312

guidelines selected by participants to answer the

question in the medical scenario are the second

metric we adopted and called it the Guidelines

Selected quality metric. The metrics relate to the

Fourth step under Select in Phase 1 of the

methodology.

5 RESULTS AND DISCUSSION

5.1 Results for Hypothesis 1

Table 3 shows the results for hypothesis H1 for the

Path Selected and Guidelines Selected metrics for

novices.

Table 3: Results for Hypothesis 1 (* = significant).

Participants

Phase 1

correct %

Phase 2

correct %

% Decrease

Correctness

Path Selected Quality Metric

Novice

Residents

46 27 41*

Novice

Nurses

70 30 60*

Guidelines Selected Quality Metric

Novice

Residents

45 18 60*

Novice

Nurses

70 30 60*

The results in Table 3 show that for the Path

Selected quality metric novice residents selected the

correct path to the keywords for 46% of their

attempts in Phase 1 (using the initial, i.e. unchanged,

ontology) and for 27% of their attempts in Phase 2

(using the changed ontology). Novice nurses

selected the correct path to the keywords for 70% of

their attempts in Phase 1 and for 30% of their

attempts in Phase 2.

A binomial test (p < 0.05) revealed that, for the

Path Selected quality metric, there is a significant

decrease between Phase 1 and Phase 2 for both

novice participant groups.

This demonstrates that the removal of an

essential part from the ontology decreased the

performance for novices. This confirms H1 for the

metric, showing that novices needed the complete

ontology due to their limited expertise.

For the Guidelines Selected quality metric,

novice residents were able to select a correct

guideline for 45% of their selection attempts in

Phase 1. They selected a correct guideline for 18%

of their selection attempts in Phase 2. Novice nurses

were able to select a correct guideline for 70% of

their selection attempts in Phase 1. They selected a

correct guideline for 30% of their selection attempts

in Phase 2.

This result confirms that an ontology changed at

the basic level is detrimental to novices that rely on

the expertise from the ontology to perform the

search application task.

A binomial test (p < 0.05) revealed that, for the

Guidelines Selected quality metric, there is a

significant decrease between Phase 1 and Phase 2

for novice participants. The change in the ontology

significantly decreased the guideline selection

performance for novices confirming H1.

5.2 Results for Hypothesis 2

Table 4 shows the results for hypothesis H2 for the

Path Selected and Guidelines Selected metrics for

experts.

Table 4: Results for Hypothesis 2 (* = significant).

Participants

Phase 1

correct %

Phase 2

correct %

% Decrease

Correctness

Path Selected Quality Metric

Expert

Residents

57 69 -

Expert

Nurses

53 47 11

Guidelines Selected Quality Metric

Expert

Residents

57 50 12

Expert

Nurses

36 24 33

The results in Table 4 show that for the Path

Selected quality metric, expert residents selected the

correct path to the keywords for 57% of their

attempts in Phase 1 and for 69% of their attempts in

Phase 2.

The change to the ontology eliminated the link

between the superordinate and subordinate disease

concepts thereby decreasing the specific disease

etiology. Nevertheless, expert residents were able to

select a path throught the ontology to the correct

keywords. Inspection of the post-participation

survey revealed that expert residents were

knowledgeable about the scenario topic based on

their “Very Familiar” and “Familiar” responses to

this question and the correct restatement of the

scenario topic in their own words.

For the Path Selected qualit metric, expert nurses

LightweightOntologiesinContext-RelationshipbetweenOntologyCharacteristicsandContextParameters

313

selected the correct path to the keywords for 53% of

their attempts in Phase 1 and for 47% of their

attempts in Phase 2.

This suggests that expert nurses obtained slightly

better support from the unchanged ontology.

A binomial test revealed that there was no

significant decrease in the correct Path Selected

quality metric for expert residents and expert nurses.

The change in the ontology decreased the

performance for only one group of experts. The

impact of the change was not significant for both

groups, confirming our hypotheses H2 for the

metric.

For the Guidelines Selected quality metric,

expert residents were able to select a correct

guideline for 57% of their selection attempts in

Phase 1, and selected the correct guideline for 50%

of their selection attempts in Phase 2. Expert nurses

were able to select a correct guideline for 36% of

their selection attempts with the initial ontology and

for 24% of their attempts using the changed

ontology. This suggests that experts obtained

moderately better support from the unchanged

ontology.

A binomial test revealed that there is no

significant decrease in the Guidelines Selected

quality metric for expert participants. The change in

the ontology did not significantly impact expert

performance confirming our hypothesis H2 for

experts.

5.3 Ontology as a Substitute

for Human Expertise

Participants in our study were either experts or

novices in order to account for our examination of

the impact of lightweight ontologies in human

expertise. The results from the study show that there

is a significant impact on the performance of an

ontology-supported task by novices when that

ontology is changed to the detriment of its ability to

support that task with respect to expertise.

Removing the ontology layer containing the

superordinate concepts to the basic level breaks the

link to the disease etiology. This causes a significant

impact on the search task for the novice groups, and

with greater effect on the nursing group.

The type of training conducted within the two

groups may play a role. Physician training has a

greater focus on diagnostic skills for the

identification of disease and its appropriate

treatment as compared to training in nursing. The

MeSH ontology is based on a taxonomy of disease.

A change in the ontology that made it more difficult

to link the symptoms to the disease presented in the

scenario impacted nurses to a greater extent than

physicians. Expert physicians were able to

compensate for the lack of knowledge during the

navigation through the ontology, and had the

smallest decrease in the final task of identifying the

appropriate guidelines.

The change to the ontology had the greatest

impact on novice nurses, who were the least

knowledgeable group about the specific medical

conditions in the scenarios presented, followed by

novice residents, expert nurses, and then expert

physicians.

6 CONCLUSIONS AND FUTURE

WORK

From the results presented in this paper, we

demonstrate that lightweight ontologies do serve as a

replacement for human expertise in a context of use.

This is important because many of the current

taxonomy and is-a hierarchies on the Web, which

are examples of lightweight ontologies, are

supporting novices in performing tasks that might

require expertise. The study shows that a change in

characteristics of the ontology impacts the support of

an application task for a context for which the

ontology was designed. Specifically, we

demonstrated that less complete knowledge

structures will interfere with the ability of novice

users who lack expertise about the domain.

We also show that there is a relationship between

ontology characteristics and context parameters as

the change in the characteristics caused a significant

impact to performance for the novice value of the

expertise variable defining the user context

parameter.

Finally, we show that our proposed methodology

to investigate the relationship between ontology

characteristics and context parameters works as

intended. The methodology does associate an

ontology with a given context of use and can

evaluate ontology supported performance of a task.

Future work will include a closer examination of

the post-study survey to determine the details of the

performance differences in the results. Additionally,

we plan to conduct similar studies outside of the

healthcare field to vary the context. Finally, we will

implement our methodology with non-structural

ontology characteristics.

KEOD2013-InternationalConferenceonKnowledgeEngineeringandOntologyDevelopment

314

ACKNOWLEDGEMENTS

This study has been approved by the Institutional

Review Board of the Drexel University College of

Medicine as of Nov 2011. Protocol # 1111000470

(19799) - "Impact of Ontology Characteristics on

Search in the Medical Domain".

REFERENCES

Brewster, C & O’Hara, K. 2007, ‘Knowledge

representation with ontologies: Present challenges -

Future possibilities’, International Journal of Human-

Computer Studies, vol. 65, no. 7, pp. 563-568.

Dey, A. K. 1998, ‘Context-Aware Computing: The

CyberDesk Project’, AAAI 1998 Spring Symposium on

Intelligent Environments, Tech. Rpt. SS-98-02:51-54.

Dey, A. K. & Abowd, G. D. 1999, ‘Towards a Better

Understanding of Context and Context-Awareness’,

GVU Technical Report; GIT-GVU-99-22.

Doerschug, K. C., Peterson, M. W., Dayton, C. S., &

Kline, J. N. 1999, ‘Asthma Guidelines: An

Assessment of Physician Understanding and Practice’,

American Journal of Respiratory and Critical Care

Medicine, vol. 159, no. 6, pp. 1735-1741.

Ericsson, K. A., Whyte I. V., J., & Ward, P. 2007, ‘Expert

Performance in Nursing Reviewing Research on

Expertise in Nursing within the Framework of the

Expert-Performance Approach’, Advances in Nursing

Science, vol. 30, no. 1, pp. E58-E71.

Gangemi, A., Catenacci, C., Ciaramita, M., & Lehmann, J.

2006, ‘Modeling Ontology Evaluation and

Validation’, In Y. Sure and J. Domingue (Eds.),

Proceedings of the ESWC 2006, LNCS 4011, pp. 140-

154.

Gómez-Pérez, A. 2001, ‘Evaluation of ontologies’,

International Journal of Intelligent Systems, vol. 16,

no. 3, pp. 391-409.

Gruber, T. R. 1995, ‘Toward principles for the design of

ontologies used for knowledge sharing’, International

journal of human computer studies, vol. 43, no. 5, pp.

907-928.

Markman, A. B. & Wisniewski, E. J. 1997, ‘Similar and

Different: The Differentiation of Basic-Level

Categories’, Journal of Experimental Psychology:

Learning, Memory and Cognition, vol. 23, no. 1, pp.

54-70.

Noy, N. F. & McGuinness, D. L. 2001, ‘Ontology

development 101: A guide to creating your first

ontology’, Technical Report KSL-01-05, Stanford

Medical Informatics.

Porzel, R. & Malaka, R. 2005, ‘A Task-Based Framework

for Ontology Learning, Population and Evaluation’, In

P Buitelaar et al. (Eds.), Ontology learning from text:

methods, evaluation and applications. pg. 107.

Ryan, N., Pascoe, J., & Morse, D. 1997, ‘Enhanced

Reality Fieldwork: the Context-Aware Archaeological

Assistant’, In V. Gaffney, M. van Leusen, & S. Exxon

(Eds.), Computer Applications in Archaeology.

Schilit, B. & Theimer, M. 1994, ‘Disseminating Active

Map Information to Mobile Hosts’, IEEE Network,

vol. 8, no. 5, pp. 22-32.

Strasunskas, D. & Tomassen, S. L. 2008, ‘Empirical

Insights on a Value of Ontology Quality in Ontology-

Driven Web Search’, In R. Meersman & Z. Tari

(Eds.), Proceedings of the OTM 08, Part II, LNCS

5332, pp. 1319-1337.

Tatir, S., Arpinar, I. B., & Sheth, A. P. 2010, ‘Ontological

evaluation and validation’, Theory and Applications of

Ontology: Computer Applications

, pp. 115-130.

Uschold, M. & Gruninger, M. 2004, ‘Ontologies and

semantics for seamless connectivity’, ACM SIGMod

Record, vol. 33, no. 4, pp. 58-64.

Uschold, M. & King, M. 1995, ‘Towards a methodology

for building ontologies’, In Workshop on basic

ontological issues in knowledge sharing, vol. 74,

Montreal, Canada.

Xu, J. & Ma, X. 2008, ‘A Web-Based Ontology

Evaluation System’, Workshop on Advanced

Language Processing and Web Information

Technology, In Proeedings of the ALPIT 2008

International Conference, pp. 104-107.

Yu, J., Thom, J. A., & Tam, A. 2009, ‘Requirements-

oriented methodology for evaluating ontologies’,

Information Systems, vol. 34, no. 8, pp. 766-791.

LightweightOntologiesinContext-RelationshipbetweenOntologyCharacteristicsandContextParameters

315