Semantic Representation of Neuroimaging Observations: Proof of

Concept based on the VASARI Terminology

Emna Amdouni

1,2

and Bernard Gibaud

1,2

E-health department, B-com Institute of Research and Technology, Rennes, France

LTSI Inserm 1099, Universit

e de Rennes 1, Rennes, France

Keywords:

Medical Domain Ontology, Knowledge Representation, Ontology Reuse and Alignment.

Abstract:

The main objective of this work is to facilitate the identiﬁcation, sharing and reasoning about cerebral tumors

observations via the formalization of their semantic meanings in order to facilitate their exploitation in both

the clinical practice and research. We have focused our analysis on the VASARI terminology as a proof of

concept, but we are convinced that our work can be useful in other biomedical imaging contexts. In this paper,

we propose (1) a methodology, a domain ontology and an annotation tool for providing unambiguous formal

deﬁnitions of neuroimaging data, (2) an experimental work on the REMBRANDT dataset to demonstrate the

added value of our work over existing methods, namely DICOM SR and the AIM model.

1 BACKGROUND AND

SIGNIFICANCE

In literature, an ontology is deﬁned as ”a formal and

an explicit speciﬁcation of a shared conceptualiza-

tion” (Gruber, 1993). Ontologies deﬁne the formal

semantics of vocabularies by specifying axioms, ex-

pressed in a logic-based language, that constrain and

structure relationships between terms. The main pur-

pose of ontologies is to enable knowledge integration

and semantic data querying. In the medical ﬁeld, se-

mantic web technologies are used to standardize, for-

malize and share the medical data (coming form both

the clinical and research context) is very important

(Scheuermann et al., 2009; Seifert et al., 2010; Ober-

kampf et al., 2012). In this paper, we focused our

interest on the domain of cerebral tumors.

In oncology clinical practice, neuroimaging fea-

tures/phenotypes play an important role; in particu-

lar they help clinicians in making their diagnosis, se-

lecting the appropriate treatment and monitoring the

therapeutic response to an intervention as for exam-

ple the Response Evaluation Criteria in Solid Tumors

(RECIST) (Eisenhauer et al., 2009). Many radiology-

pathology correlation studies have been conducted

on cerebral tumors and show that some neuroima-

ging features are associated to genetic alterations and

gene expression (Gutman et al., 2013; Grams et al.,

2014). Therefore, suitable management of neuroima-

ging phenotypes is needed to facilitate their use and

reuse in multiple studies regarding imaging biomar-

kers (Levy et al., 2012; M

oller et al., 2009; ESR,

2011; Rubin et al., 2014a).

Currently, neuroimaging features can be recorded

and stored in a formalized format such as the DICOM

(Digital Imaging and Communications in Medicine)

SR (Structured Report) (Clunie, 2000; Clunie, 2007)

and the Annotation and Imaging Markup (AIM) mo-

del (Channin et al., 2010): the DICOM SR forma-

lizes the representation of radiological observations

by introducing a set of rules that constrain concepts

organization and a vocabulary (i.e. codes and asso-

ciated code meanings) covering the domain of ima-

ging observations. DICOM SR includes measure-

ments and qualitative assessments, their relationships

with image evidence and with the clinical interpreta-

tion of the clinician. The AIM model is an informa-

tion model and an XML-based ﬁle format to describe

the minimal information necessary to record image

annotations. This information model has introduced

the most relevant entities used in image annotation.

These standard formats enable the description of

the content of medical images. Unfortunately, they

are not suitable to support logic-based reasoning;

these formats are based on coded terms and most of

them have no semantic axioms to specify their mea-

nings and the relationships between them. As a con-

sequence, only searches based on keywords can be

handled on decision making tools that exploit these

standard formats. Rubin et al., Levy MA and many

Amdouni, E. and Gibaud, B.

Semantic Representation of Neuroimaging Observations: Proof of Concept based on the VASARI Terminology.

DOI: 10.5220/0006931100630074

In Proceedings of the 10th International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management (IC3K 2018) - Volume 2: KEOD, pages 63-74

ISBN: 978-989-758-330-8

others researchers have promoted the use of formal

ontologies (Rubin et al., 2009; Levy et al., 2009;

Kahn J.R. et al., 2011; Van Soest et al., 2014) for au-

tomatic reasoning on these data models and overcome

the current limitations listed above.

This work is based on the assumption that the use

of semantic web technologies, in particular ontologies

as well as reasoning capabilities, can make the mea-

ning of neuroimaging assessments more explicit and

facilitate their advanced exploitation and interpreta-

tion. We are convinced that this approach can be be-

neﬁciary to the whole domain of neuroimaging, and

beyond to the whole domain of biomedical imaging.

However, this work focuses on a limited domain, na-

mely the domain covered by the VASARI termino-

logy (for Visually Accessible Rembrandt Images ter-

minology)

, used as a proof of a concept.

The VASARI terminology is a controlled vocabu-

lary that describes thirty observations of high grade

cerebral gliomas (glioblastoma multiform or GBM)

in conventional Magnetic Resonance Imaging (MRI)

images. Its main objective consists in standardizing

brain tumors description and facilitating their inter-

pretation by neuro-radiologists. The VASARI termi-

nology was developed by experts in neuro-radiology

who have considered the majority of possible asses-

sments based on MRI. The validation of this corpus

of imaging features was realized by eight experienced

neuro-radiologists from distinct institutions.

Our work has three main contributions. First, ma-

king the meaning of VASARI features explicit via

the design and the implementation of a specialized

ontology, called VASARI ontology. Second, provi-

ding a semantic annotation tool that automatically

translates VASARI features into instances of the VA-

SARI ontology. This tool was applied to a VASARI

corpus called REMBRANDT data set (Repository of

Molecular Brain Neoplasia Data) (Madhavan et al.,

2009). Third, performing semantic queries and rea-

soning tasks on these data to show how this semantic

description of VASARI features can facilitate the in-

terpretation of the content of medical images.

The remaining of this article is organized as fol-

lows: in Section Material and Methods, we describe

the methodology for the design of the VASARI onto-

logy. Section Results provides, ﬁrst, an overview of

the architecture and development details of both the

developed ontology and the semantic annotation tool.

Second, it demonstrates how we can automatically

store and semantically manipulate RDF (Resource

Description Framework) data of the REMBRANDT

data set. In Section Discussion, we explain some mo-

https://wiki.cancerimagingarchive.net/display/Public/

VASARI+Research+Project

deling choices that we have made regarding ontology

design and implementation, list accomplished work

and enumerate remaining problems.

2 MATERIAL AND METHODS

2.1 Design of the VASARI Ontology

The VASARI ontology was designed according to

the realism-based approach proposed by Ceusters and

Smith (Ceusters and Smith, 2005; Smith, 2006; Smith

and Ceusters, 2010). Our modeling methodology is

composed of ﬁve main steps that can be outlined as

follows. First, we analyzed for each VASARI fea-

ture Fi the meaning of the studied aspect and sorted

its possible conﬁgurations to establish the list of pos-

sible values allowed for each criterion. Second, we

identiﬁed and described the key real entities that are

involved in each criterion. Third, we related entities

to existing ontologies, most of which coming from the

OBO foundry and aligned onto the Basic Formal On-

tology (Bittner and Smith, 2004; Smith et al., 2005b).

When needed new ontology classes were speciﬁed.

Fourth, we speciﬁed and deﬁned the axioms charac-

terizing these entities and relations between them. Fi-

nally, we made sure that all possible conﬁgurations

for each feature Fi can be modeled in a formal way.

In our work, we described the thirty VASARI fea-

tures but this paper focuses on seven of them, namely:

lesion location, lesion side, enhancement quality, pro-

portion nCET (non contrast enhanced tumor), cortical

involvement, extent of resection of enhancing tumor

and lesion size (see Table 1).

After a deep analysis of the meaning of the VA-

SARI features and the identiﬁcation of the different

entities that they involve, we have proceeded with

their formal description. This step was not a trivial

task given that we faced several modeling problems

summarized in Table 2 and more detailed in (Am-

douni and Gibaud, 2016). Modeling problems con-

cern: negative ﬁndings (MP1) (Ceusters et al., 2006),

spatial knowledge (MP2) (Bennett et al., 2013) and

complex entities representation (MP3).

We have used the version 2 of the Basic Formal

Ontology (BFO) as a foundation for the VASARI on-

tology, thus facilitating the integration of specialized

ontologies that come from the Open Biological and

Biomedical Ontologies (OBO) foundry (Smith et al.,

2007). In particular, we reused the following onto-

logies: the Foundational Model of Anatomy (FMA)

(Smith et al., 2006), the Information Artifact Onto-

logy (IAO) (Ceusters, 2012), the Phenotypic Quality

Ontology (PATO) (Mungall et al., 2007), the Open

KEOD 2018 - 10th International Conference on Knowledge Engineering and Ontology Development

Table 1: Selected subset of the VASARI features deﬁned in the VASARI terminology.

Feature Feature deﬁnition feature

F1.Lesion location Location of lesion geographic epi-

center; the largest component of the

tumor either contrast enhancing or

non contrast enhancing.

frontal, parietal, temporal,

occipital, corpus callosum,

thalamus

F2.Lesion side Side of lesion epicenter. right, central, bilateral

F4.Enhancement

quality

Qualitative degree of contrast en-

hancement is deﬁned as having all

or portions of the tumor that demon-

strate signiﬁcantly higher signal on

the post contrast T1W images com-

pared to pre contrast T1W images.

n/a, none, mild, marked

F6.Proportion nCET What proportion of the entire tumor

is non enhancing? Non enhancing

tumor is deﬁned as regions of T2W

hyper intensity that are associated

with mass effect and architectural

distortion, including blurring of the

gray-white interface.

n/a, 0%, >5%, 6-33%, 34-

67%, 68-95%, >95%, 100%,

indeterminate.

F20.Cortical invol-

vement

Non-enhancing o enhancing tumor

extending to the cortical mantle, or

cortex is no longer distinguishable

relative to subjacent tumor.

no, yes

F26.Extent of re-

section of enhancing

tumor

Using the ﬁrst postoperative scan

(contrast enhancing MR imaging)

assessed for tumor residual estima-

ting the proportion of enhancing

tumor. Total resection component

should be scored to 100%, subtotal

resection of enhancing tissue should

be scored accordingly.

n/a, 0%, >5%, 6-33%, 34-

67%, 68-95%, >95%, 100%,

indeterminate

F29.Lesion size Largest perpendicular (x-y) cross

section diameter of T2 signal ab-

normality (longest dimension X

perpendicular) measured on single

sectional image only.

unidimensional, largest dia-

meter in centimeters

Biomedical Investigations (OBI) (Brinkman et al.,

2010), the Ontology for General Medical Sciences

(OGMS) (Scheuermann et al., 2009), the Unit On-

tology (UO) (Gkoutos et al., 2012) and the Relation

ontology (RO) (Smith et al., 2005a); in our work we

have considered that RO relations are integrated under

the BFO 2 ontology. Note that ontologies acronyms

will be used in the rest of the paper.

2.2 Design of the Experimental Work

In our experimental work, we have developed a se-

mantic annotation software of VASARI data using the

VASARI ontology. This software enables the user

to transform the informal description of the 30 VA-

SARI features into a formal one. This software was

applied to a corpus of VASARI data called the REM-

BRANDT repository. The resulting semantic data set

was used to evaluate the added value of our work; es-

pecially, we performed some reasoning tasks by for-

mulating semantic queries as well as some consis-

tency tests to detect inconsistent assertions.

2.3 Presentation of the REMBRANDT

Repository

The REMBRANDT repository is freely accessible

on this link16. The ultimate objective of the REM-

BRANDT data set is to facilitate the discovery of

signiﬁcant correlations between clinical and genomic

information in order to provide patients with more

personalized treatments in the clinical context. The

REMBRANDT data set contains 30 VASARI features

labeled by 3 radiologists that concern 34 patients with

GBM tumors. Features values are stored in an Excel

Semantic Representation of Neuroimaging Observations: Proof of Concept based on the VASARI Terminology

Table 2: Modeling problems.

Modeling problem VASARI expressions Use cases and examples

MP1: How to re-

present negative

neuroimaging obser-

vations that indicate

the non-existence of

a dependent conti-

nuant (category C1)

or an independent

continuant (category

C2): bfo:Quality or

bfo:Disposition?

Negative ﬁndings are expressed by

the use of negative qualiﬁers as

for example none or expressions as

indeterminate, not applicable, wit-

hout etc.

Non-existence of an entity

or a quality/disposition. Ex-

amples: C1:Johns cerebral

tumor is without an enhan-

cing region. C2:Johns cere-

bral tumor is not edematous.

Or Johns cerebral tumor is

not inﬁltrative.

MP2: How to ensure

a faithful represen-

tation of the studied

pathological struc-

ture and describe

how its components

are situated in space?

Containment is denoted by natural

language expressions like within,

portion of, comprise of whereas

overlapping is denoted by the term

invasion. The proximity of a given

entity to another one is expressed

with terms such as surrounding and

adjacency, and separation is deno-

ted by terms like not contiguous and

separated.

Spatial location of existing

entities, entities that are re-

lated to each other, enti-

ties that are separated from

each other, entities that are

adjacent to each other, etc.

Examples: Johns cerebral

tumor epicenter is in the pa-

rietal lobe. A cerebral tumor

has part a cerebral tumor

margin.

MP3: How to encode

complex entities as

for example deri-

ved measurements

(proportions of vo-

lume measurements,

length measure-

ments, etc.), and

associate them to

their corresponding

clinical ﬁndings?

Volume proportions: 0%, <5%,

6-33%, 34-67%, 68-95%, >95%,

100%. Or, two-dimensional length

(x,y) that represents cross-sectional

diameters; scores are between <0.5

cm and >8cm.

The representation of the ex-

tent of the resection of a gi-

ven cerebral tumor compo-

nent (nCET, necrotic, etc.).

Examples: 35% of Johns

cerebral tumor is enhanced,

Johns cerebral tumor of size

1cm2cm.

document where each spreadsheet contains evaluati-

ons asserted by a radiologist.

2.4 Implementation Details

We have designed the VASARI ontology in the Onto-

logy Web Language 2 (OWL2) format using the ver-

sion 5 of the Prot ´eg ´e tool (Tudorache et al., 2013).

To extract modules from the relevant OBO ontolo-

gies we used the Ontofox web interface (Smith et al.,

2007). Our semantic annotation tool was developed

with the JAVA language and created using the NetBe-

ans IDE 8.0.2 programming environment. To design

the semantic annotation software, we used the version

2.8.3 of the JENA35 semantic web programming fra-

mework and the version 2.3.2 of the JENA-Pellet re-

asoner engine (external engine of the JENA API) (Si-

rin et al., 2007) to perform automatic reasoning tasks.

To execute some SPARQL queries, we have used the

CORESE tool 3.2

. Note that instance data are seria-

lized in the RDF/XML format.

3 RESULTS

3.1 The VASARI Ontology

The VASARI ontology imports eight ontology modu-

les and contains around 570 OWL classes and 120

properties. Figure 1 shows that the four major se-

mantic aspects that constitute the VASARI domain,

namely: pathological structures, anatomical localiza-

tion, qualities and dispositions, and measurements. In

http://wimmics.inria.fr/corese

KEOD 2018 - 10th International Conference on Knowledge Engineering and Ontology Development

this section, classes are represented in Italic and rela-

tionships between them in Bold.

In our proposed ontology, a

vasari:CerebralTumor is a va-

sari:CerebralPathologicalStructure. Different

regions of the cerebral tumor are introduced with

the entity vasari:CerebralTumorComponent that is

deﬁned as a vasari:CerebralPathologicalStructure

and bfo:continuant

part at some time some

vasari:CerebralTumor. In our semantic model

and as it is deﬁned in the VASARI termino-

logy, we considered that a vasari:CerebralTumor

can be composed of four basic components that

characterize the brain tissue abnormality: va-

sari:EnhancingCerebralTumorComponent, va-

sari:NonEnhancingCerebralTumorComponent,

vasari:NecroticCerebralTumorComponent and

vasari:CerebralEdemaComponent.

The following paragraphs describe how the speci-

ﬁc domains requirements mentioned in the Material

and Methods Section were addressed in the VASARI

ontology.

Modeling Problem 1: In order to respect the basic

principles of realist ontology, no instance should be

created when no concrete entity exist in reality. The

solution that we proposed uses an equivalentclass ax-

iom (condition if and only if involving some negative

somevaluesfrom assertion). Thus, classes are deﬁned

as follows:

• Case category C1: A

vasari:CerebralTumorComponentNotLocatedIn-

BrainCortex [deﬁnition] is a va-

sari:CerebralTumorComponent and

not (bfo:located in at some time

some vasari:CerebralCortex), an

vasari:EnhancingCerebralTumorWithoutNonEn-

hancingCerebralTumorComponent [deﬁnition]

is a vasari:EnhancingCerebralTumor and not

(bfo:has continuant part at some time some

vasari:NonEnhancingCerebralTumorCompone-

nt).

• Case category C2: A va-

sari:NonCysticCerebralTumorComponent [de-

ﬁnition] is a vasari:CerebralTumorComponent

and not (bfo:has quality at some time

some pato:Cystic), a va-

sari:NonEnhancingCerebralTumorComponent

[deﬁnition] is a va-

sari:CerebralTumorComponent and not

(bfo:has disposition at some time some

vasari:DispositionToBeEnhancing).

Modeling Problem 2: Three main spatial relations

are modeled:

• Containment relation: We employed the spa-

tial relation bfo:located in at some time

and the foundational relation

bfo:part of continuant at some time.

We suppose that C1 and C2 are classes

of continuants. As asserted in BFO, C1

bfo:part of continuant at some time C2 means

that for every particular c1, if c1 instance of C1

then there is some c2 such that c2 instance of C2

and c1 bfo:part of continuant at some time

c2, C1 bfo:located in at some time C2 as-

serts that for every c1 if c1 instance of

C1, then there is some c2 instance of

C2 and c1 bfo:located in at some time

c2. In our ontology, we used the re-

lation bfo:located in at some time, for

example to associate a particular va-

sari:CerebralTumorEpicenter to its speciﬁc

vasari:LobeOfCerebralHemisphere and the rela-

tion bfo:has continuant part at some time to

deﬁne that the vasari:EdematousCerebralTumor

[deﬁnition] is a vasari:CerebralTumor and

(bfo:has continuant part at some time some

vasari:CerebralEdemaComponent).

• Overlapping vs adjacency relation: We employed

the spatial relation ro:adjacent to to express that

two continuants do not share a common spatial re-

gion and we deﬁned the relation vasari:overlaps

to represent the case of overlapping. As des-

cribed in Table 1, F20 evaluates the location

of the cerebral tumor regarding the cerebral

cortex. To describe these different situations

and ensure a correct classiﬁcation of the cerebral

tumor, we have deﬁned the following classes:

vasari:CerebralTumorInvadingBrainCortex

[deﬁnition] vasari:CerebralTumor and

(vasari:overlaps some vasari:CerebralCortex),

vasari:CerebralTumorAdjacentToBrainCortex

[deﬁnition] vasari:CerebralTumor and

(ro:adjacent to some vasari:CerebralCortex).

• Separation relation: We used the relational

quality vasari:ContiguousWithCerebralTumor

with the logical negation operator (not) to

qualify and identify cerebral component that

are separated from the cerebral tumor, as

for example: vasari:SatelliteLesion [deﬁni-

tion] vasari:CerebralPathologicalStructure

and (not (bfo:has quality at some time some

vasari:ContiguousWithCerebralTumor)) and

(bfo:has disposition at some time some va-

sari:DispositionToBeEnhancing).

• Separation relation: We used the relational

quality vasari:ContiguousWithCerebralTumor

with the logical negation operator (not) to

Semantic Representation of Neuroimaging Observations: Proof of Concept based on the VASARI Terminology

Figure 1: The basic pattern of the main classes in the VASARI ontology.

qualify and identify cerebral component that

are separated from the cerebral tumor, as

for example: vasari:SatelliteLesion [deﬁni-

tion] vasari:CerebralPathologicalStructure

and (not (bfo:has quality at some time some

vasari:ContiguousWithCerebralTumor)) and

(bfo:has disposition at some time some va-

sari:DispositionToBeEnhancing).

Modeling Problem 3: The representation of the

extent of the resection of an enhancing cerebral tumor

component appears to be simple data to represent,

but in reality it involves hidden information that

are not explicit in the deﬁnition of the feature.

To model this kind of feature, we consider that

the vasari:EnhancingCerebralTumorComponent

will not preserve its identity after and before the

surgery. Thus, we identiﬁed two distinct entities:

vasari:EnhancingCerebralTumorComponentBefore-

Surgery [deﬁnition] is

a va-

sari:EnhancingCerebralTumorComponent and

(bfo:part of continuant at some time some

vasari:EnhancingCerebralTumorComponent)

and (bfo:is speciﬁed input of some

vasari:ResectionOfEnhancingCerebralTumorComp-

onent). vasari:EnhancingCerebralTumorCompone-

ntAfterSurgery [deﬁnition] is a va-

sari:EnhancingCerebralTumorComponent and

(bfo:part of continuant at some time some

vasari:EnhancingCerebralTumorComponentBefore-

Surgery) and (bfo:is speciﬁed output of some

vasari:ResectionOfEnhancingCerebralTumorComp-

onent).

We considered that the measured qua-

lity, i.e. the volume, is the same, but

the measured volume values are distinct:

vasari:volumeMeasurementDatumOfEnhancingCer-

ebralTumorAfterSurgery [deﬁni-

tion] vasari:VolumeMeasurementDatum

and (bfo:is

about some

vasari:EnhancingCerebralTumorComponentAfterSu-

rgery), vasari:volumeMeasurementDatumOfEnhanc-

ingCerebralTumorBeforeSurgery [deﬁ-

nition] vasari:volumeMeasurementDatum

and (bfo:is about some

vasari:enhancingCerebralTumorComponentBeforeS-

urgery). The interpretation of extreme values of F26

will be as follows:

• %0 means that the enhancing tumor component

is totally preserved and that the measured volume

value before the surgery is the measured volume

value after the surgery.

• %100 means that the enhancing cerebral

tumor component is totally resected, thus the

cerebral tumor component is classiﬁed as a

vasari:NonEnhancingCerebralTumorComponen-

tAfterSurgery.

3.2 Semantic Annotation Software of

VASARI Data

The annotation software begins by reading as an

input the set of imaging features values of the

REMBRANDT repository and the VASARI ontology

schema. Then, to semantically annotate the data the

software realizes four main tasks. First, it instantia-

tes the VASARI ontology based on the VASARI la-

beled values. Second, it describes imaging features

by creating RDF triples that establish semantic links

between instances. Third, it adds these triples as sta-

tements in an RDF graph. Fourth, it serializes data in

the RDF/XML grammar and records the RDF graph

in memory or in a JENA triple database (TDB). Note

KEOD 2018 - 10th International Conference on Knowledge Engineering and Ontology Development

that the software stores separately the schema and the

instance data (denoted by the terms Tbox and Abox

in the following paragraph). It returns the generated

RDF graph of the whole REMBRANDT dataset in

1.06 s ( 0.47s per radiologist).

3.3 Semantic Exploitation of the

VASARI Annotation Data

The developed software bases its reasoning on an in-

ferred model generated with a reasoner. The infor-

mation stored in the inferred graph is contained in a

knowledge base (KB) that can be exploited in two

ways: 1) accessed via SPARQL queries to retrieve

data based on their semantics 2) checked via consis-

tency tests.

Querying a knowledge base: Let us consi-

der this example to illustrate inference capabilities

and demonstrate how the semantic format enables

the exploitation of the anatomical knowledge co-

ming from the FMA ontology. We suppose that

a given KB is composed of an Abox that con-

tains the following assertions: (A1) cte instance of

CerebralTumorEpicenter, (A2) tl instance of Tem-

poralLobe, (A3) rch instance of RightCerebralHe-

misphere, (A4) cte bfo:located in at some time tl,

(A5) tl regional part of rch. A Tbox that contains:

(T1) RightTemporalLobe [equivalentClass] Tempo-

ralLobe and regional part of some RightCerebralHe-

misphere, (T2) regional part of [equivalentProperty]

part of continuant at some time.

Based on the assertions of the Abox and on

the (T1) axiom we can deduce that: (A6) tl in-

stance of RightTemporalLobe (see Figure 3). Using

(T2), (T3) and (A6), we can infer: (A7) cte

bfo:located in at some time rtl (see ﬁgure 2).

Figure 3 presents an example of SPARQL

query that retrieves the location of cerebral tumors

(?lobe, ?cerebral hemisphere) and their correspon-

ding length measurements (?long axis value, ?per-

pendicular long axis value). The lower part of the

ﬁgure presents the results returned by the CORESE

semantic query tool.

Validation of a knowledge base: The pellet JENA

reasoner enables the detection of conﬂicts in the kno-

wledge content; thus we have exploited this capa-

city to perform a global check across the KB and

looked for inconsistencies between the radiologists

assertions. For example lets suppose that (A1) ct in-

stance of CerebralTumor and that one radiologist said

that (A2) ct is a HemorrhagicCerebralTumor howe-

ver, another radiologist said that (A3) ct is a NonHe-

morrhagicCerebralTumor. Reasoning on this know-

ledge base is impossible given because the two clas-

ses HemorrhagicCerebralTumor and NonHemorrha-

gicCerebralTumor are deﬁned as disjoint classes in

the Tbox; this means that they cannot share the same

set of instances (i.e., ct). As a consequence, a classiﬁ-

cation error should occur when the reasoner performs

inference tasks.

The result of a consistency checking is provi-

ded via the object ValidityReport of the JENA API.

This data structure encapsulates all detected incon-

sistent axioms and assertions. To generate explana-

tions about inconsistencies, we have used the met-

hod explainconsistency(). The output generated af-

ter the execution of this method consists on the lis-

ting of the set of all involved axioms; Figure 4 de-

picts an example of inconsistency that is caused by

the assignment of two different length measures to the

same cerebral tumor. This multiple attribution viola-

tes the owl:functionalProperty axiom of the property

iao:has measurement value that is declared as functi-

onal.

4 DISCUSSION

In our work, we followed the realism-based appro-

ach to describe the neuroimaging reality on the side

of the patient which appeared to us the most rele-

vant methodology in the context of the biomedical re-

search; the realism-based approach is being adopted

by a growing community of researchers in the medi-

cal context. Actually, medical terminologies such as

the DICOM SR and the AIM model do not refer to

concrete existing phenomena on the side of the pa-

tient, but they only code medical statements in a for-

mal way. The adoption of the realism-based approach

enabled us to provide a faithful representation of ima-

ging features by considering both the universals level

(e.g. cerebral tumor concept) and instances of univer-

sals level (e.g. Davids cerebral tumor). To follow this

modeling perspective, we have used two foundational

and realism-based ontologies; namely the BFO on-

tology to describe existing entities and relationships

between them. The use of BFO ontology has facilita-

ted the integration of heterogeneous knowledge from

different ontologies that are specialized in anatomy,

quality phenotypes, measurements, etc.

The developed ontology answers to some challen-

ging points that are highlighted in many papers (Ceus-

ters et al., 2006; Cimino, 2006; Levy et al., 2012)

mainly: 1) the formalization of the description of neu-

roimaging information via the use of specialized onto-

logies; we can cite the example of the FMA ontology

that allowed the description of some clinical state-

ments or the BFO and RO ontologies that helped us in

Semantic Representation of Neuroimaging Observations: Proof of Concept based on the VASARI Terminology

Figure 2: Illustrative example of semantic exploitation of anatomical knowledge from the FMA ontology (protg tool); instan-

ces names are automatically generated with our developed annotation tool, the ID of the patient 900 00 1961 is included to

refer to existing entities on the side of the patient.

Figure 3: Execution result to a SPARQL query on observations of the radiologist1 in the REMBRANDT repository (demon-

stration with the CORESE tool). FMA 61823 denotes the LobeOfCerebralHemisphere, FMA 61817 denotes the CerebralHe-

misphere and BFO 0000171 denotes the relation bfo:located in at some time.

the description of foundational (i.e., is a, part of) and

spatial relations (i.e., location, adjacent to) between

pathological structures, 2) the representation of nega-

tive neuroimaging observations via the use of OWL

axioms and 3) the representation of complex entities

was made more explicit by referring to the involved

concrete entities; but we have faced some difﬁculties

to represent mathematical expressions with the IAO

KEOD 2018 - 10th International Conference on Knowledge Engineering and Ontology Development

Table 3: Alignment of ontologies to represent VASARI features.

VASARI fea-

ture

Main involved classes Main involved relations

F1:lesion lo-

cation

vasari:CerebralTumorEpicenter,

fma:LobeOfCerebralHemisphere,

fma:Brainstem, fma:Cerebellum

bfo:located in at some time

fma:regional part of

F2:lesion side vasari:CerebralTumorEpicenter,

fma:CerebralHemisphere,

fma:MedianSagittalPlane

bfo:located in at some time

F4:enhancem-

ent quality

pato:Volume,

vasari:CerebralTumorComponent,

vasari:VolumeMeasurementDatum,

obi:ValueSpeciﬁcation

iao:is quality measurement of,

bfo:is speciﬁed input of,

bfo:is speciﬁed output of,

bfo:is about

F5: propor-

tion nCET

pato:Volume,

vasari:CerebralTumorComponent,

vasari:VolumeMeasurementDatum,

obi:ValueSpeciﬁcation

iao:is quality measurement of,

bfo:is speciﬁed input of,

bfo:is speciﬁed output of,

bfo:is about

F20:cortical

involvement

vasari:CerebralTumor,

fma:CerebralCortex

vasari:overlaps ro:adjacent to

F26:extent

of resection

of enhancing

tumor

pato:Volume,

vasari:VolumeMeasurementDatum,

uo:VolumeUnit,

vasari:RatioValueSpeciﬁcation,

vasari:ProportionOfEnhancingAndRemo-

vedTumor,

vasari:EnhancingCerebralTumor-

Component,

vasari:ResectionOfEnhancingCerebralTu-

morComponent

iao:is quality measurement of,

bfo:has measurement unit label,

iao:has measurement value,

bfo:is about,

obi:is speciﬁed input of,

obi:is speciﬁed output of,

vasari:has speciﬁed denominator -

value,

vasari:has speciﬁed nominator -

value, ro:derives from

F29 and F30:

lesion size

obi:LengthMeasurementDatum,

uo:LengthUnit, pato:Quality

iao:is quality meausrement of,

iao:has measurement value,

iao:has measurement unit label,

iao:is about

ontology, thus we think that it will be interesting to

extend it to cover this kind of information that is nee-

ded in the deﬁnition of biomedical experiments.

It is important to note that our approach is not li-

mited to the translation of VASARI features resulting

from the subjective assessment of MRI images by hu-

man neuro-radiologists. In contrast, it would express

its full value if it were implemented as a complement

to an automated or semi-automated image analysis sy-

stem (Velazquez et al., 2015). For example systems

that are described in these papers (Rubin et al., 2014b;

Porz et al., 2014) enable to segment the various parts

of the tumor, and to automatically determine their

anatomical environment (e.g. what anatomical struc-

tures they are contained in, or they overlap or they are

adjacent to). Such mereotopological properties could

be directly translated in semantic form using the rela-

tionship discussed above. Similarly, the volume me-

asurements and the derived proportions could be ge-

nerated automatically and with a better accuracy than

through the current subjective assessments. VASARI

features could be derived from the detailed image-

based observations and measurements rather precede

them, and our model provides the conceptual basis to

make such enrichment of image processing systems.

The Radiology Reading Room of the Future (Gil-

lies et al., 2015) will entail a reading room where in

practicing radiologists interact with picture archiving

and communication system software to identify, seg-

ment, and extract features from regions of interest. If

prior studies obtained in the same patient are availa-

ble, the previous regions of interest will be automa-

tically identiﬁed by the reading software. As part of

the reading, the extracted size, shape, location, and

textural features will be automatically uploaded to a

shared database and algorithmically compared with

prior images to enable more precise diagnoses.

The ﬁrst limit of our proposal is that it is imple-

mented in OWL and thus it does not generate tempo-

ralized instances (Smith et al., 2006). We think that

Semantic Representation of Neuroimaging Observations: Proof of Concept based on the VASARI Terminology

Figure 4: Example of a validation output: detection of a logical contradiction, IAO 000004 denotes the relation

iao:has measurement value and paldmd 900 00 1961 refers to the measurement value of the perpendicular longest axis

of the patient 900 00 1961.

taking into consideration the temporal aspect in the

representation of neuroimaging features is needed es-

pecially in longitudinal imaging studies to, for exam-

ple, evaluate cancer treatment response. In this con-

text, we recommend to select a logic-based language

that is capable to represent n-ary relationships. The

second limit is intrinsic to the problem of logical con-

tradiction that is due to the fact that radiologists des-

cribe what they observe based on their thoughts and

experiences. As a consequence they may describe

differently the reality and produce different clinical

records about the identiﬁed entities. To resolve the

disagreement in interpretation many medical systems

require preserving in data entry a single correct value

for the evaluated feature to facilitate the aggregation

of data. According to Rector et al., electronic medi-

cal records should allow the presence of conﬂicting

statements, multiple measurements, etc. to faithfully

reﬂect the reality of clinical practice. Thus, they pro-

pose three types of foundational information models

to describe (a) the medical records, (b) the state of

the patient and (c) clinical care (Rector et al., 1991)

and they consider that a meta-language should be used

to separate between what can be said and what actu-

ally occurs, and avoid the problem of inconsistency.

Smith et al. have mentioned that even the adoption of

a meta-language cannot remove errors because medi-

cal dialogues are also subject to error (Smith et al.,

2006). In our work, we have stored (a) observations

in separate data sets and we have not included (b and

c) the meta-observation level.

The experimental work regarding the VASARI on-

tology shows that the semantic representation of neu-

roimaging features can enhance search operations;

e.g. the exploitation of the VASARI ontology with the

DL reasoner can resort the list of cerebral tumor epi-

centers that are located in the right temporal lobe by

retrieving cerebral tumor epicenters that are located in

the temporal lobe and the right hemisphere. Certainly,

such classiﬁcation task cannot be obtained with non

semantic representations such as DICOM SR and the

AIM model. We believe that our work can be reu-

sed in other image-based reasoning contexts as for

example, RECIST criteria that base the tumor clas-

siﬁcation task (i.e. measurable and non-measurable

lesions) on the knowledge of the location of the le-

sion and the calculation of its length. Added to the

reasoning task, the consistency checking functiona-

lity offered by OWL reasoners can detect inconsistent

statements that can be caused by inappropriate or er-

roneous diagnosis or treatment, in the clinical context,

until now DICOM SR and the AIM model do not offer

this semantic capability.

5 CONCLUSION

We believe that neuroimaging data should be held in

a structured format that makes their meanings explicit

to the systems and thus facilitate their comprehension

as well as management. Semantic data about imaging

features (measurement values, qualities, lesion com-

ponents, lesion localization, etc.) are important (1)

KEOD 2018 - 10th International Conference on Knowledge Engineering and Ontology Development

to support the clinical research on the development of

new imaging biomarkers by combining clinical data

with information coming from different medical dom-

ains (2) to improve the quality of the clinical healt-

hcare that tend to provide personalized treatments to

patients via the use of clinical guidelines that are ba-

sed on evaluation criteria. In this paper, we employed

the VASARI terminology as a proof of concept for the

demonstration of the feasibility and the importance of

making RDF and OWL data available to describe ce-

rebral tumors observations and determining the key

concepts and relationships that are central in their eva-

luation. Our work can be easily expanded to answer

to other use cases; thanks to the modular aspect of

the ontology and to the OWL language that is self-

descriptive (concepts are textually and formally des-

cribed in the ontology to guide users) and extendable.

REFERENCES

Amdouni, E. and Gibaud, B. (2016). Concept-based ver-

sus realism-based approach to represent neuroima-

ging observations. In Keod: Proceedings of The

8th International Joint Conference On Knowledge

Discovery, Knowledge Engineering and Knowledge

Management-Vol. 2, pages 179–185.

Bennett, B., Chaudhri, V., and Dinesh, N. (2013). A voca-

bulary of topological and containment relations for a

practical biological ontology. In International Confe-

rence on Spatial Information Theory, pages 418–437.

Springer.

Bittner, T. and Smith, B. (2004). Normalizing medical on-

tologies using basic formal ontology.

Brinkman, R., Courtot, M., Derom, D., Fostel, J., He, Y.,

Lord, P., Malone, J., Parkinson, H., Peters, B., Rocca-

Serra, P., et al. (2010). Modeling biomedical experi-

mental processes with OBI. J. Biomedical Semantics,

1(S-1):S7.

Ceusters, W. (2012). An information artifact ontology per-

spective on data collections and associated represen-

tational artifacts. In MIE, pages 68–72.

Ceusters, W., Elkin, P., and Smith, B. (2006). Referent

tracking: The problem of negative ﬁndings. Studies

in health technology and informatics, 124:741.

Ceusters, W. and Smith, B. (2005). Tracking referents in

electronic health records. Studies in health technology

and informatics, 116:71.

Channin, D., Mongkolwat, P., Kleper, V., Sepukar, K.,

and Rubin, D. (2010). The caBIG annotation and

image markup project. Journal of digital imaging,

23(2):217–225.

Cimino, J. (2006). In defense of the desiderata. Journal of

biomedical informatics, 39(3):299–306.

Clunie, D. (2000). DICOM structured reporting. PixelMed

Publishing.

Clunie, D. (2007). DICOM structured reporting and cancer

clinical trials results. Cancer informatics, 4:33.

Eisenhauer, E., Therasse, P., B. J., Schwartz, L., Sargent,

D., Ford, R., Dancey, J., Arbuck, S., Gwyther, S., M.,

M., et al. (2009). New response evaluation criteria in

solid tumours: revised recist guideline (version 1.1).

European journal of cancer, 45(2):228–247.

ESR (2011). Medical imaging in personalised medicine:

a white paper of the research committee of the euro-

pean society of radiology (esr). Insights into imaging,

2(6):621–630.

Gillies, R., Kinahan, P., and Hricak, H. (2015). Radiomics:

images are more than pictures, they are data. Radio-

logy, 278(2):563–577.

Gkoutos, G., Schoﬁeld, P., and Hoehndorf, R. (2012). The

units ontology: a tool for integrating units of measu-

rement in science. Database, 2012:bas033.

Grams, A., Gempt, J., Ringel, F., Soehngen, E., Astner, S.,

Schlegel, J., Meyer, B., Zimmer, C., and Forschler,

A. (2014). Multimodal imaging to delineate tumor

heterogeneity in cerebral gliomas. Open Journal of

Radiology, 4(02):182.

Gruber, T. (1993). A translation approach to portable onto-

logy speciﬁcations. Knowledge acquisition, 5(2):199–

220.

Gutman, D., Cooper, L., Hwang, S., Holder, C., Gao, J.,

Aurora, T., Dunn Jr, W., Scarpace, L., Mikkelsen,

T., Jain, R., et al. (2013). Mr imaging predictors

of molecular proﬁle and survival: multi-institutional

study of the tcga glioblastoma data set. Radiology,

267(2):560–569.

Kahn J.R., C., Langlotz, C., Channin, D., and Rubin, D.

(2011). Informatics in radiology: an information mo-

del of the dicom standard. Radiographics, 31(1):295–

304.

Levy, M., Freymann, J., Kirby, J., Fedorov, A., Fennessy,

F., Eschrich, S., Berglund, A., Fenstermacher, D., Tan,

Y., G. X., et al. (2012). Informatics methods to enable

sharing of quantitative imaging research data. Magne-

tic resonance imaging, 30(9):1249–1256.

Levy, M., OConnor, M., and Rubin, D. (2009). Seman-

tic reasoning with image annotations for tumor asses-

sment. In AMIA Annual Symposium Proceedings, vo-

lume 2009, page 359. American Medical Informatics

Association.

Madhavan, S., Zenklusen, J., Kotliarov, Y., Sahni, H., Fine,

H., and Buetow, K. (2009). Rembrandt: helping per-

sonalized medicine become a reality through integra-

tive translational research. Molecular cancer rese-

arch, 7(2):157–167.

oller, M., Regel, S., and Sintek, M. (2009). Radsem: Se-

mantic annotation and retrieval for medical images.

In European Semantic Web Conference, pages 21–35.

Springer.

Mungall, C., Gkoutos, G., Washington, N., and Lewis, S.

(2007). Representing phenotypes in OWL. In Pro-

ceedings of the OWLED 2007 Workshop on OWL: Ex-

perience and Directions. Innsbruck, Austria Edited by

Golbreich C, Kalyanpur A, Parsia B.

Oberkampf, H., Zillner, S., Bauer, B., and Hammon, M.

(2012). Interpreting patient data using medical back-

ground knowledge. ICBO, 897:3.

Semantic Representation of Neuroimaging Observations: Proof of Concept based on the VASARI Terminology

Porz, N., Bauer, S., Pica, A., Schucht, P., Beck, J., Verma,

R., Slotboom, J., Reyes, M., and Wiest, R. (2014).

Multi-modal glioblastoma segmentation: man versus

machine. PloS one, 9(5):e96873.

Rector, A.L., N. W. K. S. et al. (1991). Foundations

for an electronic medical record. Methods Inf Med,

30(3):179–186.

Rubin, D., Mongkolwat, P., and Channin, D. (2009). A se-

mantic image annotation model to enable integrative

translational research. Summit on translational bioin-

formatics, 2009:106.

Rubin, D., Willrett, D., O’Connor, M., Hage, C., Kurtz, C.,

and Moreira, D. (2014a). Automated tracking of quan-

titative assessments of tumor burden in clinical trials.

Translational oncology, 7(1):23–35.

Rubin, D., Willrett, D., O’Connor, M., Hage, C., Kurtz,

C., and Moreira, D. (2014b). Automated tracking of

quantitative assessments of tumor burden in clinical

trials. Translational oncology, 7(1):23.

Scheuermann, R., Ceusters, W., and Smith, B. (2009).

Toward an ontological treatment of disease and di-

agnosis. Summit on translational bioinformatics,

2009:116.

Seifert, S., Kelm, M., Moeller, M., Mukherjee, S., Caval-

laro, A., Huber, M., and Comaniciu, D. (2010). Se-

mantic annotation of medical images. In SPIE medical

imaging, pages 762808–762808. International Society

for Optics and Photonics.

Sirin, E., Parsia, B., Grau, B., Kalyanpur, A., and Katz, Y.

(2007). Pellet: A practical owl-dl reasoner. Web Se-

mantics: science, services and agents on the World

Wide Web, 5(2):51–53.

Smith, B. (2006). From concepts to clinical reality: an es-

say on the benchmarking of biomedical terminologies.

Journal of biomedical informatics, 39(3):288–298.

Smith, B., Ashburner, M., Rosse, C., Bard, J., Bug, W.,

Ceusters, W., Goldberg, L., Eilbeck, K., Ireland, A.,

Mungall, C., et al. (2007). The OBO Foundry: coor-

dinated evolution of ontologies to support biomedical

data integration. Nature biotechnology, 25(11):1251–

1255.

Smith, B. and Ceusters, W. (2010). Ontological realism:

A methodology for coordinated evolution of scientiﬁc

ontologies. Applied ontology, 5(3-4):139–188.

Smith, B., Ceusters, W., Klagges, B., K

ohler, J., Kumar, A.,

Lomax, J., Mungall, C., Neuhaus, F., Rector, A., and

Rosse, C. (2005a). Relations in biomedical ontolo-

gies. Genome biology, 6(5):R46.

Smith, B., Kumar, A., and Bittner, T. (2005b). Basic formal

ontology for bioinformatics. Journal of Information

Systems, pages 1–16.

Smith, B., Kusnierczyk, W., Schober, D., and Ceusters, W.

(2006). Towards a reference terminology for ontology

research and development in the biomedical domain.

In KR-MED, volume 2006, pages 57–66.

Tudorache, T., Nyulas, C., Noy, N., and Musen, M. (2013).

Webprot

e: A collaborative ontology editor and kno-

wledge acquisition tool for the web. Semantic web,

4(1):89–99.

Van Soest, J., Lustberg, T., Grittner, D., Marshall, M.,

Persoon, L., Nijsten, B., Feltens, P., and Dekker, A.

(2014). Towards a semantic pacs: using semantic web

technology to represent imaging data. Studies in he-

alth technology and informatics, 205:166.

Velazquez, E., Meier, R., Dunn Jr, W., Alexander, B., Wiest,

R., Bauer, S., Gutman, D., Reyes, M., and Aerts, H.

(2015). Fully automatic gbm segmentation in the tcga-

gbm dataset: Prognosis and correlation with vasari fe-

atures. Scientiﬁc reports, 5.

KEOD 2018 - 10th International Conference on Knowledge Engineering and Ontology Development