A Semantic Frame Graph for Information Extraction

Michał Gałusza

Faculty of Cybernetics, Military University of Technology, Warsaw, Poland

Keywords:

Information Extraction, Relationship Detection, Natural Language Processing, Graphical Representation of

Text.

Abstract:

The following paper describes a graphical representation of a short text based on the semantic frames (the

Semantic Frame Graph) using Semantic Role Labeling (SRL). It can be a foundation of alternative approach

for open information extraction (OIE). The approach postprocesses the output of pretrained SRL classiﬁer

and it does not use complex rules, training sets nor signiﬁcant corpus to decompose sentences. Proposed de-

composition and representation reduces number of paths between entities dropping ones that are linguistically

unmotivated, generates sequences of frames as paths which can be controlled using dialog coherence approach

which further increases plausibility of semantic relationship between entities.

1 INTRODUCTION

Information Extraction (IE) is a process of converting

unstructured information held in text into a structured,

for example, graphical representation as in knowledge

graphs (KG). The KGs represent information as typed

relations between entities, which are fundamental for

developing methods that have the potential for sophis-

ticated reasoning (Ji, 2021).

Open Information Extraction (OIE) paradigm

aims to extract all possible semantic relationships be-

tween entities available regardless of the genre of text

(ﬁnancial, medical or technical). This is achieved

without dedicated, supervised training of anticipated

relationships. OIE systems, however, are not fully

unsupervised. They use a collection of patterns,

seeding samples, predeﬁned heuristics, scoring func-

tions and distant supervision (Fader, 2011),(Banko,

2007),(M. Schmitz, 2012) to search for relationships.

Patterns and heuristics need to be calibrated there-

fore large corpus and signiﬁcant seed samples are re-

quired. However, OIE systems (Fader, 2011),(Banko,

2007),(M. Schmitz, 2012) were created to efﬁciently

parse large web resources and automatically detect

all existing semantic relations. Efﬁcient OIE system

must possess following features (Niklaus, 2018):

• automation: relationships must be extracted with-

out any prior training of them in unsupervised

manner;

• corpus heterogeneity: relationships must be ex-

tracted in various genres of text;

• efﬁciency: extraction shall be possible in large

web corpus.

Despite their success, current OIE systems are still

faced with ﬁve main challenges pertaining to relation-

ship extractions (Niklaus, 2018):

• overlaps: where same sentence fragment may

generate multiple triples (subject, relation, ob-

ject), for example, because of shared subject;

• discontinuation: where a text spans supporting the

triples are is separated by an interval i.e. distant

direct object;

• nesting: where one relation contains other relation

as in complex sentences;

• distribution: where subject, relation and direct ob-

ject mentions break a sentence and they are in dif-

ferent contexts. It is natural for a human to iden-

tify such cases, however, this still posses a chal-

lenge for current extraction and classiﬁcation so-

lutions;

• reliability: in general they do not return meaning-

ful results on too small corpus

This paper describes a method to represent a short

texts as a Semantic Frame Graph, which will:

• provide a method to limit negative impact of over-

laps, discontinuation, nesting and especially dis-

tribution in relationship extraction;

• allow identifying relationships in short texts;

• fulﬁll automation and corpus heterogeneity re-

quirement;

Gałusza, M.

A Semantic Frame Graph for Information Extraction.

DOI: 10.5220/0011729300003411

In Proceedings of the 12th International Conference on Pattern Recognition Applications and Methods (ICPRAM 2023), pages 435-442

ISBN: 978-989-758-626-2; ISSN: 2184-4313

 2023 by SCITEPRESS – Science and Technology Publications, Lda. Under CC license (CC BY-NC-ND 4.0)

435

• use Semantic Role Labeling to pre-process a text

instead of complex dependency parsing;

• compare relation extraction efﬁciency using dia-

log coherency approach.

2 RELATED WORK

2.1 Open Information Extraction

The ﬁrst solution formally introducing OIE was Tex-

tRunner (Banko, 2007). Its primary goal was to obvi-

ate manual adjustment of the extraction rules in case

of corpora or a target relation shift. TextRunner was

making a single pass over the corpus heuristically ex-

tracting relations triple centered around a verb phrase

(VP) with a speciﬁc subject and direct object. It at-

tempted to ﬁnd the best arguments for that relation

applying additional heuristics e.g. neither head nor

target consist solely of a pronoun. A Naive Bayes

was used as an estimator of a conﬁdence function that

was then trained over a set of features on the extrac-

tions so that the system could provide calibrated con-

ﬁdence values. Comparing to the last pre OIE solu-

tion, KnowitAll, quality of extraction and its perfor-

mance signiﬁcantly improved. Apart from that in Kn-

woItAll relations had to be speciﬁed upfront (Oren Et-

zioni, 2005). ReVerb (Fader, 2011) introduced addi-

tional syntactic and lexical constraints to limit inco-

herent and uninformative extractions of TextRunner

in detecting distant relationships. For example: ”The

Obama administration is offering only modest green-

house gas reduction targets at the conference.” would

yield a relation ”is offering only modest greenhouse

gas reduction targets at” between ”The Obama ad-

ministration” and ”the conference”.

Fast dependency parsers and their ability to cre-

ate a sentence Dependency Tree (DT) allowed con-

struction of more sophisticated templates that further

increased precision and recall of extraction. OLLIE

(M. Schmitz, 2012) deﬁned ”Open Relation Patterns”,

which, using a dependency tree, were mediated by

nouns and adjectives, not just verbs. OLLIE’s pro-

cessing began with seed tuples from REVERB and

used them to build a bootstrap training set. It learned

open pattern templates applied to individual sentences

at extraction time.

The latest advancement in OIE is related to the lat-

est progress in language modeling. The Transformer

architecture leads to a novel paradigm of Neural Open

Information Extraction (NOIE), (Zhou, 2022). NOIE

approaches extractions from two major directions:

Tagging and Generation.

Tagging-based solutions use annotation of tags’

sequence corresponding to facts in the input sentence.

The Generation-based ones directly decode relations

relying on Sequence2Sequence architecture. Both

tagging and generation paradigms predicts relation-

ships auto-repressively, which means the current pre-

diction relies on the previous output (Zhou, 2022). A

skewed prediction will be inherited and magniﬁed in

the later steps. As the number of steps grows, errors

accumulate and may decrease the performance.

An exemplary graph-based approach (Yu, 2021)

breaks the auto-regressive factorization by construct-

ing a graph where nodes are text spans and edges con-

necting them indicate that they belong to the same

fact. Relationship discovery task is cast as maximal

clique detection.

2.2 Semantic Role Labeling

Frame Semantics was originally introduced by

Charles J. Fillmore (Charles, 1977) with the basic

idea that one cannot understand the meaning of a sin-

gle word without access to all the essential knowledge

related to that word, namely, its semantic frame. The

semantic frame is strictly associated with the word’s

meaning expressed in the sentence. Semantic Role

Labelling (SRL) (D.Gildea, 2000) (D.Jurafsky, 2022)

identiﬁes and models frame’s structure. SRL takes

a sentence and identiﬁes verbs and their arguments.

Then, it classiﬁes the arguments by mapping them to

roles relevant to the verb in that frame, such as agent,

patient, instrument, or benefactor. In other words,

SRL tries to identify ”Who, What, Where, When,

With What, Why, How” for each frame. A state-of-

the-art deep pre-trained SRL model (Peng Shi, 2019)

detects the simpliﬁed structure of a frame where in-

stead of an agent, a patient, an instrument it detects

generic simpliﬁed arguments of a verb: ARG0, ARG1

and others. The structure of the frame highly corre-

lates with the dependency tree (DT) of the sentence

(T.Shi and O.Irsoy, 2020), where the verb and verb’s

arguments create constituencies (noun phrases NPs

and verb phrases VPs). Moreover, it is possible to

reduce the SRL task to a Dependency Parsing task

(T.Shi and O.Irsoy, 2020) . In addition, SRL offers an

efﬁcient approach to the problem of the decomposi-

tion of complex sentences which was initially solved

by a trained, dedicated, classiﬁer splitting a sentence

into shorter utterances (Angeli, 2019).

2.3 Knowledge Graphs

Constructing an ontology from text is challenging

due complexity of human language. Initial approach

ICPRAM 2023 - 12th International Conference on Pattern Recognition Applications and Methods

436

to construct a knowledge graph relied on syntactical

parsing for terms, synonyms, concepts, relationships

between them, their hierarchies. On top of them a set

of axioms were created (or inferred from text) to be

a set of logical implications constraining the interpre-

tation of concepts and relations therefore governing

inference (D. Maynard, 2017), (Cimiano, 2006).

A semantic frame serves two purposes: it is a

means to abstract a cognitive schemata and it is this

schemata computational counterpart (A. Gangemi,

2010). Structure of semantic frame naturally identi-

ﬁes objects in the sentence (as they are frame’s ar-

guments) and relationship between them (as it is a

verb) (M. Alam, 2021). There are two limitations to

that approach. First, neither concepts nor their hier-

archies can be automatically detected. Second, con-

necting frames needs to be contextualized and sup-

ported by the text, meaning that connections mediated

by co-occuring terms often require additional valida-

tion to make sure they form a logical ﬂow. For exam-

ple sentences: ”Astra Zeneca was ﬁrst to develop a

Covid19 vaccine. Covid19 was a serious threat to a

global health in 2020 and 2021” may generate a rela-

tionship between Astra Zeneca and threat to a global

health through Covid19. Therefore a situation needs

to be reconstructured from original text in order to

validate the relationship (M. Alam, 2021). This pa-

per addressed this problem speciﬁcally using dialog

coherence approach to measure quality of connection

between frames.

3 SEMANTIC FRAME GRAPH

Formally, a Semantic Frame Graph is an undirected,

attributed, heterogeneous graph

G = (V, E)

where:

• V is a set of nodes of following types:

– Noun : nouns detected in a frame.

– Argument : a span of text describing semantic

role of a frame

– Frame : verb identifying a frame.

• E is a set of edges that represents a speciﬁc seman-

tic role type: ARG0, ARG1, ...,. An edge ’VERB’

is used if argument is further split into a frame; an

’NOUN’ edge links nouns in the argument in case

there is no more frames.

The graph is constructed by applying SRL identiﬁca-

tion (Peng Shi, 2019) on every sentence in the corpus.

A general structure of the graph is depicted in Figure

Figure 1: Semantic Frame Graph Structure.

A Semantic Frame Graph decomposes a sentence

into a hierarchical structure of its frames.

SRL groups words per detected structure of the

frame and yields a structure that correctly segments a

sentence, solving, for example, the TextRunner’s dis-

tant relationship issue (Figure 2). It does it without

implementing any additional constraints and correctly

detects that ”conference” is a location, not a direct

object of the verb: ”offer” as in exemplary sentence:

”The Obama administration is offering only modest

greenhouse gas reduction targets at the conference.”

A corresponding SFG captures the structure detected

by SRL (Figure 3).

Figure 2: Example Semantic Decomposition.

Figure 3: SFG Capturing Semantic Decomposition.

SFG decomposes complex sentences and in case

of overlaps and discontinuations properly linking re-

mote subjects and objects. Frames are always fully

deﬁned in its direct neighborhood which means that

verbs without arguments are dropped. Comparing to

the initial attempt (Angeli, 2019), where this task was

cast as a linguistically driven search problem over a

sentence DT, SFG relies in the SRL decomposition.

For example, a sentence (Angeli, 2019): ”Born in a

small town, she took the midnight train going any-

where” is parsed to SFG (ﬁgure 4):

A Semantic Frame Graph for Information Extraction

437

Figure 4: SFG Complex Sentence Decomposition.

The direct neighborhood of verbs creates com-

plete frames, which are similar to a target shorter

utterances (Angeli, 2019). It captures directly ex-

pressed relationships as: ”she born in a small town”,

”she took the midnight train going anywhere”, ”she

going anywhere”. Although the third utterance does

make sense, it is incorrect from DT parsing perspec-

tive, I will discuss it in summary. Sentence decompo-

sition correctly identiﬁes ”she” as a shared subject for

the ﬁrst and the second utterances and skipping ”took

the midnight train” completely for the third utterance

which separates subject with the verb.

The SFG captures a hierarchy of frames as in a

sentence ”A water landing of a jetliner that lost both

engines due to hitting birds became known as the Mir-

acle on the Hudson River” (Figure 5)

Figure 5: SFG Hierarchy of Frames.

SFG allows modeling distant relationships in the

text that beyond single sentence. It allows linking

mentions in the text that are outside of deﬁned con-

text window. The following example takes sentences

describing a speciﬁc type of jet fuel. ”An airplane

uses engines for ﬂying. ATF is a type of aviation fuel

designed for use in aircraft powered gas-turbine en-

gines. (...) If these supercooled droplets collide with a

surface they can freeze and may result in blocked fuel

inlet pipes.”

The relationship between entities (nouns) exists if

there is a path in SFG joining them. A path of seman-

tic frames joining them will create a passage that will

deﬁne a relationship that can be classiﬁed.

For example a path between water and an engine:

”water” → ”supercooled water droplets collide with

a surface” → ”if supercooled water droplets collide

with a surface they may result in blocked fuel inlet

pipes” → ”blocked fuel inlet pipes” → ”aviation fuel

designed for use in aircraft powered by gas turbine

engines” → ”an airplane uses engines for ﬂying” →

”engine”

The generated path may indicate that ”water” im-

pacts ”engine” that may also impact the safety of

”ﬂight”

4 SHORT TEXT

REPRESENTATION

In this section will compare an SFG graph with a Sen-

tence Graph (SG) which does not perform decom-

position. I will use a short text (12 sentences) de-

scribing news excerpt on Iraq War as in (D.Radev,

2004). The SG approach (D.Radev, 2004) constructs

a weighted graph of sentences where connections are

deﬁned by co-occurring nouns and connection weight

equals the similarity between two sentences deﬁned

by idf-modiﬁed-cosine. The threshold on the simi-

larity manages connectivity of the graphs and hence

quality of the paths.

A Table (1) summarizes structure. A sample con-

text, around noun ”Baghdad” is provided in Figures

(7) and (8). An SFG represents semantics of the text

in more compact manner and even without any thresh-

olds it can reduce number of paths almost 76 times.

In general, the problem of detecting a relationship be-

tween entities can be cast as ﬁnding a path between

them. Structure of the graph impacts the performance

of relationship extraction.

Table 1: Graphs’ structure comparison.

metric SG SFG

number of nodes 144 194

number of edges 193 219

number of noun nodes 76 76

number of verb nodes 57 60

number of paths 3901764 51040

A centrality (PageRank) of words (Table 2) shows

that 8 out of 10 top words are preserved in an SFG

representation so there is insigniﬁcant semantic drift

between both representations.

ICPRAM 2023 - 12th International Conference on Pattern Recognition Applications and Methods

438

Figure 6: SFG Distant Relationships.

4.1 Coherence of Paths and

Relationship Extraction

SFG represents frames which are singular sentences.

Therefore a path between any entities is a sequence of

generated singular sentences. Such a sequence can be

measured for its coherence.

The SFG represents a set of N frames

F : { f

, f

, ..., f

}

if the similarity of frames has the property that

s : ∀ f

, f

∈ F, 0 ≤ s( f

, f

) ≤ 1 (1)

then the coherence c of a path p containing K frames

I deﬁne as the minimum similarity of the neighboring

frames in the path:

: min

0≤ j<K

s( f

, f

j+1

) (2)

There are various realizations of function (1)

available: sentence similarity (N. Reimers, 2019),

textual entailment (Poliak, 2020).

I update (B. Grosz, 1995) approach to meet re-

quirements of function (1) and I replace a direct

noun expressions to evaluate continuation retaining

and shifting with a semantic approach.

I use the fact that frame’s subject and object (cen-

ters) are decomposed into ARG0 and ARG1 roles

A Semantic Frame Graph for Information Extraction

439

Figure 7: A ”Baghdad” SG Context.

Table 2: Top 10 key words comparison.

pageranked position sentence frame

1 Iraq Iraq

2 United United

3 Nations today

4 today Baghdad

5 Prime Prime

6 Baghdad inspector

7 Minister Minister

8 inspector Nations

9 Blair year

10 Ivanov work

and I update the original centering approach using

zero-shot label classiﬁcation. Instead of continua-

tion/retention/shift signals I evaluate cosine distance

between classiﬁcation of backward centers of current

frame vs. previous.

The Wikipedia’s page for Teton Dam http://en.

wikipedia.org/wiki/Teton Dam is used as an example

to identity if selected object (a storage tank) posed a

risk and what kind of risk to another object (a city) in

the event of Teton Dam collapse.

Percentage of coherent path give minimal thresh-

old (Figure 9) shows that RTE immediate drop in-

dicating that frames are rather not each other entail-

ments and coherent passage will have low coherence

score making it difﬁcult to ﬁnd.

Best paths with best coherence score for each eval-

uation method (tab. 3) show that modiﬁed centering

approach on the SFG linked all related sentences from

original text to form a passage conﬁrming that there

is a semantic relationship between a storage tank and

a city.

Table 3: Best paths for coherence types.

path c p type

Dozens of logs hit a bulk

gasoline storage tank a few

hundred yards away →

When the ﬂood waters hit

thousands of logs washed into

town →

When the ﬂood waters hit →

The city of Idaho Falls even

further down on the ﬂood

plain had time to prepare

0.52

semantic

similarity

Dozens of logs hit a bulk

gasoline storage tank a few

hundred yards away →

The gasoline sent ﬂaming

slicks adrift on the racing wa-

ter →

When the ﬂood waters hit →

The city of Idaho Falls even

further down on the ﬂood

plain had time to prepare

0.21 RTE

Dozens of logs hit a bulk

gasoline storage tank a few

hundred yards away →

The gasoline sent ﬂaming

slicks adrift on the racing wa-

ter →

When the ﬂood waters hit

thousands of logs washed into

town →

The force of the logs and

cut lumber and the subsequent

ﬁres practically destroyed the

city

0.85 centering

ICPRAM 2023 - 12th International Conference on Pattern Recognition Applications and Methods

440

Figure 8: A ”Baghdad” SFG Context.

Figure 9: Fraction of coherent paths given threshold.

5 CONCLUSION

A Semantic Frame Graph is an alternative represen-

tation of text. It uses SRL as pre-processing step

to identify the structure of frames. A whole set of

frames is loaded into a graph that is used as a foun-

dation for identifying relationships between selected

words. It links mentions distant in text, even across

sentences. It does not use a complex rule-based ap-

proach that requires bootstrapping nor signiﬁcant cor-

pus to validate them as in early OIE solutions. Even

without any additional edges’ weighting and thresh-

olds on them, it shows signiﬁcant reduction in number

of paths between entities so further classiﬁcation of

them to proper relationships will require less compu-

tation. Modiﬁed centering approach measures over-

all coherence of paths between entities and thresholds

enables control of the quality of the connection that

can be classiﬁed for its type afterwards.

A few issues can be solved to increase text rep-

resentation quality further. Current state-of-the-art

SRL parser (Peng Shi, 2019) can be augmented with

additional post-processing steps to correct the align-

ment of SRL tags with a sentence’s dependency tree.

In exemplary sentence (Figure 5), for detected frame

”she going nowhere” a subject is ”train” which is

visible in its dependency tree. A progress in a co-

reference resolution will further increase the quality

of the graph not only because pronouns will be prop-

erly replaced by nouns they refer to, but also descrip-

tive expressions like ”this event”, ”in this case” will

properly link the frame with a noun or noun phrase

mention in the text.

REFERENCES

A. Gangemi, V. P. (2010). Towards a pattern science for the

semantic web. In Semantic Web Journal. IOS Press.

Angeli, G. (2019). Leveraging linguistic structure for open

domain information extraction. In Proceedings of the

Association of Computational Linguistics. ACL.

B. Grosz, A. Joshi, S. W. (1995). Centering a framework for

modeling the local coherence of discourse. In Compu-

tational Linguistics. MIT Press.

Banko, M. (2007). Textrunner: Open information extraction

on the web. In ACL-HLT.

Charles, F. (1977). Scenes-and-frames semantics. In In A.

Zampolli, ed. Linguistic Structures Processing.

Cimiano, P. (2006). Ontology learning and population from

text algorithms, evaluation and applications. Springer.

A Semantic Frame Graph for Information Extraction

441

D. Maynard, K. Bontcheva, I. A. (2017). Natural language

processing for the semantic web. In SYNTHESIS LEC-

TURES ON THE SEMANTIC WEB: THEORY AND

TECHNOLOGY #15. Morgan & Claypool.

D.Gildea, D. (2000). Automatic labeling of semantic roles.

In Association for Computational Linguistics. ACL.

D.Jurafsky, J. (2022). Speech and Language Processing.

DRAFT, Stanford, 2nd edition.

D.Radev, G. (2004). Lexrank: Graph-based lexical central-

ity as salience in text summarization. In Journal of

Artiﬁcial Intelligence Research. ACL.

Fader, Soderland, E. (2011). Identifying relations for open

information extraction. In ACL.

Ji, S. (2021). A survey on knowledge graphs: Represen-

tation, acquisition and applications. In IEEE Trans-

actions on Neural Networks and Learning Systems.

IEEE.

M. Alam, A. G. (2021). Semantic role labeling for knowl-

edge graph extraction from text. In Progress in Artiﬁ-

cial Intelligence.

M. Schmitz, S. S. (2012). Open language learning for in-

formation extraction. In ACL.

N. Reimers, I. G. (2019). Sentence-bert: Sentence em-

beddings using siamese bert-networks. In EMNLP.

EMNLP.

Niklaus, C. (2018). A survey on open information extrac-

tion. In 27th International Conference on Computa-

tional Linguistics. ACL.

Oren Etzioni, A. Y. (2005). Unsupervised named-entity ex-

traction from the web: An experimental study. In Ar-

tiﬁcial Intelligence.

Peng Shi, J. J. L. (2019). Simple bert models for relation

extraction and semantic role labeling. In ArXiv. AL-

LENNLP.

Poliak, A. (2020). A survey on recognizing textual entail-

ment as an nlp evaluation. In EMNLP. EMNLP.

T.Shi, I. and O.Irsoy (2020). Semantic role labeling as syn-

tactic dependency parsing. In ACL.

Yu, B. (2021). Maximal clique based non-autoregressive

open information extraction. In EMNLP.

Zhou, S. (2022). Survey on neural open information extrac-

tion: Current status and future directions. In IJCA.

ICPRAM 2023 - 12th International Conference on Pattern Recognition Applications and Methods

442