Incremental Bidirectional Transformations: Evaluating Declarative and

Imperative Approaches using the AST2Dag Benchmark

Matthias Bank, Sebastian Kaske, Thomas Buchmann and Bernhard Westfechtel

Chair of Applied Computer Science I, University of Bayreuth, Universit

atsstrasse 30, 95440 Bayreuth, Germany

Keywords:

Model-Driven Development, Model Transformations, Bidirectional Transformations, Incremental

Transformations, BX, Benchmark, Evaluation, QVT-R, BXtend.

Abstract:

Model transformation are the core of model-driven software engineering. Typically an initial model is reﬁned

throughout the development process using model transformations to derive subsequent models until eventually

code is generated. In round-trip engineering processes, these model transformations are performed not only

in forward, but also in backward direction. To this end, bidirectional transformation languages provide a

single transformation deﬁnition for both directions. This paper evaluates the transformation languages QVT

Relations (QVT-R) which allows to specify incremental bidirectional transformations declaratively at a high

level of abstraction and BXtend - a framework for procedural speciﬁcation of both forward and backward

transformation in a single rule set. Both languages have been used to implement the AST2Dag transformation

example. The benchmarx framework was used for a quantitative and qualitative evaluation of the obtained

results.

1 INTRODUCTION

Model-Driven Software Engineering (MDSE)

(Schmidt, 2006) aims at increasing the productivity

of software engineers by replacing low-level program

code with high-level models. Typically, modeling

languages are deﬁned with the help of metamodels,

e.g., the Meta Object Facility standard MOF (Object

Management Group, 2016b) provided by the Object

Management Group (OMG). A subset of MOF

called Essential MOF (EMOF) is available, which

has been implemented in the Eclipse Modeling

Framework (EMF) (Steinberg et al., 2009). In the

context of model transformations, a vast variety of

languages and tools has been developed (Czarnecki

and Helsen, 2006). They differ with respect to the

underlying computational paradigm (procedural,

functional, rule-based, object-oriented), application

and scheduling strategies, directionality (uni- vs.

bidirectional), support for in-place or out-place, batch

or incremental transformations, etc.

This paper focuses on bidirectional transforma-

tions. It compares two languages that have been de-

signed for bidirectional transformations: QVT-R (Ob-

ject Management Group, 2016a), a declarative lan-

guage in which transformations are deﬁned by re-

lations between source and target model elements,

and BXtend (Buchmann, 2018), an object-oriented

framework for bidirectional transformations in which

transformations are programmed in Xtend. In QVT-

R, it is possible to deﬁne a bidirectional transforma-

tion by a single set of relations, while both trans-

formation directions have to be speciﬁed separately

in BXtend. The comparison is performed with the

help of a benchmark which we implemented on top

of the Benchmarx framework for evaluating bidirec-

tional transformations (Anjorin et al., 2017b; Anjorin

et al., 2019). The evaluation is performed with re-

spect to the following properties: size of the transfor-

mation deﬁnitions, functionality (measured in terms

of passed test cases), and performance (run time).

For our comparison, we selected the AST2Dag

transformation case that was described ﬁrst in (West-

fechtel, 2018) and was added later to the bx wiki

for bidirectional transformations

. An abstract syn-

tax tree represents an arithmetic expression as a tree,

while a directed acyclic graph representation of the

same expression includes each subexpression only

once. Even though both models are equivalent rep-

resentations of the same information, the AST2Dag

case proved to be challenging for bidirectional model

transformation languages and tools.

http://bx-community.wikidot.com

Bank, M., Kaske, S., Buchmann, T. and Westfechtel, B.

Incremental Bidirectional Transformations: Evaluating Declarative and Imperative Approaches using the AST2Dag Benchmark.

DOI: 10.5220/0009206602490260

In Proceedings of the 15th International Conference on Evaluation of Novel Approaches to Software Engineering (ENASE 2020), pages 249-260

ISBN: 978-989-758-421-3

249

2 BIDIRECTIONAL

TRANSFORMATIONS

2.1 Overview

A broad range of different languages and tools ad-

dressing model transformations has been developed

in the past decade. They differ with respect to com-

putational paradigm (rule-based, functional, object-

oriented, procedural), directionality (unidirectional

vs. bidirectional), support for in-place, out-place,

incremental or batch transformations, etc. Numer-

ous model transformation languages emerged accord-

ingly, ranging from general ones, like ATL (Jouault

et al., 2008), graph-based languages like Henshin

(Arendt et al., 2010) or eMoﬂon (Anjorin et al., 2011)

to domain-speciﬁc languages such as, e.g., the Ep-

silon family of transformation languages (Rose et al.,

2014) or EMG (Popoola et al., 2016).

The vast majority of transformation languages

supports only unidirectional batch transformations

with accompanying fairly well developed tool sup-

port. However, bidirectional transformations are re-

quired in a variety of use cases of industrial relevance.

For example, round-trip engineering integrates for-

ward and reverse engineering into a coherent process

which needs to be supported by bidirectional transfor-

mation tools: After transforming a source model into

a target model, additions or modiﬁcations of the tar-

get model may be necessary. Consequently, changes

to the source model need to be propagated in a way

which allows to retain the modiﬁcations of the target

model. These change propagations call for incremen-

tal rather than batch transformations. Additionally,

changes to the target model may have to be propa-

gated back to the source model, resulting in bidirec-

tional transformations. Furthermore, integration of

heterogeneous tools calls for data converters which

ideally perform lossless transformations in all direc-

tions.

In the light of this diversity, the OMG issued

the QVT standard (Queries, Views, and Transforma-

tions) which deﬁnes a family of model transforma-

tion languages located at different levels of abstrac-

tion (Object Management Group, 2016a). The most

high-level language deﬁned in the QVT standard is

QVT Relations (QVT-R). QVT-R provides a language

for the declarative speciﬁcation of transformations

between MOF-based models. Furthermore, QVT-R

reuses the Object Constraint Language (OCL), an

expression language for MOF models (Object Man-

agement Group, 2014). QVT-R addresses a wide

spectrum of model transformation scenarios, includ-

ing enforcing and checking, unidirectional and bidi-

rectional, batch and incremental, and n:1 transforma-

tions. However, bidirectionality is not a unique fea-

ture of QVT-R (Czarnecki et al., 2009). The most

prominent competitors are languages and tools based

on triple graph grammars, which were introduced

much earlier (Sch

urr, 1994) and have been developed

further in numerous research contributions (K

onigs

and Sch

urr, 2006; Becker et al., 2007). In contrast

to the grammar-based approach, QVT-R follows a re-

lational paradigm where a transformation is speciﬁed

by a set of relations among patterns to be instantiated

in the participating models.

2.2 Languages and Tools

In the following, we describe the languages and tools

used for our AST2Dag benchmark.

QVT-R. While QVT-R provides many interesting

and promising features, the language has not been

adopted widely. This stands in sharp contrast to other

OMG standards such as MOF and OCL. To the best of

our knowledge, currently no tool is available that fully

conforms to the standard deﬁned by the OMG. For

the benchmark, we used medini QVT

, which runs

only under old Eclipse versions, but still can be down-

loaded from the cited web site. The tool QVT Declar-

ative

is under development, but supports the QVT-R

standard only partially so far.

Medini QVT conforms to the QVT-R standard

syntactically, but implements a deviating semantics.

In medini QVT, a persistent trace is stored which

records relations between source and target elements.

The trace is used to support incremental transforma-

tions. When source elements have been inserted, they

are transformed because they do not appear in the

trace so far. When source elements have been deleted,

deletion is propagated to target elements via the trace

elements. Finally, certain changes to source elements

(modiﬁcations of attribute values) may be propagated

to target elements such that consistency is restored.

BXtend. The BXtend (Buchmann, 2018) frame-

work was created as a result to a case study in round-

trip engineering between UML class models and Java

source code. Approaches using TGGs (Buchmann

and Westfechtel, 2016) and QVT-R (Greiner and

Buchmann, 2016) clearly revealed the drawbacks of

highly declarative approaches, when right-hand sides

of rules contain variability in creation patterns (e.g.,

http://projects.ikv.de/qvt

https://projects.eclipse.org/projects/modeling.mmt.qvtd

ENASE 2020 - 15th International Conference on Evaluation of Novel Approaches to Software Engineering

250

it is not possible to specify rules between abstract su-

perclasses and create the corresponding subtypes dur-

ing rule execution). This leads to massive copying

and pasting similar rules. As a consequence BXtend

uses an object-oriented approach, where the frame-

work infrastructure comprising abstract rules, generic

creation of source and target elements and handling

the correspondence model is generated automatically.

The transformation developer only needs to specify

transformation patterns for forward and backward di-

rections respectively using the Xtend programming

language

, a fully Java compatible, less verbose lan-

guage which comes with high-level constructs such as

powerful lambda expressions.

3 THE TRANSFORMATION

CASE: EXPRESSION TREES

AND DAGS

3.1 Description

We study a problem from compiler construction:

Given an abstract syntax tree of an expression, a di-

rected acyclic graph (dag) is constructed which con-

tains common subexpressions only once (Figure 1).

From the dag created by this folding transformation,

the original tree may be reconstructed by unfolding

the dag. In contrast to a case study performed ear-

lier (Westfechtel and Buchmann, ), where both source

and target model contained different information, the

same information is represented in different ways

here. A pair of a tree and a DAG is consistent, if

traversing both the tree and the DAG in the same or-

der (e.g., inorder) results in the same expression rep-

resented by the data structure.

Figure 2 displays the metamodels for trees and

dags. In the tree metamodel, left and right are con-

tainment references from operators to left and right

operands, respectively. In contrast, these references

do not own the containment property in the dag meta-

model because an expression may occur at multiple

locations. Thus, in the dag representation of an ex-

pression all nodes are immediate components of the

root object of the dag model, and the hierarchy of

expressions is represented by non-containment refer-

ences.

This transformation case is highly problematic for

rule-based and grammar based approaches and poten-

tially highlights current limitations. While the tree is

best constructed in a top-down way by unfolding the

https://www.eclipse.org/xtend/

a 10 b 10 a 10 b 10

/ ‐ / ‐

+ +

Expressiontree

a b 10

/ ‐

Expressiondag

Figure 1: Tree and dag for (a/10 + (b-10))*(a/10 + (b-10)).

Model

Expression

Operator

op :ArithmeticOperator

<<enumeration>>

ArithmeticOperator

‐ Add

‐ Subtract

‐ Multiply

‐ Divide

model

0..1

expr

Operand

Variable

name :String

Number

value :Integer

left

leftInverse

right

Inverse

0..1

Expressiontrees

Model

Expression

Operator

op :ArithmeticOperator

<<enumeration>>

ArithmeticOperator

‐ Add

‐ Subtract

‐ Multiply

‐ Divide

1..*

model

expr

Operand

Variable

name :String

Number

value :Integer

left

leftInverse

right

Inverse

0..*

Expressiondags

Figure 2: Metamodels for Expression Trees and Dags.

DAG, the DAG is best constructed bottom-up by fold-

ing the tree.

Let us brieﬂy summarize the formal properties

of forward and backward transformations: Trees and

dags are equivalent representations of expressions.

Both the forward and the backward transformation are

total, deterministic, injective, and surjective.

3.2 Benchmarx

In order to evaluate the solutions provided in this pa-

per, we use the benchmarx framework (Anjorin et al.,

2017b; Anjorin et al., 2019). It is designed as a

generic framework for benchmarking bx tools and

uses JUnit test cases designed as synchronization di-

alogs. Such a synchronization dialog always starts

from the same agreed upon initial consistent state, and

then applies a sequence of edits to the source and tar-

get models. After the edits, the transformation is ex-

ecuted and only the resulting models are directly as-

serted by comparing them with predeﬁned expected

versions. In this paper, we provide a benchmark for a

Incremental Bidirectional Transformations: Evaluating Declarative and Imperative Approaches using the AST2Dag Benchmark

251

speciﬁc bx problem (Ast2Dag) by providing a suite of

test cases which are used to evaluate the solutions to

the transformation problem speciﬁed in QVT-R and

BXtend. The test suite contains test cases for both

forward and backward direction as well as batch and

incremental tests for each direction. Batch test cases

always start with an empty model and an edit delta

which creates the starting state of the source model

of the transformation (AST model for forward direc-

tion, DAG model for the backward direction). After

the model state is established, the transformation is

started and the obtained result is compared with a pre-

deﬁned expected state. Incremental test cases on the

other hand start with an initial consistent state of both

models. Afterwards a series of edit deltas is applied

to one of those models and the transformation is re-

run. The test result is obtained by comparing the ac-

tual state of the models with the expected one. Please

note that the incremental test cases also check if the

propagated change is minimal.

3.3 Test Cases

Batch Forward. The batch forward test cases com-

prise the following scenarios:

Initialization: An empty model with the root ele-

ment Model is created.

Single Expression: This test case creates an expres-

sion with a single number in an empty AST. The

expected result is an expression DAG with a cor-

responding single number.

Complex Expression: Extension of the previous test

case: a complex expression tree is created, which

is transformed to a corresponding DAG.

Multiple Expressions: An expression tree with mul-

tiple expressions is created. Expressions appear

more than one time. The corresponding DAG

should contain multiple references to correspond-

ing expression elements.

Multiple Complex Expressions: Analogously to

the previous test case multiple expressions over

several hierarchy layers which now have iden-

tical subtrees are created. The corresponding

DAG should contain multiple references to

corresponding elements.

Batch Backward. Analogously to the forward test

cases, the batch backward test cases address the fol-

lowing scenarios:

Single Expression: An expression with a single

number in an empty DAG is created. The ex-

pected result is a single number expression in the

corresponding AST.

Multiple Expressions: Multiple expressions are cre-

ated in the DAG, its references are single valued.

The expected result are multiple expressions in

the corresponding AST.

Multiple Complex Expressions: Multiple expres-

sions in the DAG which have multi-valued

references. The corresponding unfolding oper-

ation should create multiple occurrences of the

expressions in the AST.

Multiple Expressions with Identical Subtrees:

Similar to the previous test case, but containing

expressions that have identical subtrees.

Incremental Forward. The incremental test cases

start with a consistent pair of AST and DAG mod-

els. Afterwards, the AST is modiﬁed and the changes

need to be propagated to the DAG.

Incremental Inserts: Leaves are replaced with a

complex expression in an existing AST (which

has been transformed to a corresponding DAG in

a previous step). The expected result is a corre-

sponding DAG.

Incremental Deletions: Nodes from an existing

AST are removed and replaced with a number.

Please note, that inner nodes are removed, as re-

moving only leaves would result in an invalid

AST. Children of deleted nodes are removed re-

cursively. The expected result is an accordingly

updated DAG.

Incremental Modiﬁcations: An existing AST is

modiﬁed. A modiﬁcation is either changing an

attribute of a node or switching between variable

and number. The expected result is a correspond-

ing DAG.

Incremental Modiﬁcations Resulting in Deletions:

An existing AST is modiﬁed. The modiﬁcation

leads to merging of elements in the DAG.

Stability: This test re-runs the transformation after

an idle source delta has been applied. The ex-

pected result is an unchanged target DAG.

Incremental Backward. Incremental backward

test cases also start with a consistent pair of DAG

and AST models. After a change to the DAG,

corresponding updates are propagated to the AST.

Incremental Inserts: New nodes are inserted into

the DAG. Changes are propagated to the AST.

Incremental Deletions: Nodes are deleted from the

DAG. The resulting AST must be valid, i.e. Chil-

dren of affected AST nodes need to be deleted re-

cursively.

ENASE 2020 - 15th International Conference on Evaluation of Novel Approaches to Software Engineering

252

Incremental Modiﬁcactions: An existing DAG is

modiﬁed. A modiﬁcation comprises changes of

node values and switching the type of a node

(variable and number). The corresponding AST

needs to be updated accordingly.

Stability: The stability of the transformation is

tested. An idle delta to the DAG is propagated

to the AST, resulting in an unchanged AST.

4 SOLUTIONS

4.1 QVT-R

We failed in the construction of a single bidirectional

transformation. Instead, we developed separate uni-

directional transformations operating bottom-up and

top-down, respectively. The solution was described

in (Westfechtel, 2018) and is summarized brieﬂy be-

low to make this paper self-contained. Please note the

separate speciﬁcation of both transformation direc-

tions renders round-trip engineering impossible be-

cause forward and backward transformation employ

different trace models.

The forward transformation employs backward

chaining of relations through relation calls in when

clauses (Listing 1). The relations Variable2Variable

and Number2Number collapse multiple occurrences

of the same atomic operand. Collapsing of multiple

occurrences works recursively since it also applies to

operators. The key declarations in lines 2–4 ensure

that only one element per key value is created in the

dag model.

In contrast, the backward transformation makes

use of forward chaining through relation calls in

where clauses (Listing 2). The speciﬁcation contains

three top-level relations which handle three cases of

expression root nodes (variable, number, and opera-

tor). Starting from the root, the expression tree is built

top-down. For each occurrence of a subexpression,

exactly one relation call is executed. For each type of

nested operand, there are two relations transforming

it as a left or right operand of the enclosing operator,

respectively.

4.2 BXtend

The BXtend solution to our example scenario con-

sists of four rule classes, one for each concrete class

in the respective source and target models (Model,

Number, Operator, and Variable). Within those rule

classes bodies for the automatically generated method

stubs for sourceToTarget() and targetToSource() are

supplied by the user, specifying transformation pat-

terns for each execution direction separately. Since

the transformation problem describes folding/unfold-

ing the respective data structures, a n:1 correspon-

dence model is required. Since the correspondence

model is shared between both transformation direc-

tions, round-trip engineering may be supported (in

contrast to the solution in medini QVT).

Since the transformation developer has full con-

trol over rule orchestration, we speciﬁed a bottom-

up traversal of the AST for the forward direction

and a top-down creation of the AST for the back-

ward direction. Listing 3 depicts a cutout of the

rule class responsible for handling numbers. The

method sourceToTarget() (lines 3-25) is called from

the framework whenever a forward transformation is

executed. We iterate over all instances of Number

in the model and check if there is already a corre-

spondence model element available (line 6). If no

correspondence model element is found the method

addToTargetElem is called. Since a folding of multi-

ple AST elements with same values to a single DAG

element is performed in this transformation direction,

the method ﬁndTargetElem is called to retrieve match-

ing DAG elements. If no matching element is found, a

new target model element dag.Number is created (line

30). Otherwise the current source model element is

added to the list of matching elements (line 33). The

incremental behavior of the transformation is realized

in lines 11-18. A check is performed, if all source cor-

respondences which contain the current source model

element contain the same value attribute (line 11-12).

If the current source value was updated, this check

fails and the transformation proceeds in line 19, where

the source element is removed from the correspon-

dence list and addToTargetElem is called to establish

a correct matching.

Listing 4 depicts the method targetToSource from

the rule class responsible for transforming Operators.

Since the corresponding AST is constructed in a top-

down way, an ordering of the elements is required

(lines 4-9). An Xtend switch statement is used to map

the operator types of both models. Since variables and

numbers may only occur at the leaves of the AST, the

hierarchy of operators is established by traversing the

left and right references of the current element (lines

33-47) and creating or updating corresponding AST

elements.

Incremental Bidirectional Transformations: Evaluating Declarative and Imperative Approaches using the AST2Dag Benchmark

253

Listing 1: Transformation of trees to dags.

1 transformation as t2 da g ( as tM o d e l : ast , d ag Mo de l : dag ) {

2 key da g :: V ar ia bl e { m odel , na me };

3 key da g :: N u mb er { mo del , v al ue };

4 key da g :: O pe ra to r { m odel , op , left , r ig ht };

5 top relation M od e l 2 Mo de l { .. .}

6 relation E x pr e ss i on 2 Ex p r e s s i on {

7 checkonly domain a st Mo de l a s t_ ex p re s si on : a st :: Ex pr e ss io n {} ;

8 enforce domain d ag Mo de l d a g_ ex p re s s i on : d ag :: Ex pr e ss io n {} ; }

9 top relation V a r i ab l e2 V ar i a b le {

10 na me : S t ri ng ; a s t : a st : : M od el ; dag : da g :: M o de l ;

11 checkonly domain a st Mo de l a s t_ va r ia bl e : as t :: V ar ia bl e { na me = na me };

12 enforce domain d ag Mo de l d a g_ va r ia bl e : da g :: V ar ia bl e {

13 mo de l = dag ,

14 na m e = na me };

15 when { ast = r oot ( a st _ va ri a bl e ) . m od el ;

16 M od el 2 M o de l ( ast , dag ); }

17 where {

18 if d a g _ va r i a bl e . o cl Is U nd e f i ne d () then false

19 else E x pr e ss i on 2 Ex p r e s s i on ( a st _v ar ia bl e , d ag _v a ri ab l e )

20 endif; }

21 }

22 top relation N um b er 2N u mb er {. .. }

23 top relation O p e r at o r2 O pe r a t or {

24 as t_ op : a s t :: A r it h me ti c Op e ra t or ; a s t : a st : : M od el ; dag : da g :: M o de l ;

25 checkonly domain a st Mo de l a s t_ op e ra to r : as t :: O pe ra to r {

26 op = as t_o p , l eft = a s t_ le ft : a st : : E x p r es si o n {} ,

27 ri gh t = as t_ r ig ht : a st : : E x p r es si o n {}

28 };

29 enforce domain d ag Mo de l d a g_ op e ra to r : da g :: O pe ra to r {

30 mo de l = dag , o p = c o r r e s p o n d i n g D a g O p e r a t o r ( as t_ op ) ,

31 le f t = d ag _l ef t : dag :: E xp re ss i on {} ,

32 ri gh t = da g_ r ig ht : d ag : : E x p r es si o n { }

33 };

34 when { ast = ro ot ( a st _o p er at o r ) . m od e l ;

35 M od el 2 M o de l ( ast , dag );

36 E xp r es s io n 2E x pr e s s io n ( as t_ lef t , d ag _l ef t ) ;

37 E xp r es s io n 2E x pr e s s io n ( a s t_ ri ght , d a g_ ri gh t ) ; }

38 where { if d a g_ op e ra to r . o cl I sU nd e fi n e d () then false

39 else E x pr e ss i on 2 Ex p r e s s i on ( a st _o pe ra to r , d ag _o p er at o r )

40 endif; }

41 }

42 ...

5 EVALUATION

5.1 Size of the Transformation

Speciﬁcation

Both solutions presented in this paper support a tex-

tual concrete syntax with which transformation deﬁ-

nitions can be speciﬁed. Thus, a quantitative impres-

sion of the size of the transformation deﬁnitions can

be obtained by counting the number of lines of code

(excluding empty lines and comments), the number

of words (character strings separated by whitespace)

in these lines, and the number of characters in these

words. The values obtained for these metrics are de-

picted in Table 1. In general, the solution in QVT-R is

expected to be shorter for three reasons: (1) It is pos-

sible to deﬁne both transformation directions simulta-

neously by a single relational speciﬁcation. (2) The

speciﬁcation is declarative, at a high level of abstrac-

tion. (3) Incremental consistency restoration need

(and can) not be speciﬁed at all. In the AST2Dag case,

the ﬁrst reason does not hold because even in QVT-R

both transformation directions have to be deﬁned sep-

arately. Even then, we expect savings for the other

reasons. In contrast, the size metrics for the solutions

in QVT-R and BXtend are in the same order of mag-

nitude. Thus, BXtend’s procedural approach does not

ENASE 2020 - 15th International Conference on Evaluation of Novel Approaches to Software Engineering

254

Listing 2: Transformation of dags to trees.

1 transformation da g2 as t ( da gM o d e l : dag , a st Mo de l : ast ) {

2 top relation R o o t V a r ia b le 2 Va r ia b le {. .. }

3 relation L e ft V ar i ab l e2 V a r i a b le {

4 nam e : St ri ng ;

5 checkonly domain d ag Mo de l d a g_ op e ra to r : da g :: O pe ra to r {

6 le f t = d ag _ v a ri ab l e : dag :: Va r i a bl e {

7 nam e = na me }} ;

8 enforce domain a st Mo de l a s t_ op e ra to r : as t :: O pe ra to r {

9 le f t = a st _ v a ri ab l e : ast :: Va r i a bl e {

10 nam e = na me }} ;

11 }

12 relation R i gh t Va r ia b le 2 Va r ia b l e {. .. }

13 top relation R o o t Nu m be r 2N u m b er {. .. }

14 relation L e ft N u m be r 2N u mb e r { . .. }

15 relation R i gh t N u mb e r2 N um b er {. .. }

16 top relation R o o t O p e ra t or 2 Op e ra t or {

17 da g_ op : d a g :: A r it h me ti c Op e ra t or ;

18 as t_ op : a st : : A r it h me t ic O p e ra t or ;

19 checkonly domain d ag Mo de l da g : d a g :: M ode l {

20 ex pr s = da g _o pe r at or : d ag :: Op er at or {

21 op = d ag _o p } };

22 enforce domain a st Mo de l as t : a s t :: M ode l {

23 ex p r = a st _ o p er at o r : ast :: Op e r a to r {

24 op = a st _o p } };

25 when { d a g _o p e r at o r . l ef tI nv er se -> is Em pt y () ;

26 d ag _o p er at o r . r ig ht In ve rs e -> is Em pt y ( ) ; }

27 where { as t_ op = c or r e s p o n d i n g A s t O p e r a t o r ( d ag _o p ) ;

28 L ef t Va r ia b le 2 Va r i a bl e ( d ag _o pe r a t or , a s t _o p e r at o r ) ;

29 L ef t Nu mb e r2 N um b er ( d ag _o pe ra t o r , a st _o p er at o r ) ;

30 L ef t Op e ra t or 2 Op e r a to r ( d ag _o pe r a t or , a s t _o p e r at o r ) ;

31 R ig h tV a ri a bl e 2V a ri a bl e ( da g_ op er at or , a s t_ o p e ra to r );

32 R ig h tN u m b er 2 Nu m be r ( da g_ op er at or , as t _ o pe r a t or ) ;

33 R ig h tO p er a to r 2O p er a to r ( da g_ op er at or , a s t_ o p e ra to r ); }

34 relation L e ft O pe r at o r2 O p e r a t or {. .. }

35 relation R i gh t Op e ra t or 2 Op e ra t o r {. .. }

36 }

increase the size of the transformation deﬁnition.

Table 1: Size of the Transformation Deﬁnitions of Both So-

lutions.

medini QVT BXtend

Lines of code 353 282

Number of words 702 832

Number of characters 7576 9974

5.2 Qualitative Analysis

In order to perform a qualitative analysis, test cases

for the different transformation directions have been

speciﬁed and executed for both batch and incremen-

tal mode of operation. We assume a test case to be

passed, if the resulting model matches a predeﬁned

expected model state. The BXtend solution is able

to pass all tests speciﬁed in Section 3.3. The QVT-

R solution passes batch forward and backward tests

and also incremental backward tests, but it fails in two

incremental forward tests: incremental modiﬁcations

and incremental modiﬁcations resulting in deletions.

Expressiontree

a 5

a 7

Expressiondag

a 5

(1)

Expressiontree

a 5

(2)

Expressiondag

a 5

Expressiondag

a 5

(3)

Actualresult Expectedresult



Figure 3: Test Case failing in Medini QVT.

The failures are due to the fact that the incremental

Incremental Bidirectional Transformations: Evaluating Declarative and Imperative Approaches using the AST2Dag Benchmark

255

Listing 3: Transformation of trees to dags.

1 class N u m be r 2N u mb e r extends E le m 2E l em {

3 override def s o u rc e To T a r ge t () {

4 so u rc eM o de l . al l Co n te n ts .filter(typeof( N u mbe r ) )

5 .forEach[ n |

6 val co r r = n. g et C or rM o de l E l em as Mu l ti E le m

7 if( c orr == null) {

8 n . a dd To T ar g e t El e m

9 } else {

10 val t = corr . t a rg et E le m en t as dag . Nu m ber

11 if( c orr . s o ur ce E le m en t s .forall[ i t instanceof N umb e r &&

12 ( it as Numb e r ) . v a lue == n . v a lue ] ) {

13 val n e wT a rg e t = n . fi n d T ar g et E le m

14 if( ne w Tar g et ! = null)

15 ( n e wTa r ge t . g e tC o rr Mo d el E l e m as M ul t iEl e m ) . so u rc eE l em e nt s += n

16 else

17 t . v a lue = n . va l ue

18 }

19 if(t . v alu e ! = n . v alu e ) {

20 c o rr . s o ur c eE l e m en t s -= n

21 n . a dd To T ar g e t El e m

22 }

23 }

24 ]

25 }

27 def private a d d To T ar ge t El e m ( ast . Nu m ber e ) {

28 var n e wT a rg e t = e . fi n d T ar g et E le m

29 if( ne w Tar g et = = null) {

30 n ew T ar g et = c r e a te T a r ge t E l em e n t ( d a g . D ag P ac k ag e . eI N ST A NC E . n um b e r ) as d a g . N um b e r

31 }

32 val n e wCo r r = ne w Ta r ge t . ge t O r Cr ea t e C or rM o d e lE l e m en t ( ru l eID ) as M u lt i El e m

33 ne w Cor r . so u r c eE l em e nt s += e

34 ne w Ta r ge t . v alu e = e. v a l ue

35 ne w Ta r ge t . m ode l = e. m o d el . g e tC or r Mo d e l El e m . t a r g et E le m en t as dag . Mo d el

36 el em e nt s To Co r r . put ( ne w Co r r )

37 }

39 def private f i n dT a rg e t E le m ( a s t . N um b e r e) {

40 (e. m o del . g et C or r Mo de l El em . t ar g et E le m e n t as dag . M o d el ) . e xpr s

41 .filter(typeof( dag . Num b er ) ) .findFirst[ it . va l u e == e . va l ue ]

42 }

44 ...

45 }

behavior built into medini QVT deviates from the re-

quired behavior. One of the failing test cases is illus-

trated in Figure 3. In an initial batch transformation

(1), an expression tree for (a+5)*(a+7) is transformed

successfully into a dag, in which variable a is shared.

In step (2), the number 7 is replaced with the number

5. Since both operands of the multiply operator are

the same, the incremental transformation (3) should

result in the dag displayed at the right-hand side. In

contrast, medini QVT merely propagates the modi-

ﬁcation of the attribute value, without checking for

common subexpressions. The second test case fails

for the same reason, but even results in a dag which

represents a different expression than the tree. These

failures are symptoms of a general problem with the

design of QVT-R (note that this is not a problem spe-

ciﬁc to medini QVT): In QVT-R, relations between

source and target elements are speciﬁed declaratively;

incremental behavior is provided “for free”. However,

the provided behavior may deviate from the required

behavior. In contrast, in BXtend incremental behav-

ior has to be speciﬁed explicitly, but can tailored to

speciﬁc requirements. For example, when a number

is changed, it is checked whether the same number is

already present in the dag. In that case, the dag is re-

organized such that the number is present only once;

this reorganization is applied recursively updwards in

the dag.

5.3 Performance Analysis

In order to evaluate the efﬁciency and scalability of

the resulting transformation with respect to increasing

model size, two experiments were conducted in both

forward and backward directions resulting in four sets

of measurements: (1) batch transformations in for-

ward and backward directions, and (2) incremental

transformations in forward and backward directions.

The batch transformations test how the solutions scale

when creating the corresponding opposite model of

increasing size, starting with an AST with only one

single expression up to a tree with a depth of 20 hier-

archy layers. The incremental transformations apply

the exact same edit to master models of increasing

size. After correspondence has been established, one

single value of a leaf is changed. The time required

to locate and propagate this change to the dependent

model is measured. The tests were performed on the

ENASE 2020 - 15th International Conference on Evaluation of Novel Approaches to Software Engineering

256

Listing 4: Transformation of dags to trees.

1 class O p e ra t o r 2O p er at o r extends E le m 2E l em {

2 override def t a r ge t To S o u rc e () {

3 var List < d a g . O p erat o r > p re O rd e r = new A r ray L i st < da g . O p e rato r >() ;

4 for (var it = ta r ge t Mo d el . a ll C on t en t s .filter(typeof( dag . Op e ra t or ) ) ; it . h asN e xt ( ) ; ) {

5 val dag . O p era t or o p = it . n e x t () ;

6 if ( op . l e ft I nv e rs e . e mpt y & & o p . r i g h tI n ve r se . e m pty ) {

7 p re O rd e r += op ;

8 }

9 }

10 if ( p re O rd e r . size == 0) {

11 return;

12 } else if ( p r eO r der . s i ze > 1) {

13 throw new As s er t i o nE r ro r (" Da g h a s m u lti p le r oot e le m en t s .") ;

14 }

16 var List < O p erat o r > pre O rd e r S rc = new Arra y L ist < O p erat o r >() ;

17 val c o rr R oot = p r eO r de r . g e t (0) . g e t O rC r e a te Co r r M od el E l e me n t ( r u leI D ) as M ul t iE l em ;

18 val s r cRo o t = co r rRo o t . g e t O rC r e a te S o u rc eE l em ( s o u rc e Pa c ka ge . op e r ator , [ tru e ] ) as Ope r at o r ;

19 pr e Or de r Sr c . a d d ( s rc R o ot ) ;

20 sr c Roo t . m ode l = p r eO r de r . g e t (0) . mo d e l . c o rr M od e lE l e m . so u rc e El em e nt as Mode l ;

22 whi l e (! p r eOr d er . e m pty ) {

23 val dag . O p er a t or cu r re n t = pre O rd e r . re m o ve ( 0) ;

24 val O p er a tor c u rr e nt S rc = p r eO r de r Sr c . re m o ve ( 0 ) ;

25 switch ( c ur r ent . op ) {

26 case ADD : c u rr e nt S rc . op = a s t . A ri th m et ic O pe ra t or . A D D

27 case D I VID E : c ur r en t Sr c . op = as t . A r i th m e t ic O p e ra t o r . D IVI D E

28 case M U LT I PLY : c ur r en t Sr c . op = ast . Ar it h me ti c Op er a to r . MU L TI P LY

29 case S U BT R ACT : c ur r en t Sr c . op = ast . Ar it h me ti c Op er a to r . SU B TR A CT

30 }

31 if ( c ur r ent . r i ght instanceof d a g . O pe r at o r ) {

32 p re O rd e r . a dd (0 , c ur r ent . r i ght as d a g . O pe r at o r );

33 val c o rr R ig h t = pr e O rd e r . get (0 ) . g e t Or C r e at eC o r r Mo d e l E l em e n t ( rul e ID ) as Mu l ti E le m ;

34 val s r cR i ght = c o rr R ig h t . g e t O rC r e a te S o u rc e E l em (

35 s ou rc e Pa c ka g e . op e r ator , [ e | ( e as O pe r at o r ). r ig h tI n ve r s e = = c u rr e nt S rc ] ) as O pe r at o r ;

36 p re O rd er S rc . add (0 , sr c Ri g ht ) ;

37 c ur r en t Sr c . rig h t = s rc R igh t ;

38 }

39 if ( c ur r ent . l e f t instanceof dag . O p era t or ) {

40 p re O rd e r . a dd (0 , c ur r ent . l e f t as dag . O p era t or ) ;

41 val c o rr L eft = p r eO r de r . g e t (0) . g e t O rC r e a te Co r r M od el E l e me n t ( r u leI D ) as M ul t iE l em ;

42 val s r cLe f t = co r rLe f t . g e t O rC r e a te S o u rc eE l em (

43 s ou rc e Pa c ka g e . op e r ator , [ e | ( e as O pe r at o r ). l ef t In v er s e == c ur r en t Sr c ]) as Op e rat o r ;

44 p re O rd er S rc . add (0 , sr c Le f t );

45 c ur r en t Sr c . left = sr c Le f t ;

46 }

47 }

48 }

49 ...

50 }

same machine and in isolation for each solution. A

standard machine with an Intel Core i7-4770 CPU

was used, running at 3.40 GHz, with 16 GB of DDR3

RAM and with Microsoft Windows 10 64-bit as oper-

ating system. We used Java 1.8.0 191, Eclipse Neon

(4.6.3), and EMF version 2.12.0 to compile and exe-

cute the Java code for the scalability test suite. Each

test was repeated 5 times and the median measured

time was computed. The four measurement results are

depicted in Fig. 4, 5, 6, and 7. A ﬁgure is composed

of two plots – a plot with linear/linear scale to the

left, and a plot with log/log scale to the right. While

the linear plot provides a realistic impression for the

actual complexity of each solution, the logarithmic

one zooms into ﬁner details for smaller models and

zooms out for larger models, allowing to qualitatively

present large differences in runtime. The BXtend so-

lution outperforms the Medini QVT solution in ev-

ery benchmark. The gap between both solutions be-

comes clearly evident in the Figures showing linear/-

linear scale. Medini QVT shows sharp increases in

runtime kicking in already at relatively small models

of around 10k elements, while the BXtend solution is

able to handle signifcantly larger models.

6 RELATED WORK

While a broad range of languages and tools exist for

model transformations in general and bidirectional

transformations in particular, thorough evaluations of

different approaches applied to various transforma-

tion problems are still missing in the literature. The

transformation tool contest (TTC)

, aims at bringing

together tool developers in a competition-like annual

event. Several predeﬁned transformation problems

are given which have to be solved by the participants

and their respective tools. However, to the best of our

knowledge so far only one case for bidirectional trans-

formations has been posed and solved, so far (see be-

https://www.transformation-tool-contest.eu/

Incremental Bidirectional Transformations: Evaluating Declarative and Imperative Approaches using the AST2Dag Benchmark

257

# model elements

time in s

1000

2000

3000

4000

5000

50000 100000 150000 200000 250000 300000 350000

BXtend Medini QVT

Batch Forward (lin - lin scale)

# model elements

time in s

0.1

100

1000

10 100 1000 10000 100000

BXtend Medini QVT

Batch Forward (log - log scale)

Figure 4: Forward Batch Transformation: Linear/linear Scale (Left) and Log/log Scale (Right).

# model elements

time in s

5000

10000

15000

25000 50000 75000 100000 125000 150000 175000

BXtend Medini QVT

Incremental Forward (lin - lin scale)

# model elements

time in s

0.01

100

10000

10 100 1000 10000 100000

BXtend Medini QVT

Incremental Forward (log - log scale)

Figure 5: Forward Incremental Transformation: Linear/linear Scale (Left) and Log/log (Right).

# model elements

time in s

10000

20000

30000

40000

50000 100000 150000 200000 250000 300000 350000

BXtend Medini QVT

Batch Backward (lin - lin scale)

# model elements

time in s

0.01

100

10000

10 100 1000 10000 100000

BXtend Medini QVT

Batch Backward (log - log scale)

Figure 6: Backward Batch Transformation: Linear/linear Scale (Left) and Log/log Scale (Right).

# model elements

2500

5000

7500

10000

12500

25000 50000 75000 100000 125000 150000 175000

BXtend Medini QVT

Incremental Backward (lin - lin scale)

# model elements

time in s

0.01

100

10000

10 100 1000 10000 100000

BXtend Medini QVT

Incremental Backward (log - log scale)

Figure 7: Backward Incremental Transformation: Linear/linear Scale (Left) and Log/log Scale (Right).

ENASE 2020 - 15th International Conference on Evaluation of Novel Approaches to Software Engineering

258

low).

The benchmarx (Anjorin et al., 2017b) frame-

work is an important step towards providing a uniﬁed

platform for assessing and benchmarking incremental

model transformations. For the transformation tool

contest 2017, a bidirectional transformation problem

has been accepted, which addresses the transforma-

tion of family and person databases (Anjorin et al.,

2017a). A wide range of solutions was submitted,

which allowed for a comparison of seven signiﬁcantly

differing tools (Anjorin et al., 2019). An additional,

more detailed comparison of BXtend and QVT-R ap-

plied to the Families to Persons case is described in

(Westfechtel and Buchmann, 2018).

The transformation problem discussed in this pa-

per differs signiﬁcantly from the Families to Persons

case. While the latter involves information loss and

asymmetric speciﬁcations of both transformation di-

rections, the AST2Dag benchmark is concerned with

a symmetric transformation problem without any loss

of information. Both the AST and the Dag represent

the same information in different ways. Transforma-

tions in both directions require restructuring (folding

in forward direction and unfolding in backward di-

rection). Interestingly, currently there is no bidirec-

tional language or tool known that is able to solve

this bidirectional transformation problem in a sin-

gle speciﬁcation (please recall that our solutions in

BXtend and QVT-R both require two unidirectional

transformations that are speciﬁed separately). Thus,

the AST2Dag benchmark constitutes a real challenge

for bidirectional languages and tools.

7 CONCLUSION

In this paper we provide an evaluation of declarative

and imperative approaches to solve a bidirectional

transformation problem. The problem is taken from

compiler construction and addresses expressions rep-

resented as abstract syntax trees (AST) and directed

acyclic graphs (DAG). Incremental and bidirectional

transformations between the two data structures rep-

resented as EMF models have been created using

QVT-R as a representative of declarative approaches

and BXtend, an imperative framework speciﬁcally de-

signed for incremental bidirectional transformations.

The experiments revealed that we failed to specify

a single bidirectional transformation with QVT-R.

Rather, two unidirectional transformations were re-

quired. As a consequence, the quantitative analysis

shows that the declarative approach does not provide a

beneﬁt in terms of size of the transformation descrip-

tion. The qualitative analysis is done using Bench-

marx, a framework for evaluating bx tools. A set of

test cases for each transformation direction and for

each mode of operation had to be solved with each

tool. While the BXtend solution was able to pass all

test cases, the QVT-R solution failed in two incremen-

tal forward ones. Furthermore, the performance tests

revealed that the QVT-R solution shows signiﬁcant in-

creases in runtime for even smaller models, while the

BXtend solution scales much better with increasing

model sizes.

As pointed out in the section on related work,

the AST2Dag case constitutes a challenge for bidi-

rectional transformation languages and tools. To gain

further insights from this case, it would highly desir-

able to have this case implemented in further bidirec-

tional tools (again using the benchmarx framework).

REFERENCES

Anjorin, A., Buchmann, T., and Westfechtel, B. (2017a).

The Families to Persons case. volume 2026 of CEUR

Workshop Proceedings, pages 27–34, Marburg, Ger-

many.

Anjorin, A., Buchmann, T., Westfechtel, B., Diskin, Z., Ko,

H.-S., Eramo, R., Hinkel, G., Samimi-Dehkordi, L.,

and Z

undorf, A. (2019). Benchmarking bidirectional

transformations: theory, implementation, application,

and assessment. Software and Systems Modeling.

Anjorin, A., Diskin, Z., Jouault, F., Ko, H., Leblebici, E.,

and Westfechtel, B. (2017b). Benchmarx reloaded:

A practical benchmark framework for bidirectional

transformations. In Eramo, R. and Johnson, M., ed-

itors, Proceedings of the 6th International Workshop

on Bidirectional Transformations co-located with The

European Joint Conferences on Theory and Prac-

tice of Software, BX@ETAPS 2017, Uppsala, Swe-

den, April 29, 2017., volume 1827 of CEUR Workshop

Proceedings, pages 15–30. CEUR-WS.org.

Anjorin, A., Lauder, M., Patzina, S., and Sch

urr, A. (2011).

Emoﬂon: leveraging EMF and professional CASE

tools. In Heiß, H., Pepper, P., Schlingloff, H.,

and Schneider, J., editors, Informatik 2011: Infor-

matik schafft Communities, Beitr

age der 41. Jahresta-

gung der Gesellschaft f

ur Informatik e.V. (GI), 4.-

7.10.2011, Berlin, Deutschland (Abstract Proceed-

ings), volume 192 of LNI, page 281. GI.

Arendt, T., Biermann, E., Jurack, S., Krause, C., and

Taentzer, G. (2010). Henshin: Advanced Concepts

and Tools for In-Place EMF Model Transformations.

In Petriu, D. C., Rouquette, N., and Haugen, Ø., ed-

itors, Proceedings 13th International Conference on

Model Driven Engineering Languages and Systems

(MODELS 2010), Part I, volume 6394 of Lecture

Notes in Computer Science, pages 121–135, Oslo,

Norway. Springer-Verlag.

Becker, S. M., Herold, S., Lohmann, S., and Westfechtel,

B. (2007). A graph-based algorithm for consistency

Incremental Bidirectional Transformations: Evaluating Declarative and Imperative Approaches using the AST2Dag Benchmark

259

maintenance in incremental and interactive integration

tools. Software and Systems Modeling, 6(3):287–316.

Buchmann, T. (2018). Bxtend - A framework for (bidi-

rectional) incremental model transformations. In

Hammoudi, S., Pires, L. F., and Selic, B., edi-

tors, Proceedings of the 6th International Confer-

ence on Model-Driven Engineering and Software De-

velopment, MODELSWARD 2018, Funchal, Madeira

- Portugal, January 22-24, 2018., pages 336–345.

SciTePress.

Buchmann, T. and Westfechtel, B. (2016). Using triple

graph grammars to realise incremental round-trip en-

gineering. IET Software, 10(6):173–181.

Czarnecki, K., Foster, J. N., Hu, Z., L

ammel, R., Sch

urr, A.,

and Terwilliger, J. F. (2009). Bidirectional transforma-

tions: A cross-discipline perspective. In Paige, R. F.,

editor, Proceedings of the Second International Con-

ference on Theory and Practice of Model Transforma-

tions (ICMT 2009), volume 5563 of Lecture Notes in

Computer Science, pages 260–283, Zurich, Switzer-

land. Springer-Verlag.

Czarnecki, K. and Helsen, S. (2006). Feature-based sur-

vey of model transformation approaches. IBM Sys-

tems Journal, 45(3):621–646.

Greiner, S. and Buchmann, T. (2016). Round-trip engineer-

ing UML class models and java models: A real-world

use case for bidirectional transformations with QVT-

R. International Journal of Information System Mod-

eling and Design (IJISMD), 7(3):72–92.

Jouault, F., Allilaire, F., B

ezivin, J., and Kurtev, I. (2008).

ATL: A model transformation tool. Sci. Comput. Pro-

gram., 72(1-2):31–39.

onigs, A. and Sch

urr, A. (2006). Tool integration with

triple graph grammars - a survey. In Heckel, R., ed-

itor, Proceedings of the School of SegraVis Research

Training Network on Foundations of Visual Modelling

Techniques (FoVMT 2004), volume 148 of Electronic

Notes in Theoretical Computer Science, pages 113–

150, Dagstuhl, Germany. Elsevier Science.

Object Management Group (2014). Object Constraint Lan-

guage Version 2.4. Needham, MA, formal/14-02-03

edition.

Object Management Group (2016a). Meta Object Facility

(MOF) 2.0 Query/View/Transformation Speciﬁcation

Version 1.3. Needham, MA, formal/16-06-03 edition.

Object Management Group (2016b). OMG Meta Object Fa-

cility (MOF) Core Speciﬁcation Version 2.5.1. Need-

ham, MA, formal/19-10-01 edition.

Popoola, S., Kolovos, D. S., and Rodriguez, H. H. (2016).

EMG: A domain-speciﬁc transformation language for

synthetic model generation. In Gorp, P. V. and Engels,

G., editors, Theory and Practice of Model Transfor-

mations - 9th International Conference, ICMT 2016,

Held as Part of STAF 2016, Vienna, Austria, July 4-5,

2016, Proceedings, volume 9765 of Lecture Notes in

Computer Science, pages 36–51. Springer.

Rose, L. M., Kolovos, D. S., Paige, R. F., Polack, F. A. C.,

and Poulding, S. M. (2014). Epsilon ﬂock: a model

migration language. Software and System Modeling,

13(2):735–755.

Schmidt, D. C. (2006). Guest editor’s introduction: Model-

driven engineering. IEEE Computer, 39(2):25–31.

Sch

urr, A. (1994). Speciﬁcation of graph translators with

triple graph grammars. In Mayr, E. W., Schmidt, G.,

and Tinhofer, G., editors, Graph-Theoretic Concepts

in Computer Science, 20th International Workshop,

WG ’94, Herrsching, Germany, June 16-18, 1994,

Proceedings, volume 903 of Lecture Notes in Com-

puter Science, pages 151–163. Springer.

Steinberg, D., Budinsky, F., Paternostro, M., and Merks,

E. (2009). EMF Eclipse Modeling Framework. The

Eclipse Series. Addison-Wesley, Upper Saddle River,

NJ, 2nd edition.

Westfechtel, B. (2018). Case-based exploration of bidirec-

tional transformations in QVT Relations. Software

and Systems Modeling, 17(3):989–1029.

Westfechtel, B. and Buchmann, T. Incremental bidirectional

transformations: Comparing declarative and procedu-

ral approaches using the families to persons bench-

mark. In Damiani, E., Spanoudakis, G., and Maci-

aszek, L. A., editors, Evaluation of Novel Approaches

to Software Engineering - 13th International Con-

ference, ENASE 2018, Funchal, Madeira, Portugal,

March 23-24, 2018, Revised Selected Papers, pages

98–118.

Westfechtel, B. and Buchmann, T. (2018). Incremental bidi-

rectional transformations: Comparing declarative and

procedural approaches using the families to persons

benchmark. In Damiani, E., Spanoudakis, G., and

Maciaszek, L. A., editors, Evaluation of Novel Ap-

proaches to Software Engineering - 13th International

Conference, ENASE 2018, Funchal, Madeira, Portu-

gal, March 23-24, 2018, Revised Selected Papers, vol-

ume 1023 of Communications in Computer and Infor-

mation Science, pages 98–118. Springer.

ENASE 2020 - 15th International Conference on Evaluation of Novel Approaches to Software Engineering

260