it is frequently stated as a best practice to test and
find defects as early as possible (Voelter, 2009), since
this has been shown to increase quality and reduce the
total time and effort required to develop or maintain
software (Melleg
˚
ard et al., 2016; Broy et al., 2012).
There are several ways to improve the quality of
models and ensure correctness. One best practice is
to review models, just like any other piece of soft-
ware (Voelter, 2009). This is currently done by many
practitioners to build confidence in the quality of code
generators and the generated code (Broy et al., 2012;
Mooij et al., 2013). In (Baker et al., 2005), the quality
and correctness of models is established by simulating
the models against an executable test suite. Another
best practice is to use model-level validation (Voelter,
2009) to verify that the model is a valid instance of
the language, but perhaps more importantly, to vali-
date that the model makes sense in the domain where
it will be used.
3.4 Quality of Generated Code (Pain 12)
This pain is phrased rather broadly, since software
quality can mean a lot of different things (Interna-
tional Organization for Standardization, 2011). A ter-
tiary study, i.e., a study of literature surveys, in the
area of quality in MBE is presented in (Goul
˜
ao et al.,
2016). The study considers as many as 22 litera-
ture surveys, many of which choose maintainability
as the quality metric of choice. They conclude that
the field is not yet fully mature as most surveys tar-
get researchers and focus on classifying work, rather
than targeting industry practitioners and aggregating
quantitative evidence according to established quality
metrics. We proceed by discussing a few relevant pri-
mary studies, most of which conclude that quality of
generated code is actually a gain rather than a pain.
A case study (Melleg
˚
ard et al., 2016) in the Dutch
IT-industry showed that introducing MBE in the
maintenance phase of a software project can improve
software quality. More specifically, they showed that
a lower defect density was achieved using modeling,
although at the expense of increasing time to fix a de-
fect. However, the total result of these effects was
a decrease in the total effort spent on maintenance of
versions of the software. A reduction of defects is also
observed in (Mohagheghi and Dehlen, 2008), but it is
not supported by any quantitative evidence. A similar
observation was made by Motorola in (Baker et al.,
2005), which states that it is sometimes faster and
sometimes slower to find the root cause of a software
defect when using MBE. They also provide quantita-
tive estimates suggesting a reduction in the time to fix
defects encountered during system integration, over-
all reduction of defects, and improvements in phase
containment of defects (i.e. that defects are more
likely to be detected and fixed in the development
phase in which they are introduced) and productivity.
Another aspect of generated code quality is the ex-
tent to which it is readable by humans. Best prac-
tices state that generated code should follow accept-
able style guides. This may seem like a waste of
time, since other best practices suggest that generated
code should not be modified (Voelter, 2009). How-
ever, people still interact with generated code in sev-
eral ways. For example, just like for any other code,
generated code is inspected by developers trying to
track down the root cause of a defect and this goes
faster if it is clear what the code is doing. Secondly,
manual code reviews of generated code are part of the
development practice in many places to ensure cor-
rectness of the code and its generators (Broy et al.,
2012; Mooij et al., 2013). Since code generators gen-
erate code in a structured way, this means that the
confidence in their correctness is increased over time.
This argument is consistent with a best practice stated
in (Voelter, 2009). Lastly, for certification of safety-
critical software in e.g., the automotive and avionics
domains (RTCA, Inc., 2012), it may furthermore be
more cost-efficient to manually inspect the code than
to qualify the code generator, which is very expensive
and time-consuming.
4 APPROACH TO INVESTIGATE
MITIGATION OF PAINS
This section explains the organization of the practi-
cal investigation of the pains for an MBE approach
based on DSLs in an industrial case study. We start
by motivating our choice of modeling technology, be-
fore presenting the case study. Lastly, we present our
approach to investigate pain-mitigation techniques.
4.1 Modeling Technology
The potential pains of MBE are investigated by means
of DSLs, since the discussion in Section 3 suggests
that it has the potential to successfully mitigate many
of the chosen pains. It is also a technology that is al-
ready used within the partner company, and we have
many years of experience of transferring it to indus-
try and applying it, e.g., (Kurtev et al., 2017; Mooij
et al., 2016). There are many approaches (Mernik
et al., 2005) and tools (Erdweg et al., 2015) for de-
veloping DSLs. This work uses Xtext as DSL devel-
opment tool. Xtext is a mature language workbench
that has been around for more than a decade and has
Pain-mitigation Techniques for Model-based Engineering using Domain-specific Languages
755