Specifying Industrial System Requirements using Speciﬁcation Patterns:

A Case Study of Evaluation with Practitioners

Predrag Filipovikj and Cristina Seceleanu

School of Innovation, Design and Engineering, M

alardalen University, V

aster

as, Sweden

Keywords:

Speciﬁcation Patterns, Pattern-Based Requirements Speciﬁcation, Industrial Case Study.

Abstract:

With the ever-increasing size and complexity of the industrial software systems there is an imperative need for

an automated, systematic and exhaustive veriﬁcation of various software artifacts, such as system speciﬁca-

tions, models, code, etc. A potential remedy for this need might lie in a pool of techniques for computer-aided

veriﬁcation of software related artifacts, including system speciﬁcations. The Achilles’ heel of these tech-

niques, and the main hinder for their wider adoption in the industrial development process are the complexity

and the specialized skill-set required for the formal encoding of speciﬁcations. To alleviate this problem,

Speciﬁcation Patterns that are based on the observation that the system speciﬁcations are framed within reoc-

curring solutions have been proposed. The approach has been shown to be expressive enough for capturing

requirements in the automotive domain, however, there is a lack of empirical data that can be used to judge its

practical usefulness. In this paper, we involve an existing speciﬁcation-patterns-based tool, called PROPAS,

and propose a small-size evaluation of the approach with practitioners, on a case study conducted in coopera-

tion with Scania, Sweden, one of the world’s leading manufacturers of heavy-load vehicles. Our results show

that the speciﬁcation patterns that are supported by an adequate tooling have the potential to be practically

usefull for the non-experts in formal methods.

1 INTRODUCTION

Developing and maintaining system requirements

speciﬁcations (system speciﬁcations from now on) for

complex software systems is a daunting task. Ac-

cording to the current state-of-the-practice, the system

speciﬁcations are written using free text, predomi-

nantly in English. While the free text natural language

speciﬁcations are easy to capture and read, such way

of specifying systems suffers from a number of limita-

tions. First, the unrestricted natural language speciﬁ-

cations lack structure, thus leading to situations where

the same requirement can be interpreted in multiple

ways, usually depending on the domain knowledge of

the person who is reading it. Second, using the un-

restricted natural language can potentially lead to an

inconsistent encoding. The most prominent example

of this kind is when a same concept is referred to by

different names.

Such reasons, combined with the ever-increasing

safety-critical importance of the functions, call for

more precise and unambiguous ways of encoding re-

quirements that will also enable rigorous veriﬁcation

*corresponding author

of various models. Existing techniques for computer-

aided analysis of software-related artifacts, including

the system speciﬁcations (Barnat et al., 2016; Post

et al., 2011; Filipovikj et al., 2018) are mostly based

on formal methods, which have the advantage of be-

ing completely automated, mathematically rigorous

and possibly exhaustive. Despite their potential added

value, the formal analysis techniques are still just

seldomly applied in industrial practice. One of the

main obstacles towards a broader industrial adoption

of such techniques seems to be the formal encoding

of the textual speciﬁcations in some form of tempo-

ral logic, which is a challenging task for practitioners,

without appropriate tool support.

Speciﬁcation Patterns (SPS) (Dwyer et al., 1998)

is an approach intended to help practitioners to for-

mally encode system speciﬁcations. The approach is

based on reoccurring solutions for specifying require-

ments, which have been collected in a catalogue of

patterns. The initial set of patterns has been extracted

by analyzing more than 500 requirements from vari-

ous domains. The advantage of the speciﬁcation pat-

terns is that they have a well deﬁned structure and se-

mantics, and as such can be encoded in many differ-

Filipovikj, P. and Seceleanu, C.

Specifying Industrial System Requirements using Speciﬁcation Patterns: A Case Study of Evaluation with Practitioners.

DOI: 10.5220/0007726600920103

In Proceedings of the 14th International Conference on Evaluation of Novel Approaches to Software Engineering (ENASE 2019), pages 92-103

ISBN: 978-989-758-375-9

ent ways, including the constrained natural language

and various formal notations that accurately capture

the semantics of the system behavior described by the

particular requirement.

Besides the body of work focusing on enrich-

ing the speciﬁcation pattern catalogue (Dwyer et al.,

1998; Konrad and Cheng, 2005; Grunske, 2008; Au-

tili et al., 2015) and providing specialized tool sup-

port (Smith et al., 2002; Mondragon and Gates, 2004;

Remenska et al., 2014), there exist several endeav-

ors targeting the expressiveness of the speciﬁcation

patterns in the automotive domain, showing that the

speciﬁcation patterns are expressive enough for for-

malizing the requirements in the domain (Post et al.,

2012; Filipovikj et al., 2014). However, in all stud-

ies the generation of the formal system speciﬁcations

based on the speciﬁcation patterns is performed by

formal methods experts. Consequently, there is a lack

of empirical data when it comes to applying the ap-

proach in industrial settings, by the intended users,

that is, the engineers.

In this paper, we report the results from an ex-

ploratory case study, which is intended to collect data

about the practical usefulness of the speciﬁcation pat-

terns in industry. The exploratory nature of the case

study means that the generated results are to be used

for proposing hypotheses that can then be tested in

future studies. The case study is performed in coop-

eration with Scania, one of the world’s leading manu-

facturers of heavy-load vehicles (trucks and buses),

as part of an ongoing academic-industrial coopera-

tion. In this case study, we involve ﬁve different stake-

holders of the system development process, as case-

study subjects (CSS), who use the speciﬁcation pat-

terns and the existing tool called PROPAS (Filipovikj

et al., 2018) to specify a predeﬁned set of require-

ments of an operational automotive system. For the

purpose of our study, we collected both quantitative

and qualitative data. The quantitative data consists

of: i) speciﬁcation correctness, which is measured

in terms of how many of the requirements are cor-

rectly speciﬁed using the speciﬁcation patterns and

the PROPAS tool, and ii) the required time for spec-

iﬁcation. The qualitative data on the other hand, is

collected via a survey of ten statements, which can be

answered using ﬁve predeﬁned options.

The paper continues as follows. In Section 2 we

present an overview of the speciﬁcation patterns (Sec-

tion 2.1) and the PROPAS tool (Section 2.2). Next,

in Section 3 we provide details on the case study re-

search method, followed by Section 4 in which we

show the details about the setup of our case study. In

Section 5 we present the results of the evaluation and

the threats to validity, followed by the discussion of

the results in Section 6. We compare to related work

in Section 7, and in Section 8 we outline the conclu-

sions and directions for future work.

2 SPECIFICATION PATTERNS

AND PROPAS TOOL

In this section, we give an overview of the speciﬁca-

tion patterns (Section 2.1) and the PROPAS tool (Sec-

tion 2.2).

2.1 Speciﬁcation Patterns

The speciﬁcation Patterns System (Dwyer et al.,

1998) is an approach proposed to facilitate the for-

mal speciﬁcation of system properties for practition-

ers who are not experts in formal methods. It is based

on the assumption that the systems’ speciﬁcations are

framed within reoccurring solutions, from which a set

of patterns can be extracted and saved for future reuse.

Each pattern is characterized by a behavior that it cap-

tures, and an extent of the program execution, called

scope, within which the behavior must hold. The pat-

terns are expressed as a combination of literal and

non-literal terminals. The non-literal terminals can be

either Boolean expressions that describe system prop-

erties, or integer values that capture timing aspects.

The rest of the pattern is made of literal terminals,

which represent the ﬁxed part of the pattern and can-

not be changed.

The original SPS catalog proposed by Dwyer

et al. (Dwyer et al., 1998) is compiled by analyz-

ing more than 500 examples of property speciﬁca-

tions from various domains. It contains 13 quali-

tative patterns, which for easier navigation are di-

vided into two categories: order and occurrence, ex-

pressed in various types of temporal logic, including

but not limited to (Timed) Computation Tree Logic

((T)CTL) (Alur et al., 1993), Real-time Interval Logic

(RTGIL) (Moser et al., 1997), Metrics Temporal Log-

ics (MTL) (Alur and Henzinger, 1993), etc. The pat-

terns in the later category describe the occurrence of

a given event in the system, while the patterns from

the ordering category are used to capture the rela-

tive ordering of the occurrence of multiple events dur-

ing system execution. In order to be able to specify

real-time systems properties, Konrad and Chen (Kon-

rad and Cheng, 2005) have subsequently extended the

speciﬁcation patterns catalog with a new category of

patterns called real-time. Consequently, the new and

extended catalog was named Real-time Speciﬁcation

Pattern System (RTSPS). Besides the introduction of

the new set of patterns, in the same work, Konrad and

Specifying Industrial System Requirements using Speciﬁcation Patterns: A Case Study of Evaluation with Practitioners

Cheng proposed an additional encoding of the speciﬁ-

cation patterns in controlled natural language (CNL).

The intended purpose of the CNL representation is

to enable various stakeholders who are not trained

in formal methods to read and interpret the system

speciﬁcation expressed via patterns. For instance, a

bounded-response pattern over events P and S with

global scope expressed in CNL, reads as follows:

Globally, it is always the case P is followed by S

within T seconds.

Based on the formal deﬁnitions for both the

scope (Globally) and the behavior (bounded-

response) (Dwyer et al., 1998; Konrad and Cheng,

2005), one can automatically generate the following

TCTL property speciﬁed using the CNL pattern

above:

AG(P ⇒ AF

≤T

The introduction of the real-time patterns did not

yield the catalog of patterns complete. In order to sat-

isfy the different purposes, the catalog has since been

extended in many directions. The most notable ones

include the introduction of probabilistic patterns by

Grunske (Grunske, 2008), which provide means for

one to specify probabilistic system properties. The

most comprehensive catalog compiled to date is by

Autili et al. (Autili et al., 2015). In our work, we

use the following subset of real-time speciﬁcation pat-

terns:

P1: Globally, Universally: Globally, it is always the case

that P holds.

P2: Timed Globally, Universally: Globally, it is always the

case that if P held for T time units, then S holds.

P3: Globally, Real-time Response: Globally, it is always

the case that if P holds, the S eventually holds within T

time units.

P4: After Q Until R Universally P: After Q, it is always the

case case that P holds until R holds.

P5: Timed After Q Timed Until R Universally P: After Q

holds for T time units, it is always the case that P holds

until R holds or at most T

time units.

2.2 ProPaS Tool

The PROPAS tool (Filipovikj et al., 2018) is an open-

source tool that is used for the formal system speciﬁ-

cation using speciﬁcation patterns. In the following,

we overview in details the user interface (UI), as well

as other non-UI-related features that have been intro-

duced to support our experiment.

The UI of the PROPAS tool is given in Figure 1,

and it represents an improvement of an earlier tool

called SeSAMM Speciﬁer (Filipovikj et al., 2016). In

the top left part of the UI we have the list of all the

Figure 1: ProPaS Tool User Interface.

system speciﬁcations (denoted with 1). The list can

be manipulated via commands for deleting an exist-

ing speciﬁcation or creating a new one, which are pro-

vided by a context menu that is available on right click

of the list. Next to it, the list of properties in the cur-

rently active system speciﬁcation are displayed. Sim-

ilarly as for the list of system speciﬁcations, one can

manipulate the list of properties (2) using the context

menu that is displayed upon right-clicking the list.

The currently available actions over the list of proper-

ties include removing an existing property or adding

a new one.

The middle part of the UI (3) contains details

about the currently active system property. The

details are composed of editable and read-only at-

tributes. The ﬁrst editable attribute for the system

property is its name, which is also used as an iden-

tiﬁer. In the particular example illustrated in Figure 1,

the name of the currently active property is “req1”.

Next comes the ﬁrst read-only attribute, called de-

scription, which is deﬁned by the selected pattern.

The speciﬁcation pattern for the property is applied

from the drop-down list, which contains all the avail-

able patterns that the tool has access to. Below the list

of available speciﬁcation patterns, a list of proposi-

tions (4) characteristic for the speciﬁc pattern is given.

In the bottom part of the UI (5) there are two elements

that serve the purpose of giving feedback to the user

on the speciﬁed behavior. First, there is a text ﬁeld

in which the combination of the property and the pat-

tern speciﬁc attributes are combined to automatically

ENASE 2019 - 14th International Conference on Evaluation of Novel Approaches to Software Engineering

generate the different views for the given property. In

this case, the speciﬁcation pattern is encoded in con-

strained natural language and TCTL. Lastly, the UI

shows a visual representation (6) of the ordering of

the events as speciﬁed by the selected pattern, which

in the example given in Figure 1 is the ordering of the

events according to the bounded-response pattern.

Beside the UI, the PROPAS tool provides other

features, out of which some are relevant and some

are not for the case study. Out of the study-relevant

features, the most important one is the timer feature,

which is used to keep track of the amount of time

that is spent on specifying each property of the set.

Out of the non-relevant features worth mentioning is

the consistency analysis feature, which has been pro-

posed and implemented in the previous version of the

tool. For more details on the consistency analysis,

we refer the reader to earlier work (Filipovikj et al.,

2018).

3 RESEARCH METHOD

In this section, we present the case-study research

method, which is used in the evaluation of the speci-

ﬁcation patterns and the PROPAS tool.

A case study is a research method in software en-

gineering research that aims at studying a contempo-

rary phenomena in their natural context (Wohlin et al.,

2012). Initially the term case study was used to de-

note a demonstration case, during which an imple-

mentation of some tool or other type of concept is pre-

sented. Subsequently, it got a broader meaning and is

also used to refer to self-experienced or self-reported

investigation.

The case-study research method is mostly used to

provide deeper understanding of the phenomena un-

der study. According to Runeson and H

ost (Rune-

son and H

ost, 2009), a case-study research method

is “well suited for many kinds of software engineer-

ing research, as the objects of study are contempo-

rary phenomena, which are hard to study in isola-

tion”. In general, the case-study research method

represents a more liberal way of studying some phe-

nomena related to a single project. It is easy to plan

and execute, and as such, it is useful for evaluating

new methodologies and tools in industrial settings.

On the other side, compared to other empirical re-

search methods, such as for instance the controlled

experiment (Wohlin et al., 2012), the results obtained

from a case study are generally more difﬁcult to in-

terpret and generalize. Despite the limitations, if a

well-established case-study research methodology is

followed, then the obtained results are valuable.

Relevant literature (Runeson and H

ost, 2009) sug-

gests that any well-designed case study should fol-

low at least the following ﬁve process steps: i) case

study panning and design, ii) data collection prepara-

tion, iii) data collection, iv) data analysis v) report-

ing. In order to better plan and execute the listed

steps of a case study, there are several papers that

provide comprehensive check lists (Kitchenham et al.,

1995; Runeson and H

ost, 2009), which contain series

of questions intended to guide the researchers to cor-

rectly plan and execute the research process.

During the planning and design phase, the most

important thing is to correctly determine the objec-

tive of the study. In software engineering, in gen-

eral there are four predominant types of case-study re-

search: exploratory, descriptive, explanatory and im-

proving (Robson and McCartan, 2016). Basically, the

type of the case-study research determines the charac-

teristics of the subsequent steps. For instance, if there

is a lack of empirical data on the subject, one resorts

to an exploratory case study where the main objec-

tive is to provide information on what is happening,

and generate ideas and hypothesis for new research.

Another important aspect to consider during the plan-

ning and design phase is to account for the trade-off

between the level of control over the experiment and

the degree of realism. Given that the primary focus

of a case-study research process is to study some phe-

nomena in their natural context, it is expected to lose

some control in favour of a high degree of realism.

In a case-study research, there are two types of

variables: independent and dependent. Independent

variables are those variables that can be controlled

and changed during the experiment. They must have

some effect on the dependent variables. The depen-

dent variable(s) a.k.a. response variables are the vari-

ables that cannot be controlled in the experiment, and

they are usually measured based on the independent

variables.

The data collected in a case study is either quali-

tative or quantitative. Quantitative data is represented

with numbers, whereas the qualitative data is usually

expressed in words, descriptions, diagrams, etc. Due

to its descriptive nature, the qualitative data is usu-

ally preferred in case-study research, however, a com-

bination of qualitative and quantitative data provides

better understanding of the phenomena under study.

There are three well-established methods for data col-

lection in case study, namely: direct, indirect and in-

dependent (Runeson and H

ost, 2009). Regardless of

which method is collected for the study, it is recom-

mended that multiple data sources are considered in

order to reduce the effects of the interpretation bias.

When it comes to the data analysis, different tech-

Specifying Industrial System Requirements using Speciﬁcation Patterns: A Case Study of Evaluation with Practitioners

Table 1: Case study subjects and their roles.

CSS Role

CSS

Software architect for autonomous systems

CSS

Development engineer (former quality assurance engineer)

CSS

Development engineer/researcher

CSS

Requirements Coach

CSS

Information/safety architect

niques apply for qualitative and quantitative data. The

accepted analysis methods for quantitative data in-

clude: descriptive statistics, correlation analysis, de-

velopment of predictive models and hypothesis test-

ing, whereas the qualitative data are usually analyzed

using categorization or sorting.

Finally, the report of the case study serves the pur-

pose of communicating the case-study results. It is

often the case that the report cannot be distinguished

from the case study itself, and as such it is the main

source for judging the overall quality of the study. For

communicating results from a case study to a research

community the preferred forms of reporting include

conference or journal articles or in some cases even

technical reports.

4 CASE STUDY PLANNING AND

EXECUTION

In this section, we present the details about the plan-

ning and execution of our case study, in which we

evaluate the PROPAS tool and the speciﬁcation pat-

terns based on their application by the industrial prac-

titioners.

4.1 Case Study Design

There is a lack of empirical data on whether the spec-

iﬁcation patterns can be effectively and efﬁciently

used by the engineers and other stakeholders for spec-

ifying their systems. Given this, the objective of our

case study is to be exploratory, that is, to provide an

initial set of empirical data that can be used later to

form hypotheses and execute larger case studies in

various companies and domains. For planning and

executing the case study, we are mainly guided by the

checklists for case-study planning and execution by

Kitchenham et al. (Kitchenham et al., 1995) and Run-

neson and H

ost (Runeson and H

ost, 2009).

For the case study, we consider an operational sys-

tem, namely the Fuel Level Display (FLD), which is

installed in all heavy load-vehicles produced by Sca-

nia, Sweden. The core of the system is a software

module, which is responsible for: i) reading the value

of the sensor installed inside the fuel tank, ii) calculat-

ing the estimated total available fuel left in the tank,

and iii) showing the estimated value on the dashboard

to the driver. The speciﬁcation of the FLD system

consists of twenty four requirements. As the intended

functionality of the system is not of a primary inter-

est for this case study, we omit further details. For

a detailed system description, we refer the reader to

the paper by Westman and Nyberg (Westman and Ny-

berg, 2014).

The main objective of this study is to collect data

that can be used to form a future hypothesis regard-

ing the practical usefulness of the speciﬁcation pat-

terns and the PROPAS tool for specifying require-

ments in the automotive domain. The data gathering

phase of the case study has been executed at the com-

pany premises, involving ﬁve engineering profession-

als from Scania as case-study subjects, each having

a different role in the systems development process.

The roles for each of the subjects are given in Table 1.

For the purpose our our study, the subjects have been

asked to specify the following subset of FLD require-

ments using a predeﬁned set of speciﬁcation patterns

(see Section 2.1) and the PROPAS tool:

If actualParkingBrake is false, then the indicated-

FuelVolume showed by the fuel gauge is less or

equal to actualFuelVolume.

If CAN1 is equal to a CAN message rmsg, then

rmsg is in HWreceiveBuffer and Rcan decodeCan

is set to true.

The position of the ﬂoater sensedFuelLevel,

sensed by the fuel sensor, does not deviate more

than 10% from actualFuelVolume in the fuel tank.

If CAN1 is equal to a CAN message rmsg, then

rmsg is in FiFoBuffer within 20ms.

If the DMA channels timerCh and rﬁfoCh are en-

abled for approximately 20ms, then a RAW value

of VFuel is available in ADC RFIFO.

If actualParkingBrake is false and CAN1 is equal

to CAN message FuelEconomy and it has not

passed more than 300ms since the last time CAN1

was equal to FuelEconomy, then the CAN signal

FuelRate in FuelEconomy does not deviate more

than 1% from the derivative of actualFuelVolume.

The given requirements represent a diverse subset

from the FLD functional requirements. The require-

ments capture both the untimed (R

, R

and R

) and

timed (R

, R

) system behavior.

The case study is executed in three major phases.

In the ﬁrst phase, we compiled a list of potential case-

study subjects and contacted them personally in order

to obtain their consent for participation in the study.

ENASE 2019 - 14th International Conference on Evaluation of Novel Approaches to Software Engineering

Upon our request all of the potential case-study sub-

jects agreed to participate. Since none of the sub-

jects has previous knowledge or experience in using

the speciﬁcation patterns and the PROPAS tool, we

prepared and distributed to each of them a short tu-

torial. The tutorial, in a form of a written document,

contains a compact description about the general idea

of speciﬁcation patterns, the set of requirements in-

tended for the case study and a set of step-by-step

instructions (complemented by screen shots of each

step) on how to use the PROPAS tool. During the sec-

ond phase of the case study, which is the data collec-

tion phase, one of the researchers met with each of the

subject in person in order to carry out the given task

of specifying the predeﬁned set of requirements us-

ing the PROPAS tool. The last phase of the case study

included data analysis, drawing conclusions and pro-

ducing the case-study report.

As independent variables in our case study we

identify the following: i) the system (FLD) and the

subset of the functional requirements used for speci-

ﬁcation, ii) the PROPAS tool and the set of patterns,

and iii) the case-study subjects. The set of dependent

variables consists of the following: i) correctness of

the speciﬁcation, ii) required time for speciﬁcation,

and iii) level of domain expertise of the case-study

subjects.

4.2 Data Collection Preparation

The quantitative and qualitative data in this case study

is collected from more than one source. For collecting

the qualitative data, we use a direct method facilitated

by the “save to disk” feature of the PROPAS tool,

which enables the subjects to save their system speci-

ﬁcations on a computer hard drive. The data produced

by the PROPAS tool includes a pattern-based spec-

iﬁcation of the predeﬁned set of requirements aug-

mented with information on how much time was spent

for specifying each individual requirement.

For qualitative data, we use a survey composed of

the following ten statements:

I can specify the given set of requirements using

the speciﬁcation patterns (SPS).

The SPS formalized patterns expressed in con-

strained natural language are comprehensible.

The visual representation of the pattern is helpful

in understanding the behavior represented by the

selected pattern.

The PROPAS tool is intuitive and easy to use.

The PROPAS tool is practically useful for specify-

ing requirements using SPS.

The quality of generated speciﬁcations is high.

I feel conﬁdent using the PROPAS tool.

I am able to use the tool without any additional

support.

The tool is helping in decreasing the speciﬁcation

effort.

I can easily select the appropriate pattern for a

given requirement.

Each of the above statements is intended to cap-

ture the subjects’ opinion in terms of degree of satis-

faction, via the following set of predeﬁned options: i)

“I strongly agree”, ii) “I agree”, iii) “I neither agree

or disagree”, iv) “I disagree” and v) “I strongly dis-

agree”. For collecting the qualitative data, we use

the online Google Survey. Some of the main advan-

tages of using this tool for collecting the qualitative

data are: fast and straightforward survey generation,

distributed access to the questions, real-time access to

the data once entered by the case-study subjects and

the convenient visual representations of the results of-

fered by the tool.

4.3 Data Collection

The data collection was performed according to the

data collection plan described in Section 4.2. The

qualitative data was collected in real-time during the

study execution through the PROPAS tool, at Scania

premises. To avoid technical problems, such as soft-

ware compatibility that includes the operating sys-

tem and the run-time environment required by the

PROPAS tool, the tool was pre-installed on a laptop

provided by the researchers. Then, the machine with

the PROPAS tool was used by each of the case-study

subjects to perform the system speciﬁcation and to

ﬁll in the survey. During the data collection phase,

only the current case-study subject and one of the re-

searchers involved in the case study were present in

the room. The subjects were expected to use the tool

on their own, based on the information from the pre-

viously provided tutorial. However, during the spec-

iﬁcation process, the subjects were allowed to inter-

act with the researcher/expert in order to improve the

accuracy of their results and increase the conﬁdence

in their work. Each interaction initiated by the case-

study subject was recorded by the researcher, as part

of the qualitative data that characterizes the practical

usefulness of the method and the tool. In general, the

less the interaction between the subject and the expert,

the higher the understanding of the subject regarding

the method and the tool.

After the subjects declared that they were done

with the speciﬁcation of the predeﬁned set of require-

Specifying Industrial System Requirements using Speciﬁcation Patterns: A Case Study of Evaluation with Practitioners

ments, they were asked to ﬁll in the survey with

qualitative data. In addition to the survey, we col-

lected the reﬂections and the comments of the sub-

jects expressed while performing the system speciﬁ-

cation task using PROPAS.

After both the system speciﬁcation and the sur-

vey were completed, the data collection phase was

concluded for that particular subject. At the end of

the data collection phase, the following data was col-

lected: ﬁve system requirements speciﬁcations con-

taining the predeﬁned set of requirements from the

FLD system complemented with the amount of time

spent per requirement, and ﬁve survey responses to

our survey for the qualitative data.

4.4 Data Interpretation and Analysis

The third, and the last phase of the case study is the

data interpretation and analysis. The gathered quanti-

tative data is analyzed using statistical methods. In

order to determine the correctness of the speciﬁca-

tion, that is, to determine whether a correct speciﬁ-

cation pattern is used for a given requirement, we use

a baseline system speciﬁcation of the same set of re-

quirements, which was created by a formal methods

expert.

The gathered qualitative data on the other hand,

is interpreted in a less formal way, mainly through a

discussion of the obtained survey data. Given that the

primary purpose of the qualitative data is to provide

us with a subjective reﬂection of the case-study sub-

jects on the whole process, the discussion is mainly

intended to draw conclusions and extract suggestions

that can be useful for setting up future similar case

studies, but also for the researchers in order to further

improve the PROPAS tool or any other tool designed

for the purpose of specifying (formal) requirements.

5 RESULTS

In this section we show the results obtained from the

case study. In Section 5.1 we present the quantitative

data, followed by the qualitative data results in Sec-

tion 5.2, after which we discuss the potential threats

to validity of our results in Section 5.3.

5.1 Quantitative Data Analysis

The collected quantitative data in the case study, rep-

resented through the correctness of the speciﬁcations

produced using the speciﬁcation patterns and the time

for speciﬁcation are given in Tables 2 and 3, respec-

tively.

Table 2: Pattern-based requirements speciﬁcation correct-

ness.

CSS R

total

CSS

X X X X X X 100%

CSS

× × X X X X 66.67%

CSS

X X X X X X 100%

CSS

X X X X X X 100%

CSS

X X X X X × 83.33%

total 80% 80% 100% 100% 100% 80% 90%

Table 3: Requirements speciﬁcation time in minutes.

CSS R

total

CSS

3 2 2 2 4 13 26

CSS

3 6 2 2 1 16 30

CSS

2 2 2 2 2 16 26

CSS

3 2 2 2 2 18 29

CSS

2 3 2 2 2 10 21

avg. 2.16 2.50 1.66 1.66 1.83 12.16 26.4

The correctness of the produced system speciﬁca-

tions given in Table 2, shows the data both per case-

study subject and the individual requirements, respec-

tively. In the same table, we denote by the symbol (X)

that a given subject has speciﬁed a particular require-

ment correctly, meaning that the appropriate patterns

for speciﬁcation has been selected, and the result is

equivalent with the baseline speciﬁcation. The oppo-

site case, that is, when an incorrect pattern is used is

denoted by the symbol (×). By analyzing the speciﬁ-

cation correctness per subject, we observe the follow-

ing: three of the subjects achieve 100% correctness,

that is, for each of the six requirements a correct spec-

iﬁcation pattern is used. One of the subjects selected

an incorrect pattern for requirements R

and R

, thus

achieving (66.67%) correctness of the speciﬁcation.

Lastly, one subject used an incorrect pattern for one

requirement, namely R

, resulting in 83.33% correct-

ness of the produced speciﬁcation.

If we analyze the collected data per requirement,

we observe a single mistake (80% correctness) in the

speciﬁcation of the requirements R

, R

and R

. The

rest of the requirements are speciﬁed correctly (100%

correctness) by all of the case-study subjects. If we

consider the total number of instances of the speci-

ﬁed requirements, which is thirty and compare them

to only three mistakes, the overall conclusion is that

we have 90% correctness for the speciﬁcation of all

requirements by all subjects.

The second aspect of the collected quantitative

data is the speciﬁcation time, which is given in Ta-

ble 3. The table shows the speciﬁcation time in min-

utes. Similarly as for the correctness of the speciﬁca-

tion, the results for measured speciﬁcation time can be

analyzed both per subject and per requirement. Ana-

lyzing the speciﬁcation time per subject, shows that

ENASE 2019 - 14th International Conference on Evaluation of Novel Approaches to Software Engineering

the shortest time for specifying the complete set of

requirements is 21 minutes, whereas the slowest one

is 30. The average speciﬁcation time for the set of

the requirements per subject is 26.4 minutes. The av-

erage time per requirement is 4.4 minutes, or more

accurately around 2 minutes for R

to R

, whereas for

is around 12 minutes, which accounts for almost

half of the time spent on the entire speciﬁcation.

5.2 Qualitative Data Analysis

The qualitative data in the case study is collected in

two ways: through a survey and by taking notes of

comments by the case-study subjects during the data

collection process. While the data collected from the

survey is based on answers to a predeﬁned set of state-

ments, the notes taken during the data collection are

based on comments by the subjects and as such are in-

tended to replace the free-text question in the survey.

The intended purpose of the qualitative data collec-

tion in this case study is to give us a more clear picture

on how the subjects perceive the system speciﬁcation

using the speciﬁcation patterns and the PROPAS tool,

and to point to some directions for future work with

respect to both the method and the tool, such that their

practical usefulness is improved.

Based on the responses of the subjects, we have

the following quantitative data:

20% strongly agree, 60% agree, 20% neither

agree or disagree

60% strongly agree, 40% agree

20% strongly agree, 20% agree, 60% neither

agree or disagree

20% strongly agree, 80% agree

40% strongly agree, 40% agree, 20% neither

agree or disagree

60% strongly agree, 40% neither agree or disagree

20% strongly agree, 40% agree, 20% neither

agree or disagree, 20% disagree

40% strongly agree, 40% agree, 20% neither

agree or disagree

40% strongly agree, 60% agree

20% strongly agree, 20% agree, 60% neither

agree or disagree

The obtained qualitative data can be interpreted in

various ways, however, it is clear that the case-study

subjects are positively inclined towards the speciﬁca-

tion patterns and the PROPAS tool, which can be con-

cluded based on the fact that majority of the state-

ments are described using positive to neutral phrases.

The only negative description can be observed for the

statement S

, where 20% of the subjects express that

they do not feel conﬁdent in using the PROPAS tool.

The qualitative data obtained through the survey

is additionally complemented with unrestricted feed-

back provided by the case study subjects while they

are performing the speciﬁcation using the PROPAS

tool. Since the case-study subjects were not explic-

itly asked to provide this kind of feedback, CSS

and

CSS

completed the speciﬁcation task without giv-

ing any feedback. The rest of the subjects provided

their opinion either during the speciﬁcation process

or immediately after the speciﬁcation phase was con-

cluded. The provided feedback is not extensive, but it

is very important, as the case-study subjects were free

to elaborate on the aspects of both the method and the

tool that were not reﬂected by the survey. From the

obtained feedback, we point out the following: CSS

was positive during the speciﬁcation of the require-

ments. The subject pointed out that the method is

useful, however, the subject also added some sugges-

tions for improving the tool. Concretely, it has been

pointed out that in order for a tool such as PROPAS

to be considered for adoption in in the industrial de-

velopment process of automotive systems, it should

also provide requirements management features such

as versioning, version comparison, etc. The reﬂection

of CSS

is very interesting as the subject has phrased

his experience with the speciﬁcation patterns and the

PROPAS tool as follows: “I can write requirements

in natural language as free-text much faster, but the

pattern-based requirements speciﬁed via the PROPAS

tool are much more concise”. Lastly, similar to CSS

and CSS

, CSS

also had a positive incline towards

the method and the tool, however, the subject pointed

out that this version of the tool must be improved in

order to be considered for industrial adoption. Espe-

cially, it has been pointed out that the visual feedback

mechanism should be coupled with the speciﬁed re-

quirement and not with the pattern as it is now in the

tool. Similar comment has been given by CSS

5.3 Threats to Validity

In this section we discuss the threats to validity of the

results obtained in this case study. First, we discuss

the internal threats to validity and then the external

ones.

Internal Threats to Validity are often deﬁned as

inﬂuences that can affect the independent variables

with respect to causality without the knowledge of

the researchers. Based on this deﬁnition, we iden-

tify the following internal threats to validity for our

case study: i) case-study subjects selection, and ii)

the number of requirements used in the case study.

Specifying Industrial System Requirements using Speciﬁcation Patterns: A Case Study of Evaluation with Practitioners

The selection of the case study subjects is done in

such a way as to satisfy a number of criteria, includ-

ing the following: i) have at least one representative

of each category of stakeholders involved in the sys-

tem development process, and ii) have subjects with

various levels of experience and expertise. The case

study subjects and their roles are given in Table 1.

Given that there are number of Scania employees

satisfy the above criteria, we additionally took into

consideration the following factors, which are not rel-

evant for the validity of the results: the availability of

the subjects at a speciﬁc time and their willingness to

participate in the case study.

Due to the limited time that the case-study sub-

jects were assigned for the participation in the case

study, we were required to isolate a small subset of all

the possible requirements. For that purpose, we com-

piled a set of requirements that are feasible to be spec-

iﬁed within the given time frame of around one hour.

Regarding the selection of the system and the require-

ments to be used in the case study, we had to take into

consideration several factors such as: complexity of

the system and its speciﬁcation, and the possibility to

make the requirements public.

External Threats to Validity represent the condi-

tions that limit the ability to generalize the results

from the case study over the whole domain. In our

case study, we consider the following two factors as

external threats to validity: i) the selection of the par-

ticular system and the particular set of requirements

used in the study, and ii) the relative domain expertise

of the case study subject compared to their peers from

the same company or from the companies within the

automotive domain.

The system used in the case study and the rather

small set of requirements hinder us from the general-

izing the obtained results, however, the obtained re-

sults serve well the purpose of an exploratory case

study aimed at gathering empirical data. There are

several concerns that arise from the system and the

mentioned selections, such as: will the results be sim-

ilar if a different system would be used? Will results

be the same if more requirements of the same sys-

tem were used or more requirements but from differ-

ent system(s)? In order to answer these questions, a

larger case study that includes more systems and case

study subjects is required.

The second external threat to the validity of the re-

sults is the relative expertise of the case study subjects

compared to their peers, be they from the same com-

pany or from a different one. We have no means nor a

set of well-deﬁned parameters on how to measure the

expertise of the case-study subjects. Consequently, in

order to be able to pose and test hypotheses for the

whole domain, one needs to include more case study

subjects preferably from different companies in the

same domain, or at least to ﬁnd an objective metrics

to quantify the expertise of the case study subjects.

6 DISCUSSION

In this section, we discuss both the quantitative and

the qualitative data obtained from the case study.

Based on the discussion, we draw some conclusions

which can be useful in deﬁning future hypothesis.

The collected quantitative data shows that on av-

erage the industrial practitioners achieve 90% correct-

ness of the speciﬁcation, while spending 4.4 minutes

on average per requirement. However, a more in-

depth analysis of the results shows some surprising

trends. The selected group of requirements are not

of equal complexity. For instance, the simplest re-

quirements specify instant causality (such as in R

whereas the most complex one (R

) speciﬁes com-

plex behavior captured by occurrence of two events.

Intuitively, one would expect that most of the mis-

takes in the speciﬁcation to occur in specifying the

most complex requirements, nevertheless, our data

does not provide evidence to support this expectation.

If we refer back to Table 2, we can see that three mis-

takes made in the speciﬁcation have been connected

to different requirements, with only one being for R

When it comes to required time for speciﬁcation, the

data given in Table 3 shows that the time required for

specifying the requirements is proportional to their

complexity. The average time required for specifying

is 12.1 minutes, which is almost half of the total

average time for the whole speciﬁcation.

Some of the subjects employed interesting strate-

gies/heuristics when it comes to selecting the appro-

priate pattern and identifying the propositions for a

given requirement. For instance, CSS

selected the

wrong pattern, that is, P3 instead of P1 for both R

and

. Given the fact that P3 by deﬁnition incorporates

three propositions, compared to only one in P1, the

subject introduced new data that does not exist in the

original requirement (the maximum response time) in

order to ﬁll in the missing propositions. CSS

also

employed an interesting heuristic for selecting a pat-

tern for a speciﬁc requirement, as follows: ﬁrst, when

reading the requirement, the subject tried to determine

whether the requirement incorporates timing informa-

tion or not. Given that in the case study only one of

the pattern is untimed (does not encode time explic-

itly), for requirements that do not incorporate time the

subject selected P1 without much hesitation. For the

rest of the requirements, which are timed, the subject

ENASE 2019 - 14th International Conference on Evaluation of Novel Approaches to Software Engineering

100

analyzed the structure of the pattern by propositions

that model events and time and tried to match it with

the information encoded in the given requirement.

The analysis of the qualitative data shows that

the industrial practitioners are generally favourable

for the speciﬁcation patterns. This is supported by

the fact that they have expressed positive opinions

about the method and the quality of the generated sys-

tem speciﬁcations. However, based on the qualitative

feedback, some issues, primarily related to the tool

support must be address in order for the pattern-based

speciﬁcation to have more signiﬁcant penetration in

industrial settings.

Overall, the speciﬁcation patterns and the

PROPAS tool seem to be a promising combination for

a pattern-based speciﬁcation of requirements in in-

dustrial settings. The method and the tool in general

are easy to use and the quality of the speciﬁcations is

perceived as good by the industrial practitioners. The

qualitative data also suggests that the combination of

speciﬁcation patterns and PROPAS tool seems to re-

duce the speciﬁcation effort for the engineers. On the

other hand, the main challenge for the pattern-based

speciﬁcation of requirements for engineers who are

not skilled in formal methods is the validation of the

expressed behavior. This is observed by the fact that

the subjects express neutral and in some cases nega-

tive opinion (S

and S

) on the quality of the generated

speciﬁcations and the conﬁdence in what they are do-

ing.

7 RELATED WORK

In the literature, there exists a body of works that used

the speciﬁcation patterns for capturing industrial re-

quirements from various domains. In the following,

we list some of the recent attempts.

To the best of the authors’ knowledge, there are

only two reported cases in the literature that are fo-

cused on accessing the suitability of the speciﬁcation

patterns for specifying requirements in the automo-

tive domain. The ﬁrst case study was reported by

Post et al. (Post et al., 2012). In their work, the au-

thors conduct a case study at BOSCH, one of the lead-

ing suppliers in automotive domain, in which they

aim to study whether the speciﬁcation patterns are

expressive enough to formalize requirements in the

automotive domain. The results from the case study

suggest that the speciﬁcation patterns are expressive

enough for capturing most of the requirements used

in the study. Moreover, they also observed that the

small subset of patterns are enough to formalize most

of the requirements. Since the results were obtained

based on a rather limited set of requirements and in-

volved only one company, it is hard to generalize

the results to the entire domain. In order to further

strengthen or disprove the claims of Post et al., a fol-

low up case study was carried out at Scania by Fil-

ipovikj et al. (Filipovikj et al., 2014). The case study

involved one hundred requirements from four differ-

ent systems. The results of the study are aligned with

the ones of Post et al., and as such represent one step

close towards being able to generalize for the whole

domain. Compared to our current study, both of the

previous attempts were completely oriented towards

accessing the expressiveness of the speciﬁcation pat-

terns, and as such were performed by formal meth-

ods experts and not actual engineers. Also, neither of

the studies involved a specialized tool support since

the experts where comfortable with using general pur-

pose text editors or pen and paper.

In order to support potential users of the spec-

iﬁcation patterns, be they from academia or indus-

try, number of research tools of various maturity

have been proposed. The Property Speciﬁer tool

(Prospec) (Mondragon and Gates, 2004) is one of the

ﬁrst tools proposed to facilitate the usage of speciﬁca-

tion patterns for users not trained in formal methods.

It provides a complex interface that supports many

degrees of freedom when specifying complex prop-

erties. Another similar tool is the Property Elucidator

(PROPEL) tool (Smith et al., 2002). Both tools rep-

resent the pattern-expressed requirements as a ﬁnite

state machines, however, none of them has been eval-

uated by industrial practitioners, thus no data on their

practical usefulness is available. Another tool for as-

sisting non-experts in formal methods is the Property

Assistant (PASS) proposed by Remenska et al. (Re-

menska et al., 2014). In comparison to the afore-

mentioned tools, PASS uses UML constructs to show

the speciﬁed behavior captured by the patterns. The

tool also provides a multi-feature interface that can

be used to specify complex system properties, how-

ever, similar to the previous two tools, it has not been

evaluated by industrial practitioners. Lastly, the PSP-

Wizard tool (Autili et al., 2015) represents the latest

attempt to produce a speciﬁcation-pattern-based tool.

Compared to the previous tools, the PSPWizard in-

corporates the most comprehensive catalog of speciﬁ-

cation patterns and as such can potentially be used to

specify the widest pallet of different properties. The

PSPWizard tool and the pattern-based speciﬁcation

framework proposed in (Autili et al., 2015) have suc-

cessfully been applied on an industrial use case, how-

ever, the assessment was performed by researchers

(presumably experts in formal methods) and not the

actual practitioners (engineers).

Specifying Industrial System Requirements using Speciﬁcation Patterns: A Case Study of Evaluation with Practitioners

101

Compared to all of the aforementioned reports, in

this case study we speciﬁcally aim at accessing the

practical usefulness of the speciﬁcation patterns via

the PROPAS tool in industrial settings. As such, the

purpose of our exploratory case study is to generate

an initial set of empirical data about the potential us-

ability of the speciﬁcation patterns in industrial set-

tings, which should serve a good starting point for

conducting bigger case studies of similar type in order

to empirically access the usefuleness of the speciﬁca-

tion patterns.

8 CONCLUSIONS

In this paper, we presented an exploratory case study

through which we gather empirical data for the prac-

tical usefulness of the speciﬁcation patterns via the

PROPAS tool for specifying system requirements in

industrial settings, more precisely in the automotive

domain.

The reported case study was performed in collab-

oration with Scania, one of the world’s leading manu-

facturer of heavy-load vehicles, including trucks and

buses. As case-study subjects, we used ﬁve stakehold-

ers from the embedded software development pro-

cess at Scania. The case-study subjects belong to the

following roles: two development engineers, one re-

quirements coach, one safety architect and one sys-

tems architect. The case study was executed in three

major phases: preparation, data gathering and analy-

sis, and reporting. In the ﬁrst phase, the subjects were

provided with a short tutorial on the speciﬁcation pat-

terns and the PROPAS tool. In the second phase, we

visited the subjects at the company premises where

they performed the speciﬁcation of the requirements

using the PROPAS tool and answered a survey, such

that both quantitative and qualitative data is collected.

For the quantitative data we have considered the fol-

lowing: correctness of the speciﬁcation and time for

speciﬁcation, whereas for collecting the qualitative

data we used a survey of ten statements, which was

completed by the case-study subjects immediately af-

ter the speciﬁcation of the requirements.

The analysis of the quantitative data shows the fol-

lowing: the overall speciﬁcation correctness is 90%,

meaning that only three out of thirty requirements

speciﬁcation instances were incorrectly speciﬁed by

the subjects. The correctness results do not show a

direct causal relationship between the complexity of

the requirements from the system speciﬁcation and

the correctness of the same. The average time for

formalizing the predeﬁned set of requirements is 26.4

minutes, or 4.4 minutes per requirement. If we look at

the speciﬁcation time per individual requirement (see

Table 3), it is clear that the complexity of the require-

ment has a direct effect on the speciﬁcation time. For

instance, the average speciﬁcation time for the most

complex requirement (R

) is 12.16 minutes, which

is very close to the time spend for the speciﬁcation

of the rest of the ﬁve requirements combined. The

qualitative data, which was obtained via survey (see

Section 4.1) shows that the engineers are positively

inclined towards system speciﬁcation via speciﬁca-

tion patterns supported by an adequate tool support,

such as the PROPAS tool. However, the same data re-

veals that in order to have more penetration and real

industrial impact, the specialized tool support must

be signiﬁcantly improved, especially in the following

aspects: provide a validation mechanism that should

help non-experts in formal methods to validate the

speciﬁed behavior, as well as requirements manage-

ment features such as traceability, versioning, etc. De-

spite the identiﬁed drawbacks, our results show that

the speciﬁcation patterns that are supported by an ad-

equate tooling can be practically useful for formaliz-

ing requirements by engineers and other stakeholders

who are not experts in formal methods.

Based on the results from the case study, we en-

vision the following directions for future research.

First, we aim to deﬁne a set of hypotheses to be tested

in future case studies, which should involve more sys-

tems, subjects and possibly include other companies,

either from the same or different domain. The second

direction for future work is to further improve the spe-

cialized tool support, meaning to enrich the PROPAS

tool with new features that address the recommenda-

tions and suggestions extracted from the qualitative

data. This direction involves both engineering (tool

implementation) and research efforts (ﬁnding a suit-

able method for validation of speciﬁed behavior).

ACKNOWLEDGEMENTS

This work has been funded by the Swedish Govern-

mental Agency for Innovation Systems (VINNOVA)

under the VeriSpec project 2013-01299.

REFERENCES

Alur, R., Courcoubetis, C., and Dill, D. (1993). Model-

checking in dense real-time. Information and compu-

tation, 104(1):2–34.

Alur, R. and Henzinger, T. A. (1993). Real-time log-

ics: Complexity and expressiveness. Information and

Computation, 104(1):35–77.

ENASE 2019 - 14th International Conference on Evaluation of Novel Approaches to Software Engineering

102

Autili, M., Grunske, L., Lumpe, M., Pelliccione, P., and

Tang, A. (2015). Aligning qualitative, real-time, and

probabilistic property speciﬁcation patterns using a

structured english grammar. IEEE Transactions on

Software Engineering, 41(7):620–638.

Barnat, J., Bauch, P., Bene

s, N., Brim, L., Beran, J., and

Kratochv

ıla, T. (2016). Analysing sanity of require-

ments for avionics systems. Formal Aspects of Com-

puting, 28(1):45–63.

Dwyer, M. B., Avrunin, G. S., and Corbett, J. C. (1998).

Property speciﬁcation patterns for ﬁnite-state veriﬁca-

tion. In Proceedings of the second workshop on For-

mal methods in software practice, pages 7–15. ACM.

Filipovikj, P., Jagerﬁeld, T., Nyberg, M., Rodriguez-Navas,

G., and Seceleanu, C. (2016). Integrating pattern-

based formal requirements speciﬁcation in an indus-

trial tool-chain. In Computer Software and Applica-

tions Conference (COMPSAC), 2016 IEEE 40th An-

nual, volume 2, pages 167–173. IEEE.

Filipovikj, P., Nyberg, M., and Rodriguez-Navas, G. (2014).

Reassessing the pattern-based approach for formal-

izing requirements in the automotive domain. In

Proceedings of the 22nd IEEE International Re-

quirements Engineering Conference (RE), volume 00,

pages 444–450, Los Alamitos, CA, USA. IEEE Com-

puter Society.

Filipovikj, P., Rodriguez-Navas, G., Nyberg, M., and Sece-

leanu, C. (2018). Automated SMT-based consistency

checking of industrial critical requirements. ACM

SIGAPP Applied Computing Review, 17(4):15–28.

Grunske, L. (2008). Speciﬁcation patterns for probabilis-

tic quality properties. In 2008 ACM/IEEE 30th Inter-

national Conference on Software Engineering, pages

31–40. IEEE.

Kitchenham, B., Pickard, L., and Pﬂeeger, S. L. (1995).

Case studies for method and tool evaluation. IEEE

software, 12(4):52–62.

Konrad, S. and Cheng, B. H. C. (2005). Real-time speciﬁca-

tion patterns. In Proceedings of the 27th International

Conference on Software Engineering, ICSE ’05, pages

372–381, New York, NY, USA. ACM.

Mondragon, O. A. and Gates, A. Q. (2004). Support-

ing elicitation and speciﬁcation of software properties

through patterns and composite propositions. Inter-

national Journal of Software Engineering and Knowl-

edge Engineering, 14(01):21–41.

Moser, L. E., Ramakrishna, Y., Kutty, G., Melliar-Smith,

P. M., and Dillon, L. K. (1997). A graphical envi-

ronment for the design of concurrent real-time sys-

tems. ACM Transactions on Software Engineering

and Methodology (TOSEM), 6(1):31–79.

Post, A., Hoenicke, J., and Podelski, A. (2011). rt-

inconsistency: a new property for real-time require-

ments. In International Conference on Fundamen-

tal Approaches to Software Engineering, pages 34–49.

Springer.

Post, A., Menzel, I., Hoenicke, J., and Podelski, A. (2012).

Automotive behavioral requirements expressed in a

speciﬁcation pattern system: a case study at bosch.

Requirements Engineering, 17(1):19–33.

Remenska, D., Willemse, T. A., Templon, J., Verstoep, K.,

and Bal, H. (2014). Property speciﬁcation made easy:

Harnessing the power of model checking in uml de-

signs. In International Conference on Formal Tech-

niques for Distributed Objects, Components, and Sys-

tems, pages 17–32. Springer.

Robson, C. and McCartan, K. (2016). Real world research.

John Wiley & Sons.

Runeson, P. and H

ost, M. (2009). Guidelines for conduct-

ing and reporting case study research in software en-

gineering. Empirical Softw. Engg., 14(2):131–164.

Smith, R. L., Avrunin, G. S., Clarke, L. A., and Osterweil,

L. J. (2002). Propel: an approach supporting property

elucidation. In Proceedings of the 24th International

Conference on Software Engineering, pages 11–21.

ACM.

Westman, J. and Nyberg, M. (2014). Environment-Centric

Contracts for Design of Cyber-Physical Systems. In

MODELS, pages 218–234.

Wohlin, C., Runeson, P., H

ost, M., Ohlsson, M. C., Reg-

nell, B., and Wessl

en, A. (2012). Experimentation in

software engineering. Springer Science & Business

Media.

Specifying Industrial System Requirements using Speciﬁcation Patterns: A Case Study of Evaluation with Practitioners

103