Measuring Architecture Principles and Their Sets in Practice:

Creating an Architecture Principle Measurement Instrument

Challenged in a Case Study at the Dutch Tax Agency

Michiel Borgers and Frank Harmsen

School of Business and Economics, Maastricht University, Tongersestraat 53, Maastricht, The Netherlands

Keywords: Architecture Principles, Measurement Instrument, Characteristics, Attributes, Information System, Case

Study.

Abstract: Although architecture principles are important in the implementation of information systems requirements,

empirical evidence of the effect of architecture principles is lacking. To find this empirical evidence, we need

an instrument to measure architecture principles in the first place. This paper is the result of creating an

architecture principle measurement instrument challenged in a case study. We describe both the measurement

instrument and the related measurement method, including the test in a real-life case. Based on the outcome

we extended the instrument with extra architecture principles characteristics and attributes. We also made

some improvements on the measurement method as well.

1 INTRODUCTION

As we described in (Borgers and Harmsen, 2018),

architecture principles play, in theory, a key role in

guiding the design and the implementation of

information system (IS) requirements. Examples of

architecture principles are: A fact is stored only once,

or Reuse, before Buy, before Make. But the question

is: are architecture principles effective in practice, i.e.

do they have a – hopefully positive – effect on the

implementation of IS requirements?

To answer this overarching question of our

research, we consider the use of architecture

principles during the implementation of IS

requirements. We measure both the architecture

principles and the success of the implementation of

the information system requirements, in order to

determine a correlation. Our research is an

investigation into the practical value of architecture

principles: does the theoretical promise of

architectures principles have a positive effect on the

implementation of information system requirements

in practice?

The current paper is the result of creating an

architecture principle measurement instrument

challenged in a case study. We have used the

definition as well as the framework for defining and

describing architecture principles introduced in

(Borgers and Harmsen, 2018), to create a

measurement instrument and method. With this

instrument we measure architecture principles in

practice, testing the definition and model in a real-life

case.

We start this paper with the research methodology

in section 2. In the third section we provide an

architecture principle measurement instrument. The

validation results of the measurement instrument,

based on a case study, are described in section four.

We finish this article with limitations, conclusions

and further research.

2 RESEARCH METHODOLGY

2.1 Research Question

Our literature study (Borgers and Harmsen, 2018)

resulted in a theoretical framework for architecture

principles. The next step is to challenge this

framework in practice: Is it, in a real-life situation,

useful to measure the architecture principles?

Therefore, we phrased the research question:

"How to measure architecture principles and their

sets in practice?"

To answer this question, we need an instrument to

measure both the individual architecture principles as

534

Borgers, M. and Harmsen, F.

Measuring Architecture Principles and Their Sets in Practice: Creating an Architecture Principle Measurement Instrument Challenged in a Case Study at the Dutch Tax Agency.

DOI: 10.5220/0007787905340543

In Proceedings of the 21st International Conference on Enterprise Information Systems (ICEIS 2019), pages 534-543

ISBN: 978-989-758-372-8

well as the set of principles used. This instrument

should be able to identify architecture principles and

their sets in the first place. Secondly, using the

theoretical framework, we have to determine the

possible values of the characteristics and attributes

out of this framework. In determining the

characteristics and attributes, it is important that the

description is coherent and complete. So, the first sub

question is:

I: “What does an Architecture principle measurement

instrument look like?"

To be able to answer the overall research question,

a measurement instrument in itself is not enough. We

also need a method to collect and to analyze the

information. The second sub question, therefore, will

be:

II: "How to use the Architecture principle

measurement instrument in practice?"

To answer these two research questions, we used

the approach described below.

2.2 Case Study Approach

We have chosen the case study approach for

answering our research questions. There are, in

general, two reasons for using the case study

approach:

1. Early phases of research: case studies are useful

“in early phases of research where there may be

no prior hypotheses or previous work of

guidance” according to (Steenhuis and De Bruijn,

2004), but also stated by (Eisenhardt, 1989).

2. Context-related: if a phenomenon is strongly

related to its context, case study research “is used

to investigate a specific phenomenon through an

in-depth limited-scope study” (Eisenhardt, 1989).

Yin states that case studies are necessary “when

the boundaries between phenomenon and context

are not clearly evident” (Yin, 1984).

Architecture principles have been studied since

the early 1990’s. This would imply that the first

reason does not apply to our research. But in

(Borgers, 2016) we already identified that there is a

lack of research about the practical use of architecture

principles. And although there is a scientific basis laid

out by this theory, we would like to challenge its

adequacy, “because they have little empirical

substantiation” as Eisenhardt (1989) would call it.

And, although some of the past publications

introduced new definitions and descriptions, they all

confirmed the conclusions of previous publications

(Borgers and Harmsen, 2018). It is time to

juxtaposition theory and practice.

For answering our research question the second

reason to choose for the case study approach, is valid

as well. In our literature review we cited several

authors that “the context in which the architecture

principles are used, is important as well, in particular

for the effect of a principle.” (Borgers and Harmsen,

2018). So, architecture principles are conceptual

instruments used by people in the context of the

design and implementation process.

2.3 Research Steps

A research approach has to be reliable and valid

(Babbie, 2015a). Therefore, especially in a case study

approach, it is important to guarantee the objectivity

of fact findings. Therefore, we looked at the steps

defined by (Eisenhardt, 1989). She based her steps on

literature from authors like (Yin, 1981; 1984; Miles

and Huberman, 1984; Strauss, 1987) and experience

from authors conducting case studies like (Harris and

Sutton, 1986; Eisenhardt and Bourgeois, 1988;

Gersick, 1988). We grouped the steps of Eisenhardt

in the following three phases:

Figure 1: Case study approach.

1. Preparing the Case Study: in this phase we

defined the research question, as described above.

Secondly, we selected a case. Because our overall

research program is focusing on Dutch

government organizations, we chose a case from

the Dutch Tax Agency. One reason to choose this

case study was, of course, that the project did use

architecture principles in the first place. Besides,

the chosen project had to be finished: we could see

the use of principles during the entire

implementation life cycle. We crafted the

instruments and protocols to be used during the

case study research. We defined our research team

with subject matter experts, defined our survey,

and built our measurement instrument and

method.

2. Doing the Research: this was the iterative phase

of data collection, analysis and theory building.

We collected the data from documents, surveys,

interviews, and a site visit. Based on the results of

our desk research, we aimed at specific subjects

Measuring Architecture Principles and Their Sets in Practice: Creating an Architecture Principle Measurement Instrument Challenged in a

Case Study at the Dutch Tax Agency

535

during our interviews. We added all collected data

to the spreadsheets that contained our

measurement instrument. During this phase we

sharpened our measurement instrument, adding

new characteristics and attributes. After all data

were collected, the research team evaluated both

the measurement instrument and method and

made suggestions for improvement.

3. Closing the Research: the last phase compared the

results with existing literature and to end the case

study because there were no new data sources to

investigate. We ended the case study after

evaluating all possible data sources in our

spreadsheet and reporting the results of the case

study back to the Dutch Tax Agency. We closed

the research by answering the research question.

2.4 Boundaries of Research

In literature, different kinds of architecture principles

are defined: design and representation principles

(Stelzer, 2009; Winter and Aier, 2011; Haki and

Legner, 2013), syntactic and semantic architecture

principles (Lindström, 2006), and architecture

management principles (Lumor et al., 2016). In our

literature review we already took the view that these

different types of principles are different perspectives

on the same kind of architecture principles, all

meeting our definition.

As mentioned in section 2.2, the context in which

architecture principles are used, is relevant. With

narrowing the scope to Dutch government

organizations, we are scaling down the research

scope, resulting in more reliable research results.

3 THE MEASUREMENT

INSTRUMENT

To be able to measure architecture principles in

practice, we need both a measurement instrument and

a corresponding method to use in practice. In this

section we start describing the measurement as

answer on research question one. In the second part

we are answering the second research question, by

explaining the measurement method.

3.1 The Measurement Instrument

To measure architecture principles and related

architecture principle sets in practice we need a well-

defined measurement instrument. This measurement

instrument should be able to:

1. Identify the architecture principles and the

architecture principle sets related to them;

2. Describe Architecture principle and the

architecture principle sets with their

characteristics and attributes.

Based on comprehensive literature review we

redefined the definition of an architecture principle.

We also extended a framework for describing both

the architecture principles and related sets.

3.1.1 Definitions of Principle and Set

We defined an architecture principle as: “An

architecture principle is a declarative statement,

based on, at least, business and IT strategy. It

normatively describes a property of the design of an

information system, which is necessary to ensure that

the information system meets its essential

requirements.” Of course, to be an architecture

principle all elements of the definition should be

fulfilled. So, in the measurement instrument we

designed a definition check for the elements

declarative statement, based on business and IT

strategy, normatively, describes a property of the

design, and necessary meeting its essential

requirements. Each of these elements has to be

present, including explanatory facts.

For an architecture principle set we use the

definition “a group of architecture principles defined

and presented as a collection”. Because this

definition is more generic, we do see every group of

architecture principles described together as a set.

3.1.2 Descriptions

For describing both the individual architecture

principles and the sets of architecture principles we

defined a framework of characteristics and related

attributes (see fig. 2). This framework is based on

research of Richardson et al. (1990), Fischer, et al.,

(2010) and Winter and Aier (2011).

Using our framework, we listed all characteristics

named in literature (Borgers and Harmsen, 2018). For

each characteristic and its attributes, we provided a

definition (see appendix) and defined the relationship

with the architecture principle. In our measurement

instrument we list all characteristics and related

attributes, so we can collect the data for the

characteristics and attributes found in practice.

In our literature review we also found all kinds of

characteristics and documents in the context of the

architecture principles of which we are calling

artefacts. We described the context of the architecture

principle and set by defining the relationship with the

ICEIS 2019 - 21st International Conference on Enterprise Information Systems

536

artefacts, like ‘design’ and ‘requirements’.

Figure 2: Framework for describing the architecture

principle and set.

3.2 Measurement Method

To be able to use the measurement instrument, we

also need a reliable and valid measurement method to

measure the architecture principles. The

measurement method helps with collecting and

analyzing the right data and finally with measuring

the architecture principles. The measurement method

is an iterative three-step approach (see fig. 3).

Figure 3: The measurement method.

3.2.1 Collect Data

The objective of this step is to collect data about

architecture principles and the architecture principle

set. Therefore, data about the related artefacts are also

relevant: IS architectures, IS designs, requirements

and business & IT strategies. We use different

methods to collect data: desk research, surveys,

interviews and site visits.

For the desk research all kind of documents might

be useful, like architecture descriptions, requirements

specifications, test reports, and so on. All these

address elements of the architecture principles and/or

the related artefacts. The survey is used to collect data

before the interview sessions, to be able to focus on

specific items during the interview. The survey

consists of open questions based on the characteristics

in the framework. The interviews take place with at

least two members of the research team. All

interviews are recorded, and the minutes of the

interview are sent to the interviewee for feedback.

Interviewees are architects, software engineers, the

test manager, the project leader, and the system

owner. Site visits are useful to see the information

system running in the daily operation and to consider

to what extent the essential requirements are

implemented.

We record all relevant data per architecture

principle and per source, in order to have different

facts about the same characteristic or attribute

available. This is useful for the analysis of data, when

differences or even conflicts among the data about a

specific characteristic or attribute occur.

3.2.2 Analyze Data

In this step we analyze the data to check the precision

and accuracy of the data and to find exceptions and

trends. For all data collected we check for

inconsistencies between sources. If so, we have to go

back for data collection to find the right or new data.

If, afterwards, data conflicts remain, we have to

explain the differences or decide not to use the data.

Secondly, we check whether or not so-called

architecture principles are in accordance with the

architecture principle definition. We determine if the

principle satisfies each element of the definition and

write down that reasoning in a spreadsheet. If not, the

so-called architecture principle is declared be out of

scope.

We then analyze the qualitative data on

exceptions. The analysis has to be done per principle,

but also between different principles. We look at

remarkable differences between attributes or

characteristics of a principle or between principles.

For instance, the key action cannot be related to the

prerequisite of the principle. Or, one architecture

principle is fulfilled completely, while another one is

not, although those two are strongly related.

The final action is to quantify the data and find

specific trends from the quantified data. We simplify

the analysis between principles out from different

cases. We quantify the data of each principle and set

as follows.

 We review the different sources per attribute and

set the numerical score;

 The score reflects the level of fulfilment of the

definition of the attribute: ‘0’ is no fulfilment, ‘1’

is partly fulfilment and, ‘2’ is complete fulfilment.

We call this our code scheme (Babbie, 2015b).

The reasoning for the score is described in the

spreadsheet;

Measuring Architecture Principles and Their Sets in Practice: Creating an Architecture Principle Measurement Instrument Challenged in a

Case Study at the Dutch Tax Agency

537

 The score of the characteristic is the average of the

score of its attributes;

 Only for the “classification” characteristic of the

principle set we use an alternative score: the

scores used refer to the specific values the

attribute can have. See the appendix for all

attribute values of principle sets;

 Finally, we calculate the average score of each

characteristic over all principles and sets.

We now are able to make cross-section analyses, to

create graphics and to search for trends.

3.2.3 Measure the Principle

This final step is to evaluate the exceptions and

trends. We describe the architecture principles and

the architecture principle sets, including the overall

conclusions about their state.

Based on the qualitative and quantified analysis

we evaluate the exceptions and trends. We explain

those exceptions and trends and draw our conclusions

as subject-matter experts. Of course, we add evidence

supporting those conclusions.

We end up with describing the architecture

principles and the architecture principle sets by

describing their characteristics and attributes. In this

description we add the qualitative and quantified

analysis, including the conclusions.

4 CASE STUDY RESULTS

We started challenging our measurement instrument

with one case study only. Before we use the

instruments for many cases simultaneously, we first

would like to test to what extent the instrument is

useful in practice. Depending on the outcome of the

first case study, we can decide how to continue. If

there is a big misfit with the instrument itself, we will

focus on improving the instrument. If it is working in

practice quite well, we can start directly with the

research itself and optimize the measurement

instrument where necessary.

We used the ‘Teruggaaf Dividendbelasting’

(TDi), in English: ‘Return of Dividend Tax’, of the

Dutch Tax Agency as case in our research. TDi is an

information system supporting the return of tax on

dividend payed to legal entities. TDi was rebuilt in an

agile project, which we investigated until system

release in December 2017. For this case study we

reviewed nineteen documents, conducted five

interviews with six different stakeholders and

examined the TDi system itself during a site visit.

These activities were done by our research team

consisting of three researchers.

4.1 Architecture Principles of the TDi

Case

In the rebuilding of the TDi system 55 potential

architecture principles were used. According to our

definition (see section 3), only 36 architecture

principles could be identified as such. In fig. 4 we see

the level of completeness of all those 36 architecture

principles together, while the individual scores may

differ between the principles.

Looking at the specification characteristic of the

architecture principles we did recognize that none of

the principles included a rationale, while the

statement and implications were worked out well. A

reference to the rationale, as they were described in

other architecture documents, was missing as well.

Figure 4: Level of completeness of the 36 architecture

principles.

80% of the principles has been fulfilled partly

(36%) or fully (44%). Only one principle was not

followed (“from object based to subject based

working”) and in 17% we could not determine

whether or not the principles had been fulfilled

because of missing resource data (see fig. 5).

Interestingly, developers didn’t respect some of the

principles as meant to be and implemented parts of

the system in other directions.

With respect to the prerequisites, for seven

principles specific preconditions were defined, while

for all, some overall preconditions were defined. Not

all of the preconditions have been fulfilled

(completely) at the start or during the project, like

“B/CAO building blocks available”. Therefore, not all

principles could be fulfilled, as we did see.

Surprisingly, there were no “key actions” identified,

to get the preconditions fulfilled.

All meta data was in place, so that managing the

architecture principles was no issue. Information

about author, status, version, users and much more

was easy to find.

ICEIS 2019 - 21st International Conference on Enterprise Information Systems

538

The architecture principles were meeting the

quality attributes as well. The main reason why the

overall score of the quality is not 100 percent, is that

the rationale was missing. Therefore, we were not

able to determine the principle’s intention and

relevance as described in the “specific” and

“relevant” attribute. All architecture principles, even

when they were imposed from outside the project,

were translated to the TDi context.

Figure 5: Level of fulfilment of the principles.

The architecture principles originate from two

documents: a High Level Design (HLD) and a Project

Start Architecture (PSA). The HLD describes the

process, application and technical infrastructure of

the TDI system, while the PSA focuses on the

application and technical infrastructure only. In the

HLD, twelve architecture principles are defined

explicitly but most principles are only addressed by

referring to other architecture documents. In the PSA,

nine ICT principles are described, including

directives for using in the TDi system

implementation. In both documents many meta data

attributes can be found, like authors, administrator,

status, target audience, etc.

Given the many architecture principles mentioned

in both sets we conclude that the sets are not

representative for meeting the essential requirements.

There are too many architecture principles adopted

from the overall architectures, resulting in overlap.

Those principles are not translated to a single

principle specific for the TDi system. Although there

are many overlapping architecture principles, there

are no contradictory principles in the sets. The

accessibility of the sets themselves is good, because

they were managed by the architects of the TDi

project. Because the sets refer to other documents, the

accessibility of the original sets of principles is less

evident.

4.2 Evaluation of the Measurement

Instrument in the Case

For evaluating the measurement instrument, we have

to test for both the identification and the description

of the architecture principles, whether they are

complete and coherent. To start with the

identification of the architecture principles, the

instrument was helpful in determining which of the

so-called architecture principles are fulfilling the

elements of the definition. As a result, nineteen of the

55 so-called architecture principles did not pass

verification. On the other hand, we did not find any

other statements, not explicitly called architecture

principles, that did fulfil the elements of the

definition.

The definition of the architecture principle set was

not differentiating enough. There are many ways to

present a group of principles together. In our case we

analyzed the different kinds of sets to understand the

interrelationships between those sets. We did see in

this case study that the presentation of the architecture

principles was related to other architecture

documents, which were already in place. So, the way

of presenting the principles is not necessarily related

to the system itself but influenced by external factors.

Therefore, our case study resulted in a changed

definition of the architecture principle set: “a group

of architecture principles defined and presented as a

collection based on a similar type or scope of the

architecture principles.”

Although we might state that - in this case - the

identification of all individual principles was done,

we also learned that the identification is also related

to the essential requirements. In our case study the

essential requirements were defined at a high level,

e.g. “the system has to be future-proof”, so it was

quite easy to link architecture principles to the

essential requirements. So, in next cases more in-

depth research to the essential requirements is

necessary.

The coherence of the architecture principle

definition was already theoretically explained in our

literature review with the WH-questions approach

(Borgers and Harmsen, 2018). During the case study

we did not find any inconsistencies between the

elements, which might suggest that the elements of

the definitions are incorrect.

The second part of the measurement instrument

describes the architecture principles. Looking at the

completeness of the instrument we found in the case

all but two of the attributes defined in the model. The

“rationale” attribute was defined in other documents

and, although no “key actions” were defined, in some

interviews necessary key actions to take were brought

forward. So, none of the attributes are irrelevant.

In our case study we detected some omissions in

the model. The attribute “degree of acceptance” has

Measuring Architecture Principles and Their Sets in Practice: Creating an Architecture Principle Measurement Instrument Challenged in a

Case Study at the Dutch Tax Agency

539

to be added to the “measure” characteristic, because

in our case this element was addressed by several

sources. It describes an aspect relevant to the

fulfilment of the principle and will be defined as:

“level of acceptance of the principle by all of its

users”. The attribute “preconditions fulfilled”, related

to the “prerequisites” characteristic, is also relevant

to add. We explicitly saw in the case that, when

preconditions were set, it was also relevant to know

whether the preconditions were fulfilled. The

definition of this attribute can be described as “the

level of fulfilment of the preconditions defined.”

For the architecture principle set we will add an

extra characteristic: “prerequisites”. We discovered

in the case study that some prerequisites were not

related to a specific principle, but to a group of

principles. Besides the “precondition” attribute,

“basic assumptions” were described for some sets as

well. Basic assumptions are “relevant criteria for

successful use of the principle”.

In this case we did not find any inconsistencies in

the coherence of the description model. Some of the

relationships as described in the model, e.g. “depends

on” or “level of fulfilment”, were described explicitly

in the documents or way mentioned during the

interviews. The amount of information, though, is

insufficient to make fact-based statements about the

consistency of the coherence. In this case it was clear

that there are interrelationships between attributes,

e.g. the missing of the rationale and therefore a lower

score at quality, as we already described in (Borgers

and Harmsen, 2016). More research data is necessary

to make clear statements about the coherence.

4.3 Evaluation of the Measurement

Method

We evaluate the measurement method by discussing

the reliability and validity of the case study results.

To challenge the reliability of the results, we want to

know to what extent the results would be consistent

when doing the case study research again. In the

measurement method are different protocols defined

to assure the reliability of the outcome: using

different kinds of data collections, working with a

research team, minutes including feedback, etc. All

these mechanisms are important, because this case

demonstrates the subjectivity of facts collected. Two

architects, for example, were working closely

together during the project, but had different opinions

about the fulfilment of some of the architecture

principles. The research team could, based on all

different sources, make an expert judgement about

the fulfilment.

Although we used different ways of collection

data, in this case study we were lacking some in-depth

information about the essential requirements. As a

result, it was difficult to see to what extent

architecture principles were adding value in meeting

the essential requirements. Additional sources related

to the essential requirements, e.g. interviewing extra

business owners, would help to bridge this gap.

In evaluating the validity of the measurement

method, we concluded that the description of the

architecture principles reflects the real situation of

TDi. Although we found some contradictory data,

especially in the interviews, we were able to explain

the differences in the data. In this specific case we

were not always able to go into details of specific

architecture principles. Because the case used quite

some principles, 36 in total, it was difficult to address

all individual principles. So, in following cases we

need mechanisms to get more in-depth information

about the individual principles.

5 LIMITATIONS, CONCLUSIONS

AND NEXT STEPS

The aim of this research was to build and test the

architecture principle measurement instrument. To do

so, we chose the case study approach to juxtaposition

theory with practice.

5.1 Limitations

Although the arguments for using the case study

approach are still valid, there are some limitations

important to address in this case study.

We are aware that one case cannot prove the

completeness of the measurement instrument. As

discussed in section 4, the objective of this research

is to test to what extent the instrument is useful in

practice in the first place. For an extended test on

completeness and coherence of the measurement

instrument, we need more test cases.

A second limitation might be that the researchers,

although all kind of protocols are defined, are biased

in searching for characteristics and attributes. The

moment we are introducing a model as a description

of our research object, we see architecture principles

through this model. We tried to avoid this prejudice

by avoiding naming of attributes during the survey

and interviews. The fact that we identified new

attributes and characteristics, shows that we were

open for new elements as well.

Finally we can also note that there is currently no

ICEIS 2019 - 21st International Conference on Enterprise Information Systems

540

other instrument that measures architectural

principles, so we cannot compare this instrument with

other instruments. Studies have been done on the

added value of architecture in general and we could

apply that approach to the same case to assess

whether comparable results will be achieved.

5.2 Conclusions

To answer the overall research question, we first had

to investigate how an architecture principle

measurement instrument looks like. Based on a

theoretical framework we described a measurement

instrument tested in a real-life case study. Based on

the experiences from practice we can conclude that

the measurement instrument is fit for purpose,

although the instrument can be extended with extra

characteristics and attributes. These extensions are

‘degree of acceptance’ and ‘preconditions fulfilled’

for describing the individual principle and the

characteristic ‘prerequisites’, including the attributes

‘precondition’, and ‘basic assumption’ for the

architecture principle set. Although we know that we

tested the instrument with one case only, we are

beyond doubt that with these add-ons the model has

added value in measuring architecture principles and

in measuring related architecture principle sets.

Secondly, we had to determine how to use this

measurement instrument in practice. We defined a

three-step method to collect, analyze and measure the

architecture principle. We carried out this

measurement method in our case study, with a

description of the architecture principles and

principles sets for the TDi case as a result. We have

the opinion that the method yields reliable and valid

results, although we discovered in this case that more

information on the requirements and the individual

principles, would strengthen the results of the case

study.

5.3 Next Steps

In summary, the conclusions and limitations

combined confirm that it is feasible to enlarge the

number of tests of architecture principle

measurements with more cases. With more cases we

are able to test the ability to compare architecture

principles and principle sets between case studies.

When carrying out new cases, we would prefer the

number of cases to be as large as possible. However,

the substantial scaling up of the number of cases

requires automatic processing of the data. We are

going to investigate whether automatic processing is

possible, since architectural principles are written in

human language and we have found that this requires

human interpretation.

Secondly, we can use the new case studies to test

the completeness and coherence of the measurement

instrument and method. We need to investigate

whether or not the vision of using architecture

principle is a characteristic in itself and we also have

to see how we can elaborate on the essential

requirements. So, new case studies will give new

insights in the use of the instrument and method and

help in optimizing them both.

ACKNOWLEDGEMENTS

We would like to thank Yvette Hoekstra and Henk

van den Berg for participating in the research team

and giving all kinds of comments on both the

measurement instrument and the case study results.

Secondly, we want to thank the Dutch Tax Agency,

and Jelco Bosma specifically, for enabling this case

study. Special thanks to the senior experts Saco

Bekius (M&I/partners) and Paul Oude Luttighuis (Le

Blanc Advies) for reviewing the draft version of this

article.

REFERENCES

Babbie, E. (2015a and b) The Practice of Social Research..

Borgers, M. A. C. (2016) ‘Do IT architecture principles

contribute to IT system ’ s requirements realisation ?’,

in DCEIS 2016 – Doctoral Consortium on Enterprise

Information Systems. Maastricht: SCITEPRESS, p.

pages 3-8.

Borgers, M. and Harmsen, F. (2016) ‘Case Report of

Identifying and Measuring IT Architecture Principles in

the Dutch Tax Agency’, 2016 IEEE 18th Conference on

Business Informatics (CBI), (November), pp. 100–110.

doi: 10.1109/CBI.2016.57.

Borgers, M. and Harmsen, F. (2018) ‘Strengthen the

architecture principle definition and its characteristics:

A survey encompassing 27 years of architecture

principle literature’, in ICEIS 2018 - Proceedings of the

20th International Conference on Enterprise

Information Systems.

Eisenhardt, K. and Bourgeois, L. J. (1988) ‘Politics of

strategic decision making in high velocity

environments: Toward a mid-range theory’, Academy

of Management Journal, 31, pp. 737–770.

Eisenhardt, K. M. (1989) ‘Building Theories from Case

Study Research.’, Academy of Management Review,

14(4), pp. 532–550. doi: 10.5465/AMR.1989.4308385.

Eisenhardt, M. (1989) ‘Building Theories from Case

Research’, The Academy of Management Review,

14(4), pp. 532–550. doi: 10.5465/AMR.1989.4308385.

Measuring Architecture Principles and Their Sets in Practice: Creating an Architecture Principle Measurement Instrument Challenged in a

Case Study at the Dutch Tax Agency

541

Fischer, C., Winter, R. and Aier, S. (2010) ‘What Is an

Enterprise Architecture Principle?’, Computer and

Information Science 2010, (Ieee 2000), pp. 193–205.

doi: 10.1007/978-3-642-15405-8_16.

Gersick, C. (1988) ‘Time And Transition in Work Teams:

Towards a New Model Of Group Development.’,

Acedamy of Management Journal, 31, pp. 9–41.

Haki, M. K. and Legner, C. (2013) ‘Enterprise Architecture

Principles in Research and Practice: Insights from an

Exploratory Analysis’, Ecis, (2013), pp. 1–12.

Available at: http://works.bepress.com/

mohammadkazem_haki/1.

Harris, S. G. and Sutton, R. I. (1986) ‘FUNCTIONS OF

PARTING CEREMONIES IN DYING

ORGANIZATIONS.’, Academy of Management

Journal. Academy of Management, 29(1), pp. 5–30.

doi: 10.2307/255857.

Lindström, Å. (2006) ‘On the Syntax and Semantics of

Architectural Principles’, in Proceedings of the 39th

Hawaii International Conference on System Sciences.

IEEE, pp. 1–10.

Lumor, T., Chew, E. and Gill, A. Q. (2016) ‘Exploring the

Role of Enterprise Architecture in IS-enabled OT: An

EA Principles Perspective’, in Enterprise Distributed

Object Computing Workshop. doi: 10.1109/EDOCW.

2016.7584360.

Miles, M. and Huberman, A. M. (1984) ‘Drawing valid

meaning from qualitative data: Toward a shared craft’,

Educational researcher, 13(5).

Richardson, G. L., Jackson, B. M. and Dickson, G. W.

(1990) ‘A Principles-Based Enterprise Architecture:

Lessons from Texaco and Star Enterprise’, MIS

Quarterly, 14 (4), pp. 385–403.

Steenhuis, J. and De Bruijn, E. (2004) ‘Building theories

from case study research: the progressive case study’,

Journal of Chemical Information and Modeling, 53(9),

pp. 1689–1699. doi: 10.1017/CBO9781107415324.

004.

Stelzer, D. (2009) ‘Enterprise architecture principles:

literature review and research directions’, Proceedings

of the 2009 international conference on Service-

oriented computing, pp. 12–21. doi: 10.1007/978-3-642

-16132-2_2.

Strauss, A. L. (1987) Qualitative analysis for social

scientist. Cambridge: Cambridge University Press.

Winter, R. and Aier, S. (2011) ‘How are enterprise

architecture design principles used?’, Proceedings -

IEEE International Enterprise Distributed Object

Computing Workshop, EDOC, (September 2016), pp.

314–321. doi: 10.1109/EDOCW.2011.27.

Yin, R. (1984) Case Study Research: Design and Methods.

Beverly Hills: Sage Publications.

Yin, R. K. (1981) ‘The case study crisis: Some answers’,

Administrative Science Quarterly, 26(1), pp. 58–65.

APPENDIX

Table 1: Characteristics for Architecture principles.

Characteristic

Attribute

Definition

Specification

Statement

Succinctly and

unambiguously

communicates the

fundamental rule to the

user of the principle

Rationale

Highlights the business

benefits of adhering to the

principle

Implications

Highlights the

requirements for carrying

out the principle

Measure

Level of the fulfilment of

the statement

Prerequisites

Precondition

Preconditions and

requirements to be fulfilled

before the principle can be

applied

Key action

Guidelines for

implementing the principle,

giving the preconditions

Meta data

Several

Specifications to be able to

govern the principle

Quality

Specific

The user can understand its

intention and its effects to

use it in his work

Measurable

Possible to determine

whether or not a given

behaviour is in line with

architecture principle

Achievable

The implications of it can

all be performed by or

adhered to by all those

affected

Relevant

The principle should lead

to a improvement of the

system meeting the

essential requirement

Time framed

Principle should be stable

in context and time

ICEIS 2019 - 21st International Conference on Enterprise Information Systems

542

Table 2: Characteristics for architecture principle set.

Characteristic

Attribute

Definition

Classification

Type

The principles in the

set are related to one of

the architecture layers.

Scope

Level of use of the

principle.

Meta Data

Several

Specifications to be

able to govern the

principle set.

Quality of the

set

Representative

The set covers all

relevant requirements

in a specific problem

domain.

Accessible

Users can find and

retrieve the set of

principles and they can

comprehend the

principles.

Consistent

No contradictions

between the

architecture principles

in the set.

Table 3: Possible values for the attributes of the

classification characteristic.

Attribute

Score

Value

Type: The principles in the

set are related to one of the

architecture layers.

Infrastructure

Application

Information system

Scope: Level of use of the

principle.

Part of the target

organisation.

Full target organisation.

More than the target

organisation.

Table 4: The 36 architecture principles used within the TDi

case (translated from Dutch).

Number

Architecture principle

Organisational units do specialize

Collaboration based on services

We share proven services within the Dutch

government

We communicate digitally with citizen and

companies, if possible

Data administration is done digitally only

Digital workspaces offer customized

information

Number

Architecture principle

We connect with the activities of citizens and

companies

We develop knowledge about laws and

regulations and share them

We strengthen the information position of

citizens and companies

Design modularity carefully

Unique management and multiple use of data

Design the continuity of business operations

completely

Use standards

Use services available (re-use, before buy,

before build)

Use ICT products as intended

Deliver robust ICT services

Take security risks consciously

Solve problems at the source

Employee centrally, tailor-made information

Standard building blocks

Client-oriented payment and management of

data

Establish source data

Exchange of information

Process characteristics

From object-oriented to subject-oriented

Maximize compliant behaviour

Integral production control

Data is used across contexts

Event-driven transactions exist alongside

periodical transactions

The handling time of transactions matches the

expectation of the customer

We are preparing for settlement of positive

and negative claims

Advances can be partially paid

Operational Excellence is for Customer

Intimacy

Decoupling of risk detection and

determining legal consequences

Sensible reuse of process patterns and ICT

facilities

Where possible, we shift functionality for

transaction processing to the interaction

process

Measuring Architecture Principles and Their Sets in Practice: Creating an Architecture Principle Measurement Instrument Challenged in a

Case Study at the Dutch Tax Agency

543