Practical Application of Maturity Models in Healthcare: Findings

from Multiple Digitalization Case Studies

Anja Burmann

and Sven Meister

1,2 b

Digitization in Healthcare, Fraunhofer Institute for Software and Systems Engineering, Dortmund, Germany

Health Informatics, Faculty of Health, Witten/Herdecke University, Witten, Germany

Keywords: Maturity Model, Digitalization, Hospital, Literature Review, Case Study Research.

Abstract: Maturity models (MMs) are widely applied means for describing a current status of development across

numerous domains, but also within the healthcare sector. They offer orientation for systematic development

and improvement regarding the aspect examined. However, experience from the practical application of MMs

is scarcely described. Within this article two projects in which MMs were used in collaboration with

practitioner groups from hospital environments are presented. The general project intentions, motivation for

incorporating MMs and generated results are described. By deriving observations across the two cases,

general tensions between healthcare practice and research concerning maturity modelling are identified.

Additionally, the suitability of existing MMs to support especially hospitals in coping with the challenges of

the digital transformation is discussed. This study’s findings may be incorporated into development and

refinement of MMs and may thus contribute to increasing practical value created by such means.

1 INTRODUCTION

Manufacturing industries all over the world are facing

the challenges of the digital age: a change of

production, services as well as business models itself

is required (Bauernhansl, 2014). The same challenge

applies to healthcare provision: the increased

availability of electronically held information and

data offers new ways of providing healthcare. Digital

interfaces between care providers, sectors, divisions,

wards and functional areas offer a seamless

availability of information, but also require new ways

of organizing and steering the health provision

process. The peculiarity of the digital transformation

is the prospective designation as an industrial

revolution, what has usually been named as such in

historical review. Regardless of whether the scope

was correctly anticipated, this withholds the

opportunity and the burden for societies, industry,

research, but also healthcare providers to actively

shape the future (Hermann et al., 2016). Hitherto,

practitioners and researchers pursued the

systemization of change processes, in order to

constitute a phenomenon, and to be able to

https://orcid.org/0000-0002-6989-1230

https://orcid.org/0000-0003-0522-986X

structurally evolve it (Meister et al., 2019). The

shaping of this as a revolution labelled future working

world requires the disruptive rearrangement of

processes according to a constantly developing

vision (Deiters et al., 2018).

Up to this point, maturity models (MMs) have

been widely established as tools for describing a

specific development status, circumstance or

condition of an organization, a process or a structure

(Wendler, 2012). At the same time, they usually offer

a path of evolution for systematic development and

improvement with regard to the assessed aspect

(Pöppelbuß et al., 2011; Pöppelbuß & Röglinger,

2011). Current research efforts focus on the

development of increasingly rigorous MMs

particularly addressing the challenges of the digital

transformation in various domains. An increasing

number of MMs is dealing with digitalization aspects

in healthcare settings. At the same time, there is little

narrative about the implementation and effectiveness

of applying MMs in highly complex environments

like hospitals (Waring & Currie, 2009). However,

such experience should have sustainable impact on

the development of such models itself (Blondiau et

100

Burmann, A. and Meister, S.

Practical Application of Maturity Models in Healthcare: Findings from Multiple Digitalization Case Studies.

DOI: 10.5220/0010228601000110

In Proceedings of the 14th International Joint Conference on Biomedical Engineering Systems and Technologies (BIOSTEC 2021) - Volume 5: HEALTHINF, pages 100-110

ISBN: 978-989-758-490-9

al., 2016). Barret and Oborn describe the division

between practice and research as twofold: on the one

side researchers are “lost in translation” struggling in

explaining the relevance of theories to practice. On

the other side research is “lost before translation”

developing practically irrelevant theories (Barrett &

Oborn, 2018). In order to bridge the mentioned

research-practice-division, within this article we

present experiences from applying MMs as means for

achieving specific practical purposes in two separate

research projects. According to Braa & Vidgen (Braa

& Vidgen, 1999) the studying of IS artefacts in-

context is able to contribute to understanding the

“organizational laboratory”, the artefact itself and

integration mechanisms. We thus endeavour to

contribute to answer the following research questions

by investigating two practical applications of MMs:

1) What challenges does the practical field face

when applying MMs as tools for structured

development of healthcare provision?

2) What challenges arise from using MMs as

means to structurally cope with the digital

transformation in healthcare?

For this purpose, we firstly outline the relevant

conceptual background regarding maturity modelling

and modelling digital maturity in the healthcare

domain. Secondly, we present the research approach

and data collection methods, as well as the two

projects we derive our findings from. Subsequently,

we depict the project results and the observations

made from incorporating MMs into practice. We

conclude with a summary of the contributions,

limitations and potential for future research.

2 BACKGROUND

2.1 Maturity Modelling

The concept of modelling maturity originates from

the classification of software development processes

(Humphrey, 1988) within the Capability Maturity

Model (CMM) (Paulk et al., 1993). Since then, a

continuously rising number of publications with

regard to MMs have been made (Lee et al., 2019;

Wendler, 2012).

The modelling of maturity distinguishes between

general approaches and domain specific descriptions

of maturity aspects (Wendler, 2012). This is due to

the challenge of distinctive MMs of on the one hand

addressing individual challenges and generating a

hands-on benefit for the assessed organization, and at

the same time provide general expressiveness and

abstraction, for which the research community is

striving (Becker et al., 2009; Blondiau et al., 2016).

MMs mostly categorize a defined context in discrete

stages (Gottschalk, 2009; Schuh et al., 2017), and

differ in their degree of descriptivity and

prescriptivity (de Bruin et al., 2005).

Most of the existing measurement means were

developed following a top-down approach, which led

to the criticism regarding relevance and rigor of these

procedures. Following that, bottom-up methods and

tools have been proposed (Lahrmann et al., 2011;

Rönkkö et al., 2008).

Within this paper the focus lies on the depiction

of maturity with regard of general healthcare

provision, institutions and processes, and in particular

regarding digitalization aspects within this domain.

2.2 Digital Maturity in Healthcare

Healthcare provision substantially differs from

providing conventional services, or from production

in the manufacturing sector. This is obvious in the

requirements dynamics, interdisciplinarity and

human-centred process control (Söylemez & Tarhan,

2016). Furthermore, and contrary to other

organizations, actors in healthcare organizations are

not subordinate to a central strategy, but follow

diverging aims (e.g. different aims in different clinics,

conflicting economic and medical aims). For this

reason, the healthcare domain puts its trust in domain

specific models rather than adopting general models

from industry.

In 2016 Carvalho et al. identified 14 maturity

models dealing with information systems technology

in healthcare (Carvalho et al., 2016), and since then

others have been added (Gomes & Romão, 2018;

Kolukısa Tarhan et al., 2020).

The probably most widely applied MM approach

regarding digitalization in healthcare is the Electronic

Medical Record Adoption Model (EMRAM),

developed and provided by the Healthcare

Information and Management Systems Society

(HIMMS Analytics, 2017). EMRAM measures the

degree of the integration of an electronic health

record (EHR) of hospitals. This model was first

introduced in 2005, revised in 2018 and categorizes

hospitals in eight levels of maturity from 0 (no

digitalization) to 7 (paperless hospital) (Stephani et

al., 2019).

Other exemplary approaches in the hospital sector

dealing with digitalization aspects set a focus on the

maturation and evolvability of PACS (van de

Wetering & Batenburg, 2009) or digital imaging

Practical Application of Maturity Models in Healthcare: Findings from Multiple Digitalization Case Studies

101

(Studzinski, 2017), the adoption of data analytics

(Sanders et al., 2013), the degree of correspondence

of the IT architecture with strategic goals (Mettler et

al., 2014; Mettler & Pinto, 2018), or the ability to

implement IT innovation (Esdar et al., 2017).

While a considerable number of methods

operationalize and measure particular maturity

aspects of healthcare institutions, Carvalho et al. in

their literature review remark, that no identified

model covers all organizational areas and systems of

healthcare organizations. This notation is e.g. picked

up by the regularly published “IT Healthcare Report”

(Hübner et al., 2015; Hübner et al., 2018; Hübner et

al., 2020), which points out one particular aspect per

issue. One of these aspects is the clinical information

logistics, operationalized in the workflow composite

score (WCS) (Liebe et al., 2015). The score breaks

down the availability of data along and across clinical

processes.

Most of the mentioned approaches neglect the

dependence of healthcare processes on interaction

(purely social as well as socio-technical) and

commitment of human actors. Krasuska et al. i.e.

identify multimodal capabilities relevant for “digital

excellence” in hospitals (Krasuska et al., 2020). Pak

& Song (Pak & Song, 2016) explicitly address the

human interaction aspect, while Burmann et al.

suggest a combination of technical and human factors

(Burmann et al., 2019).

However, successful implementation and

deployment in practice is scarcely described

(Blondiau et al., 2016). Mettler et al. addressed this

issue by discussing experiences and pitfalls from the

translation of their intervention-cycle into application

(Blondiau et al., 2016; Mettler, 2010). They conclude

that implementation of maturity assessments in

practice requires especially in hospitals a high level

of support (Blondiau et al., 2016; Caldwell & Atwal,

2003; Conwell et al., 2000).

3 METHODOLOGY

The presented research methodologically builds on a

structured literature review (SLR) and case study

research (CSR). The SLR provides a comprehensive

overview of the current status of research and

application. Thereby, substantial developments of

maturity modelling in general and particularly in the

healthcare domain were identified. The findings

presented in this article are based on the retrospective

analysis of two projects, which were conducted from

2017 to 2020. Both projects followed the assumptions

regarding case study research made by Yin (Yin,

1987). The presented analysis combines aspects from

the pragmatism and interpretivism paradigms in IS

research as explained by Goldkuhl (Goldkuhl, 2012).

The “data generation” followed the former since the

two projects were carried out in joint teams with

representatives from research and practice, while the

retrospective “data analysis” can be merely assigned

to interpretivism. The SLR served as a continuous

knowledge basis for both of the presented case

studies.

3.1 Literature Review

In order to gain an overview of the literature in the

field of maturity modelling, a structured literature

review, as suggested by Webster and Watson

(Webster & Watson, 2002) and vom Brocke et al.

(vom Brocke et al., 2009) was conducted. This search

was carried out accompanying both projects and

updated regularly. Since literature in the field of

Information Systems (IS) is gathered across a large

number of databases, a combination of search sources

is advised (Levy & J. Ellis, 2006). The databases

Scopus, AIS eLibrary and IEEEXplore were chosen

in order to incorporate the leading journals and

conference proceedings. Firstly, the authors aimed at

collecting the conceptual foundation of maturity

models in general and therefore used the search string

“maturity model” OR “maturity assessment” OR

“maturity measurement”. Subsequently, languages

other than English and were excluded, and the results

were limited to secondary sources. From initially over

6800 identified publications 118 documents

remained, of which 4 sources referred to

digitalization MMs. Via a backward and forward

search of the relevant literature we identified the

major works and contributions to general maturity

modelling and modelling the status of digitalization.

Additionally, a summary of activities with particular

regard to maturity modelling in the healthcare domain

was generated by combining the search string above

with AND “Hospital” OR “Healthcare” OR “clinical

workflow”. This search lead to a total number of 146

results, which were subsequently screened for actual

models presented. The total number of maturity

models particularly for the healthcare domain rose

from around 30 in 2017 to more than 60 in 2020

(Kolukısa Tarhan et al., 2020). In order to also

incorporate approaches which might not be described

on a peer reviewed research level, but are part of

practical application, the search was conducted in

GoogleScholar as well. The identified maturity

models served as a knowledge basis for solving

domain problems within several projects related to

HEALTHINF 2021 - 14th International Conference on Health Informatics

102

hospital digitalization since 2017, and particularly the

two referred to in this article. Section 3.4 describes

the two cases in further detail.

3.2 Case Study Research

In order to derive general findings associated with the

practical application of MMs in healthcare context

and contribute to answering the formulated research

questions the recent project past of the authoring

working group was screened for cases suitable to be

included into this analysis. Since we aimed at breadth

and generalization across different practical

scenarios, we identified two projects as heterogenous

as possible regarding timely distance to each other,

data collection means, combination of applied MMs,

type of action (intervention or description) and

number of maturity assessments. Both projects were

assigned to the field of case study research. A case

study generally analyses a real-world phenomenon,

and contributes to understanding a complex problem

(Yin, 2003). The investigation is defined in time and

location, and follows a specific research question

(Baxter & Jack, 2008; Ridder, 2016)., related to the

description of digitalization of healthcare processes,

were carried out under the CSR notion. Both projects

were carried out and analysed separate from each

other (Gustaffson, 2017). Section 3.4 describes the

adduced projects, while Table 1 compares the case’s

parameter.

3.3 Data Collection Means

Within the two projects primary qualitative data

collection methods were used. That encloses in-depth

interviews and focus groups (Gill et al., 2008) with

designated experts as well as guest visits and

observations of the daily working environment by one

of the researchers (shadowing) (Quinlan, 2008). The

in-depth interviews followed a predefined semi-

structured guideline, and were conducted by phone

and documented by the interviewer (Longhurst,

2010). The focus groups were carried out with

healthcare representatives and scientists from the

research institute on the site of the respective project

partner. Focus groups strengthen the relevance as

they enable joint problem-oriented research activity

with practitioners (Gill et al., 2008). Within these

focus groups processes were modelled and analysed

with regard to the current status of digitalization and

optimization potentials. Therefore, tools such as the

Business Process Model and Notation (BPMN)

(Allweyer, 2010), group discussions or the Digital

Imaging Adoption Model (DIAM) (Studzinski, 2017)

were used. The focus groups were documented by the

research representative (doctoral candidates in the

subject area) with whiteboards, flipcharts and

protocols.

3.4 Case Description

The two mentioned cases were driven by user needs

and aimed for different purposes. Common ground is

that both incorporated the application of maturity

modelling concepts into practice. They were carried

out independently and included into this analysis

retrospectively. The projects were initiated from

institutions of healthcare provision and can be

assigned to two specific and disparate functional

areas: diagnostic imaging (DI) and oral and

maxillofacial surgery (OMS). The project objectives

are summarized in the following, and the project

parameter relevant for this article are comparatively

depicted in table 1.

3.4.1 Case 1: Diagnostic Imaging

The scope of Project 1: Diagnostic Imaging was to

support the substitution of an analogue radiology

imaging system with a digital workplace solution and

to assess the impact of an increase of the degree of

digitalization on workflow efficiency. In order to

achieve that, six radiology departments (four within

hospitals and two in resident practice) were

identified, all of which were just about to upgrade

from an analogue to a digital imaging system. in a

first step the initial workflow and the stage of

development with regard to digitalization were

modelled. Therefore, as MM means parts from the

EMRAM were combined with aspects from DIAM.

Additionally, the classic process modelling notation

BPMN (Allweyer, 2010) was used. By using these

means, the initial process was modelled and assessed

regarding workflow and status of digitalization.

Additionally, workflow efficiency was monitored by

a person from the research team in a time frame of

two days of shadowing the everyday work routine.

Following that, the current workflow was analysed

regarding optimization potential through

digitalization. Based on this a digital imaging system

was selected according to the customers’ needs and

configured correspondingly to the greatest possible

extent. The respective imaging system was then

integrated into each of the six departments, and the

department teams were trained on the system.

Following an initial familiarization phase of 6-8

weeks, the new workflow was assessed again with

regard to process flow by applying the same means as

Practical Application of Maturity Models in Healthcare: Findings from Multiple Digitalization Case Studies

103

used in the initial analysis (two days of shadowing).

Both assessments were comparatively examined and

changes in workflow efficiency were identified.

3.4.2 Case 2: OMS

Within Project 2: Oral and Maxillofacial Surgery the

goal was to describe the current status of

digitalization in this particular medical discipline,

with the goal to describe and share this with the

medical community. This description was supposed

to reveal fields of action, structural internal and

external barriers as well as best practices. In order to

achieve that goal, in a first step the standard

procedures and information flows of the discipline

were modelled along the intersectoral patient

pathway. Additionally, the literature review with

regard to modelling digital maturity in healthcare

settings was updated. Therefrom, suitable aspects of

MM tools were identified and merged into an

interview guideline, which was supposed to extract

the relevant information from direct interaction with

medical professionals from the discipline. The

interviews were restricted to a timeframe of 40

minutes. Subsequently, the interview protocols were

pseudonymized and confirmed by the person

interviewed. Following that, a combination of open

and axial coding protocol as suggested by Wiesche et

al. (Wiesche et al., 2017) was carried out. The coding

and grouping of text passages led to a content

classification and analysis. The analysis was split into

general findings about process bottlenecks of the

discipline, the digitalization status of the field OMS

Table 1: Categorization of the two projects implementing MM means.

Name Diagnostic Imaging Oral and Maxillofacial Surgery

Project goal Assess the initial radiology workflow based on

an analogue imaging system. Integrate a digital

imaging system and adapt customizable settings

suitable to an envisaged digital workflow.

Assess efficiency changes between workflows.

Assess the current status of digitalization of the

medical discipline “oral and maxillofacial

surgery”. Identify internal and external barriers

remarkable structural achievements and easily

accessible adjusting screws for leveraging the

potential of digitalization for the discipline.

Scenarios

investigated

3 overall: 1

in hospital, integration of a mobile

imaging system, 2

in hospital, stationary

imaging system, 3

in resident practice,

stationary imaging system

2: 1

in resident practice, 2

in hospital

MM goal Assess, improve MM level, reassess, display

improvement

Assess, interpret, suggest actions

MMs used EMRAM, DIAM WCS, DHMI, EMRAM

MM proceeding Identification of meaningful parameter from the

mentioned models, reduction and fusion of the

models into a project specific set of maturity

assessment points

Identification of meaningful parameter from the

mentioned models, reduction, fusion, and

transfer to the relevant data points and interfaces

in OMS

Type intervention description

Assessing person External (researcher), in close collaboration

with the participants

External (researcher), in close collaboration

with the participants

Data collection

method

Focus groups and guest visit (shadowing) In-depth interviews

Number of

participants

number of participants according to team size

and availability with alternating line-up,

according to roster: 2 departments per scenario,

leading to a total number of 6 departments

8 in total, 4 per scenario

Profile of

participants

Mainly medical technical radiology assistants

(MTRA), and radiologists

Medical specialists categorized by years of

working experience (>10, <10), medical

assistants (at least one representative of each

role per scenario)

Data collection

points

3 per intervention: “before” (focus group and

shadowing), “during” installation (shadowing)

and “after” replacing an analogue with a digital

radiology system (shadowing)

1 interview per participant

Project Year 2017 2020

HEALTHINF 2021 - 14th International Conference on Health Informatics

104

and best-practices in both respects. Additionally,

external and internal influencing factors on successful

digitalization were clustered.

3.5 Retrospective Analysis

The two described projects were retrospectively

analysed with regard to the formulated research

questions. In a first step the project content, its goals

and the applied means and methods were

summarised. Following that, the conducted steps and

tools were comparatively displayed on a timeline.

The project documentation (interview transcripts,

focus group documentation and notes, workflow

analysis and notes from the shadowing sessions) was

then inductively analysed by a coding protocol

combining open and axial coding as suggested by

Wiesche et al. (Wiesche et al., 2017). The axial

grouping led to 5 classes of findings observed in both

cases associated with the formulated research

questions. The findings are described in section 4.3.

4 RESULTS

The project’s timelines of steps and methods

conducted showed a common proceeding of the

following steps: 1. to assess the specific user goals, 2.

To narrow down the object that was to be described,

3. to identify a suitable tool to depict the respective

field of interest, 4. to adapt available tools to the

specific need, 5. to translate this into a real

application, 6. to derive knowledge from this

application and optional (only for the diagnostic

imaging project) 7. to integrate an intervention in

order to increase the digital maturity, and 8. to

reassess and examine efficiency differences between

the initial and final workflow. The application of

MMs was thus not an end in itself, and not

particularly scope of the observation, but served as a

tool for achieving a practical goal. However, the

translation of formalized MMs into practice is still a

field of research these projects contribute to.

Therefore, the following sections briefly outline the

two project’s results regarding their general scope,

and present central findings from the application of

MMs in these two healthcare contexts.

4.1 Case 1: Diagnostic Imaging

The efficiency of diagnostic imaging workflows in

resident practice as well as in hospitals based on an

analogue imaging system was assessed. Following

the replacement of the analogue with a digital

imaging system, the efficiency of the old workflow

was compared to the new workflow in the same

setting. In the resident practices a stationary imaging

system and in the hospital environment stationary and

mobile systems were examined. For each scenario,

two systems from differing manufacturers were

compared to a digital system from one manufacturer

(who also fabricated one of the initial systems in each

scenario). The digital workflow was found to be more

efficient compared to the analogue workflow to a

varying extent in all the scenarios. The comparison of

two different manufacturers of the initial system

showed no significant divergence by contrast with the

digital systems for none of the scenarios. Since this

project was commissioned by the digital system’s

manufacturer and the results intended for internal

purposes only, the study has not in all its details been

made available to the community.

4.2 Case 2: MOS

Based on the conducted interviews, the general

workflow, involved parties and actors, interfaces,

data and transfer points were described. The

digitalization status of the working environment,

workflow and the organizations was modelled based

on the set of maturity parameter merged from WCS,

DHMI and EMRAM for each interviewee as well as

abstracted across all participants. General

digitalization hurdles like literacy and sovereignty in

handling digital services and workflows, or the lack

of integrated process support were identified.

Particularly within hospitals, the competition of

disciplines for investment budgets was mentioned as

an inhibiting factor. The development status of

digitalization especially between resident practices

was highly heterogeneous. The close interrelationship

with the field of diagnostic imaging was identified as

a driver for development: innovations often diffuse

from this area to the discipline of OMS. The

comprehensive analysis was made available to the

medical discipline and the scientific community in a

separate article (Meister et al., 2020).

4.3 Retrospective Findings across

Cases

Only sparse experience from practical application of

MMs in healthcare has been reported thus far.

Blondiau et al. (Blondiau et al., 2016) made their

experiences available, based on the intervention-

cycle presented by Mettler (Mettler, 2010),

differentiating between findings related to MM

design and implementation. Referring to the latter, the

Practical Application of Maturity Models in Healthcare: Findings from Multiple Digitalization Case Studies

105

authors brought MMs into application, and identified

challenges exceeding the ones addressed by the

experiences and suggestions by Blondiau et al. In this

section the authors present and formalize 5

observations noted in both of the two case studies

with regard to translating MMs into real-world

application in healthcare context. The observations

are as follows:

1) Continuous Translation between Research and

Practitioners is Required.

Not only the knowledge of the relevant state of the art

regarding MMs was found to be fairly limited on the

practitioners’ side (which is understandable since it is

a huge field and not particularly the core business of

the hospitals’ representatives). At the same time,

although the interest to be involved and contribute to

the arrangement was present, the comprehensibility

of the available literature seemed rather formalized

and impractical to the user groups. This led to the

necessity of intensive collaboration between research

and practice for identification, extraction and merging

of suitable means into a depiction protocol. Also

during the protocol execution, a close guidance and

affirmation was asked for. This emblematically

shows the misunderstanding between research and

practice. Researchers are striving for abstraction and

formalization and not necessarily provide an

overview of explicit technical or processual

innovations and best practices. On the other hand,

practitioners are hoping for precise details and

guidance. The interface in our two projects was

bridged by human efforts. Both sides need to move

towards each other by the means of education and

simplification.

2) Implementation of MMs and Taking Actions

based on This Requires Support of All Affected

Professional Disciplines and Hierarchy Levels.

In Hospitals we have the special situation, that

managerial and medical leaders work detached from

each other in terms of content and are not authorized

to issue instructions to each other with regard to the

specific professional matters handled by the

respective division. Since interests between these

areas not always fully overlap, joint efforts require the

involvement and support of all affected stakeholders.

Vice versa, that implies that MMs which explicitly

target a particular profession, while the object of

examination (or especially the evolution of it) affects

areas or actors beyond the included group, are hardly

able to contribute to mutual interests. While there are

reasonable intentions of addressing an explicit

profession with a MM (e.g. for tailoring the language

or conceptual level for the needs of the user group

(Blondiau et al., 2016), or strengthening the

negotiation position of individuals), the concomitant

non-consideration of other interest groups may even

affect the implementation of measures adversely.

3) Structural Development of Hospitals Requires

Modular and Holistic MMs.

Numerous MMs with differing level of detail and

focus exist, some of which were found suitable for

depicting a project-relevant aspect of maturity within

very specific boundaries. However, neither a

conceptually holistic and encompassing MM nor one

suitable approach for that very specific project goal

could be identified. This can of course vary in

conjunction with the respective area or goal one

wants to examine or achieve. Nonetheless, for the two

projects we had to extract different aspects from

different MMs and combine them into a presentation

appropriate for the respective group of practitioners.

We certainly did not expect to identify a perfectly

suitable MM able to model the two specific medical

disciplines, workflows or data exchange points.

Addressing the conducted effort in both project of

merging suitable modules of existing MMs into a

project-specific set of parameters by the development

of a modular and holistic MM can significantly

reduce the application hurdles for representatives

from practice. From a scientific point of view the

focus on depth rather than breadth in MMs makes

sense: comparability and schematization of similarly

parameterizable aspects naturally leads to narrowing

the observation field. At the same time, it impairs

practical application, since seldom only one particular

aspect is part of a real-world project. In order to

increase value creation for the hospital domain

multivariate and modular analysis means, or an

integration of specialized solutions are required.

4) A Differentiation between “Visible Result” and

Necessary Action to Achieve That Result is

Required.

MMs usually offer a path of evolution with regard to

the specific aspect of examination. This presentation

suggests necessary actions for achieving a higher

level of maturity by displaying the required

characteristics for the next stage. However, especially

MMs dealing with digitalization tend to derive a

maturity model from a set of “checkable

characteristics”. This seems intuitive since

digitalization becomes obviously technologically

verifiable. At the same time, it neglects that the digital

transformation requires processual and organizational

action, which eventually leads to a technological

HEALTHINF 2021 - 14th International Conference on Health Informatics

106

status. So checkable maturity parameter and

evolutionary actions not necessarily need to be the

same. Most MMs do not clearly distinguish between

these “checkable characteristics” and necessary

actions to achieve such a state. The development of

an organization like a hospital is multivariate and

complex itself. Correlation and causation remain

largely undescribed. Especially MMs with a focus on

data flows and degree of information integration

suggest a rather technical approach to developing a

state of digital maturity. From our experience, the

depiction of such “measurable” indicators may be

mistaken by practitioners for the necessary actions

required to achieve the next level. Especially in such

an interprofessional environment this outcome may

not in all cases be achieved by “just” implementing

technological requirements for the next maturity

level. E.g. in rather technical MMs the processual,

organizational and human prerequisites are necessary

to shape the digital transformation successfully and

encompassing. We suggest a stronger emphasis on

the differentiation between measurable maturity level

parameter and evolutionary actions required to

achieve the next stage.

5) The Static Definition of Maturity States in Times

of Disruptive Change Needs to be Questioned.

Some MMs provide a firm definition of maturity

levels, and some comprise a certain degree of

dynamics and adaptability to future developments.

The general concept of maturity modelling in the past

decades was mainly intended to support incremental

development. Today, great uncertainty engages the

healthcare provision domain on how to shape the

disruptiveness of the digital revolution. A clear and

encompassing vision of the future digital hospital is

neither by practice nor by research formulated yet.

This is also reflected in the respective definitions of

high sophistication in existing MMs. Therefore, the

depiction of the current status of maturity of a certain

skill, process or organization seemed capable within

the two applications. On the other hand, the

evolutionary paths provided by the selected MMs

could not cope with the expectations regarding

disruptive reorganization of healthcare provision.

Some of the selected MMs even acknowledge this

uncertain target state and leave the room for future

definition. That certainly reflects the current situation

of the domain in a sufficient way from the research

perspective. At the same time, it leaves practice alone

with the empty room the anticipated disruptiveness of

the digital revolution provides. Statically defined

maturity states are only able to describe our current

knowledge and imagination. However, they set

boundaries when they are used as means to

prospectively shape disruptive change. It could be

incorporated explicitly that current depictions only

reflect current knowledge, and that the digital

transformation requires solutions and arrangements

exceeding that. Nonetheless, in order to increase

value creation for practitioners in the future, this

uncertainty needs to be addressed by joint efforts to

foster clarification (this probably exceeds research

and practice and also requires politics and society) of

how healthcare delivery should be transformed in the

digital age.

5 CONCLUSIONS

With this article we intend to support building the

bridge between the developers and implementers of

MMs. The growing number of publications with

regard to maturity models emphasizes developments

and discussions of design methodologies rather than

reports on practical application and efficacy of that

implementation itself and potential change efforts

(Blondiau et al., 2016).

The need for application guidance, as described

by Caldwell and Atwal as a result of the

interdisciplinary organization “hospital” (Caldwell &

Atwal, 2003), was found to be still very high.

In both depicted cases, the definition of an

envisaged digital target state required a joint effort

between practitioners and researchers. A systematic

uncertainty within the domain of how exactly the

“digital hospital” as a maturity status can be defined

remains. For the purpose of guiding through the

disruptiveness and dynamics of the digital

transformation, existing MMs were perceived as too

focused on incremental development and static states.

With this prevailing design the capability of MMs of

guiding the healthcare domain through the

prospectively postulated and not yet fully described

digital revolution is arguable. However, applying

MMs with the aim of monitoring progress as

described in case 1 was found appropriate.

5.1 Contribution

The presented article provides various practical and

managerial implications. The observations derived

from projects applying MMs in practice may serve

practitioners and collaborative implementing teams

in their decision making when it comes to reflecting

the practical capabilities and expectations with

applying MMs. Furthermore, insights into challenges

to expect in the process of identifying, adapting and

Practical Application of Maturity Models in Healthcare: Findings from Multiple Digitalization Case Studies

107

implementing suitable means for a specific maturity

purpose are provided.

At the same time, the observations presented are

equally suited to be taken into account by the

developers of MMs. The consideration of these

during the creation of MM solutions may contribute

to the practical applicability and thus on the value

creation for the practitioner’s scope of application as

well as the structural development of healthcare

organizations in general.

5.2 Limitations

At this stage, however, it is necessary to also share the

limitations of the presented study. This particularly

refers to the generalizability of the results. Firstly, the

findings result from two case studies from very

delimitable medical disciplines, and the

transferability to other fields or wider scopes is not

necessarily given. Secondly, the observed

implications arose from the interaction of humans

during the projects. We acknowledge the theory

formulated by Walsham, that the enquirer’s

interaction with the study object or environment and

the perception of all involved parties not only

influences the sensing of events, but also the reality

itself (Walsham, 1995). Perception and interaction

impact the reality created within a situation and such

observations are thus not compulsory applicable to

other social settings or general environments.

5.3 Future Research

Prospectively, it will be essential to invest further

research into the determination of success factors and

capabilities a hospital must have to be able to

collaboratively shape the change for the benefit of the

patient. The expert organization itself is only partially

described and can thus hardly be evolved structurally.

Therefore, further knowledge about the underlying

causal relationships of communicated and subliminal

aspirations of individuals and professional groups in

this special setting of an expert organization is

required.

Besides, further invest is needed in the definition

of a common ground of a potential “target state” of

the digital hospital.

Furthermore, since the transferability of the

presented observations is limited due to the two cases,

insights into the work of other research or practitioner

groups with regard to practical application of MMs

would be beneficial in the future.

ACKNOWLEDGEMENTS

Parts of the research presented in this paper stems

from the Center of Excellence for Logistics and IT

located in Dortmund (Leistungszentrum Logistik &

IT).

REFERENCES

Allweyer, T. (2010). Bpmn 2.0: Introduction to the

standard for business process modeling. Books on

Demand.

Barrett, M., & Oborn, E. (2018). Bridging the research-

practice divide: Harnessing expertise collaboration in

making a wider set of contributions. Information and

Organization, 28(1), 44–51. https://doi.org/10.1016/j.

infoandorg.2018.02.006

Bauernhansl, T. (2014). Die Vierte Industrielle Revolution

– Der Weg in ein wertschaffendes

Produktionsparadigma. In T. Bauernhansl, M. ten

Hompel, & B. Vogel-Heuser (Eds.), Industrie 4.0 in

Produktion, Automatisierung und Logistik (pp. 5–35).

Springer Vieweg. https://doi.org/10.1007/978-3-658-

04682-8_1

Baxter, P., & Jack, S. J. (2008). Qualitative Case Study

Methodology: Study Design and Implementation for

Novice Researchers. The Qualitative Report, 13, 544–

559.

Becker, J., Knackstedt, R., & Pöppelbuß, J. (2009).

Developing Maturity Models for IT Management.

Business & Information Systems Engineering, 1(3),

213–222. https://doi.org/10.1007/s12599-009-0044-5

Blondiau, A., Mettler, T., & Winter, R. (2016). Designing

and implementing maturity models in hospitals: An

experience report from 5 years of research. Health

Informatics Journal, 22(3), 758–767. https://doi.org/

10.1177/1460458215590249

Braa, K., & Vidgen, R. (1999). Interpretation, intervention,

and reduction in the organizational laboratory: a

framework for in-context information system research.

Accounting, Management and Information

Technologies, 9(1), 25–47. https://doi.org/10.1016/

S0959-8022(98)00018-6

Burmann, A., Deiters, W., & Meister, S. (2019). Digital

Health Maturity Index: Analyse des

Digitalisierungsgrades im Krankenhaus. In M. A.

Pfannstiel, P. Da-Cruz, & H. Mehlich (Eds.), Digitale

Transformation von Dienstleistungen im

Gesundheitswesen VI (pp. 3–18). Springer Fachmedien

Wiesbaden.

Caldwell, K., & Atwal, A. (2003). The problems of

interprofessional healthcare practice in hospitals.

British Journal of Nursing (Mark Allen Publishing),

12(20), 1212–1218. https://doi.org/10.12968/bjon.

2003.12.20.11844

Carvalho, J. V., Rocha, Á., & Abreu, A. (2016). Maturity

Models of Healthcare Information Systems and

HEALTHINF 2021 - 14th International Conference on Health Informatics

108

Technologies: A Literature Review. Journal of Medical

Systems, 40(6), 131. https://doi.org/10.1007/s10916-

016-0486-5

Conwell, C. L., Enright, R., & Stutzman, M. A. (2000,

December). Capability Maturity Models support of

modeling and simulation verification, validation, and

accreditation. In 2000 Winter Simulation Conference

Proceedings (Cat. No.00CH37165) (pp. 819–828).

IEEE. https://doi.org/10.1109/WSC.2000.899880

de Bruin, T., Freeze, R., Kulkarni, U., & Rosemann, M.

(2005). Understanding the Main Phases of Developing

a Maturity Assessment Model. Australasian

Conference on Information Systems.

Deiters, W., Burmann, A., & Meister, S. (2018).

Digitalisierungsstrategien für das Krankenhaus der

Zukunft [Strategies for digitalizing the hospital of the

future]. Der Urologe. Ausg. A, 57(9), 1031–1039.

https://doi.org/10.1007/s00120-018-0731-2

Esdar, M., Liebe, J.-D., Weiß, J.-P., & Hübner, U. (2017).

Exploring Innovation Capabilities of Hospital CIOs:

An Empirical Assessment. Studies in Health

Technology and Informatics, 235, 383–387.

Gill, P., Stewart, K., Treasure, E., & Chadwick, B. (2008).

Methods of data collection in qualitative research:

Interviews and focus groups. British Dental Journal,

204(6), 291–295. https://doi.org/10.1038/bdj.2008.192

Goldkuhl, G. (2012). Pragmatism vs interpretivism in

qualitative information systems research. European

Journal of Information Systems, 21(2), 135–146.

https://doi.org/10.1057/ejis.2011.54

Gomes, J., & Romão, M. (2018). Information System

Maturity Models in Healthcare. Journal of Medical

Systems, 42(12), 235. https://doi.org/10.1007/s10916-

018-1097-0

Gottschalk, P. (2009). Maturity levels for interoperability in

digital government. Government Information Quarterly,

26(1), 75–81. https://doi.org/10.1016/j.giq.2008.03.003

Gustaffson, J. T. (2017). Single case studies vs. multiple

case studies: A comparative study. Sociology.

Hermann, M., Pentek, T., & Otto, B. (2016, January).

Design Principles for Industrie 4.0 Scenarios. In 2016

49th Hawaii International Conference on System

Sciences (HICSS) (pp. 3928–3937). IEEE.

https://doi.org/10.1109/HICSS.2016.488

HIMMS Analytics. (2017). „Electronic Medical Record

Adoption Model“. http://www.himss.eu/healthcare-

providers/emram

Hübner, U., Esdar, M., Hüsers, J., Liebe, J.-D., Naumann,

L., Thye, J., & Weiß, J.-P. (2020). IT-Report

Gesundheitswesen: Wie reif ist die Gesundheits-IT aus

Anwenderperspektive? Schriftenreihe der Hochschule

Osnabrück.

Hübner, U., Esdar, M., Hüsers, J., Liebe, J.-D., Rauch, J.,

Thye, J., & Weiß, J.-P. (2018). IT-Report

Gesundheitswesen: Wie reif ist die IT in deutschen

Krankenhäusern? Hochschule Osnabrück - IGW.

Hübner, U., Liebe, J.-D., Hüsers, J., Thye, J., Egbert, N.,

Hackl, W., & Ammenwerth, E. (2015). IT-Report

Gesundheitswesen: Pflege im Informationszeitalter.

Schriftenreihe der Hochschule Osnabrück.

Humphrey, W. S. (1988). Characterizing the software

process: a maturity framework. IEEE Software, 5(2),

73–79. https://doi.org/10.1109/52.2014

Kolukısa Tarhan, A., Garousi, V., Turetken, O., Söylemez,

M., & Garossi, S. (2020). Maturity assessment and

maturity models in health care: A multivocal literature

review. Digital Health, 6, 2055207620914772.

https://doi.org/10.1177/2055207620914772

Krasuska, M., Williams, R., Sheikh, A., Franklin, B. D.,

Heeney, C., Lane, W., Mozaffar, H., Mason, K., Eason,

S., Hinder, S., Dunscombe, R., Potts, H. W. W., &

Cresswell, K. (2020). Technological Capabilities to

Assess Digital Excellence in Hospitals in High

Performing Health Care Systems: International eDelphi

Exercise. Journal of Medical Internet Research, 22(8),

e17022. https://doi.org/10.2196/17022

Lahrmann, G., Marx, F., Mettler, T., Winter, R., &

Wortmann, F. (2011). Inductive Design of Maturity

Models: Applying the Rasch Algorithm for Design

Science Research. In H. Jain, A. P. Sinha, & P.

Vitharana (Eds.), Lecture Notes in Computer Science.

Service-Oriented Perspectives in Design Science

Research (Vol. 6629, pp. 176–191). Springer Berlin

Heidelberg. https://doi.org/10.1007/978-3-642-20633-

7_13

Lee, D., Gu, J.‐ W., & Jung, H.‐ W. (2019). Process

maturity models: Classification by application sectors

and validities studies. Journal of Software: Evolution

and Process, 31(4), e2161. https://doi.org/

10.1002/smr.2161

Leistungszentrum Logistik & IT. http://leistungszentrum-

logistik-it.de

Levy, Y., & J. Ellis, T. (2006). A Systems Approach to

Conduct an Effective Literature Review in Support of

Information Systems Research. Informing Science: The

International Journal of an Emerging Transdiscipline,

9, 181–212. https://doi.org/10.28945/479

Liebe, J.-D., Hübner, U., Straede, M. C., & Thye, J. (2015).

Developing a Workflow Composite Score to Measure

Clinical Information Logistics. A Top-down Approach.

Methods of Information in Medicine, 54(5), 424–433.

https://doi.org/10.3414/ME14-02-0025

Longhurst, R. (2010). Semi-structured interviews and focus

groups. In N. J. Clifford & G. Valentine (Eds.), Key

methods in geography (2nd ed., pp. 103–115). Sage

Publications.

Meister, S., Burmann, A., & Deiters, W. (2019). Digital

Health Innovation Engineering: Enabling Digital

Transformation in Healthcare: Introduction of an

Overall Tracking and Tracing at the Super Hospital

Aarhus Denmark. In N. Urbach & M. Röglinger (Eds.),

Management for Professionals. Digitalization Cases

(pp. 329–341). Springer International Publishing.

https://doi.org/10.1007/978-3-319-95273-4_17

Meister, S., Haßfeld, S., & Burmann, A. (2020).

Digitalisierung in der Mund-, Kiefer- und

Gesichtschirurgie: Vom Status-Quo zur Vision. Der

MKG-Chirurg.

Mettler, T. (2010). Supply-Management im Krankenhaus:

Konstruktion und Evaluation eines konfigurierbaren

Practical Application of Maturity Models in Healthcare: Findings from Multiple Digitalization Case Studies

109

Reifegradmodells zur zielgerichteten Gestaltung (1.

Aufl.). Sierke.

Mettler, T., Fitterer, R., Rohner, P., & Winter, R. (2014).

Does a hospital’s IT architecture fit with its strategy?

An approach to measure the alignment of health

information technology. Health Systems, 3(1), 29–42.

https://doi.org/10.1057/hs.2013.10

Mettler, T., & Pinto, R. (2018). Evolutionary paths and

influencing factors towards digital maturity: An

analysis of the status quo in Swiss hospitals.

Technological Forecasting and Social Change, 133,

104–117. https://doi.org/10.1016/j.techfore.2018.03.

009

Pak, J., & Song, Y.-t. (2016). Health Capability Maturity

Model: Person-centered approach in Personal Health

Record System. In Surfing the IT innovation wave:

22nd Americas Conference on Information Systems

(AMCIS 2016) : San Diego, California, USA, 11-14

August 2016. Curran Associates Inc.

Paulk, M. C., Curtis, B., Chrissis, M. B., & Weber, C. V.

(1993). Capability maturity model, version 1.1. IEEE

Software, 10(4), 18–27. https://doi.org/10.1109/52.

219617

Pöppelbuß, J., Niehaves, B., Simons, A., & Becker, J.

(2011). Maturity Models in Information Systems

Research: Literature Search and Analysis.

Communications of the Association for Information

Systems, 29. https://doi.org/10.17705/1CAIS.02927

Pöppelbuß, J., & Röglinger, M. (2011). What makes a

useful maturity model? A framework for general design

principles for maturity models and its demonstration in

business process management. 19th European

Conference on Information Systems (ECIS).

Quinlan, E. (2008). Conspicuous Invisibility: Shadowing as

a Data Collection Strategy. Qualitative Inquiry, 14(8),

1480–1499. https://doi.org/10.1177/1077800408318318

Ridder, H.-G. (2016). Sozialwissenschaftliche

Forschungsmethoden. Case Study Research:

Approaches, Methods, Contribution to Theory (B.

Kittel, C. F. Altobelli, C. Strobl, G. Tutz, H. Hinz, I.

Borg, J. Reinecke, J. Wagner, K.-U. Schnapp, M. Spieß,

M. Berlemann, M. Kraft, P. Sedlmeier, R. Schnell, R.

Oesterreich, S. Liebig, U. Jirjahn, W. Seidel, & W.

Matiaske, Eds.). Rainer Hampp Verlag.

Rönkkö, M., Järvi, A., & Mäkelä, M. M. (2008). Measuring

and Comparing the Adoption of Software Process

Practices in the Software Product Industry. In Q. Wang,

D. Pfahl, & D. M. Raffo (Eds.), Lecture Notes in

Computer Science. Making Globally Distributed

Software Development a Success Story (Vol. 5007,

pp. 407–419). Springer Berlin Heidelberg.

https://doi.org/10.1007/978-3-540-79588-9_35

Sanders, D., Burton, D., & Protti, D. (2013). Healthcare

Analytics Adoption Model: A Framework and Roadmap.

https://www.healthcatalyst.com/white-

paper/healthcare-analytics-adoption-model/

Schuh, G., Anderl, R., Gausemeier, J., & Hompel, M. ten

(2017). Industrie 4.0 Maturity Index.

Söylemez, M., & Tarhan, A. (2016). The Use of

Maturity/Capability Frameworks for Healthcare

Process Assessment and Improvement. In P. M. Clarke,

R. V. O'Connor, T. Rout, & A. Dorling (Eds.),

Communications in Computer and Information Science.

Software Process Improvement and Capability

Determination (Vol. 609, pp. 31–42). Springer

International Publishing. https://doi.org/10.1007/978-

3-319-38980-6_3

Stephani, V., Busse, R., & Geissler, A. (2019).

Benchmarking der Krankenhaus-IT: Deutschland im

internationalen Vergleich. In J. Klauber, M. Geraedts, J.

Friedrich, & J. Wasem (Eds.), Krankenhaus-Report

2019 (pp. 17–32). Springer Berlin Heidelberg.

https://doi.org/10.1007/978-3-662-58225-1_2

Studzinski, J. (2017). Bestimmung des Reifegrades der IT-

gestützten klinischen Bildgebung und Befundung mit

dem Digital Imaging Adoption Model: Ist Ihre klinische

Bildgebung bereit für das digitale Zeitalter?

[Evaluating the maturity of IT-supported clinical

imaging and diagnosis using the Digital Imaging

Adoption Model: Are your clinical imaging processes

ready for the digital era?]. Der Radiologe, 57(6), 466–

469. https://doi.org/10.1007/s00117-017-0253-8

van de Wetering, R., & Batenburg, R. (2009). A PACS

maturity model: A systematic meta-analytic review on

maturation and evolvability of PACS in the hospital

enterprise. International Journal of Medical

Informatics, 78(2), 127–140. https://doi.org/10.1016/

j.ijmedinf.2008.06.010

vom Brocke, J., Simons, A., Niehaves, B., Riemer, K.,

Plattfaut, R., & Cleven, A. (2009). Reconstructing the

Giant: On the Importance of Rigour in Documenting

the Literature Search Process. http://www.Alexandria.

Unisg.Ch/Publikationen/67910.

Walsham, G. (1995). The Emergence of Interpretivism in

IS Research. Information Systems Research, 6(4), 376–

394. https://doi.org/10.1287/isre.6.4.376

Waring, J., & Currie, G. (2009). Managing Expert

Knowledge: Organizational Challenges and Managerial

Futures for the UK Medical Profession. Organization

Studies, 30(7), 755–778. https://doi.org/10.1177/

0170840609104819

Webster, J., & Watson, R. T. (2002). Analyzing the Past to

Prepare for the Future: Writing a Literature Review.

MIS Quarterly, 26(2), xiii–xxiii. www.jstor.org/stable/

4132319

Wendler, R. (2012). The maturity of maturity model

research: A systematic mapping study. Information and

Software Technology, 54(12), 1317–1339.

https://doi.org/10.1016/j.infsof.2012.07.007

Wiesche, M., Jurisch, M. C., Yetton, P. W., & Krcmar, H.

(2017). Grounded Theory Methodology in Information

Systems Research. MIS Quarterly, 41(3), 685–701.

https://doi.org/10.25300/MISQ/2017/41.3.02

Yin, R. K. (1987). Case study research: Design and

methods (6. print). Applied social research methods

series: Vol. 5

. Sage Publ.

Yin, R. K. (2003). Case study research: Design and

methods (3. ed.). Applied social research methods

series: Vol. 5. Sage. http://www.loc.gov/catdir/

enhancements/fy0658/2002152696-d.html

HEALTHINF 2021 - 14th International Conference on Health Informatics

110