Translating a Clinical Workflow into a Computer-executable Model

User Needs Discovery, Challenges and Lessons Learned

Ana Leitão, Jennifer Caffarel and Erin Stretton

Royal Philips - Research, High Tech Campus 34, Eindhoven, The Netherlands

Keywords: User Requirements, Healthcare IT, User Confrontation, Clinical Pathways.

Abstract: Getting to technical requirements from user input is already a hard task in an environment where the workflow

processes are very well defined. When trying to extract a unique process from the users of such a variable

work environment such as a healthcare institution can be very challenging. In this paper, we share our

experience with extracting user requirements from clinical users by presenting the specific example of

transforming workflows into models that can then be used as part of an IT solution to support workflow

guidance. Here we present not only some of our main challenges when approaching different institutions and

professionals with different roles, but also some of the methods we find most useful to establish

communication and extract as much relevant information possible. In the end we explain some of the

differences between a workflow as explained by the users and a computer–executable model and how to make

the connection between the two.

1 INTRODUCTION

In 2004 Ash et Al. said that “we should strive to have

a national system of Electronic Health Record (EHR)

that can share information on any patient in any

health care setting”. While some institutions have

the financial capabilities to have the latest technology

which allows them to automate and make a wide

range of tasks digital, there are still institutions who

depend very highly on paper, especially when it

comes to documenting clinical processes and

pathways.

From our learnings this persistence on the use of

paper to document clinical processes and pathways

within a hospital is not only driven by lack of

resources and financial capabilities to go digital but

in many cases, paper is seen as an easier alternative

to implement than modifying the IT system to support

such tasks. Also, the IT systems are still having

negative impact on the work of the clinicians by

increasing the documentation time and being

incompatibility with clinical workflow. This leads to

a higher amount of interruptions in the medical work

and system-introduced errors in patients care.

(Ologeanu-Taddei, R. et al., 2015; Jamoom, E. W. et

al., 2016)

Another limitation of paper supported processes is

that although this may make the administrative task

of collecting most relevant information in one

location easier, they generally do not aid the clinical

users to adhere to the recommended steps in a

pathway in real-time – they mainly serve as a

reminder of which data to collect so that the

management can at a later date evaluate how well the

care was delivered. And while with an EHR this

collection of information is easier and more

automated, there are still gaps on the usage of these

that limit the step by step tracking of clinical events.

One of our research aims is to investigate how we can

proactively support the clinical staff to adhere in real-

time to clinical pathways, with a greater focus on

delivery of care than on care delivery documentation.

We want to do this by going beyond the simple

digitization of the paper process.

Even though it is possible to identify processes in

healthcare, these are different from the ones that

could be found in an industrial manufacturing

environment (Mans et al., 2015). In the healthcare

environment, users are dealing with the care of

patients, who are not predictable machines with

predictable failure modes. Therefore enforcing a

process can be an extremely challenging task, let

alone having a system trying to follow the steps

making up that process. It is extremely difficult to

identify all the steps healthcare professionals follow

in a process performing retrospective data analysis

LeitÃ

co A., Caffarel J. and Stretton E.

Translating a Clinical Workﬂow into a Computer-executable Model - User Needs Discovery, Challenges and Lessons Learned.

DOI: 10.5220/0006275005940600

since EHRs are rarely pre-programmed to support

clinical workflows in a structured manner. Therefore

any evidence we collect from these, represent point

measures of the process either indicating partial

activities in the process or post-documentation

information. Our research challenge was therefore to

understand how the users incorporate the clinical

processes in their routine, how they interact with the

systems, and how they could optimally be supported

with technology in the future to support them in real-

time to adhere to clinical pathways as they deliver

care to patients. There are no specific guidelines to

support extraction of requirements from end-users in

such situations, with most publications in the area

focusing on modelling of clinical processes, but not

giving insights onto the differences one might expect

across hospitals or recommendations how to translate

user specifications to technical requirements.

(Böckmann and Heiden, 2013; Dadam et al., 2000;

Latoszek-Berendsen et al. 2010; Staccini et al. 2001;

Oosterhout et al. 2005)

In this article we share some of our experiences

and methods used to go from an expressed wish from

users to actual user and technical requirements, using

the specific example of transforming workflows into

models that can then be used as part of an IT solution

to support workflow guidance.

In the specific study that we are using as example

we have confronted 4 different institutions with

different levels of understanding and complexity

regarding the same type of care delivery pathways

and associated workflows. We had a total of 51

participants in which 13 where physicians, 21 where

nurses (some with coordination roles within the

departments), 11 where quality staff and 6 had other

roles within the Hospitals. The Hospitals involved

have between 200 and 350 beds each and cover most

specialties inside the institution. Associated to this

they all have implemented an EHR but only one was

considered a paperless Hospital. Our expectation is

that models can allow us to create a unique translation

of realities, bringing together the workflows of the

different institutions. Therefore, we focus on the

specification of requirements for a technology

solution to assist the adherence to a Clinical Pathway

that involves multiple stakeholders, sometimes across

departments; and for a variety of care settings.

2 USER CONFRONTATION

2.1 Preparation Phase

Our team conducted user confrontations to validate

hypotheses, developed from literature reviews and

existing knowledge on the topic, on what users would

want in a workflow product. When researching how

to design these products to fit the needs of a specific

population of healthcare professionals, doing many

iterations of user confrontations is important. The

aim with the first user confrontations is to understand

what aspects of the problem cause the user the most

hindrance, and from there prioritize aspects of the

future product and create a research roadmap. For

each iteration after this, the scope of the

confrontations becomes more granular to specify

wishes for components of the proposed product.

Preparation for these confrontations consists of 3

phases: (i) identifying the activities to be done with

the clinicians and what type of professionals need to

be included, (ii) preparing legal and regulatory

documentation with our company and the hospitals

with which we will collaborate, (iii) contacting

hospitals and agreeing on an agenda.

(i) Identifying activities and participants: For

every round of user confrontations, we need to

identify:

a. What information we need to retrieve from

the activities (proving or disproving our

hypotheses) [more on this topic in the next

sub-section]

b. Who we should involve in the

confrontations in terms of professional

roles

Then, we plan activities that help us gather this

information. For each activity, we usually assign 2

researchers per activity, for 1-4 participants. This way

one person is able to take notes and another leads the

exercise. When performing user confrontations, a risk

that is run is that of collecting the opinion of too few

users and using these opinions to generalise to the

overall population of users. One way to address this,

if resources are limited, is to organise group activities

which enable multiple users’ viewpoints to be

collected in one session.

(ii) Legal and Regulatory: For conducting user

confrontations, there are legal and regulatory

agreements that need to be made on the

institutions involved, covering the interviewers

and the interviewees. It is important to keep this

in mind when planning the confrontations.

Translating the relevant documents such as

Non-Disclosure Agreements and Consent

Forms into the local language and allowing

time for reading and signing by the institution

could take several weeks. We strive to send the

Participation Information Letter and the

Informed Consent in advance of the visit, to

ensure the participants have had time to read it

and consider their involvement.

(iii) Preparing the agenda: When planning the

activities it is important to keep in mind time.

Healthcare professionals will typically have

30-45 minutes to spend in the activities. Lastly,

the agenda request must be made to the

hospital. In this request, there should be an

overview of the project, goals of the interviews,

and request for specific users for specific

amounts of time (for individual interviews and

group exercises).

2.2 Exercises Used by the Team

In this paper, we focus on the exercises carried out in

the first round of confrontation sessions with the

users. For this round, our main aim was to derive the

users’ main needs and challenges when it comes to

supporting the implementation of Clinical Pathways

in practice. As we were still in our project definition

phase, our scope was large: we wanted to learn more

about the topic from the users (general information on

how this is done in practice), from a variety of users

(from administrative to clinical and managerial staff)

and from all phases of Clinical Pathways (from the

creation of pathways, the use at the point of care to

the evaluation reporting). With this in mind, we

devised a number of activities:

 Interviews to understand the users’ and

institutions’ realities and scope the landscape

of Clinical Pathways as it stands today in

clinical practice

 A model building exercise, where we asked

the participants to build a pathway of their

choice and take us through their routine to help

us identify elements common to various

pathways as well as routine bottlenecks and

deviations from the set pathway

 Confrontations of our work and assumptions:

- Review of a generic model we created, to

get some validation on the modelling of the

pathways

- Feedback on user interface designs for use

in supporting clinical and administrative

workflow to verify some assumptions we

made on information needs for clinical

users when it comes to needing to track

and care for a patient on a pathway

- Interactive activity enabling participants to

prioritise importance of pre-assumed

needs, as well as creation of unthought-of

needs

 Questionnaires to derive quantitative results

related to the concepts explored with the users.

Whenever possible, we asked the users to walk us

through their way of working, showing us how they

interact with the clinical systems and indicating when

they would do this at various points of the care.

We had a variety of one to one and group settings.

Considerations when setting up group confrontations

are the following:

 Size of group: 4-6 participants for 2 facilitators

is an appropriate size, larger groups may have

to be separated in sub-groups

 Roles within group: hierarchy and personality

may influence the interaction dynamics within

a group, for overall opinion on a concept, we

prefer to separate the groups according to role;

mixing of roles can work well when

complimentary perspectives about different

aspects of a topic are sought

 Facilitation skills: the facilitator of the activity

should direct the involvement of participants

when necessary, to ensure a fair representation

of all participants in the discussions

 Discussion material: having material to discuss

(such as a concept video, screen designs, a

conceptual poster) can help the conversation

along, as a starting point or as a way to focus

the participants on the topic matter.

Overall, we derived a lot of useful information

from the users, which ranged from scoping a

landscape of realities and challenges from various

hospitals with varying levels of maturity when it

comes to implementing Clinical Pathways; all the

way to having a much clearer picture of the roadmap

we needed to create in order to meet the most pressing

needs of the users.

Some pitfalls we encountered were:

 Broad scope and limited time meant that some

topics could not be deeply explored



Tight planning meant that not all exercises

could be conducted with all users

 Questionnaire not specific enough to provide

significant added value on top of qualitative

results

 Unexpected changes in times and personnel

available for participation in the activities.

Our recommendations include:

 Be clear within the team on the objectives of

each exercise

 Dry-run the activities with colleagues or proxy

users not involved in your project to ensure

instructions and questionnaires make sense

before finalisation

 Double check translations (back translation if

possible) to ensure that the meaning is retained.

This might seem quite obvious but it is often

dismissed specially when using official

translators. It is important to make sure the

interpretation is the same for all readers no

matter the language.

 Perform debriefing within the team as

frequently as possible, at least at start and end

of each day, and if possible, in between

activities especially in the first days to ensure

that the activities can be refined as the

interviews progress.

Finally, keep in mind that structured activities and

questionnaires are important to ensure focus is not

lost, but ability to improvise and follow the

participants’ train of thought in a semi-structured

interview format can often be invaluable to discover

user needs the team had not thought of or planned for.

2.3 Challenges of Interaction with the

Users

The first challenge of interacting with users on an

international level is communication. It is imperative

that the professionals that are participating in the

discovery activities, fully understand what is

presented so that your questions can be answered in

the end of the exercises.

To facilitate the understanding and better

communication we try to provide all the material and

conduct all the activities in the language of the users,

whenever possible.

Also, it is important to keep in mind that the realities

differ among institutions so the speech should be

adapted to the reality of each institution and

professional. You should always take into

consideration the following factors:

 Technical resources. Not all institutions have

the same resources, such as, imaging machines,

beds or medications. This has a very high

impact on how the tasks are done, meaning that

the same step in a process can be executed in

different ways and sometimes even include a

third party institution who provides the

resource. A good model based system can help

not only to optimize the existing resources but

also to find the best workflow using the

resources available at each institution.

 People. Not only is there variation in the

availability of staff, but also in the interaction

between different types of professionals among

the different institutions. As an example, in

some hospitals strict hierarchy may be the norm

(e.g. in one institution nurses may be

empowered to put a patient in a clinical

pathway whereas in another this may only be

done by a clinician). This has a big impact not

only on the identification of who should be

involved in a task but also on the attribution of

authority and permission for decision making.

This is so far the hardest factor that can affect

not only the way the exercises are done during

the user interactions but also can have a big

impact on how an IT solution will be used in

the institution. If you are looking to create a

solution which could be used in different

institutions it is important to identify all the

potential users, how they interact and who will

be the potential main users (which can include

different type of professionals)

 Impact of geographical, organisational and

legislation factors on the level of maturity of

clinical processes. By association, institutions

that are involved with certification

organizations and medical societies usually

have very clear ways of working which are

based on best practices. This is also very

closely related to differences in implementation

of health services between different

institutions, regions and countries. In countries

where there are little or no public healthcare

institutions except for the primary care

facilities, most Hospitals and private

institutions will rely on certifications to

distinguish themselves from others. In the case

of countries where the Healthcare service is

very well managed by the government and

advances, chances are that standardization and

certification processes are stimulated if not

required by the government to guarantee the

minimum quality of services.

 Knowledge. It is easy to assume that different

types of professionals have different levels of

knowledge. While that is true on a high level

and most people have greater knowledge on

their roles rather than on that of others, it is

good to not only explore their roles, but also

how they interact with and perceive the roles of

others in the organisation. When approaching

users from different institutions, cities or even

countries, we must take in consideration their

knowledge not only regarding technology (how

familiar are they with the latest technology and

how they use it in their work) but also on the

content level. As mentioned before, the

involvement of the professionals with the latest

news on best practices will also define how

prepared they are to understand the concept to

be discussed during the user interactions. And

as rewarding as it may be to involve Key

Opinion Leaders who are very much up to date

with state of the art and best practices in the

area you want to discuss, it can be even more

insightful to talk with users who have less or

little knowledge of the theoretical aspects so

that you can understand the real practical issues

the end-users are actually confronted with.

Nonetheless, it is good practice to assume the

users know very little about the topic, and be

prepared with a good but not restrictive

definition of the concept you want to present.

3 FILTERING KNOWLEDGE

INTO REQUIREMENTS

The main challenges we have had are:

 How to transform information collected from

users into requirements usable by the technical

team?

 How to ensure that the collected needs and

corresponding requirements are in some way

weighted to reflect the input from the variety of

users we interacted with?

To address the above, we employed a number of

methods, which included use of:

 Raking/Rating systems where possible, e.g.

when confronting 3 user interface designs,

beyond asking for specific feedback, we also

asked the users to classify the designs from

their preferred one to least preferred one; for

the pre-assumed needs list, we asked the users

to rate each requirement as necessary, nice to

have, or not necessary

 Quantitative data analysis wherever possible;

e.g., for the pre-assumed needs list, we

calculated a weighted average across the

groups and used this to rank the requirements

in order of importance, which gave an objective

perspective on the relative importance of the

rated requirements

 Consensus technique whereby we analysed the

results of the interviews by first having a round

of insights extraction from our notes at an

individual interviewer level, before coming

together to share findings and debate if the

insights resonated with one or more

interviewees before including this as relevant

insights for our results.

Concerning Clinical Pathways, a main insight that

was drawn from our study with users is how to bridge

the technical viewpoint and the user viewpoint: there

are really two aspects to workflow modelling. One

level are the workflow elements needed for user

interaction to support them in their daily work; the

other level are those workflow elements which are

essential to the user to follow a Clinical Pathway, but

may not be relevant to model in technical terms,

either because it is not measurable or is difficult to

model, e.g. due to lack of evidence in the clinical IT

systems.

Figure 1: Simplified clinical process example as it would be

described by the user. The boxes in grey represent the

activities in the clinical process which are essential for the

users to carry out the process but only essential to the people

carrying out the task and not the model; or not captured in

the IT system because non-measurable or not included in

the IT documentation of the process.

Taking the example of a high level description of

a clinical process as described by a user such as the

one in Figure 1, we can identify 5 steps identified in

grey, of such type, e.g., the communication

interaction whereby the conversation process is more

important than the actual data exchanged. In the same

example we have the visual triage which is not

measurable since it is done mostly following the

professionals’ instinct and experience and it is not

associated with any record in the EMR; or activities

such as “collection of samples” which are not

captured in the EMR because so far, when the EMR

is used mainly for documentation of patient medical

data, there was no need to capture such process-

related information.

The same model can be translated into a machine

executable model, including only the steps that can be

found or recorded using the EMR, which would look

more like the model presented in Figure 2. Here we

can see loops appear in the place of a step by step

flow. While the user feels the need to represent every

step of the process as being unique, when mapping

these to the EMR the distinction loses relevance. For

example we can say that “Medical Evaluation” and

“Evaluation of results” are the same task since these

are represented by a new iteration of a clinical note in

the EMR.

Another big difference between a model described

by a clinical user and a technical model as the one of

Figure 2 is the detail and grouping of steps. We can,

for example, remove the “Suspected Diagnosis” step

described by the user as this is usually included only

as part of the clinical note. Also, steps that are in

distinct areas of the EMR and can be done in different

contexts outside the flow described by the user can be

represented as sub-processes. For this we have the

example of the “Diagnostic sub-process” or the

“Treatment sub-process” which can be done in a

different order or sequence than the one of Figure 1

when used in a different patient or clinical context.

Figure 2: Simplified clinical process example as it would be

used in the backend.

In the end we are left with only the steps which

can be detected from or triggered using specific

activities of the EMR. And while this might bring

some value in terms of process evaluation using the

EMR, it is not so useful if we are trying to support

and stimulate the users to follow guidelines or

processes using an abstract process model where the

same type of task can have different meanings and

relevance.

We believe that a model that reflects the habits

and routines of the professionals and not just the steps

/ recommendations of the protocols / guidelines /

pathways is the key to make a process support tool

operational and usable in clinical practice. That is, a

model which guides the users into doing the right

thing using more than just the steps that are recorded

in the EMR but also including those necessary for

their own routines. Such an ideal model based

solution would be the one that is capable of providing

the support for the human only tasks mentioned in

Figure 1, that usually have no place for representation

in the EMR (e.g. nurse calls the lab to check status of

sample analysis), even if they are not driving the

reasoning of the process. This support can be given

not only by making the association between the

modelling tools with Clinical Decision Support

Systems (CDS) but also organizational tools just like

communication, schedule assistance tools or others.

A severe limitation of modelling clinical processes

(whether prospectively or derived from process

mining) is the ability to derive representative models

despite some essential activities not being represented

in the event dataset.

Concerning those activities that are not possible to

model due to lack of evidence in the IT system,

these are essential to be aware of as this may imply:

 an incorrect (incomplete) representation of the

process when performing process discovery.

Which consists in applying process mining

algorithms to an event log based on the

information from the Hospital’s EHR database,

to discover a process model. (van der Aalst, W.,

2016)

 a necessary change to the IT system which may

have an impact on the workflow when trying to

derive a process model for real-time tracking of

process activities.

The latter has implications that go further than the

mere addition of a few requirements to the IT

solution: if the additional events cannot be captured

automatically, this will imply additional input from

the users affecting their workflow and potentially

adding burden to the overall process. If the

workflow is affected, this would also call for other

measures such as co-creation with the users and

change management leading up to and during

introduction of the technology to ensure acceptance

and good uptake of the solution.

4 CONCLUSIONS

Deriving clinical processes based on data available in

EHRs is a challenge for a number of reasons: different

hospitals are likely to implement similar processes in

different ways due to different resources available

and local constraints; not all process activities may be

directly extractable from the data, due to lack of

documentation or impossibility to capture in a

structured format; any additional process-related data

which needs to be acquired may be seen as an

additional burden on the users and may impede the

actual process we are trying to support. When

extracting knowledge from users to determine

relevant events from data or to derive process models,

one must be aware of the different realities of each

setting and user’s role, and try to capture the overall

process by approaching the various stakeholders that

often work together to make the entire clinical

process a reality.

It is really important to find a balance between the

tasks that need to be represented and shown to the

user and the tasks that can be automated relieving

burden from the user. For a good workflow support

system we do not necessarily need to present all the

steps of the process to the user nor represent in the

model all the intermediate steps that are taken by the

user. More than a good model, you will need extra

support systems that can fill the gaps and fix the

bottlenecks of the workflows.

ACKNOWLEDGEMENTS

We would like to thank the participants in our

research study for time and valuable input, as well as

our colleagues from our Research project for making

this work possible.

REFERENCES

Ash, J., Bates, D., 2004. Position Paper. “Factors and

Forces Affecting EHR System Adoption: Report of a

2004 ACMI Discussion” Journal of the American

Medical Informatics Association 2005; 12:8–12.doi:

10.1197/jamia.M1684.

Böckmann, B., Heiden, K. 2013. “Extracting and

Transforming Clinical Guidelines into Pathway Models

for Different Hospital Information Systems.” Health

Information Science and Systems

1:13.doi:10.1186/2047-2501-1-13.

Coiera, E., 2000. “When Conversation Is Better Than

Computation.” Journal of the American Medical

Informatics Association 7 (3): 277.

doi:10.1136/jamia.2000.0070277.

Dadam, P., Manfred R., Klaus K. 2000. “Clinical

Workflows — The Killer Application for Process-

Oriented Information Systems?” In BIS 2000: 4th

International Conference on Business Information

Systems, Poznań, Poland, 12–13 April 2000, edited by

Witold Abramowicz and Maria E. Orlowska, 36–59.

London: Springer London. http://dx.doi.org/

10.1007/978-1-4471-0761-3_3.

Jamoom, E. W., Heisey-Grove, D., Yang, N., Scanlon, P.,

2016. “Physician Opinions about EHR Use by EHR

Experience and by Whether the Practice had optimized

its EHR Use” Journal of Health & Medical Informatics

7 (4) OMICS International., ISSN: 2157-7420.

Latoszek-Berendsen, A., Tange, H., van den Herik, H.,

Hasman. A. 2010. “From Clinical Practice Guidelines

to Computer-Interpretable Guidelines. A Literature

Overview.” Methods of Information in Medicine 49 (6):

550–70. doi:10.3414/ME10-01-0056.

Mans, R., van der Aalst, W., Vanwersch, R., 2015. Process

Mining in Healthcare: Evaluating and Exploiting

Operational Healthcare Processes; Springer.

Ologeanu-Taddei, R., Morquin, D., Domingo, H., &

Bourret, R., 2015. “Understanding the acceptance

factors of an Hospital Information System: evidence

from a French University Hospital.” AMIA Annual

Symposium Proceedings, 2015, 1001–1007.

Oosterhout, E., Talmon, J., Clercq, P., Schouten, H., Tange,

H., Hasman, A., 2005. “Three-Layer Model for the

Design of a Protocol Support System.” MIE 2003 74

(2–4): 101–10. doi:10.1016/j.ijmedinf.2004.04.023.

Staccini, P., Joubert, M., Quaranta, J., Fieschi, D., Fieschi.

M., 2001. “Modelling Health Care Processes for

Eliciting User Requirements: A Way to Link a Quality

Paradigm and Clinical Information System Design.”

International Journal of Medical Informatics 64 (2–3):

129–42. doi:10.1016/S1386-5056(01)00203-9.

van der Aalst, W., 2016. “Process Mining: Data Science in

Action”, Springer.