Challenges and Approaches to Enhance Usability in Healthcare

Applications: A Systematic Literature Review

Emanuel P. Vicente

1 a

, Wilamis K. N. da Silva

1 b

, Geraldo T. G. Neto

2 c

and Gustavo H. S. Alexandre

1 d

CESAR, Recife Center for Advanced Studies and Systems, Apolo Street, Recife, Brazil

IFMS, Federal Institute of Mato Grosso do Sul, Corumb

a, Brazil

{epv, wkns, ghsa}@cesar.school, geraldo.galindo@ifms.edu.br

Keywords:

Usability, Usability Method, UCD, Usage-Centered, Medical-Hospital System, Healthcare, Medicine, Patient

Safety.

Abstract:

The expansion of medical-hospital applications requires increasingly more quality in their interfaces so that

usability problems do not encourage errors and adverse events that could impact patient safety. The ecosystem

of healthcare applications has challenges in the ﬁeld of usability due to the dynamic medical environment in

which these systems are designed, developed and operated, suffering interference from social and technolog-

ical factors, and compliance with legislation. This research identiﬁes the challenges faced by the healthcare

applications ecosystem in order to map them. This work reports, through a literature review, which challenges

software engineering has faced to guarantee the usability of applications in the healthcare context. A system-

atic review of the literature was carried out using a research protocol with the purpose of identifying relevant

studies in the IEEE Xplore, ACM, Scopus, Science Direct and PUBMED databases, using criteria that adhere

to the practices of search, critical evaluation, data extraction and synthesis of works, resulting in the obtaining

of 43 relevant works. The literature search yielded 43 articles that met the study criteria. It resulted in map-

ping the main challenges found in the literature when applying existing approaches to increase the usability

of healthcare applications and reduce the impact of violations that could result in adverse health events for

patients. The study identiﬁed the challenges faced by the usability of health systems, as well as mapping

which factors contribute to these challenges. Furthermore, this work aims to impact the use of approaches that

guarantee better usability of the applications of this ecosystem and consequently reduce the risk of adverse

events to the health of patients.

1 INTRODUCTION

In recent years, several software products aimed at the

healthcare application ecosystem have emerged in the

private and government markets. This explosion of

solutions has directed the attention of software engi-

neering to issues of usability of interfaces that relate

to patient health care. According to Cho (Cho et al.,

2022), healthcare professionals are often faced with

unintended consequences related to the use of these

systems, which may negatively impact patient safety.

One of the main reasons for these unforeseen chal-

lenges associated with an increase in clinicians’ cog-

https://orcid.org/0009-0008-9859-8819

https://orcid.org/0000-0002-8530-5916

https://orcid.org/0000-0001-7244-8822

https://orcid.org/0000-0002-8347-6383

nitive workload, related fatigue using this systems,

burnout, work inefﬁciency, job dissatisfaction, and in-

tentions to leave the job arise from lack of or poor

execution usability testing of these systems.

During user interaction with Health Information

System (HIS), many usability problems can be iden-

tiﬁed, such as: lack of readability, feature confusion,

poorly differentiated alarms or alerts, lack of design

intelligent, poor feedback on the state or behavior of

the system, lack of provision for online help, poor

language support, among others. Also according to

Adams (Adams et al., 2021), these usability problems

can eventually contribute to medication errors.

For systems with usability problems, the impact

of these violations signiﬁcantly increases the chances

of errors or adverse events that can lead to a disas-

ter for the patient’s health, as in the case of wrong

medication prescription or inappropriate dosages. On

552

Vicente, E. P., Silva, W. K. N., Neto, G. T. G. and Alexandre, G. H. S.

Challenges and Approaches to Enhance Usability in Healthcare Applications: A Systematic Literature Review.

DOI: 10.5220/0013295300003929

Paper published under CC license (CC BY-NC-ND 4.0)

In Proceedings of the 27th International Conference on Enterprise Information Systems (ICEIS 2025) - Volume 2, pages 552-564

ISBN: 978-989-758-749-8; ISSN: 2184-4992

the other hand, systems with high usability help users

carry out their tasks safely, quickly, easily and with

minimal mental effort.

Adams (Adams et al., 2021) highlights that sys-

tems with strong usability have the potential to signif-

icantly reduce errors and enhance patient safety.

A study carried out in 3 US hospitals on the

safety of medication administration in pediatric pa-

tients showed that 36% of adverse events were re-

lated, among other problems, to the lack of feedback

from the system and poor visual display of informa-

tion in the Recording System. of Health(Ratwani

et al., 2018b).

Usability can be considered as one of the main

quality characteristics of a system, and is also deﬁned

by ease of use and the extent to which a product is

used efﬁciently, effectively and satisfactorily by spe-

ciﬁc users in the execution of objectives (ISO 9241-

11, 1998) .

The adoption of practices to guarantee a software

development process must be aligned with a human-

ized and user-centered perspective, and thus, direct

attention to end users during all phases of a soft-

ware project. This view is described by Lowder-

milk(Lowdermilk, 2019) as User Centered Design

(UCD), where usability processes are focused on the

end user, delivering a better quality application from

the perspective of those who use the system.

This literature review analyzed studies that ad-

dressed usability methods in the context of health-

care applications to identify the main challenges faced

when using approaches with the aim of understanding

the problem and supporting new work in proposing

more efﬁcient processes of integration between devel-

opment goodwill and usability of health systems.

2 SYSTEMATIC LITERATURE

REVIEW

The literature review is composed of three phases:

planning the review, conducing the review and doc-

umenting the review as Figure 1.

The Systematic Literature Review (SLR) requires

an extensive process. Kitchenham et al. (Kitchen-

ham et al., 2015) argued that the process of reviewing

the literature should be highly planned and structured,

thus avoiding bias on the side of researchers and pro-

viding reliability to the process.

2.1 Planning the Review

Kitchenham et al. (Kitchenham et al., 2015) refer to

SRL as a project and to the importance of planning as

Figure 1: Planning process of the Systematic Literature Re-

view (adapted from Kitchenham et. al. (Kitchenham et al.,

2015)).

a key step to success at the end of the study.

2.1.1 Establishing the Need for a Review

The authors (Kitchenham et al., 2015) emphasize the

importance of checking if there is already another

SRL approaching the same research topic. In his Lit-

erature Review, Carayon (Carayon and Hoonakker,

2019) identiﬁed old and new challenges of Human

Factors and Usability approches for Health Informa-

tion Technology. The author concludes that progress

in the application of human factors methods and prin-

ciples to the design of health IT is occurring, with im-

portant information provided on their actual impact

on care processes and patient outcomes, and points

to necessity of exam how work of health IT design-

ers and implementers would help to develop strategies

for further embedding human factors engineering in

IT design processes. Also, during the initial searches

on the topic of this study, we found the work (Mar-

tikainen, 2015), with a theme and research questions

similar to those of our study. The author of (Mar-

tikainen, 2015) examine what factors impact the us-

ability of Healthcare IT Systems (HITS) development

in Finland to understand the reasons for poor usabil-

ity.

This study brings importance to extend aim

the problem to a challenges in depploing usability

aprouches to enssure good results in satisfation of

healthcare ecosystems aplications and mitigate ad-

verse events on criticais interfaces.

2.1.2 Research Questions

Based on the objective of this review, the following

question helps to identify primary studies that high-

light the challenges faced in adopting usability ap-

proaches in the healthcare ecosystem highlighted in

the literature:

Challenges and Approaches to Enhance Usability in Healthcare Applications: A Systematic Literature Review

553

• RQ1: What main challenges are faced when

applying approaches that aim to improve the us-

ability of healthcare ecosystems applications?

Based on the central question, other secondary

question was raised to help understand the problem,

as follows:

• RQ2. What approaches are faced to mitigate

these challenges in the context of the healthcare

ecosystem ?

• RQ3. What are the UX artifacts most used to

address usability problems and mitigate adverse

patient safety events?

The aim of her PhD research was accordingly to

study what factors impact the usability of HITS as

well as the user participation in HITS development in

Finland and focus on User-centred design methods in

order to achieve good usability. The main viewpoint

of that study are ”user” and how improve its involve-

ment to solve usability issues, but this research aims

to extend the problem to other perspectives, beyond

the user’s point of view, that’s why we need to inves-

tigate challenges and deep to ensure good usability in

ecosystems health.

Between summer 2013 to autumn 2015, Mar-

tikainen(Martikainen, 2015), carried a literature re-

views while in this study we made searches by a sys-

tematic literature review between the year 2013 and

July 2023, to identify usability challenges on health-

care ecosystens, as well, its approaches and contribu-

tions factors, and possible solutions for challenges not

addressed.

2.1.3 Search Strategy

At this planning stage, a search protocol is deﬁned to

collect related primary studies that are relevant to this

subject.

The main electronic databases were searched,

such as: IEEE Xplore, Scopus, ACM, Science Di-

rect and PUBMED, and were used to obtain relevant

articles on the researched topic. The most relevant

keywords for the topic were used together with the

Boolean operator ”AND” to obtain articles from dif-

ferent areas and the operator ”OR” to expand the cov-

erage of these areas as deﬁned on Figure 2.

2.1.4 Study Selection Strategy

Once the works in the databases are identiﬁed, this

step ﬁlters the possibly relevant works that can answer

some of the research questions proposed in this study

(Kitchenham et al., 2015).

Figure 2: Search string construction (based on Silva et. al.

(da Silva et al., 2014)).

Table 1: Selection process.

Scope String

StudyContextUCD, Usage-centered, usability,

Ergonomics ,Human Factors,

user,Usability method

Fields

study

Healthcare, health, medical, medicine,

disease Health, clinical, care, patient

safety

Study

Type

Conference, Jornals

Aspect

ofstudy

Challenges, difﬁculties, barriers,

complications, obstacles,issues,

failures

Source IEEE Xplore, Scopus, ACM,

PUBMED, Science Direct

Language English

The Inclusion Criteria Are:

1. Papers published between the years 2023 and

2024

2. Papers that are related to Usability in healthcare

ecosystems;

3. Papers that present, in the title or abstract, terms

related to Usability in healthcare ecosystems;

4. Papers that identify approaches that may posi-

tively or negatively inﬂuence Usability in health-

care ecosystems;

5. Papers that identify methods or practices that

may positively or negatively inﬂuence Usability

in healthcare ecosystems.

ICEIS 2025 - 27th International Conference on Enterprise Information Systems

554

The Exclusion Criteria Are:

1. Papers that not are related to Usability in health-

care ecosystems;

2. Papers that not present, in the title or abstract,

terms related to Usability in healthcare ecosys-

tems;

3. Papers that not identify approaches that may pos-

itively or negatively inﬂuence Usability in health-

care ecosystems;

4. Papers that not identify methods or practices that

may positively or negatively inﬂuence Usability

in healthcare ecosystems;

5. Incomplete papers;

6. Fulltext is not available;

7. Duplicate papers;

8. studies not published in English;

9. Studies that were not available online;

10. Studies that are not based on research and that ex-

press only ofﬁcial opinions from governments and

experts from the area;

11. Notice of works, prefaces, conference proceed-

ings, handouts, summaries, panels, interviews and

reports.

12. Papers which do not meet the quality criteria.

2.1.5 Method to Check the Quality of the Studies

Assessing the quality of the primary studies identi-

ﬁed in the previous step is essential to ensure that low

quality studies do not reach the data synthesis stage

(Kitchenham et al., 2015).

There are several methods to evaluate the qual-

ity of primary studies and checklists are commonly

used for this purpose. There are several adaptations

of these lists to be used in different ﬁelds of research

(Kitchenham et al., 2015). Using the Critical Ap-

praisal Skills Program (CASP) checklist, (Dyb

a and

Dingsøyr, 2008) developed a quality list. This list has

eleven questions that are presented below:

1. Is the paper based on research (or is it merely

a “lessons learned” report based on expert opin-

ion)?

2. Is there a clear statement of the aims of the re-

search?

3. Is there an adequate description of the context in

which the research was carried out?

4. Was the research design appropriate to address the

aims of the research?

5. Was the recruitment strategy appropriate to the

aims of the research?

6. Was there a control group with which to compare

treatments?

7. Was the data collected in a way that addressed the

research issue?

8. Was the data analysis sufﬁciently rigorous?

9. Has the relationship between researcher and par-

ticipants been considered to an adequate degree?

10. Is there a clear statement of ﬁndings?

11. Is the paper of value for research or practice?

To answer the questions in the list we used anno-

tation of 1 for yes and 0 for no. During the analysis,

when the ﬁrst question obtained a value of 0, the anal-

ysis of the work was interrupted, and the work did not

proceed to the synthesis phase. According to (Dyb

and Dingsøyr, 2008), another example that can ex-

clude a job from the synthesis is if questions two and

three get the value 0 simultaneously.

2.1.6 Strategy for Extracting Data

We ﬁrst classiﬁed the types of research following the

classiﬁcation proposed by Wieringa et. al (Wieringa

et al., 2005). Then, we classiﬁed the type of study

according to Tonella et al. (Wieringa et al., 2005).

Lastly, we extracted the following data: (1) Usability

challenges in healthcare ecosystems; and (2) Year of

publication.

2.1.7 Strategy for Synthesizing Data

The challenges found in the selected studies were cat-

egorized in a deductive way. This study was based

on the systematic review carried out by Vandekerck-

hove et al (Vandekerckhove et al., 2020) which eval-

uated electronic health interventions. Research by

Vandekerckhove et al (Vandekerckhove et al., 2020)

identiﬁed participatory design methods in empiri-

cal eHealth studies for further development of the

methodology.

We analyze the selected works following your rep-

resentation. However, the categories were comple-

mented by inductive categories that emerged from

the review of the material. The categories can be

termed as “factual categories” according to Kuckartz

(Kuckartz and R

adiker, 2019), designating speciﬁc

facts in the included studies.

Challenges and Approaches to Enhance Usability in Healthcare Applications: A Systematic Literature Review

555

3 CONDUCTING AND

DOCUMENTING

3.1 Select Studies

Our study selection process comprised three activi-

ties. In the ﬁrst activity, the search string deﬁned in

Figure 2, returned 1370 results.

Figure 3: The ﬁltering process of the papers (adapted from

(da Silva et al., 2014) (Alves et al., 2016)).

After that, evaluating the titles and abstracts of

the papers found a total of 67 studies corresponding

to the inclusion criteria and thus moved to the third

phase. In this phase, 43 studies were selected as an-

swering at least one research question. The detailed

list of selected studies can be found in Multimedia

Appendix A. Table 2 shows the results obtained by

each database.

Table 2: Summary of search results.

Database/Search engine Search Results

Pubmed 833

IEEEXplore 211

ACM Digital 130

Science Direct 93

Scopus 63

Springer 40

Total 1370

3.2 Quality Assessment

Figure 4 shows the total score obtained for each ques-

tion in the quality list.

The criteria of objectives, context of the research,

conclusions and value of the research obtained a max-

imum grade in the evaluation.

However, forty three studies justiﬁed the selec-

tion of the research public cause having obtained

answer 1 on question one and seven studies were

eliminated after having obtained answer 0 in ques-

tion one (Middleton et al., 2013),(Hettinger et al.,

2021),(Alon and Torous, 2023),(Carayon and Sal-

wei, 2021),(Catchpole et al., 2021),(Schumacher and

Jerch, 2012),(Lowry et al., 2012). Full metadata

Quality Assessment from selected studies can be

found in Multimedia Appendix B.

Figure 4: Total obtained by quality questions (based on

(Dyb

a and Dingsøyr, 2008)).

3.3 Extract Data

We started our data extraction by classifying the se-

lected primary studies, according to Wieringa et. al.

(Wieringa et al., 2005). In our analysis, there was a

predominance of studies that propose a solution tech-

nique with eleven studies. According to (Wieringa

et al., 2005), these studies should propose a technical

solution and justify its relevance through validation

or a good set of references. Then came the studies

of validation research with ﬁve studies, evaluation re-

search with four, philosophical and experience papers

each one with two studies. Finally, one paper to expe-

rience papers, as show in Table 3.

Table 3: Types of research (based on Wieringa et. al.

(Wieringa et al., 2005)).

Method Result

Case Study 13/43 (30.23%)

Observational Study 13/43 (30.23%)

Action Research 12/43 (27.91%)

Cross-Sectional 2/43 (4.65%)

Experience report 1/43 (2.33%)

Comparative 1/43 (2.33%)

Not Speciﬁed 1/43 (2.33%)

Following our extraction protocol, we consolidate

the demographic information of the studies according

to Tonella et. al. (Tonella et al., 2007). We grouped

Case Study and Observational Study totaling 30,23%

each one. Action Research totaling 27,91% of stud-

ies. Other methods together represent 11,64% of stud-

ies, according to the (Tonella et al., 2007) approach.

In our study we analyzed studies from 2014 to

december of 2024, with the distribution of studies

throughout years presented in Table 4.

ICEIS 2025 - 27th International Conference on Enterprise Information Systems

556

Table 4: Distribution of studies by year.

Year Number Percent

2024 11 26%

2023 7 16%

2022 6 14%

2021 6 14%

2020 4 9%

2019 2 5%

2018 3 7%

2017 2 5%

2015 2 5%

Total 43 100%

Figure 5: Distribution of studies per country.

We observed in the year 2021 had the highest in-

cidence of studies with 22% related to this research

topic. Also, there was at least one qualifying study

each year.

3.4 Synthesis Protocol

To answer RQ1, we conducted a thematic analysis.

We followed the process described by Braun and

Clarke (Braun and Clarke, 2006) as detailed below:

1. Familiarising Yourself with Your Data. Braun

and Clarke (Braun and Clarke, 2006) introduce

two important concepts: data corpus and data set.

Data corpus refers to all data used for an analy-

sis; And the data set may correspond to a particu-

lar topic of interest within the data corpus; in our

case, the studies to include Usability challenges.

2. Generating Initial Codes. After an initial read-

ing of the primary studies, the ﬁrst codes emerged.

We analysed the studies in two rounds. First, all

the studies were analysed, generating a list of 231

codes. Second, another reading was performed.

Finally, 122 codes were generated in this step

3. Searching for Themes. At this stage, the codes

generated in the previous step were reﬁned and

duplicate items and items from the same context

were removed, resulting in a list of 122 chal-

lenges. These challenges were analyzed again and

grouped into 21 themes.

All selected code faced challenges in applica-

tion approaches that aim to improve usability of

healthcare ecosystem applications. The present

study identiﬁed 122 challenges and for a better

presentation of them, we grouped them into 21

main challenges themes.

4. Reviewing Themes. According to Braun and

Clarke(Braun and Clarke, 2006), the themes iden-

tiﬁed in the previous step should be reﬁned in two

levels. First, perform an analysis of all the codes

within each theme, checking if the codes should

belong to that theme. In addition, one must anal-

yse whether the codes generate a coherent pattern

for the construction of a theme.

In the second level of analysis, Braun and Clarke

(Braun and Clarke, 2006) deﬁne that “at this level,

you consider the validity of individual themes in

relation to the data set, but also whether your

candidate thematic map “accurately” reﬂects the

meanings evident in the data set as a whole”.

From our initial list of themes deﬁned in the stage

three, we observed a strong relationship between

two themes, “lack of academic studies on this

topic”, and “lack of secondary study on the topic”.

In this way, we grouped the two themes. The

theme “Lack of academic research on the topic”

has emerged.

5. Producing the Report. The identiﬁed themes are

presented in the section 4.2.

4 RESULTS

In this section, we ﬁrst provide an overview of the

studies included in this work. We ﬁnish the section

presenting the results of this SLR, according to our

three research questions.

4.1 Overview

The initial search was done using a general search

string as show on Figure 2 in every relevant base and

resulted in a total of 1370 hits (PubMed:n=833, 61%;

IEEEXplore:n=211, 15%; ACM Digital:n=130, 9%;

Science Direct:n=93, 6.70%); Scopus:n=63, 5%);

Springer :n=40, 3%). From the 43 primary studies an-

alyzed, 26% 11/43) of them were published in 2024,

Challenges and Approaches to Enhance Usability in Healthcare Applications: A Systematic Literature Review

557

16% (7/43) in 2023, 14% (6/43) in the year 2023, as

show in Table 4.

Regarding the type of venue, 91% (39/43) were

published in journals, and 9% (4/43) were published

in conferences.

4.2 RQ1: What Main Challenges Are

Faced when Applying Approaches

that Aim to Improve the Usability of

Healthcare Ecosystems

Applications?

After applying the thematic analysis process pre-

sented in synthesis protocol 3.4 session, twenty-one

themes representing challenges in healthcare ecosys-

tems applications studies have emerged.

To answer RQ1, our study identiﬁed three large

thematic areas with the greatest number of challenges:

Usability Process and Integration, technical Issues

and Knowledge and Conceptual Gaps , concentrat-

ing 37.70% of the challenges that need to be ad-

dressed, as as shown in table 5.

Also, 13 challenges were identiﬁed related to the

lack of Knowledge and Conceptual Gaps about us-

ability evaluation in healthcare projects, also related

to the understanding of nomenclatures and processes

in the healthcare application ecosystem

The Usability Process and Integration, which

addresses problems in applying usability methods and

approaches, point to 19 challenges that need to be ad-

dressed. Another major thematic area with challenges

is related to technical issues, which highlights chal-

lenges in product artefact that point to usability errors,

which highlights 14 challenges.

All other thematic areas and their respective chal-

lenges can be found in Appendix C of this work.

4.3 RQ2:What Approaches Are Faced

to Mitigate These Challenges in the

Context of the Healthcare

Ecosystem?

In our research, many works addressed the solution

of usability challenges such as those of Gettinger and

Zayas (Gettinger and Zayas-Cab

an, 2021) , which

proposes including UCD processes to address usabil-

ity challenges ”fund research to examine the effective-

ness of usability policies including UCD processes

employed by health IT developers”.

Also, Ratwani et al, propose to address usability

challenges for a user-centric approach(Ratwani et al.,

2018a) ”Use of user-centered design approach”.

Table 5: Number of challenges by themes.

Themes Challenges

USABILITY PROCESS AND

INTEGRATION

TECHNICAL ISSUES 14

KNOWLEDGE AND CON-

CEPTUAL GAPS

WORKFLOW SIMPLIFICA-

TION

TRAINING AND AWARE-

NESS

RESEARCH LACK 5

RESOURCE LIMITATIONS 5

SYSTEM AND IMPLEMEN-

TATION CHALLENGES

ACCESSIBILITY 5

NEW APPROACH ADOP-

TIONS

LACK OF USABILITY AP-

PROACHES

DOCUMENTATION 5

REGULATORY COMPLI-

ANCE AND CERTIFICA-

TION

RISK MANAGEMENT 4

USABILITY MEASURE-

MENT

COMMUNICATION AND

COLLABORATION

ENVIRONMENTAL LIMITA-

TIONS

ETHICAL AND DATA PRI-

VACY

LEADERSHIP AND ORGA-

NIZATIONAL SUPPORT

LACK OF TRANSPARENCY 1

CONSISTENCY 1

Total 122

In table 6 we list the most used approaches in the

papers selected by this literature review. The User-

Centered Design (UCD) approach was the most used

among the studies, showing that 60% of the studies

adopt the culture of user-centered design as the main

method to improve the usability and UX of healthcare

applications.

ICEIS 2025 - 27th International Conference on Enterprise Information Systems

558

Table 6: approaches must used to address UX challenges in

healthcare context.

Approuches Ocorrences

User-Centered Design (UCD) 26/43 (60%)

Usability Evaluation 13/43 (30%)

Iterative Design Processes 11/43 (26%)

Think-Aloud Protocol 10/43 (23%)

Human-Centered Design

(HCD)

9/43 (21%)

Participatory Design 9/43 (21%)

Heuristic Evaluation 9/43 (21%)

Human Factors Engineering

(HFE)

3/43 (7%)

Cognitive Walkthrough 2/43 (5%)

Feedback Mechanisms 2/43 (5%)

Risk-Based Usability Testing 1/43 (2%)

Scenario-based Evaluations 1/43 (2%)

SEIPS Model 1/43 (2%)

Workﬂow-Based Design 1/43 (2%)

DEMIGNED Principles 1/43 (2%)

Cognitive Systems Engineering 1/43 (2%)

Remote Testing and User Con-

ferences

1/43 (2%)

Formative and Summative Test-

ing

1/43 (2%)

Sociotechnical Framework 1/43 (2%)

In this paper all themes that categorize usability

challenges in healthcare systems expands the surface

area of the technical-social issues of usability and UX

within the healthcare applications ecosystem, not just

technical usability issues. It is also worth highlighting

that the scope of issues that impact usability is wide

and can be affected by processes, interpersonal rela-

tionships, compliance and privacy, risk management,

accessibility and so one, as categorized in table 6.

In our study, we assessed in which phases of the

software development life cycle (SDLC) healthcare

product usability issues were addressed ﬁrst.

4.4 RQ3:What Are the UX Artifacts

Most Used to Address Usability

Problems and Mitigate Adverse

Patient Safety Events?

We identiﬁed that 53% of the papers used the ”Think-

Aloud Protocols” tool to evaluate their healthcare

Table 7: UX artifact occurrences.

UX artifact Ocorrences

Think-Aloud Protocols 23/43 (53%)

System Usability Scale (SUS) 20/43 (47%)

Feedback Summaries 20/43 (47%)

Heuristic Evaluation Reports 19/43 (44%)

Prototypes 17/43 (40%)

Task Analysis 16/43 (37%)

Usability Test Reports 13/43 (30%)

Wireframes 3/43 (7%)

Personas 3/43 (7%)

Interviews 2/43 (5%)

User Flows 2/43 (5%)

Satisfaction Questionnaires 1/43 (2%)

Computer System Usability

Questionnaire (CSUQ)

1/43 (2%)

Design Workshops 1/43 (2%)

Mockups 1/43 (2%)

Use Cases 1/43 (2%)

Logic Models 1/43 (2%)

Focus Groups 1/43 (2%)

Scenarios 1/43 (2%)

Research Reports 1/43 (2%)

User Experience Questionnaire

(UEQ)

1/43 (2%)

Card Sorting 1/43 (2%)

Delphi Questionnaires 1/43 (2%)

software products em improve usability to mitigate

patient hazards as demonstrate on table 7.

Also, ”System Usability Scale (SUS)” and ”Feed-

back Summaries” have a signiﬁcant participation in

the improvement of healthcare software products with

47% each one.

All other tools and their occurrences on selected

academic works can be consulted in table 7.

5 DISCUSSION

This systematic review highlighted Challenges and

Approaches to Enhance Usability in Healthcare Ap-

plications.

Ratwani (Ratwani et al., 2018b) explores

usability-related challenges in Electronic Health

Records (EHRs) within pediatric settings and their

Challenges and Approaches to Enhance Usability in Healthcare Applications: A Systematic Literature Review

559

impact on patient safety. By analyzing 9,000 patient

safety event reports from three healthcare institutions,

the study identiﬁes key usability issues contributing

to medication errors and adverse events in pediatric

care.

This study highlights the importance of usability

to mitigate adverse events and increase patient safety

against harm to their health.

They identiﬁed usability challenges in EHR sys-

tems in pediatric healthcare and determined how these

challenges contribute to medication errors and ad-

verse events. Also propose strategies to address us-

ability challenges to improve patient safety.

They grouped the challenges into 4 themes: sys-

tems feedback, visual display, data entry and work-

ﬂow support.

Ratwani (Ratwani et al., 2018b) main ﬁndings

point out that system feedback issues were responsi-

ble for 82.4% of reported usability challenges. Other

challenges included visual display issues (9.7%), data

entry difﬁculties (6.2%), and workﬂow integration is-

sues (1.7%). Furthermore, inadequate dosage errors

represented 84.5% of medication errors. 18.8% of

events related to usability reached the patient; 3.3%

caused temporary damage or required monitoring.

Below we will discuss the main ﬁndings. To

do this, we will consider the 4 most relevant

groups/themes of challenges.

5.1 Challenges in Usability Process and

Integration

We identiﬁed 19 challenges related to the difﬁculty

in executing processes and integration between differ-

ent approaches that guarantee good usability in health

systems.

Ratwani (Ratwani et al., 2015), investigates the

usability challenges of electronic health records

(EHRs) and evaluates the user-centered design (UCD)

processes employed by vendors. It categorizes ven-

dors based on their UCD practices and identiﬁes bar-

riers to implementing effective UCD methodologies.

Ratwani (Ratwani et al., 2015), mention out the

challenges in conducting detailed clinical workﬂow

studies, highlighting the difﬁculty for healthcare so-

lution providers in understanding the ﬂow of live-

stock activities in the healthcare ecosystem, when

he says: ”Vendors in this category overwhelmingly

described the difﬁculties to conduct detailed, contex-

tually rich research. workﬂow studies across differ-

ent subspecialty clinical disciplines”. He also points

out challenges in recruiting participants for usability

studies, ”speciﬁc resource challenges include difﬁcul-

ties recruiting participants (”it is a real challenge to

get participants”)”.

It also makes reference to the challenge of includ-

ing expert personnel in the usability process, which in

many healthcare application providers is neglected as

says: ”Vendors in this group typically employ a few

usability experts, but the usability personnel face spe-

ciﬁc challenges that will need to be overcome in order

for a more rigorous UCD process to be employed”

and ”These vendors generally do not have usability

experts on staff, and the leadership of these vendors

often lacks an appreciation for UCD”.

Lloyd, (Lloyd et al., 2023), examines the usabil-

ity of electronic medical records (EMRs) in Aus-

tralian hospitals. The article explores physicians’ and

nurses’ perspectives on EMR implementation, high-

lighting usability challenges and their impact on clin-

ical workﬂows, communication, and patient care.

Paper show challenges to integration providers

and clinical workﬂows needs, ”The current systems

require the clinician to change his [or] her work-

ﬂow/practice/ processes to accommodate the system,

rather than the system accommodating the user”

Goerss, (Goerss et al., 2024) examines the feasi-

bility and value of involving people with dementia

in the user-centered design (UCD) of a smartwatch-

based intervention. The study focuses on usability,

user preferences, and the effectiveness of involving

people with mild cognitive impairment or demen-

tia in the design process. Through iterative feed-

back, the research identiﬁes design improvements

and highlights challenges in usability for this popu-

lation. In that paper, he ﬁnds challenges in under-

standing tasks when participants occasionally had dif-

ﬁculty understanding instructions, requiring clearer

guidance and design reﬁnements. “We found that in-

complete or irregular fulﬁllment of tasks was not rec-

ognized by the concerned participants” and “Some

participants struggled with the Likert scale, which

might be avoided by guiding them or modifying the

questionnaire” .

Harte, (Harte et al., 2017) presents a structured

Human-Centered Design (HCD) approach for devel-

oping a smartphone app in a connected health sys-

tem targeting older adults. The app integrates with

wireless insoles to assess fall risks and detect falls.

Through iterative phases of usability evaluation and

redesign, the study emphasizes optimizing the inter-

face for older adults, addressing usability challenges,

and improving user satisfaction and system perfor-

mance.

Harte points to a challenge of user-centered de-

sign processes. ”HCD is a multi-stage process that

allows for various iterations of a design and subse-

quent update to the requirements. The importance

ICEIS 2025 - 27th International Conference on Enterprise Information Systems

560

of involving end users in the design process of health

products is recognized, and different approaches have

been demonstrated in literature”.

5.2 Challenges in Technical Issues

We identiﬁed 14 challenges involves to moderating

sessions, observing user behavior, collecting feed-

back, and handling technical issues.

Kato (Kato et al., 2024) presents the implemen-

tation and evaluation of a 5-type Health Information

Technology (HIT) safety classiﬁcation framework by

the Veterans Health Administration (VHA). The study

focuses on categorizing HIT patient safety concerns

and improving risk mitigation strategies. By ana-

lyzing 140 patient safety issues from the VHA’s In-

formatics Patient Safety (IPS) database, the classiﬁ-

cation system demonstrates its utility in standardiz-

ing safety issue identiﬁcation, enabling actionable re-

sponses, and supporting organizational learning in a

high-reliability healthcare organization (HRO). This

paper cite Challenges with hybrid medical record

systems - ”Issues included notiﬁcation problems and

functional defects such as duplicate orders”.

Goerss (Goerss et al., 2024) examines the feasi-

bility and value of involving people with dementia

in the user-centered design (UCD) of a smartwatch-

based intervention. The study focuses on usability,

user preferences, and the effectiveness of involving

people with mild cognitive impairment or dementia

in the design process. Through iterative feedback,

the research identiﬁes design improvements and high-

lights challenges in usability for this population. This

paper cite ”Issues with notiﬁcations and account

management” - “Participants showed a wish for less

obtrusive notiﬁcations and dislike toward a watch

that was very conspicuous but also stated a wish for

speech output”.

Meidani (Meidani et al., 2024) evaluates the us-

ability and quality of the ”Head Computed Tomog-

raphy Scan Appropriateness Criteria (HAC)” mobile

application, designed to assist physicians in ordering

head CT scans. The evaluation uses a mixed-methods

approach, incorporating usability testing, the Mobile

Apps Rating Scale (MARS), and debrieﬁng sessions

to identify strengths and usability issues. This papar

points to challenges to Touch interaction - ”Some

participants expected more predictive capabilities...”.

McMullan (McMullan et al., 2023) examines the

development and usability testing of the PROmicsR

ePRO system designed for patients with inﬂamma-

tory diseases, focusing on its application in the PO-

LARISE clinical trial. The research highlights techni-

cal and usability challenges encountered during test-

ing, evaluates the system’s feasibility for clinical de-

ployment, and incorporates feedback to reﬁne the sys-

tem for subsequent trials. That paper mention Hard-

ware and technical limitations - ”Two participants

failed at the outset due to an outdated operating sys-

tem and compatibility issues”.

5.3 Challenges in Knowledge and

Conceptual Gaps

Some ﬁndings, 13 in total, are related to knowledge

challenges and conceptual gaps in how to apply us-

ability or UX methods and practices, these challenges

involve training deﬁciencies, lack of learning capac-

ity, guideline limitations and so on.

Lloyd, (Lloyd et al., 2023), mention Issues with

design and learnability - ”Non-intuitive. So many

steps to complete one task”.

Ghorayeb, (Ghorayeb et al., 2023) focuses on de-

veloping and validating a new usability scale specif-

ically tailored for clinical decision support systems

(CDSS). The HSUS aims to address usability chal-

lenges unique to clinical contexts, enabling the early

identiﬁcation of potential issues that may impact pa-

tient safety and workﬂow integration.

Usability metrics and guidelines limitations - ”The

SUS and existent scales are not reﬂective of the us-

ability in the real world as it does not consider the

context of use”.

For McMullan (McMullan et al., 2023), there is a

Limited reliance on training materials during us-

ability testing - ”Participants preferred consulting

researchers rather than using the provided training

manual”.

Pruitt, (Pruitt et al., 2023) investigates usability

challenges in the design of an electronic medica-

tion administration record (eMAR) system. Using

scenario-based evaluations and qualitative thematic

analysis, the researchers identiﬁed critical usability

issues impacting nurse workﬂows and patient safety.

The study underscores the need for iterative usability

evaluations and process improvements to align eMAR

functionalities with real-world nursing tasks.

For Pruitt, (Pruitt et al., 2023) there is a Inability

to document home medications - ”Nurses lacked a

mechanism to document medications not pre-listed in

the eMAR”.

For Harte, (Harte et al., 2017), there is a Older

adults struggle with modern touchscreen inter-

faces due to unfamiliarity and cognitive challenges

- ”The acquisition and comprehension of information

from interfaces become more difﬁcult as a person pro-

gresses into older age”.

Challenges and Approaches to Enhance Usability in Healthcare Applications: A Systematic Literature Review

561

5.4 Challenges in Workﬂow

Simpliﬁcation

Workﬂow simpliﬁcation speciﬁes challenges related

to interface navigation, presentation issues, and cog-

nitive overload of healthcare professionals while op-

erating the systems. This work found 8 challenges

related to workﬂow simpliﬁcation.

Heitkemper (Heitkemper et al., 2024) evaluates

the usability of SHAREdash, a Tableau-based health

equity dashboard developed for rural public health

practitioners in the Northwestern United States. The

usability evaluation, conducted in the third stage of

the System Development Lifecycle (SDLC), assessed

efﬁciency, satisfaction, and validity through think-

aloud tasks and interviews with public health profes-

sionals.

Heitkemper Lack of basic usability functions -

“Users described navigating ﬁlters as cumbersome,

particularly when searching for speciﬁc counties”

Engelsma (Engelsma et al., 2024) examines the

usability of digital health technologies (DHTs) de-

signed for individuals living with dementia. It eval-

uates the DEMIGNED principles, which are tailored

design guidelines for dementia-speciﬁc DHTs, using

heuristic evaluation and think-aloud user testing on

a mobile website for the Alzheimer Center Amster-

dam. The study reveals gaps in usability evaluations

and proposes reﬁnements to the DEMIGNED princi-

ples based on user feedback.

In his exam, a challenge of cognitive overload

was identiﬁed - “A former excessive amount of infor-

mation on a single page and in the menu (e.g. the

number of subpages to choose from) not only can it

burden users who live with cognitive problems prob-

lems, making it a challenge to process and re- con-

tent member, but also impair decision-making skills,

hindering the ability to effectively ﬁlter the relevant

information”.

Kene (Kenei and Opiyo, 2022), explores the use of

a classiﬁcation and visualization model to support the

review of clinical narrative texts in electronic health

records (EHRs). By addressing the challenges of un-

structured data in EHRs, the study presents a two-

step approach: classiﬁcation of clinical narrative texts

into structured sections and visualization of these sec-

tions as a cluster map. The solution aims to im-

prove information retrieval, support clinical decision-

making, and reduce cognitive overload on healthcare

providers.

In that paper cognitive overload is a great chal-

lenges as Engelsma, also points.

For Kene (Kenei and Opiyo, 2022), there is a Cog-

nitive Overload due ”clinical notes that are stored in

EHRs, present special challenges to analyzing them

due to their unstructured nature” and ”Although,

clinical notes provide detailed information about the

patient’s condition, using them is a challenge”.

Due to page limitations in this paper, all other

challenges grouped by themes listed in the table 5 can

be consulted in appendix C of this work.

5.5 Limitations

The focus of this research was on selected studies fo-

cusing on usability challenges faced in the healthcare

product ecosystem.

The search process was performed using a lim-

ited set of keywords with the aim of providing an

overview of usability challenges in healthcare. This

research focused on publications for a ﬁnite period

from 2014 to 2024. However, it explores several chal-

lenges faced by Usability/UCD/UX approaches used

during all stages of software development.

5.6 Future Works

The conduction of SLR studies is at an early stage of

research. We developed a research agenda that can

be followed by the empirical community of SE. We

believe that conducting such research may assist the

process of conducting SLR studies.

5.7 Threats to Validity

We classiﬁed the threats of this study using the study

proposed by Ampatzoglou et al. (Ampatzoglou et al.,

2019) that provides a set of validity threats in review

studies in the context of SE. Regarding the selection

process of the studies, we used the guideline proposed

by Kitchenham et al. (Kitchenham et al., 2015) to de-

ﬁne our research protocol. Our search string was de-

ﬁned based on the deﬁnitions of primary studies based

on Silva et. al. (da Silva et al., 2014), we would

identify all published studies, which can be consid-

ered a limitation to the study. Selection of articles

more speciﬁcally related to criteria of inclusion and

exclusion, a threat of this study may be the estab-

lishment of criteria deﬁned by only one researcher.

However, its application was conducted by two re-

searchers, and a third researcher mediated the con-

ﬂicts. We observed that 80% of the studies analysed

were published in important venues of SE, such as

Journal of the American Medical Informatics Associ-

ation, Journal of healthcare engineering, International

journal for quality in healthcare, International Confer-

ence on e-Health Networking, Applications and Ser-

vices and International Conference on Medical and

ICEIS 2025 - 27th International Conference on Enterprise Information Systems

562

Health Informatics.

6 CONCLUSION

In this paper we conducted short SLR to identify main

usability challenges in heathcare applications. We

analysed a total of 43 primary studies published be-

tween 2014 and April 2024. We observed that the

main challenges are related to usability process adop-

tion and technical issues. There were others chal-

lenges related do Research lack as mentioned by

Broekhuis et al (Broekhuis et al., 2021) ”In contrast,

much research has been conducted to create generic

instruments to obtain a rapid and very general assess-

ment of the status of usability of systems, regardless

of the system domain or context”. Our study also ad-

dressed solutions to usability challenges in healthcare

ecosystem products.

REFERENCES

Adams, K. T., Pruitt, Z., Kazi, S., Hettinger, A. Z., Howe,

J. L., Fong, A., and Ratwani, R. M. (2021). Identi-

fying health information technology usability issues

contributing to medication errors across medication

process stages. Journal of patient safety, 17(8):e988–

e994.

Alon, N. and Torous, J. (2023). Current challenges

for evaluating mobile health applications. Journal

of the American Medical Informatics Association,

30(3):617–624.

Alves, N. S., Mendes, T. S., de Mendonc¸a, M. G., Sp

ınola,

R. O., Shull, F., and Seaman, C. (2016). Identiﬁcation

and management of technical debt. Inf. Softw. Tech-

nol., 70(C):100–121.

Ampatzoglou, A., Bibi, S., Avgeriou, P., Verbeek, M., and

Chatzigeorgiou, A. (2019). Identifying, categorizing

and mitigating threats to validity in software engi-

neering secondary studies. Information and Software

Technology, 106:201–230.

Braun, V. and Clarke, V. (2006). Using thematic analysis

in psychology. Qualitative research in psychology,

3(2):77–101.

Broekhuis, M., van Velsen, L., Peute, L., Halim, M., Her-

mens, H., et al. (2021). Conceptualizing usability for

the ehealth context: content analysis of usability prob-

lems of ehealth applications. JMIR Formative Re-

search, 5(7):e18198.

Carayon, P. and Hoonakker, P. (2019). Human factors and

usability for health information technology: old and

new challenges. Yearbook of medical informatics,

28(01):071–077.

Carayon, P. and Salwei, M. E. (2021). Moving toward a

sociotechnical systems approach to continuous health

information technology design: the path forward for

improving electronic health record usability and re-

ducing clinician burnout. Journal of the American

Medical Informatics Association, 28(5):1026–1028.

Catchpole, K., Bowie, P., Fouquet, S., Rivera, J., and

Hignett, S. (2021). Frontiers in human factors: em-

bedding specialists in multi-disciplinary efforts to im-

prove healthcare.

Cho, H., Keenan, G., Madandola, O. O., Dos Santos, F. C.,

Macieira, T. G., Bjarnadottir, R. I., Priola, K. J.,

Lopez, K. D., et al. (2022). Assessing the usability

of a clinical decision support system: heuristic evalu-

ation. JMIR human factors, 9(2):e31758.

da Silva, F. Q. B., Suassuna, M., Franc¸a, A. C. C., Grubb,

A. M., Gouveia, T. B., Monteiro, C. V. F., and dos San-

tos, I. E. (2014). Replication of empirical studies in

software engineering research: a systematic mapping

study. Empirical Software Engineering, 19(3):501–

557.

Dyb

a, T. and Dingsøyr, T. (2008). Empirical studies of agile

software development: A systematic review. Informa-

tion and Software Technology, 50(9):833 – 859.

Engelsma, T., Heijmink, S., Hendriksen, H. M., Visser,

L. N., Lemstra, A. W., Jaspers, M. W., and Peute,

L. W. (2024). Capturing usability problems for people

living with dementia by applying the demigned princi-

ples in usability evaluation methods: Mixed methods

study. JMIR Human Factors, 11:e54032.

Gettinger, A. and Zayas-Cab

an, T. (2021). Hitech to 21st

century cures: clinician burden and evolving health it

policy. Journal of the American Medical Informatics

Association, 28(5):1022–1025.

Ghorayeb, A., Darbyshire, J. L., Wronikowska, M. W., and

Watkinson, P. J. (2023). Design and validation of a

new healthcare systems usability scale (hsus) for clin-

ical decision support systems: a mixed-methods ap-

proach. BMJ open, 13(1).

Goerss, D., K"ohler, S., Rong, E., Temp, A. G.,

Kilimann, I., Bieber, G., Teipel, S., et al. (2024).

Smartwatch-based interventions for people with de-

mentia: User-centered design approach. JMIR aging,

7(1):e50107.

Harte, R., Quinlan, L. R., Glynn, L., Rodr

ıguez-Molinero,

A., Baker, P. M., Scharf, T.,

OLaighin, G., et al.

(2017). Human-centered design study: enhancing the

usability of a mobile phone app in an integrated falls

risk detection system for use by older adult users.

JMIR mHealth and uHealth, 5(5):e7046.

Heitkemper, E., Hulse, S., Bekemeier, B., Schultz, M.,

Whitman, G., Turner, A. M., et al. (2024). The so-

lutions in health analytics for rural equity across the

northwest (share-nw) dashboard for health equity in

rural public health: Usability evaluation. JMIR Hu-

man Factors, 11(1):e51666.

Hettinger, A. Z., Melnick, E. R., and Ratwani, R. M.

(2021). Advancing electronic health record vendor

usability maturity: Progress and next steps. Jour-

nal of the American Medical Informatics Association,

28(5):1029–1031.

ISO 9241-11, W. (1998). 9241-11. ergonomic require-

ments for ofﬁce work with visual display terminals

Challenges and Approaches to Enhance Usability in Healthcare Applications: A Systematic Literature Review

563

(vdts). The international organization for standard-

ization, 45(9).

Kato, D., Lucas, J., and Sittig, D. F. (2024). Implementa-

tion of a health information technology safety classi-

ﬁcation system in the veterans health administration’s

informatics patient safety ofﬁce. Journal of the Amer-

ican Medical Informatics Association, page ocae107.

Kenei, J. K. and Opiyo, E. (2022). Using classiﬁcation and

visualization to support clinical texts review in elec-

tronic clinical documentation. In Proceedings of the

6th International Conference on Medical and Health

Informatics, pages 78–84.

Kitchenham, B. A., Budgen, D., and Brereton, P. (2015).

Evidence-Based Software Engineering and Systematic

Reviews. Chapman & Hall/CRC.

Kuckartz, U. and R

adiker, S. (2019). Analyzing qualitative

data with MAXQDA. Springer.

Lloyd, S., Long, K., Probst, Y., Di Donato, J., Oshni Al-

vandi, A., Roach, J., and Bain, C. (2023). Medical

and nursing clinician perspectives on the usability of

the hospital electronic medical record: A qualitative

analysis. Health Information Management Journal,

page 18333583231154624.

Lowdermilk, T. (2019). Design Centrado no Usu

ario:

um guia para o desenvolvimento de aplicativos

amig

aveis. S

ao Paulo: Novatec.

Lowry, S. Z., Abbott, P., Gibbons, M. C., Lowry, S. Z.,

North, R., Patterson, E. S., Quinn, M. T., Ramaiah,

M., Schumacher, R. M., and Zhang, J. (2012). Techni-

cal evaluation, testing, and validatiaon of the usabil-

ity of electronic health records. US Department of

Commerce, National Institute of Standards and Tech-

nology.

Martikainen, S. (2015). Towards better usability: Usabil-

ity and end-user participation in healthcare informa-

tion technology systems development. PhD thesis, It

Suomen yliopisto.

McMullan, C., Retzer, A., Hughes, S. E., Aiyegbusi, O. L.,

Bathurst, C., Boyd, A., Coleman, J., Davies, E. H.,

Denniston, A. K., Dunster, H., et al. (2023). Devel-

opment and usability testing of an electronic patient-

reported outcome (epro) solution for patients with in-

ﬂammatory diseases in an advanced therapy medici-

nal product (atmp) basket trial. Journal of Patient-

Reported Outcomes, 7(1):98.

Meidani, Z., Omidvar, A., Akbari, H., Asghari, F., Kha-

jouei, R., Nazemi, Z., Nabovati, E., and Holl, F.

(2024). Evaluating the usability and quality of a

clinical mobile app for assisting physicians in head

computed tomography scan ordering: Mixed methods

study. JMIR Human Factors, 11:e55790.

Middleton, B., Bloomrosen, M., Dente, M. A., Hashmat,

B., Koppel, R., Overhage, J. M., Payne, T. H., Rosen-

bloom, S. T., Weaver, C., and Zhang, J. (2013). En-

hancing patient safety and quality of care by improv-

ing the usability of electronic health record systems:

recommendations from amia. Journal of the Ameri-

can Medical Informatics Association, 20(e1):e2–e8.

Pruitt, Z. M., Johnson, T., and Kazi, S. (2023). Identifying

usability challenges in electronic medication admin-

istration record design through scenario-based evalu-

ations: A pilot study. In Proceedings of the Human

Factors and Ergonomics Society Annual Meeting, vol-

ume 67, pages 1368–1372. SAGE Publications Sage

CA: Los Angeles, CA.

Ratwani, R. M., Fairbanks, R. J., Hettinger, A. Z., and

Benda, N. C. (2015). Electronic health record us-

ability: analysis of the user-centered design processes

of eleven electronic health record vendors. Jour-

nal of the American Medical Informatics Association,

22(6):1179–1182.

Ratwani, R. M., Savage, E., Will, A., Arnold, R., Khairat,

S., Miller, K., Fairbanks, R. J., Hodgkins, M., and

Hettinger, A. Z. (2018a). A usability and safety anal-

ysis of electronic health records: a multi-center study.

Journal of the American Medical Informatics Associ-

ation, 25(9):1197–1201.

Ratwani, R. M., Savage, E., Will, A., Fong, A., Karavite,

D., Muthu, N., Rivera, A. J., Gibson, C., Asmonga,

D., Moscovitch, B., et al. (2018b). Identifying elec-

tronic health record usability and safety challenges in

pediatric settings. Health affairs, 37(11):1752–1759.

Schumacher, R. M. and Jerch, K. E. (2012). Measuring

usability in healthcare it: It’s a practice, not a compe-

tition. Interactions, 19(4):8–9.

Tonella, P., Torchiano, M., Du Bois, B., and Syst

a, T.

(2007). Empirical studies in reverse engineering: state

of the art and future trends. Empirical Software Engi-

neering, 12(5):551–571.

Vandekerckhove, P., De Mul, M., Bramer, W. M., and

De Bont, A. A. (2020). Generative participatory de-

sign methodology to develop electronic health inter-

ventions: systematic literature review. Journal of med-

ical Internet research, 22(4):e13780.

Wieringa, R., Maiden, N., Mead, N., and Rolland, C.

(2005). Requirements engineering paper classiﬁcation

and evaluation criteria: A proposal and a discussion.

Requir. Eng., 11(1):102–107.

APPENDIX

Multimedia Appendix A

Metadata of the included studies.

[PDF File , 106 KB-Multimedia Appendix A]

Multimedia Appendix B

Metadate of Quality Assessment.

[PDF File , 76 KB-Multimedia Appendix B]

Multimedia Appendix C

Metadate of Codes.

[PDF File , 257 KB-Multimedia Appendix C]

ICEIS 2025 - 27th International Conference on Enterprise Information Systems

564