A Systematic Review on the Use of Educational Technologies for Medical

Education

Dalmaris Lima

, Victor Sotero

, Diego Dermeval

1,2 a

, Jorge Artur

1,2

and Francisco Passos

Center of Excellence for Social Technologies (NEES), Computing Institute, Federal University of Alagoas (UFAL),

Av. Lourival Melo Mota, S/N, Tabuleiro do Martins, Macei

o-AL, Brazil

Faculty of Medicine, Federal University of Alagoas (UFAL), Av. Lourival Melo Mota, S/N,

francisco passos01@hotmail.com

Keywords:

Educational Technologies, Medical Education, Computers and Medical Education.

Abstract:

Educational technologies have been increasingly used in medical education to promote innovative pedagogical

strategies in classroom. However, the medical scientiﬁc community still lacks a comprehensive understanding

on how educational technologies are used in medical education. The objective of this work is to explore

what types of technologies are employed in medical education, aiming to identify in which domains they

have been applied and the reported evidence of using educational technologies in medical education. We

conducted a Systematic Literature Review (SLR) to identify the primary studies on the use of educational

technologies in medical education, following a pre-deﬁned review protocol. One hundred sixty-eight papers

were selected, covering nine types of education technologies, which were applied in more than 40 medical

domains. Moreover, our SLR results also show that most of the papers included in this SLR reported positive

evidence about the beneﬁts of using these technologies in medical domain.

1 INTRODUCTION

The evolution of medical education has been taking

place rapidly. Students enter in the medical school

with a high level of technological skills and high ex-

pectation regarding innovative ways of learning. As

such, many researchers and medical schools have

been investigating how to apply new educational tech-

nologies in medical education. These technologies

may be, for instance, adaptive platforms, educational

games, gamiﬁcation, mobile learning applications,

simulated virtual patients, and so on (Jabbar et al.,

2016; Ellaway and Masters, 2008) . Despite the ex-

istence of many works which rely on the use of ed-

ucational technologies in medical education, it is not

known to what extent there is evidence to state that

the use of educational technologies, indeed, beneﬁts

medical education. Moreover, it is also important to

understand: i) what are the main types of educational

technologies that have been used in medical educa-

tion; and ii) on which medical domains these tech-

nologies have been applied.

There are some works that investigated the appli-

https://orcid.org/0000-0002-8415-6955

cation of educational technologies in health education

(Kalaian and Kasim, 2017) and, speciﬁcally, its us-

age in medical education (Ellaway and Masters, 2008;

Taveira-Gomes et al., 2016). However, these stud-

ies did not capture all the aspects and evidences that

we are interested. We are interested in ﬁnding how

promising educational technologies (e.g., ITS, CSCL,

mobile learning, gamiﬁcation, and so on) have been

used in medical education.

In this article, we used the Systematic Review of

Literature (SLR) method (Kitchenham and Charters,

2007) to identify, evaluate, interpret, and synthesize

available studies to address speciﬁc research ques-

tions on the use and effectiveness of educational tech-

nologies in medical education settings and to estab-

lish evidence status with in-depth analysis. Thus, the

objective of this systematic review is to better under-

stand how these educational technologies are being

developed, applied and evaluated, as well as to iden-

tify in what way these technologies have been applied

to medical education. This paper presents the results

of a SLR on studies published from 1986 to Septem-

ber 2017 and was conducted following a pre-deﬁned

review protocol, as will be further explained. The re-

mainder of this paper is organized as follows. Section

Lima, D., Sotero, V., Dermeval, D., Artur, J. and Passos, F.

A Systematic Review on the Use of Educational Technologies for Medical Education.

DOI: 10.5220/0007678501530160

In Proceedings of the 11th International Conference on Computer Supported Education (CSEDU 2019), pages 153-160

ISBN: 978-989-758-367-4

153

2 describes the SLR method used in this review. Sec-

tion 3 presents the results of the quality assessment

and an overview the papers. Then, it reports the ﬁnd-

ings of the review along with an analysis and discus-

sion for each research question. Finally, Section V

presents conclusions, limitations and future works.

2 METHOD

A Systematic Literature Review (SLR) is a means of

identifying, evaluating and interpreting the available

research ﬁndings related to a research question, topic

area, or phenomenon. The main purpose of conduct-

ing a systematic review is to gather evidence on which

to base conclusion (Kitchenham and Charters, 2007).

Our SLR was structured based on the guide-

lines for conducting systematic reviews proposed by

(Kitchenham and Charters, 2007). The SLR was di-

vided into three main processes: SLR planning, con-

ducting the SLR, and reporting the SLR. Following

the steps: i) identiﬁcation of the need for a system-

atic review; ii) formulation of a focused review ques-

tion; iii) a comprehensive and thorough search of pri-

mary studies; iv) evaluation of the quality of included

studies; v) identiﬁcation of the data needed to answer

the research question; vi) extraction of data; vii) sum-

mary and summary of study results; viii) interpreta-

tion of results to determine their applicability; and ix)

reporting.

2.1 Research Question

The main objective of this SLR is to identify and

map the existing studies in the literature that use ed-

ucational technologies to promote medical education.

Based on this main objective, speciﬁc questions were

raised to identify the existing works in the area. The

questions along with their respective descriptions and

motivations are described below.

RQ1: What types of educational technologies

have been used in medical education? This question

aims to identify the main types of educational tech-

nologies (e.g., intelligent tutoring systems, computer-

supported collaborative learning, learning manage-

ment systems, and so on) used to support medical ed-

ucation;

RQ2: Which medical education domains have

been supported by educational technologies? This

question aims to identify the main domains, in the

medical ﬁeld, that make use of the technologies

for educational support. It is important support re-

searchers in future developments of educational tools

to aid medical education;

RQ3: Are there evidences to state that using ed-

ucational technologies beneﬁts medical education?

This question intends to analyze if such studies pro-

vide some evidence that the use of educational tech-

nologies beneﬁts medical education considering its

domains. Evidence should consider positive and neg-

ative results including empirical and non-empirical

evaluation. They are important since they form a

knowledge base about the use of educational tech-

nologies for medical education.

2.2 Inclusion and Exclusion Criteria

The deﬁnition of the inclusion and exclusion crite-

ria is directly linked to the research objective. We

deﬁned the following inclusion criteria based on our

research questions: (i) primary studies; (ii) peer-

reviewed studies, and (iii) studies evaluating or using

educational technologies in medical education. The

exclusion criteria of this review are: (i) duplicated pa-

pers; (ii) gray literature; (iii) non-English papers; (iv)

non-peer-reviewed studies; (v) position papers; (vi)

redundant article authored by the same researcher;

(vii) secondary studies; and (viii) short papers. We

also excluded (ix) papers that use educational tech-

nologies for other health areas (e.g.,nursing, physio-

therapy) and papers that received a score less than

50% of quality after conducting the quality assess-

ment step of the review, as further explained in Sec-

tion 2.4. Note that a primary study is a paper that

presents any new contribution to a ﬁeld (Kitchenham

and Charters, 2007) and secondary studies are review

papers, such as systematic reviews, meta-analysis or

surveys.

2.3 Source Selection and Search

The search was performed only in electronic

databases, the following electronic databases were

searched: ACM Digital, EI Compendex, IEEE Digital

Library, ISI Web of Science, Lilacs, PubMed, Science

@ Direct, Scopus and Springer Link. The systematic

review process is described in Figure 1 as well as the

number of studies analyzed per step.

In step 1, the studies were obtained from the elec-

tronic databases using the following string: ((”Med-

ical education” OR ”medical teaching” OR ”med-

ical school”) AND (”authoring tool” OR ”author-

ing system” OR ”intelligent authoring” OR ”learning

management system” OR ”m-learning” OR ”mobile

learning” OR ”collaborative learning” OR ”computer

supported collaborative learning” OR ”intelligent tu-

toring system” OR ”intelligent educational systems”

OR ”massive open on line courses” OR ”adaptive ed-

CSEDU 2019 - 11th International Conference on Computer Supported Education

154

Step 1: Identify and organize studies

retrieved from the electronic base.

Step 2: automatic removal of

duplicated papers using Parsifal tool

Step 3: reviewers reviewed the

titles, keywords, and publication

local and excluded those that did

not attend the inclusion criteria. If

there was insufficient data, the

a paper was left for the next step.

Step 4: reviewers analyzed the

the paper’s abstracts and excluded

those which did not attend the

inclusion criteria. If there was

insufficient data, the paper was left

for next step

Step 5: the complete text from the

papers selected at step 4 were

retrieved and reviewed and those

that attended the inclusion criteria

were selected

Step 6: Revised studies that aimed

to assess the use of technology in

healthcare education,

in general, were rejected,

the other studies met the

inclusion criteria for the next step.

Step 7: the studies reviewed with

Scores in the evaluation of quality

of less than 4.5, or less than 50% of

total quality assessment scores were

excluded, as well as closing the

studies selection stage

ACM Digital:

18 titles

EI Compendex:

145 titles

PubMed:

881 titles

ISI Web Science:

266 titles

Scopus:

578 titles

Springer Link:

999 titles

Science@Direct:

1129 titles

IEEE Digital Libary:

79 titles

Lilacs:

5 titles

4,100

titles

281

titles

291

titles

1,168

titles

2,153

titles

3,159

titles

168

titles

Figure 1: Research questions and motivations.

ucational systems” OR ”adaptive learning systems”

OR ”artiﬁcial intelligence in education” OR ”gamiﬁ-

cation”)).

The results for the string search (4100 papers)

were automatically downloaded and organized with

the help of the Parsifal tool

. In Figure 1 we describe

the selection stages, as well as the number of studies

in each stage. In the end the selection process, 168

papers remained and were included in the review.

2.4 Quality Assessment

The quality assessment (QA) of the selected studies

was based on a scoring technique to evaluate the cred-

ibility, integrity, and relevance of the selected papers.

All the papers were submitted to a set of 9 quality

criteria. Nine questions are adapted to quality assess-

ments of studies in the literature. Table 1 presents the

evaluation tool. All quality criteria were taken from

the literature. The main basis for structuring the qual-

ity assessment questionnaire was the Systematic Re-

view of the Literature conducted by (Dermeval et al.,

http://parsif.al

2017).

We used a technique of score between 3 possible

answers, Yes = 1,0, No = 0,0 and Maybe = 0,5. In

questions Q6 and Q9, it would be possible only a di-

rect answer (Yes / No), where Q6 was asked if the

study was evaluated with some empirical method, al-

ready in Q9 is questioned if the study can be put in

an industrial environment. In question Q5 brings the

question of whether these tools would be available for

download on the web, being scored with 1.0 if the tool

is present and it is available for download on the web,

0.5 if only the tool is presented, but it’s unavailable for

download and has received score 0,0 those who did

not present any type of tool. In the other questions,

this line follows, taking into account the requirement

of the question totally received 1.0, partially attend-

ing 0.5 and not meeting the requirements of question

received 0.0.

Note that, in the selection steps, authors evaluated

the studies separately to discuss disagreement regard-

ing classiﬁcation. Studies were organized in a spread-

sheet, separated per retrieved papers over electronic

databases. After this analysis, there was a cross-

referencing of the data and a discussion of the diver-

A Systematic Review on the Use of Educational Technologies for Medical Education

155

gences between the evaluations, reaching agreement

on the issues.

2.5 Data Extraction and Synthesis

The data extraction was performed through the read-

ing of the 168 articles selected in the previous steps.

During this stage, the data was extracted by complet-

ing the extraction form (Table 2). All search, selec-

tion, and extraction were done with the aid of the Par-

sifal tool.

3 RESULTS AND ANALYSIS

A total of 168 papers met the inclusion criteria and

their data were extracted. Prior to presenting the re-

sults and analysis for each research question, we re-

view the quality assessment results and give a short

overview of the general characteristics of the studies.

The quality assessment on the selected papers is

useful to increase the accuracy of the data extraction

results. This evaluation helped to determine the va-

lidity of the inferences conducted and in ascertaining

the credibility and coherent of the synthesis of results.

For the sake of space, the quality assessment results

as well as the list of the 168 papers included in this

review are available at Google drive

The scores of all papers are no less than 50% and

the average score is 74,46%. We chose the minimum

of 50% quality with the aim to establish an accept-

able quality threshold for the articles. Taken together,

these 9 criteria provided a measure of the extent to

which we could be conﬁdent that a particular study’s

ﬁndings could make a valuable contribution to this re-

view.

3.1 Overview of the Studies

In the following sections, we depict in detail publica-

tions years, local and type of paper, research methods,

and application context information.

3.1.1 Publication Year

The reviewed papers were published between 1986

and 2017. From a temporal point of view (Figure 2),

an increasing number of publications in the context of

this review is observed since 2006. 2016 (24;14.12%)

is the year with most publications, followed by 2014

and 2015 (each with 20 papers;11.76%), 2011 (12;

https://drive.google.com/ﬁle/d/1MXBbYq97z-

2euqCk Fg5e6eJda9x43v7/view?usp=sharing

7.06%), 2013 (12; 7.06%), 2008 (10; 5.88%), 2006,

2010 and 2017 (each with 9 papers; 5.29%), 2012 (8;

4.71%), 2002 and 1990 (each with 4 papers; 2.35%),

2007, 2000, and 1992 (each with 3 papers; 1.76%),

2009, 2005, 2004, 2001, and 1991 (each with 2 pa-

pers; 1.18%). There is only one paper published in

each of the following years: 1999, 1997, 1996, 1995,

1994, 1993, 1988, and 1986.

Note that there is an increasing interesting in the

publications on the use of educational technologies

in medical since 2006. It is also worth noting that,

as the search process of this review was performed

in September 2017, a slight decrease in the number of

publications would be expected in 2017 because some

papers might be in press.

1986

1988

1990

1991

1992

1993

1994

1995

1996

1997

1999

2000

2001

2002

2004

2005

2006

2007

2008

2009

2010

2011

2012

2013

2014

2015

2016

2017

1 1

1 1 1 1 1 1

2 2

20 20

Number of Publication per Year

Figure 2: Number of publications per year.

3.1.2 Articles Types

The types of sources could be journal, conference,

or book chapters. The majority of the articles are

the journal papers (63.53%, 108 studies), followed

by chapter books (20%, 34 studies), and conferences

(15.29%, 26 studies).

The venues with more contributions are: BMC

Medical Education (29 publications), Advances in

Health Sciences Education (5 publications), Medical

Science Educator and Lecture Notes in Computer Sci-

ence (including subsections in Artiﬁcial Intelligence

and Lecture Notes in Bioinformatics)(each one with 4

papers), and Medical Teacher (2 publications). How-

ever, one might note that several sources from edu-

cational technology, medicine and computer science

have been considered as possible venues for publica-

tion, which may suggest that scientiﬁc communities

are interested in this interdisciplinary ﬁeld.

3.1.3 Research Method

The classiﬁcation of the publications was based

on the categories (i.e.,controlled experiment, quasi-

experiment, case study, survey research, ethnography

and action research) deﬁned by (Easterbrook et al.,

2008). However, we have deﬁned two extra categories

to classify papers that did not ﬁt in those categories:

CSEDU 2019 - 11th International Conference on Computer Supported Education

156

Table 1: Study quality assessment criteria. These criteria are adapted from (Dermeval et al., 2017). Y=Yes,N=No, and

M=Maybe.

# Questions Possible answers

Q1 Is there a rationale for why the study was undertaken? Y=1,0 ,N=0,0 ,M=0,5

Q2 Is there a clear statement of the goals of the research? Y=1,0 ,N=0,0 ,M=0,5

Q3 Is the proposed technique properly described? Y=1,0 ,N=0,0 ,M=0,5

Q4 Is there an adequate description of the context (industry,

laboratory setting, products used and so on) in which the

research was carried out?

Y=1,0 ,N=0,0 ,M=0,5

Q5 Does the study provide or use a tool? If so, is the tool

available for download on the internet?

Y=1,0 ,N=0,0 ,M=0,5

Q6 Was the study empiricaly evaluated? Y=1,0 ,N=0,0

Q7 Is there a discussion about the results of the study? Y=1,0 ,N=0,0 ,M=0,5

Q8 Are the limitations of this study explicitly discussed? Y=1,0 ,N=0,0 ,M=0,5

Q9 Does the study also evaluate the proposal in industrial

settings?

Y=1,0 ,N=0,0

Table 2: Extraction form.

# Study data Description RQ

1 Study Identiﬁer Unique id for the study Overview

2 Author(s) Overview

3 Type of article Book chapter, Conference, Journal, Workshop Overview

4 Article Source Overview

5 Application context Industrial, academic Overview

6 Year Overview

7 Title Overview

8 Country Overview

9 Researchers genders Female, Male, Other Overview

10 Research Method Action research, Case study, Controlled experiment, Ethnography, Il-

lustrative Scenario, Not applicable, Survey

Overview

11 Type of educational

technologies

CSCL, Educational Game, Gamiﬁed, Environment, ITS, LMS, M-

learning, MOOC, Simulation, T-learning

RQ1

12 Medical education do-

mains supported

RQ2

13 Evidence Negative argumentation, Negative with empirical evaluation,Positive

argumentation, Positive with empirical evaluation

RQ3

illustrative scenario and not applicable. The ﬁrst is

appropriate for papers that just explain their contribu-

tions using small examples or argumentation. The lat-

ter refers to the papers that do not present any kind of

research method or explanation of using the proposal.

Case Study (41.18%; 70 studies) constitute the

majority of the studies, followed by Not applica-

ble (20%; 34 studies), Survey (17.85%; 30 studies),

Controlled experiment (10.59%; 18 studies), illustra-

tive scenario (5.29%; 9 studies) and action research

(3.53%; 6 studies). It is worth noting that 118 studies

(70.23%) are concerned in conducting empirical stud-

ies (i.e., controlled experiment, case study, and sur-

vey) on evaluating or creating technologies for medi-

cal education. There were no ethnography and action

research papers in our classiﬁcation.

3.1.4 Application Context

The paper settings were categorized as industry and

academic. The majority of them (93.45%; 157 pa-

pers) are in the academic category, whereas, 19 papers

(11.30%) belong to the industry category

3.2 RQ1: Types of Educational

Technologies

The purpose of this research question was to iden-

tify the main educational technologies used to support

medical education.

3.2.1 Results

The technologies were categorized into nine differ-

ent types, Computer-supported collaborative learning

(CSCL), Gamiﬁed environment, Educational Game,

learning management system (LMS), Simulation,

mobile-learning, intelligent tutoring system (ITS),

massive open online course (MOOC), tv-learning.

As seen in Table 3, the predominant type of ed-

ucational technology used in medical education is

A Systematic Review on the Use of Educational Technologies for Medical Education

157

Table 3: Studies over Technologies used.

Types of educational Tech-

nologies

Studies Freq %

LMS S11, S46, S58, S59, S91, S167, S125, S154, S88, S140, S168,

S1,S13, S16, S21, S32, S35, S36, 38, S48, S55, S56, S61, S62, S63,

S65, S66, S68, S69, S70, S71, S72, S74, S75, S77, S81, 83, S90,

S92, S94, S96, S97, S99, S102, S103, S104, S105, S108, S109,

S110, S114, S121, S124, S130, S132, S138, S139, S141, S143,

S144, S145, S148, S149, S150, S152, S157, S164, S14, S20, S127,

S158, S159, S5, S34, S19

75 44.12%

M-Learning S125, S8, S14, S20, S127, S158, S15, S22, S27, S28, S31, S67, S79,

S82, S89, S106, S107, S112, S116, S117, S119, S120, S128, S129,

S137, S142, S156, S163, S133, S165, S118

31 18.24%

Simulation S50, S101, S53, S134, S169, S5, S34, S2, S3, S7, S17, S18, S23,

S24, S39, S40, S42, S44, S54, S60, S76, S78, S87, S126, S136,

S161, S162, S166, S95

31 18.24%

ITS S10, S12, S25, S26, S29, S33, S37, S41, S47, S49, S64, S80, S93,

S100, S115, S135, S147, S151, S153, S155, S170, S140, S168,

S134, S169

25 14.71%

CSCL S4, S43, S45, S52, S73, S131, S146, S160, S85, S11, S46, S58, S59,

S91, S167, S125, S50

17 10%

Educational Game S6, S8, S30, S53, S84, S101, S154 7 4.12%

Gamiﬁed Environment S85, S84, S101, S86, S98, S88 6 3.53%

T-learning S19, S118, S95, S57, S123 5 2.94%

MOOC S159, S111, S122 3 1.76%

LMS (44.12%;75 studies), followed by Simulation

(18.24%;31 studies), M-learning (18.24%;31 stud-

ies), ITS (14.71%;25 studies), CSCL (10%; 17 stud-

ies), educational games (4.11%;7 studies), gamiﬁed

environment (3.52%;6 studies), t-learning (2.94%;5

studies) and MOOC (1.76%;3 studies). It is worth

noting that a publication could have used more than

one educational technology of the RE process. Thus,

the sum of the percentage is greater than 100%

3.2.2 Analysis and Discussion

In summary, we could note that several distinct types

of educational technologies have been used in medi-

cal education, showing the demand for different types

of educational technologies to target different medical

education needs. It is worth noting that most of the

papers (44.12%) are using LMS, which could suggest

that the studies are more focused on content reposi-

tories for medical education rather than on adaptive-

intensive educational systems (e.g., ITS). An interest-

ing result that we might mention is the frequent use

of mobile technologies for medical education, this re-

sults could indicate that mobile devices may be well-

accepted by medicine students. Moreover, educa-

tional technologies based on simulation are also fre-

quently used, which is expected since medical school

curricula uses lots of practical activities.

We also could note that some promising educa-

tional technologies might be underexplored in medi-

cal education. For example, educational games and

gamiﬁcation could be more used along with other

types of educational games to target achieving more

engagement of medicine students.

3.3 RQ2: Medical Education Domains

The purpose of this research question was to main

medical domains where educational technologies

have been used.

3.3.1 Results

The classiﬁcation of the medical domains was per-

formed after the data extraction step, that is, during

the extraction of data to answer this research question.

We identiﬁed 47 different medical domains where

these technologies were applied. Educational tech-

nologies have been used in several domains, for ex-

ample, for pediatric learning support (4.71 %, 8 stud-

ies), followed by anatomy study (4.12 %, 7 studies).

These areas are followed by teaching and practice

surgeries (3.53 %, 6 studies), diagnosis of diseases

(1.18 %, 2 studies), cardiology (3 studies, 1.76 %).

Note that a study could have targeted more than one

domain, so the sum of the percentage may be greater

than 100 %.

CSEDU 2019 - 11th International Conference on Computer Supported Education

158

Table 4: Medical education domains.

Medical Education

Domains

Studies Freq %

Anatomy S11, S17,S28 7 4.12%

Anesthesia S44 1 0.59%

Antimicrobia pre-

scribing

S8 1 0.59%

Atheromatosis S93 1 0.59%

Cardio-respiratory S132 1 0.59%

Cardiology S26, S47, S75 4 2.35%

Cardiopulmonary S142 1 0.59%

Cardiovascular Phys-

iology

S64 1 0.59%

Clinical education S102 1 0.59%

Cryosurgery S135 1 0.59%

Cytopathology S57 1 0.59%

Dental S150, S156 2 1.18%

Dermatology S76 1 0.59%

Chiropractic S151 1 0.59%

Diagnostic S151 2 1.18%

Emergency S50 1 0.59%

Endocrinology S77, S161 2 1.18%

Forensic medicine S24 1 0.59%

General practitioners S32 1 0.59%

Gynecologic S25 1 0.59%

Histology S158, S83 3 1.76%

Human Anatomy S35, S53, S165, S28 4 2.35%

Human Genetics S2 1 0.59%

Laparoscopy S54 1 0.59%

Microscopy S71 1 0.59%

Neurology S21 1 0.59%

Neuroradiology S69 1 0.59%

Oncology S13, S99, S134 3 1.76%

Operating S101 1 0.59%

Orthopedy S38, S90 2 1.18%

Pediatric S109, S23, S143, S154, S96,

S107, S133, S127

8 4.71%

Pathology S83 5 2.94%

Pharmacology S162, S28 2 1.18%

Physiology S29 1 0.59%

Pneumology S94 1 0.59%

Psychiatric S97, S124 2 1.18%

Radiology S19, S152 2 1.18%

Surgery S88, S123, S136 3 1.76%

Thoracic Surgery S87 1 0.59%

Traumatology S90 1 0.59%

Urology S30, S62, S159 3 1.76%

Venereology S76 1 0.59%

3.3.2 Analysis and Discussion

In fact, the great majority of the papers are not target-

ing any particular medical domain. This results might

suggest that researchers have been proposing generic

educational technologies not tied to speciﬁc medical

domains. However, it is worth noting that educational

technologies have been applied in a plenty of medical

domains, illustrating the need for research and devel-

opment for solutions in several areas. Another inter-

esting point that we can raise is that some technolo-

gies could be more amenable to be used in particular

domains. For example, it is expected that simulation

technologies might be more frequently use in surgery

domain that in psychiatric. As such, it would be also

interesting to investigate if there are some kind of cor-

relation between medical domains and the types of

educational technologies used.

3.4 RQ3: Supportive Evidence

The purpose of this research question was to gather

and classify evidences to state that using educational

technologies beneﬁts or not medical education.

3.4.1 Results

The classiﬁcation of the papers was deﬁned accord-

ing to the presence or absence of empirical evaluation

in the paper and by the positive or negative indication

that using educational technologies beneﬁts medical

education. The deﬁned categories are: positive with

empirical evaluation, positive without empirical eval-

uation, negative with empirical evaluation and nega-

tive without empirical evaluation (see Table 5). As

shown in Table 5, 45.29% (77 studies) of the studies

reported positive evidence after conducting empirical

evaluation and 44.29% of studies (76 studies) only

presented positive argumentation about the beneﬁts of

educational technologies. In addition, 4.71% (8 stud-

ies) presented negative evidence after running empiri-

cal evaluation and 3.53% (6 studies) argued about the

negative implications of using educational technolo-

gies in medical education.

Table 5: Supportive evidence.

Types of evidence Studies Freq %

Positive argumentation S1, S2, S7, S8, S12, S16, S20, S21, S23, S24, S25,

S26, S28, S33, S39, S40, S42, S43, S44, S45, S46,

S47, S48, S49, S50, S53, S56, S57, S58, S59, S60,

S61, S67, S69, S70, S77, S78, S80, S81, S83, S84,

S85, S88, S91, S93, S98, S100, S101, S103, S105,

S106, S107, S109, S110, S111, S112, S117, S120,

S121, S125, S130, S131, S136, S145, S147, S148,

S150, S152, S155, S159, S160, S162, S164, S165,

S167, S170

77 45,29%

Positive with empirical evaluation S3, S4, S5, S6, S9, S10, S11, S13, S14, S15, S18,

S22, S27, S29, S30, S31, S34, S36, S37, S38, S41,

S52, S54, S62, S63, S65, S66, S68, S74, S75, S76,

S79, S82, S86, S87, S89, S90, S92, S94, S95, S99,

S102, S104, S108, S115, S116, S118, S119, S122,

S123, S124, S126, S127, S128, S129, S132 ,S133,

S134, S137, S138, S139, S140, S142, S144, S146,

S149, S151, S153, S154, S156, S157, S158, S161,

S163, S166, S168,S169

78 45,88%

Negative argumentation S17, S19, S32, S64, S72, S114, S143 8 4.71%

Negative with empirical evaluation S35, S55, S71, S73, S135, S141 6 3.53%

3.4.2 Analysis and Discussion

Majority of the studies (91.07 %, 153 studies) pre-

sented positive evidence of the efﬁcacy on using edu-

cational technologies in the context of medical educa-

tion. Within these studies, 44.71% presented positive

arguments and 45.29% reported positive evidence.

These results indicate the potential of using educa-

tional technologies for medicine education. However,

negative evidence is also reported in the studies. This

result is interesting, since they are showing the crit-

icism of the scientiﬁc community publishing a work

that did not provide results as expected.

In summary, the evidence reported on studies sug-

gests that different types of educational technologies

beneﬁt medical education in several distinct domains.

A Systematic Review on the Use of Educational Technologies for Medical Education

159

However, more research is needed to better explore

promising educational technologies, for instance, ed-

ucational games, gamiﬁcation, ITS, CSCL – or the

combination of them – considering different contexts.

4 CONCLUDING REMARKS

In this work, we aimed at understanding how educa-

tional technologies have been used in medical educa-

tion through the conduction of a SLR. More specif-

ically, the major objective of this SLR was to better

understand how current educational technologies can

give support to medical education and identify in what

way this concept is being applied to this ﬁeld.

One hundred and sixty eight papers on the re-

view topic were included, in which nine types of tech-

nologies are applied in forty-two medical domains.

Among the nine types of educational technologies,

LMS is the highest technology used in medical ed-

ucation. Moreover, it is worth noting that the com-

munity is interested in using more ”intelligent“ ed-

ucational systems or promising technologies to tar-

get medical education since technologies such as ITS,

mobile learning applications, educational games and

MOOCs have also been used (despite being quite un-

derexplored so far).

As future work we intend to explore other research

questions for this SLR, for instance, to understand

which features have been incorporated in the educa-

tional technologies applied in medical education as

well as to gather metrics for evaluating the effective-

ness of the such technologies in learning improve-

ment.

REFERENCES

Dermeval, D., Paiva, R., Bittencourt, I. I., Vassileva, J., and

Borges, D. (2017). Authoring tools for designing in-

telligent tutoring systems: a systematic review of the

literature. International Journal of Artiﬁcial Intelli-

gence in Education, pages 1–49.

Easterbrook, S., Singer, J., Storey, M.-A., and Damian, D.

(2008). Selecting empirical methods for software en-

gineering research. In Guide to advanced empirical

software engineering, pages 285–311. Springer.

Ellaway, R. and Masters, K. (2008). Amee guide 32: e-

learning in medical education part 1: Learning, teach-

ing and assessment. Medical teacher, 30(5):455–473.

Jabbar, A., Militk

y, J., Kale, B. M., Rwawiire, S., Nawab,

Y., and Baheti, V. (2016). Modeling and analysis

of the creep behavior of jute/green epoxy compos-

ites incorporated with chemically treated pulverized

nano/micro jute ﬁbers. Industrial Crops and Products,

84:230–240.

Kalaian, S. A. and Kasim, R. M. (2017). Effectiveness of

various innovative learning methods in health science

classrooms: a meta-analysis. Advances in Health Sci-

ences Education, 22(5):1151–1167.

Kitchenham, B. and Charters, S. (2007). Guidelines for per-

forming systematic literature reviews in software en-

gineering.

Taveira-Gomes, T., Ferreira, P., Taveira-Gomes, I., Severo,

M., and Ferreira, M. A. (2016). What are we looking

for in computer-based learning interventions in medi-

cal education? a systematic review. Journal of medi-

cal Internet research, 18(8).

CSEDU 2019 - 11th International Conference on Computer Supported Education

160