DEVELOPMENT OF THE PROP CHART, A NEW VISUAL

MODEL TO EVALUATE THE EFFECTIVENESS OF TRAINING

WITH COMPUTERISED MANIKINS

A. F. Fransen

and S. G. Oei

1,2

Department of Obstetrics and Gynaecology, Máxima Medical Centre, Veldhoven, The Netherlands

Department of Electrotechnical Engineering, Eindhoven University of Technology, P. Eindhoven, The Netherlands

Keywords: Medical Simulation, Computerised Manikins, Training, Education.

Abstract: High-fidelity manikins and new computerised simulation methods play a key role in medical training.

Despite the on-going developments in computer technologies and widespread use of computerised

simulation methods, the most effective use of these technologies in medical training is still ambiguous. To

give insight into the effectiveness of medical simulation training of health care professionals and to design

more effective trainings, we created the Prop chart. This chart is initially developed for the training of multi

professional teams using medical simulation. A literature search for evidence based features of effective

medical simulation was combined with the opinion of experts in focus group discussions. Ten features of

medical simulation that contribute to effective learning were identified and were used in the Prop chart. The

experts agreed on the convenience of the Prop chart to evaluate and design medical simulation training

programs. Future research will focus on the applicability of the Prop chart for fields outside the medical

world and other new training methods like serious gaming.

1 INTRODUCTION

High morbidity and mortality rates, new

technologies and the desire to assess medical

professionals were main reasons to introduce

medical simulation. Besides, the importance of

medical team training using simulation is

underscored in several health care reports to improve

the number of preventable errors (Lewis and Drife,

2004); (CESDI, 1997); (Committee on Quality

Health Care in America, 2001).

For these reasons, the last two decades medical

simulation has had a growing interest and

computerised manikins have been developed,

revised and used for these purposes. This increase of

interest in medical simulation has simultaneously

brought simulator technology to a higher level and

has resulted in the development of new methods in

simulation. For example, one of the newest methods

which is currently considered for medical education,

is serious gaming. Serious gaming makes it possible

to transfer training in hospitals or simulation centres

to home settings. This in turns allows the adjustment

of the training frequency and intensity to

individual’s learning curves, which is more likely to

contribute to effective learning.

Beside simulation methods, new learning

strategies are developed. Traditional learning

strategies include large group lectures, tutorials,

team training and individual training with or without

instructor. A currently attractive medium for

distance learning is e-learning. Issenberg et al

underscored the importance of adaptability of

simulation to several learning strategies (Issenberg

et al., 2005). The chosen learning strategy or

combination of strategies must be appropriate to

achieve the training goals.

An important theory in education is the theory of

deliberate practice. Deliberate practice is proven to

be a very effective way to acquire skills. It

comprises a set of principles which lead to effective

learning and is based on learner’s engagement in the

accomplishment of learning goals (McGaghie et al.,

2011); (Ericsson, 2004).

Unless growing evidence about medical

simulation, development of new simulators and

learning strategies, a guidance to construct an

effective training design is lacking. Our objective

was to provide more insight in the design and

effectiveness of medical simulation training

programs by creating a visual model. This visual

model will represent the most important features in

287

F. Fransen A. and G. Oei S..

DEVELOPMENT OF THE PROP CHART, A NEW VISUAL MODEL TO EVALUATE THE EFFECTIVENESS OF TRAINING WITH COMPUTERISED

MANIKINS.

DOI: 10.5220/0003924402870290

In Proceedings of the 4th International Conference on Computer Supported Education (CSEDU-2012), pages 287-290

ISBN: 978-989-8565-07-5

 2012 SCITEPRESS (Science and Technology Publications, Lda.)

effective learning. The components of deliberate

practice will be This model is initially developed for

the training of multi professional medical teams.

2 METHODS

The visual model is based on evidence-based

features of an effective medical simulation. To

determine these features, we performed a literature

search in MEDLINE and the Cochrane database,

using the following search terms: medical

simulation, team training, computer simulation,

features, uses, effectiveness, effective learning.

From the literature, the most important features

that contribute to effective medical simulation are

chosen. These features are used in the visual model.

After selecting these items, a final shape of the

visual model was chosen.

The items can be applied in different

manifestations to a training. For objective

classification of these different forms of each item,

the items will be described and divided into sub-

items. Defining these sub-items was merely based

on evidence, but conclusive evidence was not

available for all items. Eventually we combined this

evidence-based knowledge with expert opinions.

After identifying the evidence based items and

potential sub-items, we incorporated expert opinions

in the model. To attain opinions of experts in the

field of both medical simulation and obstetrics, we

organised two focus groups in London and

Eindhoven (The Netherlands). Experts were invited

to share their thoughts and ideas about effective

features of medical simulation. The focus group

sessions were led by a moderator. Each member

completed an application form, identifying their

occupation and experience in medical simulation.

The defined, evidence-based, features of medical

simulation that lead to effective learning, were

presented. A group discussion was yielded by

assigning ratings to the different sub-items.

3 RESULTS

The literature search resulted in 143 articles. The

abstracts were used for selecting useful articles.

Only one article gave an overview of the most

important features of medical simulation that lead to

effective learning. This Best Evidence in Medical

Education (BEME) review by Issenberg et al.

identified the 10 most important items in effective

medical training (Figure 1) (Issenberg et al., 2005).

Figure 1: The 10 most important features and uses of high-

fidelity medical simulation that lead to effective learning.

Because of the high quality and the perfect match

with the addressed objectives of our model, these

items were used for designing the model. As stated

before, each item can be applied to training in

different ways. For example, there are several ways

of providing feedback, like self-evaluation, peer-

assessment, feedback by instructor and feedback by

video playback. The sub-items represent the

different manifestations of one of the key-items.

Since using any kind of training is better than no

training at all, the first level of each item

corresponds with a situation without any training.

Figure 2: The prop chart.

Based on these evidence-based items and their

sub-items, we created the Prop(eller) chart (Figure

2). The primary element is a circle which is divided

in 10 similar pieces, corresponding to the identified

features in effective training. Depending on the

extent to which a feature (categorised in sub-items)

will contribute to an effective learning, the piece will

grow from the centre (0%) to the outside border of

the circle (100%). A mean outcome of a design can

be calculated by summation of the total surface area

of all items. The border corresponds with the most

optimal training design.

CSEDU2012-4thInternationalConferenceonComputerSupportedEducation

288

Table 1a: Item 1-5 defined by Issenberg et al. Each item is divided in sub-items. This will give a direction for rating to

which extent the used item will contribute to an effective learning.

Feedback

Repetition

Curriculum

Integration

Difficulty

Range

Learning Strategies

Level 1 - - - - -

Level 2 No feedback Until level of novice

% of staff is trained

in edu-cational

program

1 difficulty level is

trained

1 learning strategy

Level 3 Solely self-evaluation

Until level of advanced

beginner

% of staff is trained

2 difficulty levels are

trained

2 learning strategies

Level 4 Peer-assessment Until level of competency % of staff is trained

3 difficulty levels are

trained

3 learning strategies

Level 5 Feedback by instructor Until level of proficiency % of staff is trained

4 difficulty levels are

trained

4 learning strategy

Level 6

Feedback with video

playback by instructor

Until level of expert Total staff is trained

5 levels of difficulty,

ranging from novice to

expert level

>4 learning strategies

Table 1b: Item 6-10 defined by Issenberg et al. Each item is divided in sub-items. This will give a direction for rating to

which extent the used item will contribute to an effective learning.

Clinical Variation Controlled Environment

Individualized

learning

Pre-defined outcomes Simulator validity

Level 1 - - - - -

Level 2

No variation within

clinical scenario

During patient care Passive observation

Not based on pre-

defined goals

Very low simulator

validity

Level 3

Very limited variation

within scenario

% of capabilities of

medical simulation centre

% of active

participation in

scenarios

% of training which is

based on pre-defined

goals

Low simulator validity

Level 4

Limited variation within

scenario

% of capabilities of

medical simulation centre

% of active

participation in

scenarios

% of training which is

based on pre-defined

goals

Medium simulator

validity

Level 5

Plenty variation within

scenario

% of capabilities of

medical simulation centre

% of active

participation in

scenarios

% of training which is

based on pre-defined

goals

High simulator validity

Level 6

Unlimited variation within

scenario

All capabilities of a

medical simulation centre

Active participation

in all scenarios

Completely based on

pre-defined goals

Perfect simulator

validity

For obtaining expert opinions, three interactive focus

group sessions were kept. In total 20 experts

participated (Table 2). In the end all items and sub-

items were discussed. The experts agreed on the

convenience of the PROP chart to evaluate and

design a medical simulation training.

4 DISCUSSION & CONCLUSIONS

One of the advantages of medical simulation Is the

adaptability of medical simulation to deliberate prac-

tice. Incorporation of this theory in the design of a

medical simulation training will contribute to

effective learning. Since the 10 items identified by

Issenberg et al. cover all principles of deliberate

practice, using these items for the design or

evaluation of a training will logically lead to an

effective training.

To overcome the shortcoming evidence for the

classification of several items in sub-items, the use

of expert opinion was an appropriate tool. Opinions

of experts from different continents were included in

the chart. Remarks on the different forms of the

DEVELOPMENTOFTHEPROPCHART,ANEWVISUALMODELTOEVALUATETHEEFFECTIVENESSOF

TRAININGWITHCOMPUTERISEDMANIKINS

289

features were made. The next step in the

classification of these sub-items will be the

designation of percentages to each of them.

Table 2: Fields of interest of participating experts in focus

groups.

Occupation & Experience in medical simulation N

(n)

Medical professional & medical trainer

Included medical professions:

‐ Obstetrician

‐ Gynaecologist

‐ Anaesthesiologist

‐ Midwife

Residents using/studying medical simulation 2

Director of medical simulation centre 2

Medical engineer (developing physiological models

for computerised simulation)

Simulation technician 2

Total 20

The experts, included in the focus groups, agreed

on the convenience of this Prop chart for designing

and evaluating medical simulation courses. Future

plans for the Prop chart will be the application to

new simulation methods, like serious gaming. In

addition this chart could be used for designing

medical simulation courses in different medical

fields, like anaesthesiology, cardiology, paediatrics

and surgery. But also using the Prop chart in non-

medical fields could be a possibility. However,

before establishing these goals, the Prop-chart will

need additional investigation and validation of the

sub-items.

REFERENCES

G. Lewis, J. Drife. Why mothers die 2000-2003. The Sixth

Report of the Confidential Enquiries into Maternal

Deaths in the United Kingdom. London (UK): RCOG

Press; 2004.

CESDI 6th annual report. London (UK): RCOG

Publishing; 1997.

Committee on Quality Health Care in America. Crossing

the quality chasm: a new health system for the 21st

century. Institute of Medicine. Washington DC:

National Academy Press; 2001.

L. D. de Wit-Zuurendonk, S. G. Oei. Serious gaming in

women’s health care. BJOG 2011;118 (Suppl.3):17-

21.

S. B. Issenberg, W. C. McGaghie, E. R. Petrusa, D. L.

Gordon, R. J. Scalese. Features and uses of high-

fidelity medical simulations that lead to effective

learning: a BEME systematic review. Medical Teacher

2005, 27: 10-28

W. C. McGaghie, B. Issenberg, E. R. Cohen, J. H. Barsuk,

DB Wayne. Does simulation-based education with

deliberate practice yield better results than traditional

clinical education? A meta-analytic comparative

review of the Evidence. Academic Medicine, 2011, vol

86, no 6: 706-711

K. A. Ericsson. Deliberate practice and the acquisition and

maintenance of expert performance in medicine and

related domains. Academic Medicine, 2004, vol 79:

70-81.

R. M. Fanning, DM Gaba. The role of debriefing in

simulation-based learning. Simulation in Healthcare

2007; 2.

C. L. Caraccio, B. J. Benson, L. J. Nixon, P. L. Derstine.

From the educational bench to the clinical bedside:

translating the Dreyfus Development Model to

learning of clinical skills. Academic Medicine 2008,

83.

CSEDU2012-4thInternationalConferenceonComputerSupportedEducation

290