Estimating Cognitive Overload in Mobile Applications for Decision

Support within the Medical Domain

Derek Flood

1,4

, Panagiotis Germanakos

2,3

, Rachel Harrison

Fergal Mc Caffery

and George Samaras

Dundalk Institute of Technology, Dundalk, Ireland

Department of Managements and MIS, University of Nicosia, 46 Makedonitissas Ave.,

P.O. Box 24005, 1700 Nicosia, Cyprus

Department of Computer Science, University of Cyprus, CY-1678 Nicosia, Cyprus

Oxford Brookes University, Oxford, U. K.

Keywords: Mobile, Interface Design, Complexity, Human Factors, Ehealth.

Abstract: Mobile applications have the potential to improve the quality of care received by patients from their primary

care physicians (PCP). They can allow doctors to access the information they need when and where they

need it in order to make informed decisions regarding patients’ health. They can also allow patients to better

control conditions such as Diabetes and Gaucher’s disease. However, there are a number of limitations to

these devices, such as small screen sizes and limited processing power, which can produce cognitive

overload which in turn can negatively impact upon the decision making processes. This paper introduces a

new research direction which aims to predict, during the development of mobile health care applications,

when cognitive overload is likely to occur. By identifying the user’s previous level of experience, their

working memory, the complexity of the interface and the level of distraction imposed by the user’s context,

a prediction can be made as to when cognitive overload is likely to occur.

1 INTRODUCTION

Decision making is a critical component in effective

healthcare. However in order for decisions to be

made effectively within this domain, a large volume

of information is required. Physicians typically have

less than 5 minutes to make life and death decisions

meaning that for them to be effective they require

information to be presented in an efficient manner,

sometimes within remote locations.

The proliferation of mobile devices has allowed

for software applications to be accessed in a variety

of contexts. Within the medical domain, physicians

can use these devices to access patient records,

prescribe medication and monitor patients in

hospital no matter where the physician is. In a

survey carried out by the PriceWaterhouseCoppers’

Health Research Institute (HRI)

(PriceWaterhouseCooper, 2010) it was found that

two thirds of physicians said they were using

personal devices for mobile health solutions which

are not connected to their practice or hospital IT

system. Indicatively, 56% of physicians stated that

mobile devices expedite their decision making,

while 39% claimed that the time needed for

administrative tasks is decreased significantly.

Mobile devices impose a number of limitations,

such as small screen size and limited connectivity,

which negatively affect the usability of mobile

applications (Zhang and Adipat, 2005). These

limitations, as well as complex design practices due

to a lack of design guidelines for mobile application

development will further impede decision making

within the medical domain as further cognitive load

will be placed on the user.

To minimise the impact of the user interface on

decision making within the medical domain, this

position paper proposes a method for predicting

when cognitive overload will occur. Using this

approach, application designers can determine,

during mobile application development, if cognitive

overload will occur and can redesign the interface if

necessary. The current method proposes the use of

four factors to estimate cognitive load: user

expertise, working memory, interface complexity

and level of environmental distractions.

103

Flood D., Germanakos P., Harrison R., Mc Caffery F. and Samaras G..

Estimating Cognitive Overload in Mobile Applications for Decision Support within the Medical Domain.

DOI: 10.5220/0003971501030107

In Proceedings of the 14th International Conference on Enterprise Information Systems (ICEIS-2012), pages 103-107

ISBN: 978-989-8565-12-9

 2012 SCITEPRESS (Science and Technology Publications, Lda.)

2 AIMS

The aim of this research is to design a cost effective

way for developers to determine if cognitive

overload will occur while using their mobile

application. To accomplish this, it is proposed to

determine the cognitive load of an application based

on a number of factors; the users’ level of expertise

in the area, their working memory, the complexity of

the interface and the level of distraction imposed by

the context in which the application will be used.

The following sections outline each of these factors

in more detail.

2.1 User’s Expertise

The first factor considered is the user’s level of

expertise within the task domain. It is believed that

if a user is experienced within the task domain then

he/she will be able to process more information thus

prolonging the point at which cognitive overload

will occur. In contrast to this, users without any

domain expertise will require additional cognitive

processing to understand the meaning of the

information displayed by the application, thereby

inducing cognitive overload sooner. For the

purposes of this work the user’s level of expertise

will be divided into three categories; Novice,

Intermediate and Expert. The classification of the

expected end user will be determined by the

application developer. However, there are a number

of factors that can help to determine the level of

expertise users have, for example, level of

knowledge of the task domain and experience within

the task domain.

2.2 Working Memory

Incorporating human factors in the interaction

process is always a challenge. For the scope of this

work, Working Memory (WM) has been employed,

since it is considered a vital mechanism of the

cognitive processing efficiency and has a direct

influence on the design of user interfaces. According

to Baddeley (1992), “the term working memory

refers to a brain system that provides temporary

storage and manipulation of the information

necessary for such complex cognitive tasks as

language comprehension, learning, and reasoning”.

Each individual has a specific and restricted memory

span. The aim is to decrease the possibility of

cognitive load in a mobile hypermedia environment

by altering the amount of simultaneously presented

information. This can be achieved with the

manipulation of the link or content structure of a

mobile application to achieve maximization of

comprehension capabilities while users are

performing a cognitive task.

2.3 Interface Complexity

One of the biggest challenges to the work presented

here is defining interface complexity. It is believed

that as the complexity of the interface increases so

too does the cognitive load on the user. The

complexity of the interface could be considered as

the sum of the complexity of the individual

components used. One approach to measuring the

complexity, is to assign each component type a

complexity rating, where simple components, such

as a label or radio button, are given a lower

complexity than more advanced components, such

as a graph or chart. Using these ratings the interface

complexity can then be rated as the sum of all

components on the interface. It is proposed to

determine the complexity rating that should be

assigned to each component through a series of

controlled experiments.

2.4 Distractions

When considering mobile applications, the context

in which the application is used is continually

changing. Zhang and Adipat (2005) suggested that

context is more than just the location of the user. It

also includes their interaction with nearby objects,

and environmental elements that may distract the

user’s attention. Distractions of the user’s attention

can inhibit their ability to process information and

therefore introduce extra cognitive load. The level of

distraction experienced by the user will be divided

into three categories; Low (a quiet area where a user

can concentrate completely on the task), Medium (an

environment in which the user may not be distracted

but will be subjected to some background noise) and

High (a noisy environment where the user will be

distracted by either additional tasks or by other

individuals).

2.5 Predicting Cognitive Overload

Using the factors identified above, it is believed that

a reliable estimation can be made as to when

cognitive overload is likely to occur. Through

experimentation, outlined below, this work aims to

investigate under each combination of the factors

previously identified when cognitive overload will

occur. A decision table will then be produced as a

ICEIS2012-14thInternationalConferenceonEnterpriseInformationSystems

104

reference guide for mobile medical decision support

application developers. When deciding between

different interface designs, the developers can refer

to this decision table to determine if each design will

produce cognitive overload which would reduce the

users’ experience of the application. This approach

will allow developers to determine earlier in the

development cycle, and in a less costly manner, if

the proposed design will provide a usable experience

for the end-user.

3 HYPOTHESES

In order to validate the above approach a number of

hypotheses are proposed. The first step will be to

determine if each of the factors outlined above do

impact cognitive load. Hypotheses 1 to 4 have been

formulated to investigate this. H1: A user’s expertise

is inversely proportional to cognitive load; H2:

Working memory is correlated to cognitive load;

H3: Cognitive load is proportional to interface

complexity; H4: Cognitive load is proportional to

the level of distraction to the user’s attention. The

next step of this research is to then determine the

impact of cognitive overload on effective decision

making. This is captured through hypothesis 5 – H5:

High cognitive load hinders decision making. Once

these hypotheses have been explored the next step is

to determine the combination of the identified

factors with which cognitive overload will occur.

The following research question will be used to

guide this work – RQ1: When does cognitive

overload occur in terms of User expertise, working

memory, interface complexity and level of

distraction? To test the hypotheses and answer the

research question, a series of controlled experiments

will be conducted.

4 FUTURE WORK

In order to determine when cognitive overload will

occur, the cognitive load on a user first needs to be

established. The level of cognitive load on a user is

reflected in the time it takes a user to complete a

given task effectively. A longer time to complete a

given task indicates a higher cognitive load. In

addition to this the effectiveness of the user at

completing the task will also be considered. In each

of the experiments presented below the effectiveness

and efficiency of the user to complete the task will

be taken as an indirect measure of cognitive load

(Oviatt, 2006). The point at which cognitive

overload occurs will be determined through a

structured questionnaire that will be provided to

participants at the end of each trial.

4.1 Hypothesis 1: A User’s Expertise is

Inversely Proportional to Cognitive

Load

For the first experiment twenty participants will be

recruited and divided into two groups, Novice and

Expert, based on their expertise with interpreting

blood pressure readings. The participants will be

shown five readings and asked in each case to state

whether the patient has high, low or normal blood

pressure. For each reading the effectiveness

(accuracy of their decisions) and efficiency (time

taken for participants to make a decision) will be

recorded. When the participant has completed all 5

trials they will then be given a questionnaire to

evaluate the perceived level of cognitive load they

experienced during the trial – Dependent Variables:

Effectiveness, efficiency, and perceived cognitive

load. Independent Variables: User Expertise.

4.2 Hypothesis 2: Working Memory is

Correlated to Cognitive Load

All participants will go through a series of WM span

tests (identifying the visual memory and central

executive/verbal storage) using a Web-based

environment. At first, the WMTB-C (Pickering &

Gathercole, 2001) will be used for measuring both

the central executive function and the verbal storage

ability (phonological loop span), providing an

indication of users’ WM ability. Secondly, a WM

test to measure the visuo-spatial sketchpad will be

used. In total, users are classified as “low”,

“normal”, or “high” accordingly, with respect to

their ability, based on a calculated aggregated score

of all tests. Once WM span has been identified, users

will interact with a number of mobile environments

varying in complexity. Navigation time as well as

accuracy on reaching their expected cognitive target

will be measured and calculated along with the value

of their WM levels. – Dependent Variables:

Effectiveness, Decision time. Independent

Variables: Working Memory Span.

4.3 Hypothesis 3: Cognitive Load is

Proportional to Interface

Complexity

During this experiment the participants will be

EstimatingCognitiveOverloadinMobileApplicationsforDecisionSupportwithintheMedicalDomain

105

subjected to two alternative interfaces, one with high

interface complexity (a graphical representation) and

one with low interface complexity (a textual

representation), displaying a patient’s blood

pressure. The participants will then be asked to

evaluate whether each of the patients has high, low

or average blood pressure. As with previous

experiments, effectiveness and efficiency will be

used to determine the level of cognitive load on the

user. To counter any learning effects that may occur

between trials, participants will be presented the

interfaces in a random order – Dependent Variables:

Effectiveness, Decision time, and perceived

cognitive load. Independent Variables: interface

complexity.

4.4 Hypothesis 4: Cognitive Load is

Proportional to the Level of

Distraction to the User’s Attention

In this experiment each participant will be again

asked to judge whether a patient has high, low or

average blood pressure in each of five cases. In this

experiment however, the participants will be placed

in two environments; a quiet office with no

distractions and an environment with a high level of

distraction. As with previous experiments,

effectiveness and efficiency will be used in

conjunction with a subjective questionnaire to

determine the level of cognitive load on the user. To

counter any learning effects that may occur between

trials, the participants will be presented the

interfaces in a random order – Dependent Variables:

Effectiveness, Decision time, and perceived

cognitive load. Independent Variables: level of

distraction.

4.5 Hypothesis 5: Cognitive Load is

Correlated to Effective Decision

Making

Using the data in each of the previous experiments

the participants’ effectiveness will be evaluated in

terms of the perceived cognitive load. It is believed

that participants’ effectiveness and efficiency will be

reduced as the cognitive load is increased. Statistical

methods, such as Pearson’s correlation test, will then

be used to determine the strength of the relationship

between cognitive load and effectiveness and

efficiency. For each participant in each study, the

number of correct determinations made in addition

to the time taken will be rated against the level of

cognitive load experienced by the participant.

RQ1: When will cognitive overload occur in

terms of User expertise, working memory,

interface complexity and level of distraction?

To answer this research question, a series of

experiments will be conducted which will examine

each combination of the previously identified

factors. In each experiment one instance of each

factor will be combined and the cognitive overload

will be examined. Upon completion of these studies

it will be possible to create a decision table, similar

to the one presented below, estimating when

cognitive overload will occur.

Table 1: Example cognitive overload decision table.

User

Expertise

Working

Memory

Interface

complexity

Level of

Distraction

Cognitive

Overload

Expert High Low Low No

Expert Low High High Yes

5 CONCLUSIONS

Decision making is a critical component of health

care. Ineffective decision making can have serious

consequences, sometimes resulting in fatalities. An

important component of effective decision making is

the ability to access relevant information in an

efficient manner. Through the use of mobile

technologies, PCP can access information almost

anywhere when they need it. Mobile applications,

however, suffer from a number of usability issues

which negatively impact a user’s cognitive load,

which will reduce the effectiveness of decisions that

are made with the support of these devices.

This work proposes the development of a

decision table that will support mobile application

developers in predicting if cognitive overload will

occur with a particular application.

ACKNOWLEDGEMENTS

This research is supported by the Science

Foundation Ireland (SFI) Stokes Lectureship

Programme, grant number 07/SK/I1299, the SFI

Principal Investigator Programme, grant number

08/IN.1/I2030 (the funding of this project was

awarded by Science Foundation Ireland under a co-

funding initiative by the Irish Government and

European Regional Development Fund), and

supported in part by Lero - the Irish Software

Engineering Research Centre (http://www.lero.ie)

grant 03/CE2/I303_1, the EU project CONET

ICEIS2012-14thInternationalConferenceonEnterpriseInformationSystems

106

(INFSO-ICT-224053) and by the project smarTag

(Internal funded projects of the University of

Cyprus).

REFERENCES

Baddeley, A. 1992. Working Memory, Science, Vol. 255,

pp. 556—559.

Bekker, H, Thornton, J G., Airey, C M., Connelly, J B.,

Hewison, J., Robinson M B., Lilleyman, J.,

MacIntosh, M., Maule, A J., Michie, S., Pearman, A

D. 1999. Informed Decision Making: An Annotated

Bibliography and Systematic Review, Health

Technology Assessment, Vol. 3: No. 1.

García, E., Martin, C., García-Cabot, A., Harrison, R.,

Flood, D. 2011. Systematic Analysis of Mobile

Diabetes Management Applications on Different

Platforms. USAB 2011: 379-396.

IEC. 2007. IEC 62366 - Medical devices -- Application of

usability engineering to medical devices.

Oviatt, S. 2006. Human-centered Design meets Cognitive

Load Theory: Designing Interfaces that help People

Think, Proceedings of the 14th annual ACM

international conference on Multimedia, October 23-

27, Santa Barbara, CA, USA

Pickering, S., Gathercole, S. 2001. The Working Memory

Test Battery for Children, The Psychological

Corporation.

PriceWaterhouseCooper Health Research Institute. 2010.

Healthcare unwired: new business models delivering

care anywhere. Available from: http://www.pwc.com/

us/en/health-industries/publications/healthcare-unwire

d.jhtml

Zhang, D., Adipat, B. 2005. Challenges, Methodologies,

and Issues in the Usability Testing of Mobile

Applications, International Journal of Human-

Computer Interaction 18(3): 293 - 308.

EstimatingCognitiveOverloadinMobileApplicationsforDecisionSupportwithintheMedicalDomain

107