Why My Grandfather Finds Difficulty in using Ehealth: Differences

in Usability Evaluations between Older Age Groups

Marijke Broekhuis

1,2

, Lex van Velsen

1,2

, Silke ter Stal

1,2,

Jordi Weldink

and Monique Tabak

1,2

Roessingh Research and Development, Roessinghsbleekweg 33B, Enschede, The Netherlands

Faculty of Electrical Engineering, Mathematics and Computer Science, University of Twente, Enschede, The Netherlands

Keywords: eHealth, Older Adults, Usability Evaluations, SUS, Thematic Analysis.

Abstract: Many studies emphasize the need for more in-depth analysis of how age-related barriers influence the user

acceptance of eHealth. In this study, we elaborate on existing work in this field by identifying how age

differences affect usability evaluations in eHealth. We examined how older adults between 55-64 years

(n=10) evaluated the usability of a game-based eHealth application in comparison to adults of 65 years and

older (n=19). A concurrent think aloud protocol and the System Usability Scale (SUS) were administered to

29 participants. Usability issues were elicited from the think aloud transcripts and benchmark scores were

obtained from the SUS. We conducted both: (1) a statistical analysis on the amount of usability issues and

SUS score; and (2) a thematic analysis of the usability issues. Our study found that the 55-64 age group

encountered significantly fewer usability issues compared to the 65+ age group. Furthermore, the thematic

analysis revealed that while both groups had similar problems regarding the ‘Navigation & Structure’ category

of the game-based eHealth application, there was much variation in the other usability categories of ‘Content

& Information’, ‘Design & Presentation’ and ‘Other’. Our results can improve the development of eHealth

that support healthy ageing.

1 INTRODUCTION

Many new eHealth systems focus on supporting

healthy ageing. A major health risk facing older

adults is frailty: The decline in cognitive and physical

functions that can lead to recurrent falls, hospital

visits and even death (Fried et al., 2001). Multiple

studies have found that the symptoms of frailty can be

slowed down by staying physically active (Liu and

Fielding, 2011; Theou et al., 2011) and engaging in

cognitive training (Ng et al., 2015). The group of

older adults continues to rise – since 1990 there has

been an increase of 62% of people aged 65 years or

older (CBS, 2018) – and consequently the risks of

frailty increases. This can impose a heavy load on

health care systems. EHealth can support and relieve

health care systems by motiving older adults to stay

active by providing online physical or cognitive

training. Furthermore, whereas a health professional

can only treat a limited number of patients, eHealth

can be implemented to reach for larger groups of

older adults.

However, a recurring problem in eHealth is the

successful implementation of eHealth in the daily

lives and routines of people and health care processes.

eHealth systems should be tailored to the specific

environments and skills of the intended end-user

groups to maximize the probability of successful

implementation (Broens et al., 2007). One important

pre-requisite for the acceptance and eventual

implementation of eHealth is good usability (Broens

et al., 2007; Narasimha et al., 2017). To measure

usability of systems and to identify usability

problems, it is important to conduct usability

evaluation tests involving potential end-users. In

many studies, eHealth systems are evaluated on their

usability among the target end-user group. However,

for systems to be truly effective, they must be user-

friendly for various groups of people and be able to

compensate for variability in, for example, socio-

economic status, health literacy, technology literacy

and chronic care needs (Kreps and Neuhauser, 2010;

Lyles and Sarkar, 2015).

Several factors affect any usability evaluation.

First, adults of 65 years or older have fewer computer

skills than younger generations (Chen and Persson,

2002; Gatto and Tak, 2008). However, eHealth

systems that take into account lower computer

Broekhuis, M., van Velsen, L., ter Stal, S., Weldink, J. and Tabak, M.

Why My Grandfather Finds Difﬁculty in using Ehealth: Differences in Usability Evaluations between Older Age Groups.

DOI: 10.5220/0007680800480057

In Proceedings of the 5th International Conference on Information and Communication Technologies for Ageing Well and e-Health (ICT4AWE 2019), pages 48-57

ISBN: 978-989-758-368-1

literacy could be perceived as cumbersome or even

unusable by the younger generation who are more

experienced with computers, tablets, and

smartphones. Second, with higher age the risk of

multimorbidity, defined as the prevalence of two or

more chronic conditions, increases (Salive, 2013).

Living with a chronic illness can induce higher levels

of stress (McEwen, 2008), that can affect how easily

users can perform tasks within a system or the types

of health information they need. Third, the study of

Wildenbos et al. (2018) identified cognitive (e.g.

working memory, spatial cognition), physical (e.g.

flexibility of joints, speed of performance),

perception (e.g. visual accommodation, colour

vision), and motivational (e.g. trust in own abilities,

efficiency in benefits) barriers that older adults often

experience when using a system. The first three are

the result of high age and can affect user interaction.

For example, people who have limited fine motor

skills could find difficulty in clicking on small

elements in a graphical user interface (GUI). Also,

motivational barriers can be different for the older

and younger generations (Wildenbos et al., 2018).

The study of Morey et al. (2017) describes how

younger participants saw more benefits in an app

intended for heart failure patients than older adults.

This can affect how users perceive the effectiveness

and usefulness of a system. Last, although older

adults often perceive how technology could benefit

them in healthy ageing, the technology should not be

unacceptably intrusive in either their homes and lives

(Jacelon and Hanson, 2013; Peek et al., 2016).

Younger generations are far more familiar with

technology and are likely to have a more positive

view on how technology can be integrated in their

daily routines.

In this study, we aimed to discover if and how

usability evaluations differ between age cohorts. We

conducted a usability evaluation of an eHealth

system, a game-based eHealth application called

‘Stranded’, between two groups: (1) adults of 55-64

years, and (2) adults of 65 years or older. The goal

was to examine if there are differences in the usability

perceptions between the two age groups in the types

and severity of usability issues.

2 METHODS AND MATERIALS

2.1 Case

In eHealth interventions, the challenge is to establish

and maintain engagement of the user for long-term

use of the technology, which is needed to establish the

targeted health goals of the older adult. ‘Stranded’ is

a game-based eHealth application that aims to engage

the older adult on the long term by using gaming

technologies. To the user, this game-based

application can be seen as an alternative interface for

the original eHealth application (called tele-

rehabilitation). The intended target group for the

application is aged 65-75 years, with sufficient

computer literacy to independently use a mobile

device or pc and with an interest in digital games. In

the design process, game design and the selection of

game elements were fitted to the specific preferences

and characteristics of the intended target group. This

resulted in a set of game design guidelines (as

described in de Vette et al., submitted) for older

adults, with the following characteristics: moderate—

to-high novelty (e.g. story line, enabling exploration),

moderate-to-high dedication (e.g. enabling

achievement, learning and mastery), low Discord and

Threat (e.g. relaxed atmosphere, not triggering

negative emotions) and low Social (i.e. solo player).

Figure 1: Screenshot of the Stranded home screen. By

clicking on one of the cabins, the user goes to the tele-

rehabilitation portal.

When Stranded is started for the first time, an opening

animation introduces the backstory. Every session

starts with the choice for the original or game-based

eHealth application. In the game, the main character

– a female explorer – is stranded on a deserted island

after her ship was caught in a storm. From the beach,

the player can explore the island and access huts.

These huts are connected to the training modules and

linked to the original tele-rehabilitation. In addition,

the player can go to a virtual crop field, visit the rest

of the island and play mini-games there, or find out

about a wooden quay where a boat it built. Items wash

up on the beach in bottles, which contents are adapted

based on the outcomes in the training module. For

example, the bottles can contain items that can be

stored in a trophy hut or seeds that can be planted in

the crop field. The locations on the island map show

Why My Grandfather Finds Difﬁculty in using Ehealth: Differences in Usability Evaluations between Older Age Groups

levels that can be played, which can be opened by

completing a training schedule in the huts. In these

locations, several mini-games can be played. After

finishing a level, the player receives a part of a boat.

After finishing all levels (corresponding to the

finishing of the 12 week rehabilitation programme),

your boat is built and you can leave the island.

2.2 Participants

Participants were eligible for participation in this

study if they fitted within one of the two age groups

and if they had basic computer skills, such as sending

an e-mail. We recruited participants through a Dutch

panel for adults aged between 55 and 64 and we

collaborated with local geriatric physiotherapy

practices to recruit participants aged 65 or older.

2.3 Study Procedure

Before participation, all respondents completed and

signed an informed consent form. First, participants

were asked about their demographics after which they

were given five tasks to complete within the game-

based eHealth application. The participant had five

minutes to fulfil each task. During these tasks, they

had to verbalize their thoughts. After completing

these tasks, they filled out the System Usability Scale

(SUS) (Brooke, 1996).

2.4 Data Analysis

Audio recordings and screen-capture recordings were

made during the usability evaluation sessions and

transcribed. Usability issues were identified from the

Table 1: Overview of the critical issues and corresponding usability category.

Age groups

Usability issues

N&S

C&I

D&P

Other

1.1

55-64 / 65+

The user does not know the purpose of the cabins in the

home-screen.

1.2

55-64 / 65+

The user cannot find the entrance to the kitchen.

1.3

55-64 / 65+

The user has difficulty distinguishing clickable and non-

clickable elements in the interface.

1.4

55-64 / 65+

The user cannot find the ingredient list in the kitchen

1.5

55-64 / 65+

The user does not understand that in the island overview

interface, each circle represents a mini game.

1.6

55-64 / 65+

The user cannot find the direction sign to the island

overview in the home-screen

1.7

55-64 / 65+

The system does not offer the user any support for entering

special characters while logging in

1.8

55-64 / 65+

The user believes the introduction of a physical exercise is

the actual explanation of the exercise.

1.9

55-64 / 65+

The interface does not show where the physical exercises

can be found (e.g. through the physical exercise cabin)

2.1

65+

The user wants to leave the game because he or she cannot

find the elements he or she is looking for (e.g. exercise, e-

mail, mini game)

2.2

65+

The user has difficulty understanding the connection

between the various gaming elements

2.3

65+

The system does not provide an option to erase incorrect

text from the entry boxes in the login screen without using

the keyboard

2.4

65+

The user does not understand the connection between the

gaming interface and the tele-rehabilitation portal

2.5

65+

The user does not understand the purpose of the play button

in the exercise video. He or she believes this button is used

to go to the next exercise.

ICT4AWE 2019 - 5th International Conference on Information and Communication Technologies for Ageing Well and e-Health

transcripts and classified as a minor, serious, or

critical, following the procedure by Van Velsen et al.

(2011). The following definitions of Duh et al. (2006)

were used for the severity classification:

 Minor issue: Occurred infrequently among the

participants and/or the problem only increased

task completion time slightly;

 Serious issue: Occurred frequently among the

participants and/or the problem severely

increased task completion time;

 Critical issue: Occurred when all participants

had the same problem and/or the problem

prevented participants from completing tasks.

The identification of the usability issues and

determination of the severity of each issue, was first

performed by one coder (MB). A second coder (LvV)

independently analysed a subset of the data.

Discrepancies between de codebooks were discussed

and overcome, after which the first coder (MB) again

coded the whole codebook, and finally the second

coder (LvV) reviewed the codebook.

Next, based on the framework proposed Van der

Geest (2004), each usability issue was grouped into

one of the following four categories: (1) Navigation

& Structure; (2) Content & Information; (3) Design

& Presentation; and (4) Other. This categorization

was performed by a pair of two researchers (MB &

StS). A third researcher (LvV) checked the final

categorization. After discussions between the

researchers about disparities, final alterations to the

categorization of the usability issues were made.

Table 1 shows a subset of the codebook, the critical

issues per age group with corresponding usability

category.

2.5 Statistical Analysis

Tests for normality indicated that normal

distributions could not be assumed. Therefore, the

data was analysed by applying non-parametric

methods. Mann-Whitney U tests were calculated to

measure: (1) if there was a significant difference

between the average SUS scores of the two age

groups; (2) if there were significant differences

between the number of minor, serious, and critical

issues between the two age groups; and (3) if there

were significant differences between unique minor,

serious, and critical issues between the two age

groups.

3 RESULTS

3.1 Demographics

In total, 29 older adults participated. Table 2 gives a

complete overview of the demographics of both age

groups. Ten respondents aged 55-64 participated in

the study. Six of them (60%) were male, and four

(40%) were female. The average age was 59 years.

Their educational background included lower

vocational education (20%), vocational education

(40%), and higher vocational education (40%).

A total of 22 older adults of 65+ years agreed to

participate, of which 19 completed the study. Twelve

of them were male (63.2%) and seven were female

(36.8%). Their mean age was 74 years and their level

of education consisted of elementary education

(5.3%), lower vocational education (42.1%),

vocational education (26.3%), and higher vocational

education (21.1 %). However, one participant did not

mention her educational background.

3.2 SUS Scores

With an average SUS score of 42.0, the usability of

Stranded was found to be just below the acceptability

threshold of the SUS scoring scale for 55-64 age

group. The participants of the 65+ age group gave the

game-based eHealth application ‘Stranded’ a SUS

score of 26.7, which means that the usability of

Stranded is unacceptably poor.

3.3 Usability Issues

There were in total 398 usability issues found across

all participants: 111 usability issues in the 55-64 age

group and 287 usability issues in the 65+ age group.

We deduplicated usability issues across participants

per age group, which resulted in 105 unique usability

issues. Finally, we examined which usability issues

were being present for both age groups and again

removed duplicates. This resulted in 26 unique

usability issues that were found among participants in

both age groups; 44 issues that were only found in the

65+ age group, and nine issues that were only found

in the 55-64 age group. Table 3 shows the number of

minor, serious, and critical usability issues for each

age group.

The 55-64 age group yielded a total of 12 (34.2%)

minor, 14 (40%) serious, and 9 (25.7%) critical

unique usability issues. Examples of minor issues are

‘Not sure how to use the button game modus’, and

‘Restart-button is mistaken for a start-button’.

Why My Grandfather Finds Difﬁculty in using Ehealth: Differences in Usability Evaluations between Older Age Groups

Examples of serious issues are ‘Physical exercises do

not provide information about the length of each

exercise’, and ‘Difference between gaming elements

and computer icons (e.g. game buttons and windows-

icons) is unclear’. Examples of critical issues are

‘Direction signs to the crop field cannot be found in

the home page of the game-based eHealth

application’, and ‘The purpose of the cabins in the

home-screen is unclear for the user’.

The think aloud protocol elicited 32 (45.7%)

minor, 24 (34.3%) serious, and 14 (20%) critical

usability issues for the 65+ age group. Minor issues

included ‘Avatar looks like a male rather than a

female character’, and ‘Dislikes the music’. Serious

issues were problems such as ‘The help-page

provides insufficient information to support the

playing of the game’, and ‘The application does not

explain how to build the boat’ (e.g. through

performing the physical exercises). Issues such as

‘Connection between the tele-rehabilitation portal

and the gaming interface is unclear’, and ‘The gaming

interface provides insufficient information for the

user about where the physical exercises can be

found’, were classified as critical issues.

Table 2: Demographics (age, education, technology usage) of the 55-64 age group and the 65+ age group.

Age group

Sex

Age

Education

Technology usage

55-64

Vocational

PC/Laptop, Smartphone, Tablet

Higher vocational

PC/Laptop, Smartphone

Vocational

PC/Laptop, Smartphone, Tablet

Vocational

PC/Laptop, Smartphone, Tablet

Higher vocational

PC/Laptop, Smartphone, Tablet

Higher vocational

PC/Laptop, Smartphone, Tablet

Lower vocational

PC/Laptop, Smartphone, Tablet

Lower vocational

PC/Laptop, Smartphone

Higher vocational

PC/Laptop, Smartphone

Vocational

PC/Laptop, Smartphone

65+

Lower vocational

PC/Laptop, Smartphone, Tablet

Vocational

PC/Laptop

Higher vocational

PC/Laptop, Smartphone, Tablet

Lower vocational

Smartphone

Lower vocational

PC/Laptop, Smartphone

Vocational

PC/Laptop, Smartphone

Higher vocational

PC/Laptop, Smartphone, Tablet

Vocational

PC/Laptop, Smartphone

Higher vocational

PC/Laptop

Lower vocational

PC/Laptop, Smartphone, Tablet

Elementary education

PC/Laptop, Smartphone, Tablet

Higher vocational

PC/Laptop, Smartphone

Lower vocational

Smartphone, Tablet

Lower vocational

PC/Laptop, Smartphone

n.a.

PC/Laptop

Lower vocational

Smartphone, Tablet

Vocational

PC/Laptop, Smartphone, Tablet

Vocational

PC/Laptop, Smartphone

Lower vocational

PC/Laptop, Tablet

ICT4AWE 2019 - 5th International Conference on Information and Communication Technologies for Ageing Well and e-Health

3.4 Statistical Differences between Age

Groups

Mann-Whitney U-test scores were computed between

the mean scores of the SUS and the numbers of minor,

serious, and critical usability issues. When

considering the SUS scores between the two age

groups, the 65+ age group (Mdn = 27.5, IQR = 10-

42.5), significantly differed from the 55-64 age group

(Mdn = 38.8, IQR = 30.6-48.8), U = 52, p = .05. Also,

the number of serious issues in the 65+ age group

(Mdn = 8, IQR = 6-9), significantly differed from that

in the 55-64 age group (Mdn = 4.5, IQR = 3-6.3), U

= 33, p = .004. Finally, the number of critical issues

in the 65+ age group (Mdn = 4, IQR = 3-5) showed a

significant difference with that in the 55-64 age group

(Mdn = 1, IQR = 0-3), U = 14, p = .001. The

difference in the number of minor issues was not

significant between the 55-64 age group (Mdn = 5,

IQR = 2-7.5) and the 65+ age group (Mdn = 3, IQR =

2-5), U = 118.5, p = .28. Figure 2 illustrates the

differences of the medians between age groups for the

usability issues using box plots.

3.5 Thematic Analysis of Usability

Issues

The usability issues were grouped into one of the

following four categories: (1) Navigation &

Structure; (2) Content & Information; (3) Design &

Presentation; and (4) Other. The first category

included 11 usability issues, of which six issues were

present for both age groups. The second category

contained 19 usability issues, of which six issues were

present for both age groups. In the third category,

there were 29 usability issues, of which ten issues

were present for both age groups. The final category

included 20 usability issues, of which four were

present for both age groups. Figure 3 shows the

number of minor, serious and critical usability issues

per category for each age group.

Table 3: Usability issues (minor, serious, critical) per age

group.

55-64

65+

Both

Minor

Serious

Critical

Total

3.5.1 Navigation & Structure

Participants in both age groups had difficulty

controlling game elements, such as moving the avatar

in the GUI and objects in the mini games. Also,

navigation to different locations (the tele-

rehabilitation portal and the mini games) caused

problems because the system gives little information

to users about where various elements can be found.

When a user by accident found the mini games by

chance, he or she had difficulty rediscovering those

locations in a subsequent task. Most participants

preferred the use of the browser navigation elements

to the navigation elements in the GUI. The main

difference between the age groups was that

participants in the 65+ age group sometimes decided

to quit the game because they thought they had to go

somewhere else to locate the physical exercises,

whereas the younger age group continued their search

in the GUI.

3.5.2 Content & Information

The information available in the game-based eHealth

application did not provide sufficient information for

users to feel in control of the game. For both age

groups, participants puzzled over the use and purpose

of various GUI elements, namely: (1) the cabins in the

home-screen, which are the link to the tele-

rehabilitation portal; (2) the buttons ‘game modus’ (to

switch off the gamified interface) and ‘basic modus’

(for people with color sensitivity or contrast

difficulties); (3) the functionalities in the mail inbox;

(4) the login entry fields, and (5) the use of the

vegetable garden. Furthermore, in both age groups

there was ambiguity about the overall goal, namely to

build a boat to escape the island. In the 65+ group,

participants mentioned that they did not understand

how they can build a boat, and in the 55-64 age group

they did not understand the purpose of the docks (in

which the boat will be shown). The difference is that

this was a minor issue for the 55-64 age group, but a

serious issue for the 65+ group.

The participants in the age group 55-64 required

additional information on various elements of the

game-based eHealth application and the connecting

tele-rehabilitation portal. For example, when

following the physical exercises, they needed more

explanation on the length and frequency of these

exercises. The participants in the 65+ age group

experienced more difficulty understanding the

available information. The information provided in

the help video on how to use the mail inbox runs too

quickly, and participants did not understand the

information provided in the GUI.

Why My Grandfather Finds Difﬁculty in using Ehealth: Differences in Usability Evaluations between Older Age Groups

Figure 2: Box plots of the number of minor, serious and

critical issues per age group.

3.5.3 Design & Presentation

Similar usability issues regarding the aesthetics and

design for both age groups were mostly critical and

serious issues. Participants had problems discovering

the direction signs in the home screen of the GUI,

which prevented or slowed down task completions.

Furthermore, the design of buttons in the GUI led to

confusion. For example, every time a user went back

to the home screen the information button lighted up.

This led users to believe that there was new or

additional information available for them to read,

which was not necessarily the case. Also, every

participant sometimes had trouble discriminating

between clickable and non-clickable GUI elements.

Looking at the unique issues for each age group,

there were many additional usability issues in the 65+

age group, while just two minor issues were unique

for the 55-64 age group. In the 65+ group, there were

several issues regarding the aesthetics. Participants

did not like the music and the objects in the GUI were

displayed in too small a size for them to identify.

Also, participants ran into problems because of the

layout of GUI elements. Since some elements are

placed closely together, users often clicked on the

wrong element without noticing. Moreover, one

respondent who was color blind could not understand

the color codes in the tele-rehabilitation portal.

Finally, some participants reported usability issues

regarding the design but these problems were are

actually due to their unfamiliarity with standard

design principles of computer interfaces.

3.5.4 Other

In both age groups difficulties were reported

regarding the accessibility of the system. Users had

trouble creating special characters to log on and the

game-based eHealth application is not adaptive for

people living with dyslexia or other reading

problems. There were just two minor usability issues

that were only present in the 55-64 age group. One of

these is that participants had no problems locating the

direction sign to the garden, but difficulty pressing

this sign because the clicking area only partly

Figure 3: Minor, serious and critical usability issues per usability category for both age groups (55-64 and 65+).

ICT4AWE 2019 - 5th International Conference on Information and Communication Technologies for Ageing Well and e-Health

technical aspects, such as the page load time. For

example, when participants clicked on a direction

sign, the loading of the next interface screen holds of

loading the next page until the avatar has walked to

the direction sign. Issues that were considered critical

were issues where participants had difficulty grasping

the game story and the overall goal of the game-based

eHealth application. The connection between the

game-based eHealth application and the tele-

rehabilitation portal often remained unclear and

participants did not understand the game story. These

were critical issues that prevented users from

completing the given tasks.

4 DISCUSSION

This study examined how age-related barriers affect

the usability evaluations of an eHealth system. There

were four main results. First, participants of the 55-

64 age group rated the overall system usability

significantly higher than participants of the 65+ age

group. Second, participants of the 55-64 age group

had significantly fewer usability issues than

participants for the 65+ age group. Third, the

identified usability issues reported by the 55-64 age

group were less severe than in the 65+ age group.

Last, there are differences in the types of usability

issues found by the two age groups, except for

navigation within the game-based eHealth

application. Navigation & Structure issues were quite

similar for both groups. In contrast, Design &

Presentation and Content & Information, and Other

issues had more variation in severity and content

between the two age groups.

The statistical differences between the age groups

can be explained via the groups’ digital skills. Van

Deursen et al. (2009) found that a higher age

especially affects operational and formal internet

skills, such as operating an internet browser and

maintaining sense of orientation. In this study, we

found similar results. Although in both groups

participants had orientation problems in the system,

we found differences regarding operational skills

between the age groups. The participants in the 65+

age group had difficulty with understanding technical

features and functionalities of the game-based

eHealth application. These issues were not present for

the 55-64 age group. Also, the 65+ age group had

more serious issues related to understanding the

content and the purposes of GUI elements, like

buttons, than did the older adults aged 55-64, in the

game-based eHealth application. In addition, for the

65+ age group, these issues were often more severe,

serious or critical, than for the 55-64 age group.

However, these statistical differences do not yet

explain the low SUS scores for both age groups.

Typically, a SUS score does not drop below the

threshold of 50 (Bangor, Kortum and Miller, 2008).

For older adults to use technology, it is important that

they perceive some benefits and relevance in using

the technology (Melenhorst and Bouwhuis, 2004).

However, from the usability test, it became clear that

many participants had difficulty grasping the purpose

of the cabins in the home–screen (i.e. to access the

tele-rehabilitation portal). Also, they did not

understand the connection between the gamified

interface and the portal. It could be that the gamified

interface slightly blurs the underlying goal of

improving one’s physical condition and making

progress in the game by performing physical

exercises.

The thematic analysis also revealed differences

between the age groups, in the type of usability

categories. These differences can be explained

because the categories refer to various factors of how

a user interacts with a system. Navigation & Structure

issues result from goal-driven strategies. A user must

understand and predict consequences by clicking on

a GUI element (Kitajima, Blackmon and Polson,

2000). This requires the system to have an intuitive

and logical structure. In contrast, the categories of

Content & Information and Design & Presentation

refer to functionalities of the system that support a

user in his or her quest. Users may need varying levels

of detailed information and support. For eHealth,

information needs to be tailored to users’ levels of

health literacy (Chew, Bradley and Boyko, 2004),

motivation to adopt a healthy lifestyle (Harjumaa and

Oinas-Kukkonen, 2009) and both physical and

cognitive functioning (Flores et al., 2008; Gerling et

al., 2012; Hoogendam et al., 2014). In the category

‘Other’ there were several issues related to the

accessibility of the system. The study of Huber and

Vitouch (2008) found that the accessibility of the

system can significantly affect the usability ratings.

The current usability evaluation methods do not

take these additional factors into account to

compensate for its potential effect on the perceived

usability. Usability experts and researchers working

in the field of healthy ageing could use these results

to optimize and standardize usability evaluations and

benchmarks of eHealth systems. As we found in the

thematic analysis, within each category there is a

large variance in the type of usability issues. For

example, in the Design & Presentation category the

issues varied from disliking music to problems with

Why My Grandfather Finds Difﬁculty in using Ehealth: Differences in Usability Evaluations between Older Age Groups

graphics resulting from color blindness. Also, the

Other category is as an undefined category

comprising a group of leftover usability issues, such

as technical and accessibility issues, that could not be

placed in the other three categories. We need more

information on factors that affect usability in the

eHealth domain and which aspects of the system

affect the user-friendliness of eHealth.

4.1 Study Limitations

This study applied fixed age boundaries in the

evaluation of usability. Of course, in real-life there

will be less distinctive age boundaries for users of

eHealth systems. Also, in the literature, there is a

growing body of research on how age, especially

older age, affects usability, user acceptance and

perceived intention-to-use of eHealth systems in daily

life. However, in our study we wanted to know in

more detail which aspects of a system’s usability are

affected by age-related barriers. We did this by

examining differences in the types and severity scores

of usability issues. Finally, this study used a

qualitative approach to examine how age differences

affected usability. To generalize the results to the

elderly population, we need to conduct larger studies

on representative cross sections of the ageing

population.

5 CONCLUSIONS

Current usability evaluations in eHealth do not take

into account variations in end-user populations and

their effects on the perceived usability of a system.

This study found that the perceptions of usability

differ between two consecutive age groups. Not only

were there differences in the type of usability issues

between the age groups, but also statistically

significant differences were found in the number of

serious and critical usability issues that each age

group encountered. This study has established that

variations in end-user populations affect usability

evaluations in eHealth. The next step is to examine

which factors we need to take into account to measure

effectively the user-friendliness of eHealth

applications.

ACKNOWLEDGEMENTS

This work is conducted within the context of the IMI

SPRINTT (IMI-JU 115621) project.

REFERENCES

Bakkes, S., Tan, C. T. and Pisan, Y. (2012) ‘Personalised

Gaming:A Motivation and Overview of Literature’, in

Proceedings of The 8th Australasian Conference on

Interactive Entertainment: Playing the System-IE ’12.

doi: 10.1145/2336727.2336731.

Bangor, A., Kortum, P. T., & Miller, J. T. (2008). An

empirical evaluation of the system usability scale.

International Journal of Human-Computer

Interaction, 24(6), 574–594. https://doi.org/10.1080/

10447310802205776

Broens, T., Huis in‘t Veld, R., Vollenbroek-Hutten, M.M.R.,

Hermens, H.J., van Halteren, A.T. and Nieuwenhuis,

L.J.M. (2007) ‘Original article " Determinants of

successful telemedicine implementations : a literature

study’, Journal of Telemedicine and Telecare, 13(6), pp.

303–309.

Brooke, J. (1996) ‘SUS - A quick and dirty usability scale’,

Usability evaluation in industry. doi: 10.1002/

hbm.20701.

Central Bureau for Statistics (CBS) (2018, October 31).

Statistics Netherlands: Population: key figures [Dataset].

Retrieved from: https://opendata.cbs.nl/statline/#/CBS/

nl/dataset/37296ned/table?ts=1537956882273

Chen, Y. and Persson, A. (2002) ‘Internet use among young

and older adults: Relation to psychological well-being’,

Educational Gerontology, 28(9), pp. 731–744. doi:

10.1080/03601270290099921.

Chew LD, Bradley KA, B. E. (2004) ‘Brief questions to

identify patients with\rinadequate health literacy’,

Family Medicine, 36(8), pp. 585–594. doi: 10.1186/

1471-2458-12-80.

van Deursen, A. J. A. M. and van Dijk, J. A. G. M. (2009)

‘Using the Internet: Skill related problems in users’

online behavior’, Interacting with Computers. Elsevier

B.V., 21(5–6), pp. 393–402. doi: 10.1016/j.intcom.

2009.06.005.

Duh, H. B.-L., Tan, G. C. B. and Chen, V. H. (2006)

‘Usability evaluation for mobile device: a comparison

of laboratory and field tests’, Proceedings of the 8th

conference on Human-computer interaction with

mobile devices and services, pp. 181–186.

Flores, E., Tobon, G., Cavallaro, E., Cavallaro, F.I., Perry,

J.C. and Keller, T. (2008) ‘Improving patient

motivation in game development for motor deficit

rehabilitation’, in Proceedings of the 2008

International Conference in Advances on Computer

Entertainment Technology - ACE ’08, p. 381. doi:

10.1145/1501750.1501839.

Fried, L. P., Tangen, C.M., Walston, J., Newman, A.B.,

Hirsch, C., Gottdiener, J., Seeman, T. Tracy, R., Kop,

W.J., Burke, G., McBurnie, M.A. (2001) ‘Frailty in

Older Adults: Evidence for a Phenotype’, The Journals

of Gerontology Series A: Biological Sciences and

Medical Sciences. doi: 10.1093/gerona/56.3.M146.

Gatto, S. L. and Tak, S. H. (2008) ‘Computer, Internet, and

e-mail use among older adults: Benefits and barriers’,

Educational Gerontology, 34(9), pp. 800–811. doi:

10.1080/03601270802243697.

ICT4AWE 2019 - 5th International Conference on Information and Communication Technologies for Ageing Well and e-Health

van der Geest, T. (2004) ‘Beyond accessibility: Comparing

three website usability test methods for people with

impairments’, in Proceedings of the HCI 2004, pp.

129–132.

Gerling, K. M., Schulte, F.P., Smeddinck, J. and Masuch,

M. (2012) ‘Game design for older adults: Effects of

age-related changes on structural elements of digital

games’, in Lecture Notes in Computer Science

(including subseries Lecture Notes in Artificial

Intelligence and Lecture Notes in Bioinformatics). doi:

10.1007/978-3-642-33542-6_20.

Harjumaa, M. and Oinas-Kukkonen, H. (2009) ‘Persuasive

Systems Design: Key Issues, Process Model, and

System Features’, Communications of the Association

for Information Systems, 24(28), pp. 485–500. doi:

10.17705/1CAIS.02428.

Hoogendam, Y. Y., van der Lijn, F., Vernooij, M.W.,

Hofman, A., Niessen, W.J., van der Lugt, A., Ikram,

M.A., van der Geest, J.N. (2014) ‘Older Age Relates to

Worsening of Fine Motor Skills: A Population-Based

Study of Middle-Aged and Elderly Persons’, Frontiers

in Aging Neuroscience. 2014/10/14, 6, p. 259. doi:

10.3389/fnagi.2014.00259.

Huber, W. and Vitouch, P. (2008) ‘Usability and accessibility

on the internet: Effects of accessible web design on

usability’, Lecture Notes in Computer Science (including

subseries Lecture Notes in Artificial Intelligence and

Lecture Notes in Bioinformatics), 5105 LNCS, pp. 482–

489. doi: 10.1007/978-3-540-70540-6_69.

Jacelon, C. S. and Hanson, A. (2013) ‘Older adults’

participation in the development of smart

environments: An integrated review of the literature’,

Geriatric Nursing. Elsevier Ltd, 34(2), pp. 116–121.

doi: 10.1016/j.gerinurse.2012.11.001.

Kelders, S. M, Kok, R.N., Ossebaard, H.C., van Gemert-

Pijnen, J.E.W.C. (2012) ‘Persuasive system design does

matter: A systematic review of adherence to web-based

interventions’, Journal of Medical Internet Research.

doi: 10.2196/jmir.2104.

Kitajima, M., Blackmon, M. H. M. and Polson, P. G. P.

(2000) ‘A Comprehension-based Model of Web

Navigation and Its Application to Web Usability

Analysis’, People and Computers XIV — Usability or

Else! SE - 24, pp. 357–373. doi: 10.1007/978-1-4471-

0515-2_24.

Kreps, G. L. and Neuhauser, L. (2010) ‘New directions in

eHealth communication: Opportunities and challenges’,

Patient Education and Counseling, 78(3), pp. 329–336.

doi: 10.1016/j.pec.2010.01.013.

Liu, C. K. and Fielding, R. A. (2011) ‘Exercise as an

Intervention for Frailty’, Clinics in Geriatric Medicine.

2010/11/26, 27(1), pp. 101–110. doi: 10.1016/

j.cger.2010.08.001.

Lyles, C. R. and Sarkar, U. (2015) ‘Health Literacy,

Vulnerable Patients, and Health Information Technology

Use: Where Do We Go from Here?’, Journal of General

Internal Medicine, 30(3), pp. 271–272. doi: 10.1007/

s11606-014-3166-5.

McEwen, B. S. (2008) ‘Central effects of stress hormones

in health and disease: Understanding the protective and

damaging effects of stress and stress mediators’,

European Journal of Pharmacology, 583(2–3), pp.

174–185. doi: 10.1016/j.ejphar.2007.11.071.

Morey, S. A., Barg-Walkow, L. H. and Rogers, W. A.

(2017) ‘Managing heart failure on the Go: Usability

issues with mHealth apps for older adults’, Proceedings

of the Human Factors and Ergonomics Society, 2017–

Octob, pp. 1–5. doi: 10.1177/1541931213601496.

Melenhorst, A.-S., Rogers, W. A., & Bouwhuis, D. G.

(2006). Older adults’ motivated choice for

technological innovation: Evidence for benefit-driven

selectivity. Psychology and Aging, 21(1), 190–195.

https://doi.org/10.1037/0882-7974.21.1.190

Narasimha, S., Madathil, K.C., Agnisarman, S., Rogers, H.,

Welch, B., Ashok, A., Nair, A. and McElligot, J. (2017)

‘Designing Telemedicine Systems for Geriatric Patients:

A Review of the Usability Studies’, Telemedicine and e-

Health, 23(6), pp. 459–472. doi: 10.1089/tmj.2016.0178.

Ng, T., Feng, L., Nyunt, M., Feng, L., Niti, M., Tan, B., Chan,

G., Khoo, S., Chan, S., Yap, P. and Yak, K. (2015)

‘Nutritional, Physical, Cognitive, and Combination

Interventions and Frailty Reversal among Older Adults:

A Randomized Controlled Trial’, American Journal of

Medicine. doi: 10.1016/j.amjmed.2015.06.017.

Peek, S. T. M., Luijkx, K.G., Rijnaard, M.D., Nieboer,

M.E., van der Voort, C.S., Aarts, S., van Hoof, J.,

Vrijhoef, H.J.M. and Wouters, E.J.M. (2016) ‘Older

Adults’ Reasons for Using Technology while Aging in

Place’, Gerontology, 62(2). doi: 10.1159/000430949.

Salive, M. E. (2013) ‘Multimorbidity in older adults’,

Epidemiologic Reviews, 35(1), pp. 75–83. doi:

10.1093/epirev/mxs009.

Theou, O., Stathokostas, L., Roland, K.P., Jakobi, J.M.,

Patterson, C., Vandervoort, A.A. and Jones, G.R.

(2011) ‘The Effectiveness of Exercise Interventions for

the Management of Frailty: A Systematic Review’,

Journal of Aging Research. 2011/05/18, 2011, pp. 1–

19. doi: 10.4061/2011/569194.

van Velsen, L., van der Geest, T., & Klaassen, R. (2011).

Identifying usability issues for personalization during

formative evaluations: A comparison of three methods.

International Journal of Human-Computer Interaction,

27(7), 670–698. https://doi.org/10.1080/10447318.

2011.555304

de Vette F., Tabak, M. (submitted). The 6D Framework: An

Evidence-Based Tool for Designers of Serious Games.

de Vette, F., Tabak, M., Dekker- van Weering, M. and

Vollenbroek-Hutten, M. (2015) ‘Engaging Elderly

People in Telemedicine Through Gamification’, JMIR

Serious Games. doi: 10.2196/games.4561.

Wildenbos, G. A., Peute, L. and Jaspers, M. (2018) ‘Aging

barriers influencing mobile health usability for older

adults: A literature based framework (MOLD-US)’,

International Journal of Medical Informatics. Elsevier,

114(December 2017), pp. 66–75. doi: 10.1016/j.

ijmedinf.2018.03.012.

Why My Grandfather Finds Difﬁculty in using Ehealth: Differences in Usability Evaluations between Older Age Groups