A System for Monitoring Stroke Patients in a Home Environment

Bart Klaassen

, Bert-Jan van Beijnum

, Marcel Weusthof

, Dennis Hofs

, Fokke van Meulen

Henk Luinge

, Federico Lorussi

, Hermie Hermens

1,2

and Peter Veltink

Biomedical Signals and Systems group, University of Twente, Enschede, The Netherlands

Roessingh Research and Development B.V., Enschede, The Netherlands

Xsens Technologies B.V., Enschede, The Netherlands

Interdepartmental Center “E.Piaggio”, University of Pisa, Pisa, Italy

Keywords: Telemedicine, Architecture, Sensing System, Stroke, Home Environment, Daily-life Activities, Monitoring,

Performance, Capacity.

Abstract: Currently, the changes of functional capacity and performance of stroke patients after returning home from

a rehabilitation hospital is unknown for a physician, having no objective information about the intensity and

quality of a patient's daily-life activities. Therefore, there is a need to develop and validate an unobtrusive

and modular system for objectively monitoring the stroke patient's upper and lower extremity motor

function in daily-life activities and in home training. This is the main goal of the European FP7 project

named “INTERACTION”. A complete sensing system is developed, whereby Inertial Measurement Units

(IMU), Knitted Piezoresistive Fabric (KPF) goniometers, KPF strain sensors, EMG electrodes and force

sensors are integrated into a modular sensor suit designed for stroke patients. In this paper, we describe the

systems architecture. Data from the sensors are captured wirelessly and stored in a remote secure database

for later access and processing via portal technology. In collaboration with clinicians and engineers, clinical

outcome measures were defined and the question of how to present the data on the web portal was

addressed. The first implementation of the complete system includes a basic version of all components and

is currently being extended to include all sensors within the INTERACTION system.

1 INTRODUCTION

Currently, the changes of functional capacity and

performance of stroke patients after returning home

from a rehabilitation hospital is unknown for a

physician, having no objective information about the

intensity and quality of a patient's daily-life

activities. As a consequence, the physician is unable

to monitor the prescribed training program for

sustaining or increasing the patient’s capacity and

performance and cannot give advice to the patient

outside the hospital setting. Therefore, there is a

need to develop and validate an unobtrusive and

modular system for objective monitoring of daily-

life activities and training of upper and lower

extremity motor function in stroke patients. That is

the main goal of the European FP7 project named

“INTERACTION”. A physician will be able to

continuously evaluate the patient's performance in a

home setting by using the INTERACTION system,

allowing the physician to compare the patient’s

performance at home with the patient’s capacity in

the rehabilitation hospital. Thereby, the system will

support the physician in making decisions to, for

example, alter the prescribed training programs.

The INTERACTION sensor system is composed

of Inertial Measurement Units (IMUs), Knitted

Piezoresistive Fabric (KPF) strain sensors, KPF

goniometers, EMG electrodes and force sensors.

These sensors are integrated into a custom made

modular suit for stroke patients (e-textile), which

consists of a shirt, a pair of trousers, shoes and

gloves. The iterative design process for the sensor

suit includes several usability tests as well as an

extensive user requirements analysis with medical

and technical experts.

Data are captured wirelessly on a home-gateway,

which transmits the data to a secure database. Portal

technology can access and process the data. The

results can be consulted by a clinician whenever

necessary.

125

Klaassen B., van Beijnum B., Weusthof M., Hofs D., van Meulen F., Luinge H., Lorussi F., Hermens H. and Veltink P..

A System for Monitoring Stroke Patients in a Home Environment.

DOI: 10.5220/0004805001250132

In Proceedings of the International Conference on Health Informatics (HEALTHINF-2014), pages 125-132

ISBN: 978-989-758-010-9

 2014 SCITEPRESS (Science and Technology Publications, Lda.)

In this paper, we describe the system architecture

and the requirements for presenting the outcome

measures to clinicians. Specifically, in section 2, the

system requirements are given along with an

overview of the whole system and a detailed

description of each component. In section 3, the data

processing aspects of the system will be explored in

further detail. In section 4, the design process of the

data presentation is elaborated upon. In section 5,

the current implementation of the system is

presented and finally, in section 6, the conclusions

and future work are described.

2 SYSTEM ARCHITECTURE

2.1 System Requirements

Four major requirements were set before the initial

system development:

1) The system should compute and display

capacity and performance measures to

evaluate stroke patients during daily-life

activities (for example: grasping an object)

in a home setting.

2) The INTERACTION system should be

divided into several modules: upper

extremity (shirt), lower extremity

(trousers), gloves and shoes. This will

allow clinicians to assign different modules

to different patients according to the

clinicians specific interests.

3) Analysis of the sensor data will not be done

in real time. The system should be able to

store the computed data such that it can be

accessed by a clinician when needed.

4) The system should present the performance

information of the patient to the clinician,

such that it optimally supports monitoring

the progress of the patient and decisions

about continued therapy. The clinician

should be able to inspect the information in

progressive detail from global performance

parameters to details concerning the quality

of specific movement tasks, according to

his or her needs.

2.2 System Overview

The INTERACTION system’s architecture is based

upon a generic architectural approach described by

Pawar et al. (2012). Figure 1 shows a general

overview of the current system’s architecture. The

Body Area Network (BAN) is composed of several

sensors listed in table 1 and a home gateway. The

Xsens wireless Awinda protocol is used to connect

and synchronize the sensors to the home gateway,

which captures the data and stores it in a European

Data Format (EDF). The EDF file protocol was

extended for the INTERACTION project by adding

additional signal labels to the header of the file.

Finally, the EDF file is uploaded to a secure and

remote SQL database if an internet connection is

detected. A server, installed at the University of

Twente, runs a Liferay portal software (Liferay,

Inc.) with custom made portlets and Matlab

(Mathworks, 2013). The portal obtains the data from

the database and sends the results to Matlab for

processing. The results are saved and visualized on

the web-portal on request. Each component is

explained in detail in the following sub-sections.

Figure 1: System Architecture.

2.3 Body Area Network

The Body Area Network (BAN) consists of all body

sensor components and a gateway to capture, store

and upload sensor data.

2.3.1 Sensors

The INTERACTION sensor system is divided into

four modules which comprises of a number of

sensors listed in Table 1. Each Xsens MTw sensor

box includes 10 primary signals: a 3D

accelerometer, a 3D goniometer, a 3D magnetometer

and one Pressure channel. Knitted Piezoresistive

Fabric (KPF) strain sensors and goniometers are

developed by the University of Pisa and are

integrated into the textile clothing. The EMG

electrodes are integrated into the shirt and the signal

is pre-processed by a on body front end into a

smooth rectified signal. The KPF strain sensors,

KPF goniometers and EMG electrodes are each

physically linked to an MTw sensor box by parsing

HEALTHINF2014-InternationalConferenceonHealthInformatics

126

the data to the MTw’s pressure channel. As a result,

the wireless capabilities of the MTw’s for data

transmission from the BAN to the gateway are

preserved. Figure 2 provides a global overview of

the sensing system for the upper and lower

extremity.

Table 1: Sensor overview.

Type Number

Shirt Trouser Shoes Gloves

IMU* 6 4 2 2

KPF Strain** 2

KPF

goniometer**

1 2 6

EMG

electrodes

Force *** 6

*Xsens MTw, **Developed by University of Pisa,

***Work in progress

Each MTw outputs 10 primary signals and 4 derived

signals (orientation in quaternions), each of which is

assigned a unique sensor label within the EDF file.

The data collection rate is dependent on the number

of sensors. In our case, the collection rate is set to 20

Hertz. This data collection rate has been assessed to

be adequate, since 3D kinematics is analyzed at a

higher frequency (1800 Hertz) inside the MTw

sensor units before transmission to the Awinda

station.

This local analysis provides a more accurate

estimation of acceleration and angular velocity

values and includes 3D estimation of orientation. 20

Hz is an adequate rate for transmission of 3D

orientation as well as the other sensed quantities, as

specified in table 1.

Figure 2: Sensing System overview.

2.3.2 Gateway

The home gateway has three main functions: 1)

collecting the data from the sensors, 2) storing the

sensor data inside an EDF file every five minutes

and 3) uploading the EDF file to the database. The

data storage interval of five minutes was determined

by considering the available network bandwidth as

well as the decreasing overhead of the EDF file with

measurement time (Figure 3).

With five minutes of data, the EDF data record

has a size of 2.24 MB in total according to equation

1 (also called the “payload”) and a header size of

49.25 kB according to equation 2. Therefore, the

header occupies only 2.14% of the total EDF file

space. The relation between the payload and the

header of a file is called the “overhead”.

Payload size = M

imu

signals

*2 bytes (1)

Header size = (N

imu

labels

+1)*256 bytes (2)

The input for equation 1 and 2 are as follows: N

imu

(Number of IMU’s), N

signals

: 14 (Number of

sensor signals), N

labels

: 14 (number of sensor labels),

: 300 seconds (Measurement time) and F

: 20

Hertz (data collection rate). The general EDF header

is 256 bytes in equation 2.

The data is uploaded to the database with SSL

secure data encryption over the network using

RESTful web services, and the users of the database

are authenticated using a username and password

combination. Furthermore, within the EDF file, only

a device ID is used to identify each sensor suit, so no

patient names are exchanged.

Figure 3: EDF overhead for the complete INTERACTION

sensing system.

2.4 Database

An SQL database was configured at the Roessingh

Research and Development centre (RRD) by reason

of their technical experience in secure databases.

Dedicated API’s were constructed for

communication between the home gateway and

portal using SSL. For obtaining the data, a correct

combination of username and password is required

to authenticate the user, and a separate authorization

model is in use which determines the access rights of

the user, including his or her reading and writing

100

0200400

EDF overhead (%)

data length (seconds)

EDF overhead

ASystemforMonitoringStrokePatientsinaHomeEnvironment

127

rights. A query engine is developed based on

RESTful web services to obtain EDF sensor data

from the RRD database on receiving a request from

the web portal with a start and end time.

2.5 Portal

The web-portal is responsible for controlling and

visualizing the data. We chose the Liferay portal

framework as it provides a flexible working

environment to develop portlets in a Model-View-

Controller (MVC) structure using Java, JavaScript,

CSS and JSP. Liferay includes a dedicated Content

Management System, which allows the portal to be

personalized for different users by means of a

detailed access-control scheme for assigning

different rights to different users.

The View component is responsible for

displaying the processed data to the user and

includes two visual libraries: the Highchart library

(Highcharts Solutions AS) for graphs and the

Bootstrap library (http://getbootstrap.com) for a

responsive layout and website elements. The

Controller component is connected with the View

and initiates the Model(s). The Model components

obtain the data from the database by use of multiple

queries and subsequently send the data to Matlab via

a Matlab-Java bridge (Matlabcontrol Java API

v4.1.0, http://code.google.com/p/matlabcontrol/).

The complete structure is shown in Figure 4.

Figure 4: MVC structure.

Users are able to send requests for different types of

measurement data in a specific portlet (by pressing,

for instance, a button on the web-portal). These

requests are directly forwarded to the Controller

component associated with that portlet. This

Controller initiates several Models accordingly.

With this MVC structure, we are able to process and

visualize large amounts of data in an organized way.

Different portlets are constructed, each having a

different function to show different types of data on

the same web-page or on different web-pages.

3 DATA PROCESSING

The data processing flow is shown in Figure 5 and

represents a Model within the MVC portal structure.

The Model is composed of four steps. The first step

involves obtaining the data from the database by use

of the RESTful web services and initiating Matlab to

read the EDF file and assign each sensor label to

each data record. This step is realized by a custom

made Java portlet in the Liferay Portal software. The

second step is to pre-process the raw sensor data by

the use of a biomechanical model within Matlab to

derive the angles and positional values of body

segments. The results are then fed into activity

recognition algorithms in the third step which are

able to detect the type of activities shown in Figure 6

with a high specificity. The type of activities is

based upon daily-life activities for both upper and

lower extremities.

Figure 5: Data Processing flow.

Figure 6: Activity recognition schemes for the upper and

lower extremity respectively.

In the final step, the INTERACTION clinical

outcome measures are computed. These measures

are visualized towards the clinicians and should

provide valuable insight into the patient’s capacity

and performance during daily-life activities. The

results are saved for quick access by clinicians and

for time comparison of the results by the end of the

week or month. A list of basic outcome measures is

shown in table 2.

4 USER INTERFACE

Designing a graphical user interface for clinicians to

access the web-portal and determining what

outcome parameters to present on the portal is one of

the major challenges in the INTERACTION project.

The INTERACTION system will be collecting data

that clinicians are not familiar with in current

practice and the data has to be presented in a format

that clinicians can understand and evaluate within a

HEALTHINF2014-InternationalConferenceonHealthInformatics

128

Table 2: Examples of clinical outcome measures in the

INTERACTION system.

1 Arm usage of the affected and non-affected arm

2 Maximum reach of the affected and non-affected arm

Range of Motion of the elbow and shoulder of the

affected and non-affected arm

4 Range of Motion of the trunk

Maximum grasping force of the affected and non-

affected arm

umber of grasps of the affected and non-affected

arm

umber of steps, step length and step time

8 Weight support by affected and non-affected leg

few minutes. Therefore, in close collaboration with

clinicians, we investigated which clinical outcome

measures are relevant and how to present the data in

such a way that the capacity and performance of a

patient can be easily evaluated and compared over

time.

After some interviews with clinicians and

engineers from Enschede (Roessingh research and

development rehabilitation centre, RRD) and Zürich

(University hospital in Zürich, USZ), we concluded

the following:

Clinicians can have as many as 40 stroke

patients in treatment at a given moment, all of whom

have to be evaluated within one hour by the end of

the week. This amounts to only a few minutes per

week to analyze the performance of each patient.

Hence, there is a need for a basic overview of all

patients on the web-portal with an option to

successively drilldown to a particular data set for a

particular patient. Based on these preliminarily

results, we made a set of mock-ups for the web-

portal. An overview mock-up (called a “dashboard”)

is shown in Figure 7 for three patients.

It includes a patient overview on the top, a general

INTERACTION system statistics at the bottom left

and a recent blog entry by the patients at the bottom

right. The main navigation menu is divided for the

upper and lower extremity, each presenting different

global statistics of a patient. Furthermore the

navigation includes a patient list page, a suit

performance page (to check which sensor suits are

operational and if no errors are present) and a tech

support. For the clinical trials, some modifications

will be made in the mock-ups to protect the

anonymity of the patient.

A detailed overview mock-up for a particular

patient is shown in Figure 8 (for the upper

extremity). The selection panel allows for easy

scrolling through patients, times of measurement and

specific body sections. The clinician can also click

on a specific anatomical body part to drill deeper

down into the patient’s data. The weekly statistics

gives an overview of the patient’s performance

during the selected week, and the INTERACTION

score summarizes the patient’s overall performance

over multiple weeks.

The mock-up for the final tier of the web-portal

drilldown into the patient’s data is shown in Figure

9. Specific clinical outcome measures are presented

over time on the left (for example, elbow angle,

trunk flexion and reaching distance), outcome

measures like Range of Motion (ROM) can be

extracted and can be compared during several weeks

of measurements.

Figure 7: Dashboard page.

ASystemforMonitoringStrokePatientsinaHomeEnvironment

129

Figure 8: Upper extremity overview page.

Figure 9: Drill down page of the patient data.

HEALTHINF2014-InternationalConferenceonHealthInformatics

130

5 IMPLEMENTATION

We finished the complete system architecture

structure involving all the components discussed in

this paper. The first prototype consists of three

Xsens MTw sensors, one gateway platform (a laptop

with an Xsens Awinda base station), one secure

database, one server with Liferay, Matlab and

custom-built portlets. Each data processing

component described earlier has been implemented,

but with a basic setting for processing data from

three MTw sensors. A demonstrational setup of the

sensors is shown in Figure 10.

Three MTw sensors are placed on the upper arm,

lower arm and trunk of the body. For

demonstrational purposes of the system, the

following measurement was performed. One subject

simulated a repetitive reaching motion of a stroke

patient’s non-affected arm for 10 seconds followed

by simulating the motion of the affected arm for 10

seconds. There was a 5 seconds rest period in

between.

During the first 10 seconds, the subject stands up

straight and does not move the trunk while reaching

for an object. During the last 10 seconds, the subject

utilizes the trunk to compensate for the decrease in

elbow flexion and elbow/shoulder coupling

(Dewald, 1995) to reach for an object.

Figure 10: MTw sensor placement.

The home gateway captures the measurement data,

stores the data in an EDF file and uploads the file to

the database if an internet connection is detected. If

an end-user (for example, a clinician) presses the

“analyze” button on the web portal (within a portlet),

the portlet retrieves the data from the database.

Subsequently, the data is processed according to the

data processing flow. In this demo, the

biomechanical model calculates the following:

elbow angle, shoulder abduction, hand-sternum

distance and trunk orientation. The activity

recognition functions detect the two types of

movement (the reaching movements of the non-

affected and affected arm) and the results are

visualized as graphs on the web-portal. The result of

one measurement is shown in Figure 9. During the

use of the affected arm, there is a decrease in elbow

flexion and an increase in trunk flexion while

reaching multiple times for an object.

6 CONCLUSIONS AND FUTURE

WORK

The INTERACTION project aims to develop and

validate an unobtrusive and modular system for

objectively monitoring the daily life activities of

upper and lower extremity motor function in stroke

patients. The system’s complete architecture was

developed according to the requirements identified

at the beginning of the project. The architecture,

including all its components, was validated by using

three Xsens MTw sensors in a short measurement,

during which we simulated a Stroke patient’s

reaching motion of the affected and non-affected

arm. In the first prototype of the architecture system,

we included a biomechanical model in combination

with activity recognition functions to compute

several clinical outcome measures, which are then

shown on a web-portal.

Extension of the architecture to a full on body

sensing system is a feasible task, as all components

are designed for that purpose. The system will be

extended to incorporate the full number of sensors

and each component needs to be updated

accordingly. The gateway software will be

transferred to a Smartphone and the Xsens Awinda

base station will be replaced by an Xsens dongle

connected to this Smartphone. Furthermore, the

portal’s MVC structure has been designed for

extensions and provides a flexible coding

environment for Engineers by the inclusion of a

Matlab-Java bridge.

In this project, we have identified an extensive

list of potential clinical outcome measures. The list

of clinical outcome measures given in this paper is

an example of what the INTERACTION system will

deliver. We are now in the process, together with

clinicians and engineers, to make a final selection of

the clinical outcome measures to be implemented by

the system. Finally, both the sensing system and

web-portal have to be evaluated before starting

clinical trials.

ASystemforMonitoringStrokePatientsinaHomeEnvironment

131

REFERENCES

Dewald, J. P., Pope, P.S., Given, J. D., Buchanan, T. S.

and Rymer W. Z.(1995) ‘Abnormal muscle

coactivation patterns during isometric torque

generation at the elbow and shoulder in hemiparetic

subjects’. Brain, April, 118 (Pt 2). Pp 495-510.

European Data Format (2013) A simple and flexible format

for exchange and storage of multichannel biological

and physical signals, [ONLINE] Available:

http://www.edfplus.info/ [30 Sept 2013]

Highsoft Solutions AS (2013) Highcharts JS, interactive

JavaScript charts for your website, [ONLINE]

Available: http://www.highcharts.com/ [30 Sept 2013]

INTERACTION (2013) Official INTERACTION project

website, [ONLINE] Available: http://

www.interaction4stroke.eu [30 Sept 2013]

Liferay, Inc. (2013) Liferay delivers open source

enterprise solutions for portals, publishing, content,

and collaboration, [ONLINE] Available: http://

www.liferay.com/ [30 Sept 2013]

Matlab (2013), The MathWorks, Inc., Natick,

Massachusetts, United States.

MatlabControl (2013) Matlabcontrol API for JAVA,

[ONLINE] Available: http://code.google.com/p/

matlabcontrol/ [30 Sept 2013]

Pawar, P., Jones, V., Beijnum van, B-J. F. and Hermens,

H. (2012) ‘A framework for the comparison of mobile

patient monitoring systems’. Journal of biomedical

informatics, 45 (3). pp. 544-556.

Twitter Bootstrap (2013) Sleek, intuitive, and powerful

front-end framework for faster and easier web

development,[ONLINE] Available:

http://getbootstrap.com/ [30 Sept 2013]

Xsens Technologies B. V. (2013) MTW Development KIT

Lite,[ONLINE] Available: http://www.xsens.com/en/

mtw-dk-lite [30 Sept 2013]

HEALTHINF2014-InternationalConferenceonHealthInformatics

132