A System for Monitoring Stroke Patients in a Home Environment
Bart Klaassen
1
, Bert-Jan van Beijnum
1
, Marcel Weusthof
1
, Dennis Hofs
2
, Fokke van Meulen
1
,
Henk Luinge
3
, Federico Lorussi
4
, Hermie Hermens
1,2
and Peter Veltink
1
1
Biomedical Signals and Systems group, University of Twente, Enschede, The Netherlands
2
Roessingh Research and Development B.V., Enschede, The Netherlands
3
Xsens Technologies B.V., Enschede, The Netherlands
4
Interdepartmental Center “E.Piaggio”, University of Pisa, Pisa, Italy
Keywords: Telemedicine, Architecture, Sensing System, Stroke, Home Environment, Daily-life Activities, Monitoring,
Performance, Capacity.
Abstract: Currently, the changes of functional capacity and performance of stroke patients after returning home from
a rehabilitation hospital is unknown for a physician, having no objective information about the intensity and
quality of a patient's daily-life activities. Therefore, there is a need to develop and validate an unobtrusive
and modular system for objectively monitoring the stroke patient's upper and lower extremity motor
function in daily-life activities and in home training. This is the main goal of the European FP7 project
named “INTERACTION”. A complete sensing system is developed, whereby Inertial Measurement Units
(IMU), Knitted Piezoresistive Fabric (KPF) goniometers, KPF strain sensors, EMG electrodes and force
sensors are integrated into a modular sensor suit designed for stroke patients. In this paper, we describe the
systems architecture. Data from the sensors are captured wirelessly and stored in a remote secure database
for later access and processing via portal technology. In collaboration with clinicians and engineers, clinical
outcome measures were defined and the question of how to present the data on the web portal was
addressed. The first implementation of the complete system includes a basic version of all components and
is currently being extended to include all sensors within the INTERACTION system.
1 INTRODUCTION
Currently, the changes of functional capacity and
performance of stroke patients after returning home
from a rehabilitation hospital is unknown for a
physician, having no objective information about the
intensity and quality of a patient's daily-life
activities. As a consequence, the physician is unable
to monitor the prescribed training program for
sustaining or increasing the patient’s capacity and
performance and cannot give advice to the patient
outside the hospital setting. Therefore, there is a
need to develop and validate an unobtrusive and
modular system for objective monitoring of daily-
life activities and training of upper and lower
extremity motor function in stroke patients. That is
the main goal of the European FP7 project named
“INTERACTION”. A physician will be able to
continuously evaluate the patient's performance in a
home setting by using the INTERACTION system,
allowing the physician to compare the patient’s
performance at home with the patient’s capacity in
the rehabilitation hospital. Thereby, the system will
support the physician in making decisions to, for
example, alter the prescribed training programs.
The INTERACTION sensor system is composed
of Inertial Measurement Units (IMUs), Knitted
Piezoresistive Fabric (KPF) strain sensors, KPF
goniometers, EMG electrodes and force sensors.
These sensors are integrated into a custom made
modular suit for stroke patients (e-textile), which
consists of a shirt, a pair of trousers, shoes and
gloves. The iterative design process for the sensor
suit includes several usability tests as well as an
extensive user requirements analysis with medical
and technical experts.
Data are captured wirelessly on a home-gateway,
which transmits the data to a secure database. Portal
technology can access and process the data. The
results can be consulted by a clinician whenever
necessary.
125
Klaassen B., van Beijnum B., Weusthof M., Hofs D., van Meulen F., Luinge H., Lorussi F., Hermens H. and Veltink P..
A System for Monitoring Stroke Patients in a Home Environment.
DOI: 10.5220/0004805001250132
In Proceedings of the International Conference on Health Informatics (HEALTHINF-2014), pages 125-132
ISBN: 978-989-758-010-9
Copyright
c
2014 SCITEPRESS (Science and Technology Publications, Lda.)
In this paper, we describe the system architecture
and the requirements for presenting the outcome
measures to clinicians. Specifically, in section 2, the
system requirements are given along with an
overview of the whole system and a detailed
description of each component. In section 3, the data
processing aspects of the system will be explored in
further detail. In section 4, the design process of the
data presentation is elaborated upon. In section 5,
the current implementation of the system is
presented and finally, in section 6, the conclusions
and future work are described.
2 SYSTEM ARCHITECTURE
2.1 System Requirements
Four major requirements were set before the initial
system development:
1) The system should compute and display
capacity and performance measures to
evaluate stroke patients during daily-life
activities (for example: grasping an object)
in a home setting.
2) The INTERACTION system should be
divided into several modules: upper
extremity (shirt), lower extremity
(trousers), gloves and shoes. This will
allow clinicians to assign different modules
to different patients according to the
clinicians specific interests.
3) Analysis of the sensor data will not be done
in real time. The system should be able to
store the computed data such that it can be
accessed by a clinician when needed.
4) The system should present the performance
information of the patient to the clinician,
such that it optimally supports monitoring
the progress of the patient and decisions
about continued therapy. The clinician
should be able to inspect the information in
progressive detail from global performance
parameters to details concerning the quality
of specific movement tasks, according to
his or her needs.
2.2 System Overview
The INTERACTION system’s architecture is based
upon a generic architectural approach described by
Pawar et al. (2012). Figure 1 shows a general
overview of the current system’s architecture. The
Body Area Network (BAN) is composed of several
sensors listed in table 1 and a home gateway. The
Xsens wireless Awinda protocol is used to connect
and synchronize the sensors to the home gateway,
which captures the data and stores it in a European
Data Format (EDF). The EDF file protocol was
extended for the INTERACTION project by adding
additional signal labels to the header of the file.
Finally, the EDF file is uploaded to a secure and
remote SQL database if an internet connection is
detected. A server, installed at the University of
Twente, runs a Liferay portal software (Liferay,
Inc.) with custom made portlets and Matlab
(Mathworks, 2013). The portal obtains the data from
the database and sends the results to Matlab for
processing. The results are saved and visualized on
the web-portal on request. Each component is
explained in detail in the following sub-sections.
Figure 1: System Architecture.
2.3 Body Area Network
The Body Area Network (BAN) consists of all body
sensor components and a gateway to capture, store
and upload sensor data.
2.3.1 Sensors
The INTERACTION sensor system is divided into
four modules which comprises of a number of
sensors listed in Table 1. Each Xsens MTw sensor
box includes 10 primary signals: a 3D
accelerometer, a 3D goniometer, a 3D magnetometer
and one Pressure channel. Knitted Piezoresistive
Fabric (KPF) strain sensors and goniometers are
developed by the University of Pisa and are
integrated into the textile clothing. The EMG
electrodes are integrated into the shirt and the signal
is pre-processed by a on body front end into a
smooth rectified signal. The KPF strain sensors,
KPF goniometers and EMG electrodes are each
physically linked to an MTw sensor box by parsing
HEALTHINF2014-InternationalConferenceonHealthInformatics
126
the data to the MTw’s pressure channel. As a result,
the wireless capabilities of the MTw’s for data
transmission from the BAN to the gateway are
preserved. Figure 2 provides a global overview of
the sensing system for the upper and lower
extremity.
Table 1: Sensor overview.
Type Number
Shirt Trouser Shoes Gloves
IMU* 6 4 2 2
KPF Strain** 2
KPF
goniometer**
1 2 6
EMG
electrodes
1
Force *** 6
*Xsens MTw, **Developed by University of Pisa,
***Work in progress
Each MTw outputs 10 primary signals and 4 derived
signals (orientation in quaternions), each of which is
assigned a unique sensor label within the EDF file.
The data collection rate is dependent on the number
of sensors. In our case, the collection rate is set to 20
Hertz. This data collection rate has been assessed to
be adequate, since 3D kinematics is analyzed at a
higher frequency (1800 Hertz) inside the MTw
sensor units before transmission to the Awinda
station.
This local analysis provides a more accurate
estimation of acceleration and angular velocity
values and includes 3D estimation of orientation. 20
Hz is an adequate rate for transmission of 3D
orientation as well as the other sensed quantities, as
specified in table 1.
Figure 2: Sensing System overview.
2.3.2 Gateway
The home gateway has three main functions: 1)
collecting the data from the sensors, 2) storing the
sensor data inside an EDF file every five minutes
and 3) uploading the EDF file to the database. The
data storage interval of five minutes was determined
by considering the available network bandwidth as
well as the decreasing overhead of the EDF file with
measurement time (Figure 3).
With five minutes of data, the EDF data record
has a size of 2.24 MB in total according to equation
1 (also called the “payload”) and a header size of
49.25 kB according to equation 2. Therefore, the
header occupies only 2.14% of the total EDF file
space. The relation between the payload and the
header of a file is called the “overhead”.
Payload size = M
t
*F
s
*N
imu
*N
signals
*2 bytes (1)
Header size = (N
imu
*N
labels
+1)*256 bytes (2)
The input for equation 1 and 2 are as follows: N
imu
:
14
(Number of IMU’s), N
signals
: 14 (Number of
sensor signals), N
labels
: 14 (number of sensor labels),
M
t
: 300 seconds (Measurement time) and F
s
: 20
Hertz (data collection rate). The general EDF header
is 256 bytes in equation 2.
The data is uploaded to the database with SSL
secure data encryption over the network using
RESTful web services, and the users of the database
are authenticated using a username and password
combination. Furthermore, within the EDF file, only
a device ID is used to identify each sensor suit, so no
patient names are exchanged.
Figure 3: EDF overhead for the complete INTERACTION
sensing system.
2.4 Database
An SQL database was configured at the Roessingh
Research and Development centre (RRD) by reason
of their technical experience in secure databases.
Dedicated API’s were constructed for
communication between the home gateway and
portal using SSL. For obtaining the data, a correct
combination of username and password is required
to authenticate the user, and a separate authorization
model is in use which determines the access rights of
the user, including his or her reading and writing
0
20
40
60
80
100
0200400
EDF overhead (%)
data length (seconds)
EDF overhead
ASystemforMonitoringStrokePatientsinaHomeEnvironment
127
rights. A query engine is developed based on
RESTful web services to obtain EDF sensor data
from the RRD database on receiving a request from
the web portal with a start and end time.
2.5 Portal
The web-portal is responsible for controlling and
visualizing the data. We chose the Liferay portal
framework as it provides a flexible working
environment to develop portlets in a Model-View-
Controller (MVC) structure using Java, JavaScript,
CSS and JSP. Liferay includes a dedicated Content
Management System, which allows the portal to be
personalized for different users by means of a
detailed access-control scheme for assigning
different rights to different users.
The View component is responsible for
displaying the processed data to the user and
includes two visual libraries: the Highchart library
(Highcharts Solutions AS) for graphs and the
Bootstrap library (http://getbootstrap.com) for a
responsive layout and website elements. The
Controller component is connected with the View
and initiates the Model(s). The Model components
obtain the data from the database by use of multiple
queries and subsequently send the data to Matlab via
a Matlab-Java bridge (Matlabcontrol Java API
v4.1.0, http://code.google.com/p/matlabcontrol/).
The complete structure is shown in Figure 4.
Figure 4: MVC structure.
Users are able to send requests for different types of
measurement data in a specific portlet (by pressing,
for instance, a button on the web-portal). These
requests are directly forwarded to the Controller
component associated with that portlet. This
Controller initiates several Models accordingly.
With this MVC structure, we are able to process and
visualize large amounts of data in an organized way.
Different portlets are constructed, each having a
different function to show different types of data on
the same web-page or on different web-pages.
3 DATA PROCESSING
The data processing flow is shown in Figure 5 and
represents a Model within the MVC portal structure.
The Model is composed of four steps. The first step
involves obtaining the data from the database by use
of the RESTful web services and initiating Matlab to
read the EDF file and assign each sensor label to
each data record. This step is realized by a custom
made Java portlet in the Liferay Portal software. The
second step is to pre-process the raw sensor data by
the use of a biomechanical model within Matlab to
derive the angles and positional values of body
segments. The results are then fed into activity
recognition algorithms in the third step which are
able to detect the type of activities shown in Figure 6
with a high specificity. The type of activities is
based upon daily-life activities for both upper and
lower extremities.
Figure 5: Data Processing flow.
Figure 6: Activity recognition schemes for the upper and
lower extremity respectively.
In the final step, the INTERACTION clinical
outcome measures are computed. These measures
are visualized towards the clinicians and should
provide valuable insight into the patient’s capacity
and performance during daily-life activities. The
results are saved for quick access by clinicians and
for time comparison of the results by the end of the
week or month. A list of basic outcome measures is
shown in table 2.
4 USER INTERFACE
Designing a graphical user interface for clinicians to
access the web-portal and determining what
outcome parameters to present on the portal is one of
the major challenges in the INTERACTION project.
The INTERACTION system will be collecting data
that clinicians are not familiar with in current
practice and the data has to be presented in a format
that clinicians can understand and evaluate within a
HEALTHINF2014-InternationalConferenceonHealthInformatics
128
Table 2: Examples of clinical outcome measures in the
INTERACTION system.
1 Arm usage of the affected and non-affected arm
2 Maximum reach of the affected and non-affected arm
3
Range of Motion of the elbow and shoulder of the
affected and non-affected arm
4 Range of Motion of the trunk
5
Maximum grasping force of the affected and non-
affected arm
6
N
umber of grasps of the affected and non-affected
arm
7
N
umber of steps, step length and step time
8 Weight support by affected and non-affected leg
few minutes. Therefore, in close collaboration with
clinicians, we investigated which clinical outcome
measures are relevant and how to present the data in
such a way that the capacity and performance of a
patient can be easily evaluated and compared over
time.
After some interviews with clinicians and
engineers from Enschede (Roessingh research and
development rehabilitation centre, RRD) and Zürich
(University hospital in Zürich, USZ), we concluded
the following:
Clinicians can have as many as 40 stroke
patients in treatment at a given moment, all of whom
have to be evaluated within one hour by the end of
the week. This amounts to only a few minutes per
week to analyze the performance of each patient.
Hence, there is a need for a basic overview of all
patients on the web-portal with an option to
successively drilldown to a particular data set for a
particular patient. Based on these preliminarily
results, we made a set of mock-ups for the web-
portal. An overview mock-up (called a “dashboard”)
is shown in Figure 7 for three patients.
It includes a patient overview on the top, a general
INTERACTION system statistics at the bottom left
and a recent blog entry by the patients at the bottom
right. The main navigation menu is divided for the
upper and lower extremity, each presenting different
global statistics of a patient. Furthermore the
navigation includes a patient list page, a suit
performance page (to check which sensor suits are
operational and if no errors are present) and a tech
support. For the clinical trials, some modifications
will be made in the mock-ups to protect the
anonymity of the patient.
A detailed overview mock-up for a particular
patient is shown in Figure 8 (for the upper
extremity). The selection panel allows for easy
scrolling through patients, times of measurement and
specific body sections. The clinician can also click
on a specific anatomical body part to drill deeper
down into the patient’s data. The weekly statistics
gives an overview of the patient’s performance
during the selected week, and the INTERACTION
score summarizes the patient’s overall performance
over multiple weeks.
The mock-up for the final tier of the web-portal
drilldown into the patient’s data is shown in Figure
9. Specific clinical outcome measures are presented
over time on the left (for example, elbow angle,
trunk flexion and reaching distance), outcome
measures like Range of Motion (ROM) can be
extracted and can be compared during several weeks
of measurements.
Figure 7: Dashboard page.
ASystemforMonitoringStrokePatientsinaHomeEnvironment
129
Figure 8: Upper extremity overview page.
Figure 9: Drill down page of the patient data.
HEALTHINF2014-InternationalConferenceonHealthInformatics
130
5 IMPLEMENTATION
We finished the complete system architecture
structure involving all the components discussed in
this paper. The first prototype consists of three
Xsens MTw sensors, one gateway platform (a laptop
with an Xsens Awinda base station), one secure
database, one server with Liferay, Matlab and
custom-built portlets. Each data processing
component described earlier has been implemented,
but with a basic setting for processing data from
three MTw sensors. A demonstrational setup of the
sensors is shown in Figure 10.
Three MTw sensors are placed on the upper arm,
lower arm and trunk of the body. For
demonstrational purposes of the system, the
following measurement was performed. One subject
simulated a repetitive reaching motion of a stroke
patient’s non-affected arm for 10 seconds followed
by simulating the motion of the affected arm for 10
seconds. There was a 5 seconds rest period in
between.
During the first 10 seconds, the subject stands up
straight and does not move the trunk while reaching
for an object. During the last 10 seconds, the subject
utilizes the trunk to compensate for the decrease in
elbow flexion and elbow/shoulder coupling
(Dewald, 1995) to reach for an object.
Figure 10: MTw sensor placement.
The home gateway captures the measurement data,
stores the data in an EDF file and uploads the file to
the database if an internet connection is detected. If
an end-user (for example, a clinician) presses the
“analyze” button on the web portal (within a portlet),
the portlet retrieves the data from the database.
Subsequently, the data is processed according to the
data processing flow. In this demo, the
biomechanical model calculates the following:
elbow angle, shoulder abduction, hand-sternum
distance and trunk orientation. The activity
recognition functions detect the two types of
movement (the reaching movements of the non-
affected and affected arm) and the results are
visualized as graphs on the web-portal. The result of
one measurement is shown in Figure 9. During the
use of the affected arm, there is a decrease in elbow
flexion and an increase in trunk flexion while
reaching multiple times for an object.
6 CONCLUSIONS AND FUTURE
WORK
The INTERACTION project aims to develop and
validate an unobtrusive and modular system for
objectively monitoring the daily life activities of
upper and lower extremity motor function in stroke
patients. The system’s complete architecture was
developed according to the requirements identified
at the beginning of the project. The architecture,
including all its components, was validated by using
three Xsens MTw sensors in a short measurement,
during which we simulated a Stroke patient’s
reaching motion of the affected and non-affected
arm. In the first prototype of the architecture system,
we included a biomechanical model in combination
with activity recognition functions to compute
several clinical outcome measures, which are then
shown on a web-portal.
Extension of the architecture to a full on body
sensing system is a feasible task, as all components
are designed for that purpose. The system will be
extended to incorporate the full number of sensors
and each component needs to be updated
accordingly. The gateway software will be
transferred to a Smartphone and the Xsens Awinda
base station will be replaced by an Xsens dongle
connected to this Smartphone. Furthermore, the
portal’s MVC structure has been designed for
extensions and provides a flexible coding
environment for Engineers by the inclusion of a
Matlab-Java bridge.
In this project, we have identified an extensive
list of potential clinical outcome measures. The list
of clinical outcome measures given in this paper is
an example of what the INTERACTION system will
deliver. We are now in the process, together with
clinicians and engineers, to make a final selection of
the clinical outcome measures to be implemented by
the system. Finally, both the sensing system and
web-portal have to be evaluated before starting
clinical trials.
ASystemforMonitoringStrokePatientsinaHomeEnvironment
131
REFERENCES
Dewald, J. P., Pope, P.S., Given, J. D., Buchanan, T. S.
and Rymer W. Z.(1995) ‘Abnormal muscle
coactivation patterns during isometric torque
generation at the elbow and shoulder in hemiparetic
subjects’. Brain, April, 118 (Pt 2). Pp 495-510.
European Data Format (2013) A simple and flexible format
for exchange and storage of multichannel biological
and physical signals, [ONLINE] Available:
http://www.edfplus.info/ [30 Sept 2013]
Highsoft Solutions AS (2013) Highcharts JS, interactive
JavaScript charts for your website, [ONLINE]
Available: http://www.highcharts.com/ [30 Sept 2013]
INTERACTION (2013) Official INTERACTION project
website, [ONLINE] Available: http://
www.interaction4stroke.eu [30 Sept 2013]
Liferay, Inc. (2013) Liferay delivers open source
enterprise solutions for portals, publishing, content,
and collaboration, [ONLINE] Available: http://
www.liferay.com/ [30 Sept 2013]
Matlab (2013), The MathWorks, Inc., Natick,
Massachusetts, United States.
MatlabControl (2013) Matlabcontrol API for JAVA,
[ONLINE] Available: http://code.google.com/p/
matlabcontrol/ [30 Sept 2013]
Pawar, P., Jones, V., Beijnum van, B-J. F. and Hermens,
H. (2012) ‘A framework for the comparison of mobile
patient monitoring systems’. Journal of biomedical
informatics, 45 (3). pp. 544-556.
Twitter Bootstrap (2013) Sleek, intuitive, and powerful
front-end framework for faster and easier web
development,[ONLINE] Available:
http://getbootstrap.com/ [30 Sept 2013]
Xsens Technologies B. V. (2013) MTW Development KIT
Lite,[ONLINE] Available: http://www.xsens.com/en/
mtw-dk-lite [30 Sept 2013]
HEALTHINF2014-InternationalConferenceonHealthInformatics
132