A Kinect-based Monitoring System for Stroke Rehabilitation

Min Hun Lee

, Daniel Siewiorek

, Asim Smailagic

, Alexandre Bernadino

and Sergi Bermúdez I Badia

Carnegie Mellon University, 5000 Forbes Avenue, Pittsburgh, PA, U.S.A.

Instituto Superior Técnico, Av. Rovisco Pais, 1, Lisbon, Portugal

Madeira Interactive Technology Institute and Faculdade de Ciências Exatas e da Engenharia, Universidade da Madeira,

Caminho da Penteada, 9020-105 Funchal, Portugal

1 OBJECTIVES

Therapists monitor and evaluate stroke patient’s

motor abilities with clinical tests to individualize

clinical interventions. After a clinical session, a

therapist designs task-oriented exercises for a patient

and requests self-reporting of patient’s adherence on

exercise regimen. However, outpatients, who cannot

receive feedback, often show low adherence (Proot

et al., 2005) et al, leading to sparse self-reports. It is

difficult for therapists to follow patient’s progress.

To address this challenge, this paper describes a

Kinect-based monitoring system that evaluates

performance and provides real-time feedback for

four stroke rehabilitation exercises. Our preliminary

study showed that this monitoring system can

accurately monitor in-home stroke rehabilitation

exercises.

2 METHOD

2.1 System Design

We designed a monitoring system for stroke

rehabilitation as shown in Figure 1. Even if a

therapist is not present, this system can perform

monitoring tasks. It provides feedback and guidance

to support achieving therapist’s prescribed exercise

regimen.

During an exercise movement, this monitoring

system tracks body joints in x, y, z coordinates using

a Kinect sensor. Given this time series kinematic

sensor data, it computes physical measurements and

pre-processes coordinates of joint trajectory into

normalized trajectory features. Thus, it reduces the

effect of user’s varying physical characteristics.

This system extracts various features for modelling

performance analysis. Performance analysis involves

three tasks: exercise type recognition, incorrect

movement detection, and performance evaluation.

Exercise type recognizer utilizes normalized

trajectory features and Hidden Markov Models

(HMMs) to recognize which exercise is performed.

After recognizing the type of an exercise,

incorrect movement detectors determine the

correctness of a movement with respect to three

performance metrics: precision, compensation, and

smoothness.

Precision represents the degree of alignment with

the target posture of an exercise. Compensation

calculates the extent of occurring compensatory

movements. Smoothness indicates the degree of

trembling movement patterns. This system models

Decision Trees for the precision and compensation

metrics and HMMs for the smoothness metric.

This system achieves the performance evaluation

by executing a probabilistic reasoning process. It

computes the correctness of three performance

metrics as a performance score.

For user engagement, this system provides

feedback based on performance analysis. Exercise

type recognizer enables to count the repetitions of an

exercise. If any incorrect movement is detected, this

system can correct any detected errors. It motivates a

user with a performance score.

2.2 Dataset

For a preliminary study, we utilize four stroke

rehabilitation exercises (Figure 2). Exercise 1 (E1) is

Figure 1: Overall Flow of Monitoring System for Stroke Rehabilitation.

Lee M., Siewiorek D., Smailagic A., Bernardino A. and BermÃždez I Badia S.

A Kinect-based Monitoring System for Stroke Rehabilitation.

In IcSPORTS 2017 - Extended Abstracts (icSPORTS 2017), pages 8-10

Figure 2: Four Stroke Rehabilitation Exercises.

Bring a Cup to the Mouth, Exercise 2 (E2) is Switch

a Light On, Exercise 3 (E3) is Troubled Cane, and

Exercise 4 (E3) is Two Hands Stand Up.

We collected both “correct” and “incorrect”

datasets of four exercises using a Kinect v2 sensor

(Microsoft, Redmond, USA). It was located at a

height of 0.72m above the floor and 2.5m away from

a subject.

For “correct” dataset, eleven healthy subjects (10

males and 1 female) with the average and standard

deviation age of 32.3 ± 5.81 years participated. Each

subject performed 15 correct repetitions of each

exercise. The “correct” dataset contains 165 sample

movements for each exercise.

For “incorrect” dataset, 5 healthy subjects (4

males and 1 female) with the average and standard

deviation age of 30 ± 3.52 years participated. Each

subject performed the different combinations of

incorrect movements. The “incorrect” dataset

contains 80 sample movements for each exercise.

3 RESULTS

We apply leave-one-subject-out cross validation and

evaluate the monitoring system using “correct” and

“incorrect” datasets. For exercise recognition, we

achieved 96.7% accuracy. Accuracies of incorrect

movement detectors are presented in Table 1. For

the accuracies of performance evaluation, we

calculated the percentage of computed scores within

ground truth scores ± margin in Table 2. Ground

truth scores indicate human observation scores and

margin is selected as 1.

Table 1: Accuracies of Incorrect Movement Detectors.

Metrics

Precision

91.07%

99.69%

94.15%

98.15%

Compensation

94.68%

94.26%

88.16%

95.10%

Smooth

98.00%

97.50%

96.80%

94.25%

Table 2: Accuracies of Performance Evaluation.

Accuracy

95.50%

87.50%

88.75%

91.25%

4 DISCUSSION

According to the preliminary evaluation, this

monitoring system has a potential to accurately

perform three monitoring tasks. This monitoring

system can offer detailed feedback on an exercise

performance without the presence of a therapist.

However, utilized datasets are collected from

healthy subjects, who acted incorrect movements.

Some trials of exercises involve exaggerated

movements, which may be different from post-

stroke survivors. Another limitation of this work is

lack of therapist’s observation scores. It is necessary

to compare ground truth scores from a therapist with

computed scores of this monitoring system. In

future, we plan to validate this monitoring system

using datasets from stroke survivors and therapist’s

observation scores.

ACKNOWLEDGEMENTS

This work was supported by the FCT through the

Augmented Human Assistance project (CMUP-

ERI/HCI/0046/2013) and SFRH/BD/113694/2015.

Figure 3: Plots of Computed Scores and Human Observation Scores.

REFERENCES

Proot, Ireen M., et al. "Stroke patients’ needs and

experiences regarding autonomy at discharge from

nursing home." Patient education and counseling 41.3

(2000): 275-283.