Physiotherapy Exercises Evaluation using a Combined Approach based

on sEMG and Wearable Inertial Sensors

Ana Pereira

, Duarte Folgado

, Ricardo Cotrim

and In

es Sousa

Associac¸

ao Fraunhofer Portugal Research, Rua Alfredo Allen 455/461, Porto, Portugal

Plux Wireless Biosignals S. A., Avenida 5 Outubro 70, 1050-59, Lisboa, Portugal

Keywords:

Physiotherapy, Inertial Sensors, Electromyography, Body Area Networks, Automatic Segmentation.

Abstract:

The efﬁcacy of home-based physiotherapy depends on the correct and systematic execution of prescribed

exercises. Biofeedback systems enable to accurately track exercise execution and prevent patients from un-

consciously introduce incorrect postures or improper muscular loads on the prescribed exercises. This is often

achieved using inertial and surface electromyography (sEMG) sensors, as they can be used to monitor human

motion variables and muscular activation. In this work, we propose to use machine learning techniques to

automatically assess if a given exercise was properly executed. We present two major contributions: (1) a

novel sEMG segmentation algorithm based on a syntactic approach and (2) a feature extraction and classiﬁca-

tion pipeline. The proposed methodology was applied to a controlled laboratory trial, for a set of 3 different

exercises often prescribe by physiotherapists. The ﬁndings of this study support it is possible to automatically

segment and classify exercise repetitions according to a given set of common deviations.

1 INTRODUCTION

Over the last years, the world’s ageing population and

prevalence of chronic diseases has lead to an increas-

ing demand of efﬁcient healthcare systems (Stankovic

et al., 2005). In general, physiotherapy is applied not

only to functional repair, but also in the prevention

of motor complications. In order to optimize the pre-

scribed treatment program, the exercises must be ex-

ecuted repeatedly and in the correct manner.

Home-based exercise physiotherapy allows stim-

ulating muscular activity more often, by enabling the

patient to execute the prescribed exercises at home.

However, home-based physiotherapy comes at the

cost of an additional effort to properly educate the

patients for its beneﬁts, allowing to maintain a con-

tinued adherence to the program (Bassett, 2003). The

exercises should be correctly and rigorously executed,

however, patients often demonstrate uncertainty with

regards to proper exercise technique and not remem-

bering the complete training program as presented on

the clinic (Smith et al., 2005). In these circumstances,

several deviations may occur from the ideal move-

ment: the unconscious introduction of compensation

movements or postures, insufﬁcient range of move-

ments, improper timing of muscular activation or even

biomechanical misalignment.

In order to overcome such challenges, biofeed-

back systems may be used. Biofeedback usually in-

volves measurement of a target biomedical variable

and relaying it to the user (Giggins et al., 2013). Pro-

viding patients with biofeedback during physiother-

apy can have potential therapeutic effects, as it en-

sures movements and loads are being executed ac-

cording to prescription and simultaneously engag-

ing patients in their physiotherapy programs (Ferreira

et al., 2014).

Biofeedback rely on different sensors to quantify

motion. Inertial sensors play an important role in

characterizing human motions, as they are able to re-

trieve motion characteristics such as acceleration, ro-

tation, angular velocity and posture information. On

the other hand, Surface electromyographic sensors

(sEMG) can be used to evaluate muscular activation

and contraction.

This work presents a feasibility study aiming to

combine inertial and sEMG information. The main

motivation lies on the challenges arising from real

home-based physiotherapy programs. During such

sessions, which may exhibit variability across sub-

jects and environments, sEMG might be used to pre-

cisely identify the intervals of muscular activation. In-

ertial information is then used to characterize the pos-

ture and movement correctness, allowing a more ac-

curate temporal resolution of classiﬁcation.

The rest of this paper is organized as follows:

Section 2 describes previous work on biofeedback

systems and methodologies using inertial and sEMG

Pereira, A., Folgado, D., Cotrim, R. and Sousa, I.

Physiotherapy Exercises Evaluation using a Combined Approach based on sEMG and Wearable Inertial Sensors.

DOI: 10.5220/0007391300730082

In Proceedings of the 12th International Joint Conference on Biomedical Engineering Systems and Technologies (BIOSTEC 2019), pages 73-82

ISBN: 978-989-758-353-7

senors applied to motion quantiﬁcation. Section 3 de-

scribes the methods used for evaluating physiotherapy

exercises, while Section 4 and 5 present the results

and its discussion, respectively. Section 6 concludes

this paper, highlighting some areas of future work.

2 RELATED WORK

In order to ensure the proper efﬁcacy of physical

physiotherapy programs, there is a need for contin-

uous and systematic evaluation of the execution of

prescribed exercises. Recent research has been try-

ing to explore the opportunities arising from techno-

logical advances to provide enhancements in moni-

toring physiotherapy exercises in home environment.

Outside the clinical environment, biofeedback sys-

tems rely on retrieving motion data, properly quan-

tify and characterize such data and issue recommen-

dations to the patient. Methods to achieve biome-

chanical analysis consist of force and balance plat-

forms, vision-based motion capture systems and sys-

tems based on inertial sensors. (Barandas et al., 2015)

used Microsoft Kinect to retrieve Range of Motion to

provide real-time biofeedback. (Pereira et al., 2017)

used multiple Inertial Measurement Unit (IMU) to es-

timate joint angles with a strong correlation between

the proposed approach and the groundtruth, which

was based on a video system.

For an unobtrusive evaluation of exercise quality,

studies investigated the feasibility of inertial sensors

to provide accurate classiﬁcation of exercise perfor-

mance in patients executing lower limb exercises for

rehabilitation monitoring. (Giggins et al., 2014) used

a logistic regression to classify between correctly and

incorrectly labelled variations of 7 types of exercises,

achieving an accuracy of 81-83% on binary exercise

classiﬁcation and 61-63% on multi-label classiﬁca-

tion (i.e. characterizing the type of error executed).

(Huang et al., 2016) combine accelerometer and gy-

roscope data from 3 inertial sensors located on tight,

shin and the foot, and through logistic regression, De-

cision Tree, Multilayer Perception Neural Network,

Support Vector Machines, Random Forest, and Ad-

aboost classiﬁers, achieved accuracies between 78-

97% when classifying normal vs. error, and 92-97%

when classifying the type of error occurred. More

recently, (Bevilacqua et al., 2018) applied similar

approaches to knee rehabilitation exercises on both

clinical and healthy subjects. Using a single inertial

sensor located on the shin, a binary classiﬁcation us-

ing Random Forest and Decision Trees were applied

to 4 different knee rehabilitation exercises, achieving

overall accuracies of 88-97%.

Whilst inertial sensors proved to be valuable

sources of information to characterize exercise execu-

tion, they still face some inherent limitations on dis-

tinguishing active vs. passive performance of a move-

ment or ensuring information is being extracted from

muscular loaded or unloaded performance of a given

activity. sEMG can be used to overcome such limi-

tations as the amplitude of sEMG signal is related to

muscle torque and activation (De Luca, 1997). With

this reasoning, (Roy et al., 2009) studied the feasibil-

ity of introducing sEMG data while monitoring activ-

ities of daily living in functional assessment of stroke

patients. (Liu et al., 2017) proposed the develop-

ment of an upper limb rehabilitation training system

designed to be used by children with cerebral palsy.

(Ghasemzadeh et al., 2010) studied the application of

sEMG to assess human balance, where ﬁducial fea-

tures based on the sEMG were used, with high accu-

racy, to provide signiﬁcance of each quantitative pa-

rameter applied to balance assessment.

Another speciﬁc application of wearable technol-

ogy in physiotherapy is the development of smart

gloves to be integrated into serious games. The com-

bination of IMUs to estimate orientation and piezore-

sistive force sensors to estimate ﬁngers’ compression

and ﬂexion can be used in hand movement physio-

therapy systems for stroke patients. (Sun et al., 2017)

implemented a linear discriminant analysis classiﬁer

to distinguish between a basic set of hand gestures

and key press events. (Alexandre and Postolache,

2018) proposed a virtual reality game to stimulate pa-

tients performing interactive exercises while simulta-

neously recording motion parameters.

The related work allowed to highlight evidence of

the notable properties of the IMU and sEMG sensors

in the evaluation of Human motion, more precisely in

physiotherapy contexts. Exercise execution in home

environments, in which there is no direct supervi-

sion of a physiotherapist, must be accurately moni-

tored. Patients may inadvertently perform additional

deviations from prescribed exercises by performing

compensation movements. Therefore, it is impor-

tant to evaluate whether compensation exists when the

muscular activation is being employed. This can be

achieved by means of the sEMG, which in long-term

home sessions might be used to accurately identify

the intervals of muscular activation. Over those in-

tervals, an accurate posture and exercises correctness

evaluation can be performed.

BIOSIGNALS 2019 - 12th International Conference on Bio-inspired Systems and Signal Processing

3 MATERIALS AND METHODS

A cross-sectional analytic study was conducted to ex-

amine two research questions: 1) whether sEMG sen-

sors can be used to automatically segment physio-

therapy exercises repetitions, and 2) whether phys-

iotherapy exercise classiﬁcation performance can be

improved using information regarding the posture of

the patient during the exercise.

For that purpose, 7 subjects were recruited to per-

form three different physiotherapy exercises. Two

Body Network Area (BAN)s

(depicted on Figure 1

a) and two inertial units (Fraunhofer AICOS, 2016)

(depicted on Figure 1 b) were placed on the subject’s

body in locations deﬁned to maximize the retrieval

of relevant information for each exercise. During the

acquisition, annotations were performed in real-time

regarding instants that correspond to transitions be-

tween repetitions. Sensor data was then segmented

manually (i.e. using the annotations) and automati-

cally using the methodology that will be thoroughly

presented in Section 3.3. A machine learning pipeline

was then applied for classifying the exercises into cor-

rect and incorrect executions.

Figure 1: Wearables used in this study: (a) two Body Net-

work Areas placed on the lower and upper trapezium, and

(b) two inertial units placed on the upper arm and forearm.

3.1 Data Collection

Participants

Seven healthy subjects, with an average age of 27

± 1 years old, 4 men, 3 women, participated in this

study. All participants had an active lifestyle and did

not previously executed the prescribed exercises in

physiotherapy contexts. Subjects requiring physical

physiotherapy were not included in this ﬁrst stage

of the study since only the feasibility of the listed

research questions was being explored at this stage.

All participants provided informed consent before

starting data collection.

http://www.biosignalsplux.com/en/muscleban

Sensors

Two types of wearables were used for data collec-

tion: two inertial sensor units (equipped with a tri-

axial accelerometer, gyroscope and magnetometer),

and two BAN’s equipped with an electromyography

sensor and an accelerometer. Inertial units were at-

tached to the body through bracelets and BANs were

attached to speciﬁc body locations using electrodes.

These devices communicated wirelessly with a smart-

phone via Bluetooth Low Energy. The inertial data

from both wearables was collected at 50 Hz. The raw

sEMG was acquired at 1000 Hz, the sEMG envelope

was calculated locally on the device and the resulting

sEMG envelope was streamed at 50 Hz. The devices

were placed on different locations of the body accord-

ing to the exercise: inertial units were placed on spe-

ciﬁc body segments for the assessment of posture on

upper and lower limbs and the BAN’s wearables were

placed on body locations which enabled to measure

muscular activity.

Protocol

The data collection protocol was deﬁned by a physio-

therapist to ensure the exercises are relevant in clinical

practice. Three physiotherapy exercises were selected

from the Physiotec

exercise database. Physiotec

divides physiotherapy exercises into three phases:

phase 1 consists of static exercises, phase 2 is com-

posed of dynamic and analytic exercises, and phase

3 includes dynamic and functional exercises. In or-

der to promote variability, one exercise of each phase

was selected. For each exercise, the physiotherapist

deﬁned two possible deviations which represented in-

correct human postures often occurring during exer-

cises execution. The selected exercises, respective de-

viations and wearables’ location are listed in Table 1.

All data collection was performed in laboratory

settings, where wearables were placed on each partic-

ipant by the physiotherapist. Participants performed

a variable number of repetitions (between ﬁve to ten)

of each of the three studied exercises. The exercises

were executed correctly and intentionally incorrectly,

according to the deviations deﬁned in Table 1. All the

exercises and deviations were described to each par-

ticipant prior to the start of data collection. The in-

stants corresponding to the beginning and end of each

repetition were manually annotated by a researcher

during the protocol, consisting of groundtruth for the

segmentation algorithm.

https://www.physiotec.ca/index.php

Physiotherapy Exercises Evaluation using a Combined Approach based on sEMG and Wearable Inertial Sensors

Table 1: Exercises, wearables location, and deviations studied for each exercise (I refers to the inertial units, B to the BANs,

and D represents the different deviations).

Exercise Wearables Location Deviations

1 - Isometric scapular retraction

strengthening (Phase 1)

I - wrist and arm

B - lower and upper trapezium

D1 - Forearm deviated from horizontal position (right forearm up)

D2 - Forearm deviated from horizontal position (left forearm up)

2 - Prone scapular retraction (Phase 2)

I - wrist and arm

B - serratus and upper trapezium

D1 - Compensatory projection of trunk (arms deviated from vertical)

D2 - Incorrect arm position (shoulder displaced backwards)

3 - Forward lunge (Phase 3)

I - wrist and thigh

B - knee and upper trapezium

D1 - Leg not perpendicular with the ground

D2 - Compensatory leg deviation from vertical position

Figure 2: Angles (in degrees) computed from orientation during one repetition (x-axis) for correct (C) execution and with

deviations 1 (D1) and 2 (D2). (a) Angle between the arm and forearm (elbow ﬂexion/extension angle) during exercise 2, (b)

angle with vertical of the forearm during exercise 1; (c) angle with vertical of the upper arm during exercise 2.

3.2 Signal Processing

The data obtained from the four wearable devices was

preprocessed in order to reduce undesired noise using

a low-pass ﬁlter. In order to characterize posture, the

tilt angles (roll and pitch) were obtained from the ac-

celerometer data of the BANs wearables. Addition-

ally, to have more information regarding human pos-

ture, data from the accelerometer and gyroscope of

these devices was fused using a second order com-

plementary ﬁlter, bringing together the relevant infor-

mation of each sensor to compute the orientation of

the device. Applying the methodology developed in

(Pereira et al., 2017), the orientation obtained enabled

to compute the angle between the two inertial units in

exercises 1 and 2, that is the angle between the arm

and the forearm, which corresponds to the elbow ﬂex-

ion/extension angle (example illustrated in Figure 2 a

for exercise 2), and also the angle of each inertial unit

with the vertical (example illustrated in Figure 2 b and

c for exercise 1). Angular information of anatomical

segments allows discriminating between correct exe-

cution and the prescribed deviations. Proper assess-

ment of exercise execution should not only ensure the

patient is performing adequate muscular contractions,

but also ensure the contractions are being performed

with the correct posture and without the intentional

introduction of compensatory movements.

3.3 Segmentation

As previously discussed in Section 2, the sEMG sig-

nal is related with muscular activation. The segmen-

tation of sEMG comprises the task of identifying the

temporal intervals in which activation is present, quite

often by analyzing the sEMG envelope. In this study,

we propose and validate a new method based on a

syntactic approach. We used a recent tool called

SSTS (Rodrigues et al., 2019), which is capable of ex-

ploring time series data for pattern and query search

tasks.

Human reasoning has an inherent capability for

recognizing patterns and complex structures. We

can take advantage of this characteristic to ease the

process of ﬁnding patterns in several time series

applications. SSTS aims to facilitate the interaction

between data scientists and the challenges arising

from data manipulation and knowledge extraction.

By proposing a symbolic method for pattern search,

which is tightly related to the reasoning and visual

analysis of time series data, it allows improving the

pattern and query search task productivity, which was

properly demonstrated on the aforementioned work.

In this work, we also validate that assumption moti-

vated by delivering a new automatic segmentation for

sEMG data with minimum design effort taking the

advantages of the SSTS capabilities. SSTS converts

BIOSIGNALS 2019 - 12th International Conference on Bio-inspired Systems and Signal Processing

0z0p0p0p0p1p1p1p1n1n0n0n0p0p0p1p1p1n1n1n0n0n0n0z

Pre-Processing Symbolic Connotation Search

☴ 0.1 50 ~ 25

⇞0.5 ∂ 0.01

0n0p

☴ (s, fc1, fs) – Lowpass filter of 0.1 Hz with sampling frequency

of 50 HZ

~ (s, w) – Moving average filter with window size of 25 samples

⇞(s, thr) – Amplitude thresholding. Above threshold is “1”, under thr

is “0”;

∂(s, thr) - Derivative of the signal (s) with a rounding threshold. P - is

the positive derivative, n - the negative and z - the stationary part.

(0n0p) – Searches for this occurence, which is a local minima.

Amplitude (mV)

Time(s)

Amplitude

Time

0z0p0p0p0p1p1p1p1n1n0n0n0p0p0p1p1p1n1n1n0n0n0n0z

Figure 3: Initial sEMG segmentation. The sEMG signal is divided into cycles, which give an initial estimation of the activation

period. Cycles are calculated by ﬁnding local minima using Syntactic Search for Time Series (SSTS).

0z0z0z0z1p1p1p0n0p1n1n0z0z1p1p0p0p0n1p1n1n1n0z0z

Pre-Processing Symbolic Connotation Search

☴ 1 50 ~ 25

↕0.5 ∂ 0.01

onset: 1p | offset: 1n

↕(s, thr) – Determines if amplitude from a local minima to

maximum (and vice-versa) is greater than thr.

∂(s, thr) - Derivative of the signal (s) with a rounding threshold. P - is

the positive derivative, n - the negative and z - the stationary part.

(1p) – Detects all ocurrences of “1p”.

(1n) – Detects all ocurrences of “1n”.

Amplitude (mV)

Time(s)

Amplitude

Time

☴ (s, fc1, fs) – Lowpass filter of 1 Hz with sampling frequency of

50 HZ

~ (s, w) – Moving average filter with window size of 25 samples

Figure 4: Accurate sEMG onset and offset detection. For each cycle previously detected the onsets are found by using SSTS

to query for ”a pronounced amplitude rise” and offsets by querying ”a pronounced amplitude decrease.” False positives are

eliminated by only considering the ﬁrst occurrence of the onset search regex string and last occurrence of the offset search

string.

time series data from numeric to string domain with

the resource of 3 symbolic steps: (1) preprocessing;

(2) symbolic connotation and (3) search. In the string

domain, characterized by a symbolic representation

deﬁned by the user, queries are performed using string

query methods. Whilst on the rest of this subsection

we provide guidance to follow the methods used by

SSTS to segment the sEMG, it is recommended that

for a comprehensive understanding the reader may re-

fer to the original publication.

Overview

The proposed sEMG segmentation is composed by

two stages. Firstly, the sEMG activation periods are

initially divided into cycles. This is accomplished by

analyzing the lower frequency components of the sig-

nal as depicted on Figure 3. Whilst this procedure

may at an initial glance be sufﬁcient, it does not al-

low an accurate temporal resolution of the onset and

offset events. Secondly, after the initial cycle segmen-

tation there is a more accurate onset and offset detec-

tion within the time intervals corresponding to each

detected cycle as depicted on Figure 4. This com-

bined approach also reduces the occurrence of false

positives.

Preprocessing

During this step, symbolic tokens are attributed to

common methods for preprocessing. A string con-

taining the set of tokens and their corresponding argu-

ments are passed as input, thus, corresponding to the

preprocessing methods and parameters. The sEMG

envelope was ﬁltered using lowpass ﬁltering and mov-

ing average as described in Step 1 of Figure 3.

Physiotherapy Exercises Evaluation using a Combined Approach based on sEMG and Wearable Inertial Sensors

Connotation

The symbolic connotation step generates a sequence

of symbols based on connotation methods deﬁned by

the user. Ideally, the method should be related to spe-

ciﬁc attributes of the time series that are considered

relevant for the target search procedure. Each sample

is represented by a number of tokens that correspond

to the number of connotation methods that were used.

In the current approach, two connotations methods

were applied in the initial stage - amplitude thresh-

olding and ﬁrst derivative: ”0p” represents a sam-

ple with value below a given threshold with positive

derivative; ”0n” represents a sample with value below

a given threshold with negative derivative; ”1z” would

represent a sample with amplitude above the given

threshold with zero derivative. The same reasoning

applies to other connotation methods. The sEMG ac-

tivation is characterized in the sEMG envelope by a

signiﬁcant rise in amplitude in comparison with the

baseline (onset), followed by a plateau during mus-

cular activation and terminating in a signiﬁcant de-

crease in amplitude to baseline (offset). A well estab-

lished method that is frequently used, takes into ac-

count this property. (Hodges and Bui, 1996) required

that the mean of the points in a sliding window ex-

ceed a given threshold (the value usually is a multiple

of the standard deviation). The applied connotation

methods were inspired by that approach and are de-

scribed in Step 2 of Figure 4.

After the connotation step, the signal is translated

from the numeric to the symbolic domain. A search

string is used in the form of a regular expression

(regex). The regex used in both stages is depicted in

Step 3 of Figure 3 and 4. In the initial stage, in which

the objective is to ﬁnd the occurrences of local min-

ima, the search consists of ﬁnding all the occurrences

of: ”a negative ﬁrst derivative, followed by a positive

ﬁrst derivative, below a given amplitude threshold”,

which is expressed as ”0n0p”. The accurate onset de-

tection is performed by querying for a ”pronounced

amplitude rise”, which holds true by the connotation

method - ”1p”. Offset detection is achieved by query-

ing for a ”pronounced amplitude decrease” - ”1n”. It

is worth to mention that muscular activity does vary

during contraction periods and thus, the EMG enve-

lope may in some circumstances have associated vari-

ability and noise. This fact leads to the appearance of

false positives. However, since a rough estimation of

EMG segmentation was achieved on the initial stage,

we can remove the false positives by only considering

the ﬁrst positive match of ”1p” for onset and the last

positive match of ”1n” for the offsets.

3.4 Machine Learning Pipeline

After segmentation, temporal and statistical do-

main features were extracted for each time win-

dow (Figueira et al., 2016). Statistical features such

as skewness, kurtosis and histogram, and tempo-

ral such as mean, median, maximum, minimum,

variance, temporal centroid, standard deviation, root

mean square, and auto correlation, were extracted us-

ing Python numpy v1.11.3. After feature extraction,

it was possible to conclude that many of the features

were correlated and could be removed without loos-

ing information, therefore, forward feature selection

was applied.

Supervised learning methods were used to dis-

criminate between a correct execution and a execution

with a deviation, specifying the type of the deviation

occurred. Therefore, each time window was classi-

ﬁed into correct (C), deviation 1 (D1) and deviation 2

(D2). Using scikit-learn v0.19.1, a Python Machine

Learning library, on Python 2.7.13, four classiﬁers

were tested to addressed this problem: Decision Trees

(DT), K-Nearest-Neighbours (KNN), Support Vector

Machines (SVM), and Random Forest (RF). The clas-

siﬁers trained separately manual and automatic seg-

mented time windows (repetitions) and also two dif-

ferent set of features: features extracted only from the

BANs and then features from all wearables, namely

the two inertial units and the two BANs.

For validation purposes, leave-one-user-out-cross

validation was employed in order to ensure indepen-

dence of the subject. To evaluate the performance

of each classiﬁer, accuracy, sensitivity, and speciﬁcity

were computed. While accuracy measures the overall

effectiveness of a classiﬁer, sensitivity measures the

effectiveness of a classiﬁer at identifying a desired la-

bel, and speciﬁcity measures the classiﬁers ability to

detect negative labels.

Table 2: Total number of time windows for all three classes

based on the method of segmentation.

Class Exercise 1 Exercise 2 Exercise 3

Manual Segmentation

C 63 71 57

D1 54 56 28

D2 53 55 27

Automatic Segmentation

C 55 67 55

D1 51 56 27

D2 46 41 27

BIOSIGNALS 2019 - 12th International Conference on Bio-inspired Systems and Signal Processing

Table 3: Multi-label classiﬁcation results obtained for DT, KNN, RF and SVM classiﬁers for each exercise and for each set of

features (features extracted only from BAN and from BAN and Inertial Units). Mean and standard deviation of the speciﬁcity

and sensitivity are reported in %.

DT KNN RF SVM

Exer. Set Meas. C D1 D2 C D1 D2 C D1 D2 C D1 D2

BAN

Sens 73 ± 32 98 ± 2 88± 18 78 ± 9 92 ± 5 90 ± 12 67 ± 30 87 ± 20 82 ± 17 80 ± 21 86 ± 17 97 ± 5

Spec 82 ± 28 66 ± 36 73 ± 35 73 ± 25 78 ± 17 70 ± 15 61 ± 43 52 ± 44 57 ± 41 86 ± 18 80 ± 29 63 ± 29

BAN + Inertial

Sens 100 ± 0 100 ± 0 99 ± 1 100 ± 0 100 ± 0 99 ± 26 100 ± 0 96 ± 5 99 ± 1 100 ± 0 100 ± 0 100 ± 0

Spec 100 ± 0 98 ± 3 100 ± 0 100 ± 0 100 ± 0 100 ± 0 100 ± 0 98 ± 3 93 ± 11 100 ± 0 100 ± 0 100 ± 0

BAN

Sens 76 ± 13 90 ± 14 90 ± 8 86 ± 13 87 ± 7 91 ± 9 61 ± 23 85 ± 15 88 ± 18 54± 37 85 ± 20 96 ± 5

Spec 80 ± 20 61 ± 29 75 ± 25 74 ± 17 82 ± 23 75 ± 19 74 ± 28 36 ± 22 60 ± 33 78 ± 21 53 ± 40 41 ± 40

BAN + Inertial

Sens 96 ± 10 97 ± 7 97 ± 4 99 ± 1 100 ± 0 99 ± 2 95 ± 7 100 ± 0 94 ± 9 98 ± 5 98 ± 3 99 ± 1

Spec 92 ± 14 95 ± 12 91 ± 22 98 ± 3 98 ± 3 100 ± 0 92 ± 14 89 ± 15 98 ± 3 100 ± 0 97 ± 4 95 ± 12

BAN

Sens 71 ± 21 91 ± 13 89 ± 12 48 ± 20 94 ± 5 98 ± 2 55 ± 15 79 ± 13 91 ± 13 40 ± 14 98 ± 3 98 ± 2

Spec 95 ± 7 50 ± 30 71± 34 96 ± 4 53 ± 33 27 ± 20 77 ± 16 51 ± 27 22 ± 23 98 ± 3 66 ± 41 10 ± 15

BAN + Inertial

Sens 100 ± 0 97 ± 4 100 ± 0 91 ± 16 97 ± 5 100 ± 0 95 ± 5 95 ± 6 90 ± 15 84 ± 13 100 ± 0 100 ± 0

Spec 96 ± 7 100 ± 0 100 ± 0 96 ± 7 96 ± 7 86 ± 29 91 ± 9 71 ± 28 93 ± 14 100 ± 0 100 ± 0 70 ± 25

4 RESULTS

In this study, 7 participants performed three exercises

which were labelled with 3 classes: correctly, and

with deviations 1 and 2. Table 2 details the dataset

collected applying manual and automatic segmenta-

tion, resultant from the annotations performed dur-

ing data collection, and from the automatic segmen-

tation approach described in Section 3.3, respectively.

As it can be seen, the number of detected time win-

dows (repetitions) applying automatic segmentation

is lower than the ones that were manually annotated.

Results of the leave-one-user-out-cross validation

for the assessment of the classiﬁcation performance

are presented in Table 3 and Table 4. It is presented

the performance scores obtained for each exercise

with the four classiﬁers considering the two sets of

features: features extracted from the BAN data and

features extracted from both the BAN and the inertial

units. Relatively high average accuracy scores were

achieved considering features from both wearable for

all three exercises, being the DT and the KNN classi-

ﬁers the ones that achieved higher accuracies. Addi-

tionally, sensitivity and speciﬁcity were also high in

general, however, for exercise 3, KNN, RF, and SVM

demonstrated a few confusion in distinguish classes

D1 and D2, which compromise the value of the speci-

ﬁcity for these two classes. Nonetheless, when us-

ing only features from the BANs wearables, accuracy

considerably decreased, as well as speciﬁcity and sen-

sitivity, for all four classiﬁers and exercises. For this

case, the standard deviation values of the metrics pre-

sented are high, which means that the values obtained

during validation were actually different depending

on the test user.

Table 5 compares the overall accuracy of the clas-

siﬁcation performance for the manual and automatic

segmentation, which was obtained when using fea-

tures from all wearables. As it can be seen, the values

presented are approximately identical, except in exer-

cise 2 for RF and SVM classiﬁers.

Table 4: Mean accuracy and standard deviation (%) for DT,

KNN, RF and SVM classiﬁers for each exercise and for

each set of features (features extracted only from BAN and

from BAN and Inertial Units). Mean and standard deviation

are reported in %.

Exer Set DT KNN RF SVM

BAN 76 ± 15 74 ± 8 60 ± 7 78 ± 13

BAN + Inertial 99 ± 1 100 ± 0 97 ± 3 100 ± 0

BAN 72 ± 12 77 ± 10 60 ± 13 63 ± 11

BAN + Inertial 94 ± 9 98 ± 2 93 ± 3 97 ± 5

BAN 71 ± 3 86 ± 7 57 ± 13 63 ± 6

BAN + Inertial 98 ± 4 98 ± 2 97 ± 3 92 ± 6

Table 5: Overall accuracy (%) for DT, KNN, RF and SVM

classiﬁers for each exercise and for manual and automatic

segmentation.

Exer Segmentation DT KNN RF SVM

Manual 99 ± 1 100 ± 0 97 ± 3 100 ± 0

Automatic 100 ± 0 100 ± 0 97 ± 5 99 ± 1

Manual 94 ± 9 98 ± 2 97 ± 3 97 ± 5

Automatic 97 ± 4 96 ± 7 89 ± 16 85 ± 11

Manual 98 ± 4 98 ± 2 97 ± 3 92 ± 6

Automatic 93 ± 6 99 ± 6 96 ± 6 89 ± 6

5 DISCUSSION

This research study has explored whether sEMG sen-

sors can be used to automatically detect exercises rep-

etitions and whether additional inertial units placed on

strategic segments of the body can contribute to dis-

tinguish correct exercise performance from deviations

based on the human posture.

Physiotherapy Exercises Evaluation using a Combined Approach based on sEMG and Wearable Inertial Sensors

The use of sEMG sensor for automatic segmenta-

tion was achieved using the capabilities of SSTS. In

order to ensure that the windows provided to classi-

ﬁcation stage were adequate and, consequently, the

classiﬁer was learning from representative data, a val-

idation stage was performed. The automatically de-

tected muscular activation periods (provided by the

onset/offset pairs) that were shorter or longer than the

expected duration according to the protocol weighted

with a given tolerance were discarded. The number of

discarded samples was higher for Exercises 1 and 2.

In fact, later visual analysis of sEMG signals and au-

tomatic segmentation results allow to conclude that in

those exercises noise and artifacts were more promi-

nent, lowering the difference between the amplitude

in muscular activation and baseline, hampering an ad-

equate calculation of threshold.

The results presented on Table 3 and 4 revealed

that is possible to correctly classify physiotherapy ex-

ercises performance from different phases using in-

ertial units with satisfactory levels of accuracy. The

use of inertial units showed a signiﬁcant improvement

(an increase of 27%, in average) on the accuracy of

all classiﬁers in each exercise. Since the exercises

deviations were deﬁned based on incorrect postures,

the attachment of inertial units which measure incli-

nation of body segments, enabled to identify more

accurately incorrect executions, which may be unde-

tectable if only the tilt angles of BANs were used.

Machine learning classiﬁcation techniques were used

to quantify wearable data acquired during the three

exercises studied. Multi-label classiﬁers (which de-

termine which deviation in a set of deviations) were

employed and the efﬁcacy of these classiﬁers was

quantiﬁed using three efﬁcacy scores; accuracy, sen-

sitivity, and speciﬁcity. Results showed that KNN

classiﬁer achieved a recognition accuracy of ≥98%,

which is greater than the other classiﬁers tested. How-

ever long training and testing times are required for

KNN. The DT classiﬁer achieved the second highest

recognition accuracy in this study, and its training and

testing times were lower than the KNN. Thus, the DT

classiﬁer proved to be an efﬁcient classiﬁer for de-

tecting deviations from correct execution of physio-

therapy exercises.

After identifying the improvement of the classiﬁ-

cation results through the use of inertial units in body

segments, DT, KNN, RF, and SVM classiﬁers were

also validated in the automatically segmented win-

dows. The results of Table 5 demonstrated no signif-

icant changes in the accuracy values of all classiﬁers

for each exercise, which proves the feasibility of the

syntactic sEMG onset detection of exercise repetition,

ensuring that muscular activation is being employed.

For the automatic segmented windows, DT and KNN

also achieved the best performances.

The results obtained allow a preliminary com-

parison to previous work that evaluated the use of

inertial units and machine learning to assess exer-

cises performance. The methodology presented in

our study achieved higher results than (Giggins et al.,

2014), and similar results to the (Huang et al., 2016)

and (Bevilacqua et al., 2018) studies. Additionally,

(Huang et al., 2016) and (Bevilacqua et al., 2018)

also used automatic segmentation based on a template

matching algorithm. However, it is worth to mention

that the dataset of this study is signiﬁcantly smaller

than the datasets of the aforementioned studies. The

size of the datasets was 58, 69 and 54 participants,

respectively for (Giggins et al., 2014), (Huang et al.,

2016) and (Bevilacqua et al., 2018). Therefore, the

developed methodology needs to be tested and vali-

dated against a larger dataset for a more representative

comparison with previous work. It is expected that

accuracy might decrease with an increase of dataset

size (Schnack and Kahn, 2016). However, we believe

the process of combining inertial and sEMG data, by

assuring classiﬁcation is performed in the moment of

muscular activation, provides more relevant informa-

tion to the exercise performance and the physiothera-

pist.

Besides the size of the dataset collected, there are

other limitations of this study that need to be consid-

ered. Firstly, the data collected was gathered in lab-

oratory settings, where the exercises were performed

under controlled conditions. These conditions may

differ from what may occur at home. Furthermore,

only two deviations were deﬁned per exercise. In a

home-based physiotherapy context, other deviations

in performance may occur that were not considered

when training the classiﬁers. Another limitation of

this study is that the sample selected was a group of

healthy subjects, so it was not possible to validate

whether the classiﬁers performed differently for dif-

ferent populations.

Nevertheless, the results obtained in this study are

important as they provide further evidence to suggest

that sEMG signal could be used to detect exercise

repetitions, and that features based on human pos-

ture could support the assessment of exercises perfor-

mance. Moreover, the exercises selected in this study

were from a worldwide database developed by phys-

iotherapists, and were not speciﬁc for a single limb

as the studies found in the literature. The exercises

selected were one from each of the three phases of

the physiotherapy process, which proves that the ap-

proach developed could be adapted for a wide range

of exercises.

BIOSIGNALS 2019 - 12th International Conference on Bio-inspired Systems and Signal Processing

6 CONCLUDING REMARKS

This paper presents the development of a combined

approach based on sEMG and inertial sensors for

the evaluation of physiotherapy exercises. The ap-

plicability of our approach lies in the implementa-

tion on biofeedback systems to optimize home-based

exercise execution. sEMG signal was used to iden-

tify temporal intervals in which muscular activation

was present. This way, exercise repetitions were seg-

mented into time windows where features related with

human posture were extracted. Then, these features

were fed to DT, KNN, RF, and SVM classiﬁers, which

were able to distinguish between correct execution

and deviations with an accuracy ≥92%. As part of our

ongoing research, we will validate the proposed sys-

tem on more extensive datasets. The sEMG segmen-

tation will be assessed in a more controlled environ-

ment, using simulated data, to permit the evaluation

of the temporal misalignment between the detected

onset/offsets and groundtruth. The models proposed

will be tested on a more extended dataset, comprising

variability in terms of age and clinical history.

ACKNOWLEDGEMENTS

We acknowledge all participants who participated

in data collection. We would like to acknowl-

edge the ﬁnancial support obtained from the project

Physio@Home: Extending Physiotherapy Programs

to People’s Home, co-funded by Portugal 2020,

framed under the COMPETE 2020 (Operational Pro-

gramme Competitiveness and Internationalization)

and European Regional Development Fund (ERDF)

from European Union (EU), with operation code

POCI-01-0247-FEDER-017863.

REFERENCES

Alexandre, R. and Postolache, O. (2018). Wearable and iot

technologies application for physical rehabilitation. In

2018 International Symposium in Sensing and Instru-

mentation in IoT Era (ISSI), pages 1–6. IEEE.

Barandas, M., Gamboa, H., and Fonseca, J. (2015). A real

time biofeedback system using visual user interface

for physical rehabilitation. Procedia Manufacturing,

3:823–828.

Bassett, S. F. (2003). The assessment of patient adherence

to physiotherapy rehabilitation. New Zealand journal

of physiotherapy, 31(2):60–66.

Bevilacqua, A., Huang, B., Argent, R., Caulﬁeld, B., and

Kechadi, T. (2018). Automatic classiﬁcation of knee

rehabilitation exercises using a single inertial sensor:

a case study.

De Luca, C. J. (1997). The use of surface electromyography

in biomechanics. Journal of applied biomechanics,

13(2):135–163.

Ferreira, C., Guimar

aes, V., Santos, A., and Sousa, I.

(2014). Gamiﬁcation of stroke rehabilitation exercises

using a smartphone. In Proceedings of the 8th Inter-

national Conference on Pervasive Computing Tech-

nologies for Healthcare, pages 282–285. ICST (In-

stitute for Computer Sciences, Social-Informatics and

Telecommunications Engineering).

Figueira, C., Matias, R., and Gamboa, H. (2016). Body

location independent activity monitoring. In BIOSIG-

NALS, pages 190–197.

Fraunhofer AICOS (2016). A Day with Pandlets. Technical

report, Fraunhofer Portugal AICOS.

Ghasemzadeh, H., Jafari, R., and Prabhakaran, B. (2010).

A body sensor network with electromyogram and in-

ertial sensors: Multimodal interpretation of muscular

activities. IEEE transactions on information technol-

ogy in biomedicine, 14(2):198–206.

Giggins, O. M., Persson, U. M., and Caulﬁeld, B. (2013).

Biofeedback in rehabilitation. Journal of neuroengi-

neering and rehabilitation, 10(1):60.

Giggins, O. M., Sweeney, K. T., and Caulﬁeld, B. (2014).

Rehabilitation exercise assessment using inertial sen-

sors: a cross-sectional analytical study. Journal of

neuroengineering and rehabilitation, 11(1):158.

Hodges, P. W. and Bui, B. H. (1996). A compari-

son of computer-based methods for the determina-

tion of onset of muscle contraction using electromyo-

graphy. Electroencephalography and Clinical Neu-

rophysiology/Electromyography and Motor Control,

101(6):511–519.

Huang, B., Giggins, O., Kechadi, T., and Caulﬁeld, B.

(2016). The limb movement analysis of rehabilita-

tion exercises using wearable inertial sensors. In En-

gineering in Medicine and Biology Society (EMBC),

2016 IEEE 38th Annual International Conference of

the, pages 4686–4689. IEEE.

Liu, L., Chen, X., Lu, Z., Cao, S., Wu, D., and Zhang,

X. (2017). Development of an emg-acc-based upper

limb rehabilitation training system. IEEE Transac-

tions on Neural Systems and Rehabilitation Engineer-

ing, 25(3):244–253.

Pereira, A., Guimar

aes, V., and Sousa, I. (2017). Joint an-

gles tracking for rehabilitation at home using inertial

sensors: a feasibility study. In Proceedings of the

11th EAI International Conference on Pervasive Com-

puting Technologies for Healthcare, pages 146–154.

ACM.

Rodrigues, J., Folgado, D., Belo, D., and Gamboa, H.

(2019). Ssts: A syntactic tool for pattern search on

time series. Information Processing & Management,

56(1):61–76.

Roy, S. H., Cheng, M. S., Chang, S.-S., Moore, J., De Luca,

G., Nawab, S. H., and De Luca, C. J. (2009). A

combined semg and accelerometer system for mon-

itoring functional activity in stroke. IEEE Transac-

Physiotherapy Exercises Evaluation using a Combined Approach based on sEMG and Wearable Inertial Sensors

tions on Neural Systems and Rehabilitation Engineer-

ing, 17(6):585–594.

Schnack, H. G. and Kahn, R. S. (2016). Detecting neu-

roimaging biomarkers for psychiatric disorders: sam-

ple size matters. Frontiers in psychiatry, 7:50.

Smith, J., Lewis, J., and Prichard, D. (2005). Physiother-

apy exercise programmes: Are instructional exercise

sheets effective? Physiotherapy theory and practice,

21(2):93–102.

Stankovic, J., Cao, Q., Doan, T., Fang, L., He, Z., Kiran,

R., Lin, S., Son, S., Stoleru, R., and Wood, A. (2005).

Wireless sensor networks for in-home healthcare: Po-

tential and challenges. In High conﬁdence medical de-

vice software and systems (HCMDSS) workshop, vol-

ume 2005.

Sun, Q., Gonzalez, E., and Abadines, B. (2017). A wear-

able sensor based hand movement rehabilitation and

evaluation system. In Sensing Technology (ICST),

2017 Eleventh International Conference on, pages 1–

4. IEEE.

BIOSIGNALS 2019 - 12th International Conference on Bio-inspired Systems and Signal Processing