VIRTUAL-PHYSIO: A Virtual Assistant for Home Physiotherapy

Rehabilitation

Nicoletta Balletti

1,2

, Antonella Cascitelli

, Patrizia Gabrieli

, Emanuela Guglielmi

Gennaro Laudato

, Aldo Lazich

, Marco Notarantonio

, Rocco Oliveto

2,3

, Stefano Ricciardi

Simone Scalabrino

and Jonathan Simeone

Center for Biotechnology, Institute of Biomedical Sciences of the Ministry of Defense, Rome, Italy

University of Molise, Pesche (IS), Italy

Datasound srl, Pesche (IS), Italy

Atlantica Digital spa, Rome, Italy

Keywords:

Virtual Assistant, Home Rehabilitation, Artiﬁcial Intelligence, Motion Capturing.

Abstract:

Mobility impairments reduce the ability of patients to complete daily activities. Physio-therapeutic exercises

help patients address such limitations. Correctly executing these exercises is crucial, often requiring a phys-

iotherapist’s guidance. To address this need, combining advanced sensors with artiﬁcial intelligence offers a

promising solution for home rehabilitation, enabling remote monitoring and reducing stress. In this paper, we

introduce VIRTUAL-PHYSIO, a virtual assistant for remote rehabilitation integrated into a home-deployable

low-cost physiotherapy monitoring system 2VITA-B PHYSICAL. VIRTUAL-PHYSIO provides real-time feed-

back during rehabilitation exercises and evaluates entire sessions, allowing physiotherapists to focus on critical

cases. We experimented with VIRTUAL-PHYSIO on 51 individuals whose performances were also evaluated

by a physiotherapist as a reference. The results (i) highlight good patient acceptability for the virtual assistant,

and (ii) show that the proposed machine learning approach can effectively perform an automated evaluation

of rehabilitative movements.

1 INTRODUCTION

Rehabilitation, like prevention, promotion, treatment,

and palliation, is an important health service both in

the community and in hospitals. Physical rehabilita-

tion aims to (i) achieve complete recovery, in the case

of patients with transient motor deﬁcits, and (ii) re-

lieve suffering and provide a higher level of indepen-

dence, in the case of patients with permanent dysfunc-

tion. The proper execution of rehabilitation exercises

is crucial to recover quickly.

Physical therapy has received attention from the

computer science research community over the years,

with a special focus on home-based rehabilitation.

Home rehabilitation allows patients to complete reha-

bilitation exercises in the comfort of their own homes

by reducing the hassle and cost of commuting on a

daily or weekly basis. This helps to improve treatment

quality and hasten recovery (Maclean et al., 2002),

while also lowering hospitalisations and, as a result,

healthcare costs (Han et al., 2005). Although remote

monitoring of patients solves a logistical problem for

patients, this solution still requires the active presence

and intervention of a human expert (physiotherapist).

The reason is that only a small percentage of patients

with motor disabilities complete exercises as recom-

mended (Shaughnessy et al., 2006). The physiother-

apist is in charge of carefully observing patients per-

forming the exercises and making them correct their

movements when necessary. The natural unbalance

between the number of patients and the number of

physiotherapists constitutes a problem: The physio-

therapist needs to schedule meetings with patients.

Ideally, human experts should intervene only when

necessary, so that they can better focus on cases that

require particular attention.

To tackle this problem, we introduce VIRTUAL-

PHYSIO, a virtual assistant for home rehabilitation.

We integrated VIRTUAL-PHYSIO in the 2VITA-B

PHYSICAL system (Antico et al., 2021a), which uses

Balletti, N., Cascitelli, A., Gabrieli, P., Guglielmi, E., Laudato, G., Lazich, A., Notarantonio, M., Oliveto, R., Ricciardi, S., Scalabrino, S. and Simeone, J.

VIRTUAL-PHYSIO: A Virtual Assistant for Home Physiotherapy Rehabilitation.

DOI: 10.5220/0013127400003911

Paper published under CC license (CC BY-NC-ND 4.0)

In Proceedings of the 18th International Joint Conference on Biomedical Engineering Systems and Technologies (BIOSTEC 2025) - Volume 2: HEALTHINF, pages 467-474

ISBN: 978-989-758-731-3; ISSN: 2184-4305

467

a low-cost easy-to-install motion tracking sensor (i.e.,

the Azure Kinect DK) which proved to be suited

to perform physiotherapy monitoring at home(Antico

et al., 2021b).

VIRTUAL-PHYSIO aims to (i) guide patients while

they perform exercises by giving them feedback about

their movement, and (ii) evaluate a whole exercise

session so that a physiotherapist can be notiﬁed about

the cases that might require attention. To achieve the

ﬁrst goal, we use a virtual 3D body model that visu-

ally mirrors the patients’ movements, possibly high-

lighting limbs not correctly positioned at a given time.

To achieve the second goal, we exploit machine learn-

ing algorithms to compare the optimal movement and

the actual one to automatically distinguish sessions

that a physiotherapist would consider well-performed

from the ones containing relevant errors.

We veriﬁed the effectiveness of VIRTUAL-

PHYSIO in a controlled experiment involving 51 par-

ticipants, who used the virtual assistant during reha-

bilitation sessions. The results achieved show that (i)

the level of real-time audio-visual feedback provided

by the system during the remote rehab session greatly

increases the patient’s conﬁdence in such a system

and their willingness to attend a home rehabilitation

plan, and (ii) the proposed automated evaluation mod-

els, according to the different machine learning tech-

niques considered, achieves up to 84% accuracy.

2 RELATED WORK

In this section, we discuss previous work to support

home rehabilitation using machine learning (ML).

Recent advancements in home-based rehabilita-

tion leverage machine learning and sensor technolo-

gies to enhance patient outcomes and accessibility.

Chae et al. (Chae et al., 2020) demonstrated that

a system using a smartwatch and machine learning

could improve motor function and shoulder mobility

in chronic stroke patients, achieving notable accuracy

in performance assessments. Similarly, Osgouei et al.

(Osgouei et al., 2020) explored how different algo-

rithms suit various rehabilitation stages, with Hidden

Markov Models excelling in early-stage performance

monitoring and Dynamic Time Warping providing de-

tailed analysis later. Adans-Dester et al. (Adans-

Dester et al., 2020) utilized wearable sensor data and

machine learning to estimate functional ability and

Fugl-Meyer scores accurately during motor task per-

formance. Imura et al. (Imura et al., 2021) identiﬁed

key variables for predicting home discharge outcomes

using a classiﬁcation and regression tree model. Liao

et al. (Liao et al., 2020) reviewed machine learning

Table 1: Summary of studies on the design and implementa-

tion of home rehabilitation systems. The column Part. indi-

cates the number of participants involved in each study, RT-

Feed speciﬁes whether real-time feedback was provided,

and ML denotes the use of machine learning techniques.

Reference Goal of the Study # Part. RT-Feed ML

(Lee, 2018) Build computer-assisted stroke rehabilitation

using Kinect and ML

26 x x

(Chae et al., 2020) Develop and evaluate a web-based upper limb

home rehabilitation system using a smartwatch

and ML model

38 - x

(Osgouei et al., 2020) Use of Motion Sensing and ML to Quantify

Exercise Performance in Healthy Volunteers

16 - x

(Adans-Dester et al., 2020) Enabling precision rehabilitation interventions

using wearable sensors and ML to track motor

recovery

37 - x

(Liao et al., 2020) Review computational approaches for the

evaluation of rehabilitation exercises

54 x x

(Lee et al., 2020) Combine machine and human intelligence for

personalized rehabilitation assessment

26 - x

(Ahammad et al., 2020) Spinal cord disorder classiﬁcation for patient

wellness and remote monitoring

950 - x

(Kashi et al., 2020) Automatic detection of movement compensa-

tion in stroke patients

30 x x

(Imura et al., 2021) Identify stroke patients after rehabilitation us-

ing functional and environmental predictors

1125 - x

(Biebl et al., 2021) Show that the interrater agreement between

physiotherapists and Motion Coach is nonin-

ferior to physiotherapists’ interrater agreement

for exercise evaluations

24 x x

(Ranasinghe et al., 2021) Introduce a system for people to perform phys-

ical exercise at home

16 x x

(Seifallahi et al., 2022) Alzheimer’s disease detection based on video

data using ML

85 - x

(Bijalwan et al., 2022) Guide patients to perform real-time upper limb

physiotherapy

25 - x

This work Introduce VIRTUAL-PHYSIO, a virtual assis-

tant to support home rehabilitation

51 x x

approaches for motion capture systems, emphasizing

their effectiveness in quantifying movement quality in

home-based settings.

Interactive machine learning approaches, such as

the one described by Lee et al. (Lee et al., 2020),

combine expert input with data-driven models to as-

sess rehabilitation exercise quality. Ahammad et al.

(Ahammad et al., 2020) focused on spinal cord dis-

order classiﬁcation using sensor data, demonstrating

improved efﬁciency in remote care. Biebl et al. (Biebl

et al., 2021) validated the MotionCoach app, showing

strong agreement with physiotherapist evaluations in

osteoarthritis patients.

Other works emphasize innovative feedback

mechanisms. Ranasinghe et al. (Ranasinghe et al.,

2021) proposed a muscle-strength-based exercise dif-

ﬁculty measurement method, enabling remote patient

guidance through instructional videos. Kashi et al.

(Kashi et al., 2020) developed a ML model to pro-

vide feedback on stroke patient movements, achiev-

ing 85% precision in detecting compensations.

Low-cost systems have also gained traction. Lee

et al. (Lee, 2018) introduced Virtual Coach, a post-

stroke rehabilitation system with 78% agreement with

clinicians. Seifallahi et al. (Seifallahi et al., 2022) and

Bijalwan et al. (Bijalwan et al., 2022) applied Kinect

v2 sensors to detect Alzheimer’s disease and catego-

rize rehabilitation exercises, achieving accuracies of

97.75% and over 98%, respectively.

Table 1 shows that the evaluation of VIRTUAL-

PHYSIO involved a participant count comparable to

prior studies. Unlike previous work, VIRTUAL-

HEALTHINF 2025 - 18th International Conference on Health Informatics

468

PHYSIO uniquely integrates real-time feedback and

automatic exercise evaluation through machine learn-

ing, offering a novel approach to home rehabilitation.

3 2VITA-B PHYSICAL IN A

NUTSHELL

The 2VITA-B PHYSICAL (Antico et al., 2021a) sys-

tem is the foundation of the VIRTUAL-PHYSIO, the

proposed virtual assistant for home-based physiother-

apy. 2VITA-B PHYSICAL is designed to monitor and

support the physical rehabilitation process by creat-

ing a link between the therapist and the patient. In

2VITA-B PHYSICAL, each user plays a distinct role

in ensuring the successful functioning of the rehabil-

itation process, in which various professionals con-

tribute to the recovery process of the patient.

The admin manages access and creates user pro-

ﬁles. The physiatrist enrolls new patients and devel-

ops an Individual Rehabilitation Project (IRP). The

physiotherapist designs and updates exercises based

on the patient’s condition and progress. The psychol-

ogist monitors the patient’s psychological state dur-

ing rehabilitation. Lastly, the patient follows a per-

sonalized rehabilitation plan, consisting of exercises

tailored to their therapy.

Patient enrollment is the process of designing a

personalized rehabilitation plan. When a new pa-

tient begins rehabilitation, the physiatrist registers

them and creates an Individual Rehabilitation Project

(IRP), entering all relevant health information. The

physiotherapist then develops the rehabilitation plan

by selecting a set of exercises from a manageable list,

tailored to the patient’s speciﬁc needs.

The core of 2VITA-B PHYSICAL is represented

by the Rehab Station, a dedicated station used by

the patient during rehabilitation activities. Rehabilita-

tion activities are conducted using the Rehab Station,

which includes a laptop running 2VITA-B PHYSICAL

software, a Microsoft Azure Kinect motion sensor

for movement analysis, and a Polar OH1 heart rate

monitor. In detail, for each exercise, a physiothera-

pist must record the ideal execution, which is used by

VIRTUAL-PHYSIO to give real-time feedback to the

patient and to evaluate the execution. This means that,

once a patient has been assigned to a new rehabilita-

tion plan, the 2VITA-B PHYSICAL system could be

used also without the presence of the physiotherapist.

The system also registers a video of the movement

and the automatic evaluation provided by VIRTUAL-

PHYSIO. Such an evaluation is used by the 2VITA-B

PHYSICAL to alert the physiotherapist that the patient

wrongly executed an exercise. This enables the phys-

iotherapist to review the execution, provide feedback,

or reach out to the patient for further clariﬁcation. Pa-

tients can also provide self-assessments, and both pa-

tient and physiotherapist evaluations are used to con-

tinuously train VIRTUAL-PHYSIO.

4 THE VIRTUAL ASSISTANT

FOR HOME REHABILITATION

VIRTUAL-PHYSIO is a virtual assistant integrated

into the 2VITA-B PHYSICAL system, designed to

help patients perform rehabilitation exercises cor-

rectly and assist physiotherapists by automatically as-

sessing exercise performance. Once the physiatrist

creates an Individual Rehabilitation Project (IRP),

patients interact with VIRTUAL-PHYSIO through a

straightforward, touch-friendly interface. This inter-

face guides them through the rehabilitation process,

allowing them to watch a demonstration video of the

exercise, perform it, review their performance, and

provide a self-assessment.

After the exercise is completed, VIRTUAL-

PHYSIO evaluates the execution, enabling physiatrist

and physiotherapist oversight to ensure progress and

address any issues. A rehabilitation session includes

multiple exercises, each consisting of several repeti-

tions. The term exercise refers to the complete set

of repetitions, while repetition describes a single in-

stance of the movement, highlighting the structured

approach VIRTUAL-PHYSIO brings to the rehabilita-

tion process.

4.1 Guiding Patients in Rehab Exercises

VIRTUAL-PHYSIO provides real-time support to pa-

tients during the execution of rehabilitation exercises

by utilizing a dual-avatar system. Patients view two

3D avatars: a real avatar replicating their movements

and an ideal avatar showcasing the pre-recorded, cor-

rect movements provided by medical staff. These

avatars, modeled with a skeleton and rendered skin,

are designed to visually guide the patient in aligning

their movements with the ideal execution.

Both avatars are 3D models of the human body,

composed of a skeleton and a skin, i.e., a polygonal

mesh. The skeleton is built of bones (a rigged sys-

tem controlling the mesh deformation) and the skin is

the rendered part of the avatar. The motion tracking

system, Microsoft Azure Kinect DK, detects 19 key

body joints for movement analysis, including arms,

legs, spine, and head, while excluding ﬁner details

like facial, ﬁnger, and toe joints. The customizable

interface allows patients to adjust the visualization,

VIRTUAL-PHYSIO: A Virtual Assistant for Home Physiotherapy Rehabilitation

469

such as superimposing avatars or displaying them sep-

arately, and to select speciﬁc movements to focus on.

Additionally, the ideal movement can be displayed in

the interface for reference. During the exercise, the

sidebar displays essential information, including the

number of completed repetitions, elapsed time, and

the patient’s heart rate. These metrics help monitor

progress, assess effort, and identify potential issues

such as overexertion, ensuring a comprehensive and

user-friendly experience during rehabilitation.

VIRTUAL-PHYSIO provides two types of feed-

back to the patient: (i) a precision score, to report

how well the exercise is going; (ii) some correction

hints, to suggest how to improve the movement.

Both the feedback are provided by comparing the

ideal movement (ideal avatar) with the actual move-

ment of the patient (real avatar). The comparison

is based on the similarity between the bones of the

two avatars, represented as quaternions. Formally, let

( f ) = (b

, b

, ..., b

) and B

( f ) = (b

, b

, ..., b

)

be the set of acquired bones—in a speciﬁc frame f —

of the real avatar and the ideal avatar, respectively.

Given two quaternions, qr and qi, representing the j

bone of both real (b

) and ideal (b

) avatar, respec-

tively, we can compute the similarity between the two

bones through the cosine similarity between the two

quaternions.

Once obtained the ∆ for each bone it is possible to

deﬁne the precision score for a speciﬁc frame f .

The precision score evaluates movement accuracy

at speciﬁc moments (frames) during an exercise, of-

fering continuous feedback to patients. This score

is visually represented using emoticons based on its

value: a smile emoticon for scores above 0.7, a neu-

tral face for scores between 0.5 and 0.7, and a sad

face for scores below 0.5. This system allows patients

to easily interpret their performance and make adjust-

ments in real-time during rehabilitation (Figure 1).

As said before, VIRTUAL-PHYSIO also provides

some correction hints during the execution of a move-

ment. Such hints include (i) suggestions, in the form

of both natural language statements and audio mes-

sages, and (ii) visual feedback (see Figure 1). The

less correctly moving bone b

bad

is the bone with the

highest ∆ that is also higher than a ﬁxed threshold t.

If the identiﬁed less correctly moving bone b

bad

re-

mains the same for more than ﬁve seconds, a warn-

ing is generated. Speciﬁcally, we mark in red b

bad

the real avatar and we display an arrow showing how

to correct the movement (Figure 1). We also gener-

ate a suggestion by reporting the name of the bone

If the highest ∆ is lower than the threshold t, we do not

generate any hints. Such a threshold has been empirically

deﬁned through a trial & error approach.

Figure 1: An example of feedback provided by VIRTUAL-

PHYSIO during the execution of an exercise.

bad

(e.g., “right arm”) and the action that needs to

be taken, based on the pointing direction of the arrow

(e.g., “raise”).

4.2 Evaluating Rehabilitation Exercises

The machine learning pipeline of VIRTUAL-PHYSIO

is composed of several steps. The ﬁrst step of the

pipeline is the extraction—for each bone—of the mo-

tion tracks from the recording. At the end of the

extraction process, we have 19 tracks, one for each

tracked bone.

After completing an exercise, VIRTUAL-PHYSIO

automatically evaluates the patient’s performance and

assigns a numeric score. This evaluation is pow-

ered by a machine learning model trained on expert-

evaluated exercises, utilizing features derived from

comparisons between the ideal and patient move-

ments. Detailed information about the model and its

accuracy is available in Section 5.

The evaluation pipeline begins by extracting mo-

tion tracks for each of the 19 tracked bones. These

tracks undergo pre-processing to ensure reliability.

Smoothing ﬁlters reduce noise caused by environ-

mental factors, while synchronization aligns the pa-

tient’s movements with the ideal execution, account-

ing for potential timing delays. The recording is

then sectioned into individual repetitions, excluding

the ﬁrst and last repetitions, which are often incom-

plete or imprecise, as advised by physiotherapists. In-

complete repetitions due to missing frames are also

discarded, ensuring the evaluation focuses on high-

quality data for accurate scoring.

Once the pre-processing is complete, features are

extracted for the ML model using cosine and Eu-

clidean distances between the ideal and real move-

ments, calculated at the bone level for every frame.

These values are aggregated (mean, maximum, stan-

dard deviation), resulting in 114 features (19 (bones)

x 2 (distance metrics) x 3 (aggregations)). Since not

HEALTHINF 2025 - 18th International Conference on Health Informatics

470

all bones are equally important for every exercise,

physiotherapists classify bones as relevant or irrele-

vant based on the exercise. Based on this considera-

tion, for each exercise, we divide the bones into two

sets B

(i.e., the relevant bones) and B

(i.e., the irrel-

evant bones). Then, we added other features that con-

sider the overall similarity of bones belonging to these

two sets. Especially, for each set of bones, we com-

pute the mean, maximum, and standard deviation of

the similarity of the bones. This results in the addition

of other 12 features, 2 (sets of bones) x 2 (distance

metrics) x 3 (aggregations). Therefore, in total, we

considered 126 features. The machine learning model

uses these features to assign a score from 1 (lowest)

to 5 (highest), reﬂecting movement accuracy. A score

of 1 indicates compensatory movements, 2 highlights

deﬁciencies in important bones, 3 suggests partial ex-

ecution, 4 denotes slight inaccuracies, and 5 repre-

sents perfect execution. This scoring system provides

a detailed assessment of performance accuracy.

5 EMPIRICAL EVALUATION

This section reports the design and the results of

the empirical study we run to validate VIRTUAL-

PHYSIO. The experimentation was approved by the

Ethics Committee of Celio Army Medical Center

(Rome, Italy)—Prot.n. CE/2021u/03/a-31/03/2021-

08.a.

5.1 Study Deﬁnition and Context

The goal of our study is to understand to what ex-

tent VIRTUAL-PHYSIO is able to help the two types

of users it is aimed at, i.e., patients and physiothera-

pists. This study is steered by the following research

questions:

• RQ

: To what extent is VIRTUAL-PHYSIO able to

identify imperfections in the execution of an exer-

cise?

• RQ

: To what extent is VIRTUAL-PHYSIO able

to quantify imperfections in the execution of an

exercise?

The controlled experiment involved 51 partici-

pants (32 males and 19 females). All the subjects

were healthy and without any motor disabilities. Par-

ticipants were selected by using convenience sam-

pling. All the participants were recruited from the

Institute One and from the Institute Two (see Table 2).

Table 2: Age, height, and weight of the participants.

Mean Median Std. Dev. Min Max

Age 35 30 13 19 63

Height (cm) 174 175 8 156 191

Weight (kg) 74 74 13 50 101

5.2 Experimental Procedure

The equipment required to conduct the experiment in-

cluded a workstation

, the Azure Kinect, the Polar

heart-rate wrist band, an adjustable ankle weights kit,

and a classical gym step. We set up the system, in-

cluding both hardware and software components, at

the Institute One in a dedicated room big enough to

allow participants to perform all the exercises.

The experimental protocol provided for the execu-

tion of ﬁve rehabilitative exercises was:

• Shoulder rehabilitation: standing, with the

weight on the wrist and keeping the arm straight,

abduct up to reach 90 degrees, without exceeding

shoulder height. Slowly return to the starting po-

sition;

• Elbow rehabilitation: standing, with your hands

crossed behind your neck, take your hands away

from your head, extending the elbows upwards,

hold for 4 seconds and return to the starting posi-

tion and hold the position for 4 seconds;

• Hip rehabilitation: standing, with the weight at

the ankle, ﬂex the hip until it reaches 90 degrees

and return to the starting position;

• Knee rehabilitation: standing, with a weight on

both wrists, step forward with the right foot, rest-

ing it on a step. Focus on the left leg, bending it

down until your knee touches the ﬂoor. The leg to

be bent is the left and the right leg ﬂexes conse-

quently, not the other way. Return to the starting

position by pushing with the front foot;

• Vertebral column rehabilitation: standing, with

your arms at your sides, slide your left hand along

the left thigh, tilting the trunk, up to the maximum

possible width, maintain for 4 seconds and return

to the starting position. Repeat on the right.

Participants were asked to repeat each exercise ten

times, always in the same order. The execution started

with the participant placed in front of the motion

tracking sensor. Before beginning the exercise, a hu-

man assistant explained the VIRTUAL-PHYSIO GUI

and the steps that the participant would have executed.

AMD Ryzen 7 3800X 8-core 3.9 GHz, 32 GB RAM,

Nvidia GTX 1660 6GB, with Windows 10 PRO

VIRTUAL-PHYSIO: A Virtual Assistant for Home Physiotherapy Rehabilitation

471

After that, the human assistant—through VIRTUAL-

PHYSIO—played the execution video to let the par-

ticipant understand the movement to be performed

for each repetition. Once the participant was ready,

the execution started. From that moment on, the hu-

man assistant did not give any feedback to the partic-

ipant living this task to VIRTUAL-PHYSIO. The hu-

man assistant was allowed to take action only in case

of very bad movements that could impact the partici-

pants’ health. When the exercise was completed, the

human assistant showed the participant the recorded

movement. At the end of each exercise, the partic-

ipant was asked to rate her execution giving a score

from 1 to 5 stars.

5.3 Analysis Procedure

To answer RQ

and RQ

, we ﬁrst collected all the ex-

ecutions of the participants and we asked a physio-

therapist with more than ten years of experience to

manually evaluate all of them both with a binary clas-

siﬁcation (with/without imperfections, for RQ

) and

on a scale from 1 to 5 (RQ

). As expected, the major-

ity of the labels are located in the most positive label

since all the participants were health.

Once obtained the ground truth, i.e., a dataset of

movement manually labelled, we trained VIRTUAL-

PHYSIO in two different scenarios: (i) a binary classi-

ﬁcation problem (RQ

), in which VIRTUAL-PHYSIO

aims at distinguishing perfectly conducted exercises

from the ones conducted with imperfections; (ii) a re-

gression problem (RQ

), in which VIRTUAL-PHYSIO

aims at providing a score from 1 to 5 which embeds

the type of imperfection, i.e., compensatory or deﬁci-

tary movements (see Section 4.2).

We experimented with seven different machine

learning models (Alpaydın, 2021) for each exercise:

(i) random forest (RF), (ii) multi-layer perceptron

(MLP), (iii) logistic regression (RG), (iv) Gaussian

Naive Bayes (GNB), (v) Linear Support Vector Clas-

siﬁcation/Regression (LSVM), (vi) C-Support Vec-

tor Classiﬁcation/Regression (CSVC), and (vii) k-

Nearest Neighbors (KNN). Note that we decide to

have a prediction model for each exercise (local pre-

diction) instead of a single model for each exercise

(global prediction) because of the differences among

the exercises. This choice was supported also by

the physiotherapists. To mitigate the problems due

to class imbalance we experimented with the use of

the SMOTE (Chawla et al., 2002) oversampling tech-

nique. Also, we experimented with several automatic

feature selection techniques and correlation analysis

to ﬁlter the features correlated to each other with a

correlation index greater than 0.95.

The validation process was based on the leave one

out cross validation (Alpaydın, 2021). This process

consists in iterating over each instance (participant),

using the i-th instance as a test set, and using the re-

maining as a train set.

For the binary classiﬁcation, we measure the qual-

ity of the classiﬁcation of VIRTUAL-PHYSIO through

the following metrics widely used for machine learn-

ing models (Alpaydın, 2021): (i) accuracy; (ii) recall;

(iii) precision; (iv) F1-score. While for the regression

problem, we evaluate the performance of VIRTUAL-

PHYSIO through the Mean Absolute Error (MAE)

(Alpaydın, 2021).

5.4 Analysis of the Results

Table 3: Performance of VIRTUAL-PHYSIO in classifying

an exercise as correct or incorrect (RQ

Exercise Model Accuracy Recall Precision F1

Shoulder MLP or KNN 0.84 0.84 0.91 0.86

Elbow MLP 0.76 0.76 0.76 0.76

Hip KNN 0.82 0.82 0.82 0.81

Knee MLP 0.57 0.57 0.32 0.41

Vertebral column MLP 0.71 0.71 0.71 0.69

Table 4: Performance of VIRTUAL-PHYSIO in evaluating

an exercise on a ﬁve-point score (RQ

Exercise Model MAE

Shoulder Random forest or KNN 0.31

Elbow MLP 0.47

Hip GNB 0.29

Knee SVM 1.16

Vertebral column Random forest 0.63

The results in Table 3 and Table 4 show the perfor-

mance of VIRTUAL-PHYSIO in classifying exercises

as correct or incorrect (RQ

) and scoring exercises on

a ﬁve-point scale (RQ

). No single machine learn-

ing model consistently outperforms others. MLP and

KNN perform best for classiﬁcation without over-

sampling, while Random Forest achieves top results

for regression in two exercises. These ﬁndings sup-

port the use of exercise-speciﬁc models over a global

model for all exercise types.

The achieved results indicate that VIRTUAL-

PHYSIO has accuracy in evaluating the correctness

of an exercise (RQ

) higher than 70%, with the ex-

ception of knee rehabilitation exercise where the ac-

curacy is 57%. Similar results were achieved when

evaluating the exercise on a ﬁve-point scale: In this

scenario, VIRTUAL-PHYSIO achieved an error always

lower than 0.5, with the exception of the knee and ver-

tebral column rehabilitation exercises where the MAE

is 1.16 and 0.63, respectively.

The knee rehabilitation exercise presented the

HEALTHINF 2025 - 18th International Conference on Health Informatics

472

highest error rate, while the shoulder rehabilitation

showed the fewest errors. This discrepancy may re-

sult from the greater complexity of movements in the

knee and spine exercises, which involve more joints,

making the evaluation more challenging for the ma-

chine learning model. In addition, the different rea-

sons why the physical therapists negatively evaluated

the knee exercises resulted in fewer similar instances

evaluated negatively in the training set, complicating

the model’s ability to identify incorrect execution pat-

terns.

5.5 Case Analysis and Discussion

This section analyzes speciﬁc cases to explore factors

behind VIRTUAL-PHYSIO’s correct and incorrect pre-

dictions.

A correct exercise classiﬁed by VIRTUAL-

PHYSIO as correct. In Figure 4 (Balletti et al., 2024)

VIRTUAL-PHYSIO identiﬁed a correctly performed

exercise, as no abnormalities were observed in the leg

or pelvis graphs, indicating proper leg extension and

absence of compensatory movements.

An incorrect exercise classiﬁed by VIRTUAL-

PHYSIO as incorrect. In Figure 5 (Balletti et al.,

2024) several abnormalities can be found. The leg

graphs revealed incomplete movements in repetitions

3, 5, and 8, and the pelvis graph showed compen-

satory torso rotation throughout the exercise, aligning

with the physiotherapist’s evaluation.

A correct exercise classiﬁed by VIRTUAL-

PHYSIO as incorrect. From the analysis of the leg

graphs in Figure 6 (Balletti et al., 2024), it is pos-

sible to observe approximately complete movements

for most of the repetitions. Although most repetitions

were performed correctly, a temporary loss of bal-

ance in the second repetition caused abnormal move-

ments, which the physiotherapist disregarded when

assessing overall performance. Instead, VIRTUAL-

PHYSIO weighted the abnormal movements more

heavily, leading to an imperfect rating.

An incorrect exercise classiﬁed by VIRTUAL-

PHYSIO as correct. In Figure 7 (Balletti et al., 2024)

VIRTUAL-PHYSIO incorrectly classiﬁed an exercise

as correct, despite the physiotherapist identifying con-

sistent failure to achieve 90-degree leg extension and

trunk ﬂexion. This execution defect, absent in other

exercises, likely contributed to the model’s inability

to classify it negatively.

A perfect exercise evaluated by VIRTUAL-

PHYSIO correctly. In Figure 8 (Balletti et al., 2024)

the physiotherapist and VIRTUAL-PHYSIO assigned

to the exercise a score equals to 5. From the analy-

sis of the graphs of shoulder and arm movement, it

is possible to observe no serious differences between

actual and ideal movements.

A good exercise evaluated by VIRTUAL-

PHYSIO correctly. In this case (Figure 9 (Bal-

letti et al., 2024)), a good but imperfect exercise

was correctly rated 4 by both VIRTUAL-PHYSIO and

the physiotherapist. The model identiﬁed an incom-

plete arm extension in the initial phase, visible in

the graphs, demonstrating its capability to detect and

score minor imperfections accurately.

A bad exercise evaluated by VIRTUAL-PHYSIO

as perfect. VIRTUAL-PHYSIO misclassiﬁed a bad ex-

ercise as perfect in Figure 10, assigning a score of 5

while the physiotherapist rated it 3. The physiothera-

pist noted incomplete shoulder movement in the ﬁrst

repetitions. The graphs indicated near-perfect arm

movement, likely inﬂuencing the model’s overly pos-

itive rating, but conﬁrmed the physiotherapist’s obser-

vation of minimal vertical shoulder movement.

5.6 Threats to Validity

The main threats to the validity of this study resulted

from the ML techniques used and the population of

participants. Regarding the models, while not all ML

approaches were explored, the study included repre-

sentatives from key algorithm categories, such as tree-

based models, logistic regression, Support Vector Ma-

chines, and neural networks. Concerning participants,

the sample may not be fully representative, a common

challenge in human-involved studies due to the time

commitment required for participation. However, as

discussed above, our study involved a number of par-

ticipants in line with the other studies in the litera-

ture. Another limitation is that the study considers

only a single session, whereas rehabilitation typically

involves multiple sessions.

6 CONCLUSION

In this paper, we presented VIRTUAL-PHYSIO, a vir-

tual assistant integrated into 2VITA-B PHYSICAL

system, designed to guide patients during exercises

by providing feedback and evaluating entire sessions

to notify physiotherapists about cases requiring atten-

tion. Leveraging affordable motion tracking devices,

visual feedback, and machine learning-based evalu-

ation, VIRTUAL-PHYSIO aims to enhance home re-

habilitation. A controlled experiment with 51 par-

ticipants demonstrated high conﬁdence and willing-

ness to use VIRTUAL-PHYSIO for home rehabilita-

tion. These results provide a valid premise for the

further enhancement of home rehabilitation using mo-

VIRTUAL-PHYSIO: A Virtual Assistant for Home Physiotherapy Rehabilitation

473

tion capture and ML technologies. As a future work,

we plan to integrate into 2VITA-B PHYSICAL a hand

tracking device aiming at supporting speciﬁc hand re-

habilitation exercises. Furthermore, we aim to im-

prove the motion-tracking capabilities by incorporat-

ing multiple Kinect sensors.

ACKNOWLEDGMENTS

This research was funded by Ministry of Defence

grant number 20536 (December 13, 2019) “2VITA-

B PHYSICAL: Veteran Virtual Training for Aging

Blockchain” – Proposal a2018.137.

REFERENCES

Adans-Dester, C., Hankov, N., O’Brien, A., Vergara-Diaz,

G., Black-Schaffer, R., Zafonte, R., Dy, J., Lee, S. I.,

and Bonato, P. (2020). Enabling precision rehabili-

tation interventions using wearable sensors and ma-

chine learning to track motor recovery. NPJ digital

medicine, 3(1):1–10.

Ahammad, S. H., Rahman, M. Z. U., Rao, L. K., Sulthana,

A., Guptha, N., and Lay-Ekuakille, A. (2020). A

multi-level sensor based spinal cord disorder classi-

ﬁcation model for patient wellness and remote moni-

toring. IEEE Sensors Journal.

Alpaydın, E. (2021). Machine Learning, Revised And Up-

dated Edition. The MIT Press.

Antico, M., Balletti, N., Ciccotelli, A., Ciccotelli, M.,

Laudato, G., Lazich, A., Oliveto, R., Notaranto-

nio, M., Ricciardi, S., Scalabrino, S., et al. (2021a).

2ViTA-B Physical: An intelligent home rehabilitation

system based on Microsoft Azure Kinect DK. Fron-

tiers in Human Dynamics, page 54.

Antico, M., Balletti, N., Laudato, G., Lazich, A., Notaran-

tonio, M., Oliveto, R., Ricciardi, S., Scalabrino, S.,

and Simeone, J. (2021b). Postural control assessment

via Microsoft Azure Kinect DK: An evaluation study.

Computer Methods and Programs in Biomedicine,

209:106324.

Balletti, N., Cascitelli, A., Gabrieli, P., Guglielmi, E.,

Laudato, G., Lazich, A., Notarantonio, M., Oliveto,

R., Ricciardi, S., Scalabrino, S., and Simeone, J.

(2024). Appendix ﬁgure. https://bit.ly/3DnLPkT. 18

dicembre 2024.

Biebl, J. T., Rykala, M., Strobel, M., Bollinger, P. K., Ulm,

B., Kraft, E., Huber, S., and Lorenz, A. (2021). App-

based feedback for rehabilitation exercise correction

in patients with knee or hip osteoarthritis: Prospective

cohort study. Journal of Medical Internet Research,

23(7):e26658.

Bijalwan, V., Semwal, V. B., Singh, G., and Mandal, T. K.

(2022). Hdl-psr: Modelling spatio-temporal features

using hybrid deep learning approach for post-stroke

rehabilitation. Neural Processing Letters, pages 1–20.

Chae, S. H., Kim, Y., Lee, K.-S., and Park, H.-S. (2020).

Development and clinical evaluation of a web-based

upper limb home rehabilitation system using a smart-

watch and machine learning model for chronic stroke

survivors: prospective comparative study. JMIR

mHealth and uHealth, 8(7):e17216.

Chawla, N. V., Bowyer, K. W., Hall, L. O., and Kegelmeyer,

W. P. (2002). Smote: synthetic minority over-

sampling technique. Journal of artiﬁcial intelligence

research, 16:321–357.

Han, D., Kuschner, D., and Wang, Y.-f. (2005). Upper limb

position sensing: A machine vision approach. In Con-

ference Proceedings. 2nd International IEEE EMBS

Conference on Neural Engineering, 2005., pages 490–

493. IEEE.

Imura, T., Iwamoto, Y., Inagawa, T., Imada, N., Tanaka,

R., Toda, H., Inoue, Y., Araki, H., and Araki, O.

(2021). Decision tree algorithm identiﬁes stroke pa-

tients likely discharge home after rehabilitation using

functional and environmental predictors. Journal of

Stroke and Cerebrovascular Diseases, 30(4):105636.

Kashi, S., Polak, R. F., Lerner, B., Rokach, L., and Levy-

Tzedek, S. (2020). A machine-learning model for

automatic detection of movement compensations in

stroke patients. IEEE Transactions on Emerging Top-

ics in Computing, 9(3):1234–1247.

Lee, M. H. (2018). A technology for computer-assisted

stroke rehabilitation. In 23rd International Confer-

ence on Intelligent User Interfaces, pages 665–666.

Lee, M. H., Siewiorek, D. P., Smailagic, A., Bernardino,

A., and Berm

udez i Badia, S. (2020). Interactive hy-

brid approach to combine machine and human intel-

ligence for personalized rehabilitation assessment. In

Proceedings of the ACM Conference on Health, Infer-

ence, and Learning, pages 160–169.

Liao, Y., Vakanski, A., Xian, M., Paul, D., and Baker, R.

(2020). A review of computational approaches for

evaluation of rehabilitation exercises. Computers in

biology and medicine, 119:103687.

Maclean, N., Pound, P., Wolfe, C., and Rudd, A. (2002).

The concept of patient motivation: a qualitative

analysis of stroke professionals’ attitudes. Stroke,

33(2):444–448.

Osgouei, R. H., Soulsby, D., and Bello, F. (2020). Rehabili-

tation exergames: Use of motion sensing and machine

learning to quantify exercise performance in healthy

volunteers. JMIR Rehabilitation and Assistive Tech-

nologies, 7(2):e17289.

Ranasinghe, I., Dantu, R., Albert, M. V., Watts, S., and

Ocana, R. (2021). Cyber-physiotherapy: Rehabili-

tation to training. In 2021 IFIP/IEEE International

Symposium on Integrated Network Management (IM),

pages 1054–1057. IEEE.

Seifallahi, M., Mehraban, A. H., Galvin, J. E., and Gho-

raani, B. (2022). Alzheimer’s disease detection using

comprehensive analysis of timed up and go test via

kinect v. 2 camera and machine learning. IEEE Trans-

actions on Neural Systems and Rehabilitation Engi-

neering, 30:1589–1600.

Shaughnessy, M., Resnick, B. M., and Macko, R. F. (2006).

Testing a model of post-stroke exercise behavior. Re-

habilitation nursing, 31(1):15–21.

HEALTHINF 2025 - 18th International Conference on Health Informatics

474