Modelling Physiological Sensor Noise to Movement-Based Virtual Reality

Activities

Phil Lopes

1 a

, Nuno Fachada

2 b

, Micaela Fonseca

1 c

, Hugo Gamboa

3 d

and Claudia Quaresma

3 e

Lusofona University, HEI-Lab, Campo Grande, 376, 1749-024 Lisboa, Portugal

Lusofona University, COPELABS, Campo Grande, 376, 1749-024 Lisboa, Portugal

LIBPhys, NOVA School of Science and Technology, NOVA University Lisbon, 2829-516 Caparica, Portugal

Keywords:

Virtual Reality, Data Processing, Biofeedback, Machine Learning.

Abstract:

This position paper proposes the hypothesis that physiological noise artefacts can be classiﬁed based on the

type of movements performed by participants in Virtual Reality contexts. To assess this hypothesis, a detailed

research plan is proposed to study the inﬂuence of movement on the quality of the captured physiological

signals. This paper argues that the proposed plan can produce a valid model for classifying noisy physiolog-

ical signal features, providing insights into the inﬂuence of movement on artefacts, while contributing to the

development of movement-based ﬁlters and the implementation of best practices for using various associated

technologies.

1 INTRODUCTION

The use of physiological monitoring has become an

essential tool for multidisciplinary studies leverag-

ing interactive virtual reality (VR) devices and digi-

tal games for the purpose of studying human-based

behaviours. Some examples include neuroscience

(Bohil et al., 2011), the study of emotions (Mon-

tana et al., 2020), human-computer interaction (Li-

apis et al., 2015), affective computing (Lopes et al.,

2017), learning (Bavelier et al., 2012), and healthcare

(e.g., rehabilitation (Howard, 2017), therapy (Nor-

rholm et al., 2016; Bohil et al., 2011) and pain man-

agement (Hoffman et al., 2011)). VR in particular

provides high-ﬁdelity immersive experiences, allow-

ing individuals to physically interact with virtual en-

vironments, which can be advantageous for rehabil-

itation (Howard, 2017) and exposure-type therapies

(Bohil et al., 2011). The main constraint, however,

is that physiological devices are not prepared for this

type of intensive interaction, a pattern that has been

observed in several studies using these tools (Kritikos

et al., 2019; Solbiati et al., 2021; Dey et al., 2022;

https://orcid.org/0000-0002-9567-5806

https://orcid.org/0000-0002-8487-5837

https://orcid.org/0000-0001-7946-4825

https://orcid.org/0000-0002-4022-7424

https://orcid.org/0000-0001-9978-261X

Higuera-Trujillo et al., 2017; Petrescu et al., 2020).

Literature suggests several methodologies on how

to mitigate this problem (Lopes and Boulic, 2020;

arvel

a et al., 2014), but such methods often rely on

minimising movement as much as possible, which is

counterproductive to the core advantages of using VR

in the ﬁrst place.

Given the integral nature of movement and inter-

action required from VR experiences, this position

paper argues that the inﬂuence of movement on the

signal quality of recorded physiological data should

be adequately explored, thus proposing a step-by-step

research plan towards that goal. In particular, this pa-

per asks the question: “Given a set of pre-deﬁned

movements, can we associate each movement type

with a classiﬁable artefact pattern?”. Thus, the hy-

pothesis is that the noise artefacts themselves, which

are speciﬁcally obtained within this context, could be

classiﬁed based on the type of movement that was

conducted by the participant.

For the resolution of this problem, the develop-

ment of a VR application is needed to enable the

collection of both physiological and participant body

movement data. This VR application serves as a stim-

ulus, offering the participant objectives and incen-

tives to move repetitively over the course of a game

playing session. Physiology can be collected through

high-quality physiological sensors, while body move-

778

Lopes, P., Fachada, N., Fonseca, M., Gamboa, H. and Quaresma, C.

Modelling Physiological Sensor Noise to Movement-Based Virtual Reality Activities.

DOI: 10.5220/0012424200003657

Paper published under CC license (CC BY-NC-ND 4.0)

In Proceedings of the 17th International Joint Conference on Biomedical Engineering Systems and Technologies (BIOSTEC 2024) - Volume 1, pages 778-785

ISBN: 978-989-758-688-0; ISSN: 2184-4305

ment can be captured through high-deﬁnition video

cameras and motion capture (mocap) systems, paired

with internal information from the VR system head-

set and controllers movement information. The dif-

ferent granularities offered by these technologies pro-

vide sufﬁcient detail on how a participant moves their

body in relation to the appearance of a noise artefact.

By analysing the captured signals and the different

levels of noise over time, the hypothesis considers that

patterns will emerge based on how participants move

their bodies during the game playing session, which in

turn can be modelled accordingly. Furthermore, such

patterns may be user-dependent, providing additional

solutions for user-centric ﬁltering techniques.

The expected outcome of such a research plan

would be a model capable of classifying noisy physi-

ological signal features while accounting for different

granularities of participant movement. Furthermore,

an extensive study and compilation of the inﬂuence

of movement on the appearance of artefacts in physi-

ological data could be constructed based on all obser-

vations collected from this type of experimentation.

We argue that such study could provide (i) models for

noise detection identifying noise source, (ii) a founda-

tion for the construction of speciﬁc movement-based

ﬁlters through pattern recognition, and, (iii) method-

ologies and best practice design patterns for systems

employing such technology.

2 BACKGROUND

The use of VR for the study of human-based phe-

nomena is a comprehensively explored subject (Lopes

and Boulic, 2020; Jawed et al., 2021; Hoffman et al.,

2011; Norrholm et al., 2016). This is unsurprising

when considering the potential of such technology

due to its immersive capabilities, which allow indi-

viduals to physically interact and walk within virtual

environments. Furthermore, these environments are

highly malleable and offer an extensive degree of con-

trol, which in turn allows therapists, psychologists

and other professionals to safely tailor these treat-

ments to each individual patient (

Cosi

c et al., 2010).

The use of physiological sensors is often a crucial

aspect when studying the inﬂuence of certain stimuli

on humans. VR is attractive for these studies due to

the immersion offered to its participants and the close

relationship individuals can have with their virtual

avatars. This psychological phenomenon is known as

embodiment (Kilteni et al., 2012), where the partici-

pant associates their own body with that of the virtual

avatar, which can increase the effectiveness of stimuli

presented in the virtual environment (Slater, 2017).

Thus, the effectiveness of using such a technology to

study speciﬁc human behaviours, while maintaining

patient safety and controlling all possible outcomes of

the virtual environment, is clear. Examples of VR ap-

plications being used as a stimulus include the study

of human emotion (Montana et al., 2020), learning

effectiveness (Bavelier et al., 2012), brain activity for

neuroscientiﬁc phenomena (Bohil et al., 2011), psy-

chological therapies (Norrholm et al., 2016) and re-

habilitation (Hoffman et al., 2011; Howard, 2017).

The advantage of using VR is also a core rea-

son why it can be so difﬁcult to collect physiologi-

cal data. Participant movement can lead to several

problems when it comes to collecting this data, as mi-

cro to macro movements can induce noise and arte-

facts, requiring thorough methods for ﬁltering the sig-

nal (Bian et al., 2019), and in the worst case having to

discard them (L

ecuyer et al., 2008). Through our ex-

tensive analysis of the literature, we observed a con-

stant trend in dealing with this exact recurring prob-

lem, where signals often had to be discarded, despite

the use of state-of-the-art physiological recording de-

vices (Kritikos et al., 2019; Solbiati et al., 2021; Dey

et al., 2022; Higuera-Trujillo et al., 2017; Petrescu

et al., 2020). Given that head tracking and 3D in-

teraction are fulcral components of the embodiment

experience, it is not possible to remove them with-

out compromising the use of VR in the ﬁrst place.

It is also worth noting that, even for interactive sys-

tems that present a lower degree of movement than

VR applications—such as traditional desktop gaming

applications—the problem persists, albeit to a lesser

extent (Lopes and Boulic, 2020; J

arvel

a et al., 2014).

The most common solutions in the literature of-

ten focus on ﬁltering these artefacts with traditional

methodologies such as low- and high-pass ﬁltering

techniques (Bian et al., 2019; Kritikos et al., 2019;

Solbiati et al., 2021; Dey et al., 2022; Higuera-

Trujillo et al., 2017; Petrescu et al., 2020), thresh-

old ﬁltering, and signal decomposition (Yuan et al.,

2019). Despite these methods being used extensively,

they are still sensitive to speciﬁc muscle contractions

and movements, forcing experimenters to exclude

these signals completely (Kritikos et al., 2019; Sol-

biati et al., 2021; Dey et al., 2022; Higuera-Trujillo

et al., 2017). Alternative electrode placement, and

even sensor types such as bracelets, have also been

explored (Borrego et al., 2019; Sra et al., 2019), as

certain parts of the body can experience less stress and

muscle contractions than parts subject to traditional

electrode placement (Lopes and Boulic, 2020). The

downside of placing electrodes on unconventional lo-

cations is that the signals collected present a lower

signal to noise ratio than when collected in common

Modelling Physiological Sensor Noise to Movement-Based Virtual Reality Activities

779

locations. Such locations are also not devoid of po-

tential noise, and given a lower amplitude signal, can

still result in discarded data points (Borrego et al.,

2019). To address this problem, some have suggested

to completely avoid or minimise the presence of noise

through meticulous protocol and task design where

certain movements and conditions are avoided (Lopes

and Boulic, 2020; J

arvel

a et al., 2014). However, this

is quite limiting, as movement and interaction are a

crucial aspect of VR applications (Jawed et al., 2021;

Hoffman et al., 2011; Kilteni et al., 2012; Slater,

2017; Howard, 2017), and limiting these features will

severely hinder the experience and put into question

its use in the ﬁrst place.

Classifying noise is not a novel approach, with

research often focusing on determining if noisy pat-

terns consistently emerge from signals (Park and Lee,

2020; Sweeney et al., 2010; Chowdhury et al., 2013).

If such patterns are proven to be consistent and allow

for a means of classifying them with a high degree

of accuracy, methods for ﬁltering this noise become

possible through the use of pattern recognition tech-

niques.

Previous studies have attempted to ﬁnd patterns in

movement-based noise classiﬁcation through the use

of accelerometers (Chavarriaga et al., 2013; Kunze

and Lukowicz, 2008) and electromyography (Yang

et al., 2017), with various degrees of success. This

position paper proposes an alternative solution for

studying movement-based noise patterns. This alter-

native consists of a VR experiment where participants

are asked to perform a series of repetitive minor tasks

speciﬁcally targeting body locomotion and muscle ac-

tivity, forcing them to move in very precise ways. Ex-

tracted data, consisting of common physiological sig-

nals (ECG, EDA, EMG and Respiration), mocap mea-

surements, and depth camera recordings, can then be

analysed to determine the relationship between spe-

ciﬁc movements and the observed noise artefacts.

3 RESEARCH PLAN AND

METHODS

The objective of the proposed research plan is to

model noise artefacts of physiological data recordings

generated by participant body movement. The opti-

mal ﬁnal result would be the construction of a model

capable of classifying these noise artefacts based on

the type and intensity of movement that generated

them. Furthermore, considering the objective of clas-

sifying movement speciﬁcally, an additional ques-

tion can be made relative to “how movement data

granularity can inﬂuence the accuracy of such mod-

els”. Therefore, the proposed study requires collect-

ing movement data from a wide range of sources,

such as real-time mocap data, depth cameras, and VR

tracker technologies. By leveraging these different

types of movement-based data, various types of clas-

siﬁers can be constructed. Lastly, by exploring the

relationship between physiological noise and move-

ment, a compilation of all the phenomena, difﬁcul-

ties, and best-practices can be made available to the

research community. This type of document can have

extensive beneﬁts for future experimental design so-

lutions using these technologies, ideally reducing the

amount of data that has to be discarded.

Although the proposed plan is focused on VR ex-

perimentation, its output can have implications be-

yond this scope, namely in biomedicine and health-

care. More speciﬁcally:

• Noise artefact detection.

• Immediate ﬁltering in real-time applications.

• Associating movements to noise patterns, offering

contextual information to experimenters.

• Personalised signal ﬁlters to each individual, us-

able for post-processing analysis rather than more

general methods (e.g., low- and high-pass ﬁlter).

• Measure and monitor physiological sensor sensi-

tivity according to body type, movement and limb

articulation, for development of more robust sen-

sor technology.

This paper proposes ﬁve milestones to achieve this

research plan:

M1 Experimental design and activity deﬁnition.

M2 VR experiment implementation.

M3 Motion capture integration.

M4 Data collection.

M5 Pre-Processing, feature decomposition, data

analysis and modelling.

The following subsections describe each mile-

stone in detail in addition to its sub-objective mile-

stones, demonstrating possible temporal overlap be-

tween these and the core milestones.

M1. Experimental Design and Activity

Deﬁnition

The goal of this milestone is to survey the require-

ments needed for accomplishing the proposed re-

search goals. These requirements include: deﬁning

the ﬁrst phases of the experimental protocol; an effec-

tive development and deployment strategy of all soft-

ware developed and hardware integration; and safe-

BIOSIGNALS 2024 - 17th International Conference on Bio-inspired Systems and Signal Processing

780

guarding the privacy and conﬁdentiality of participant

data.

The experimental protocol and methodology will

need to be submitted to the relevant ethics board to

ensure compliance with General Data Protection Reg-

ulations (GDPR) .

M1 is divided into three sub-milestones:

M1.1. Requirements Survey. For a successful ex-

periment it is ﬁrst necessary to evaluate what and how

data will be collected. We will start by deﬁning a se-

ries of movement types and classify them based on

their intensity (“is it a low-key or an assertive move-

ment?”, for example), degrees of freedom (e.g., up-

per or lower body movement, articulations) and mus-

cle activation. In-game triggers (or ﬂags) will also

need to be deﬁned to distinguish between the differ-

ent activities and effectively parse the data in post-

processing. Lastly, development considerations—

such as surveying the relevant libraries, software and

hardware integration methods needed for the develop-

ment of the applications—are also a requirement.

M1.2. Experiment Design. Given an initial set of

movements, a series of VR playful activities will be

designed and developed (see M4). These consist of

visual and active metaphors that will be used to help

the participants establish the intended movement, to

common menial tasks repurposed for VR interaction.

Examples include the movement of pushing or pulling

a door; pressing a button at different heights; walking

through a sidewalk; moving away from an incoming

object; and using a laser pointer to emphasise certain

words in a presentation. To further aid participants, it

is necessary to deﬁne how hints and visual aids will

be displayed and animated, where the objective is to

reduce ambiguity. Such activities can be disguised as

a playful game experience for the most part, offering

participants the opportunity to engage with the task

for more natural movement. Furthermore, given that

interaction using a mocap environment is slightly dif-

ferent than that of a traditional VR experience, it is

important to consider both approaches in the design

process. More speciﬁcally, to guarantee experimental

consistency, it is necessary to determine how move-

ments in one approach translates to the other, possi-

bly giving preference to movements that are easier to

translate between approaches.

Expected Outputs

1. A document with ethical procedures, data treat-

ment and hosting solutions compliant with GDPR.

2. Core movements survey effectively tagged based

on articulations, intensity and muscle activation.

3. Survey of initial VR design, including interface

and movement mockups detailing consistency of

the experimental design between the traditional

and mocap versions of the VR application.

4. Prototyping and reﬁnement of the application de-

sign.

M2. VR Experiment Implementation

The core objective of this milestone is the develop-

ment and integration of external devices into the main

experimental application (or game). This milestone

also includes the testing of mocap integration and

the piloting/deployment strategy, in parallel with M3

and M4, respectively. It is divided into three sub-

milestones:

M2.1. Design. This milestone addresses the de-

sign aspect of the application, deﬁning how partici-

pants and experimenters interact with it and how in-

formation is communicated to them. If certain design

choices end up not working within the testing and pi-

loting stage (milestone M2.3), these will return to this

design phase to be reﬁned using the knowledge ob-

tained from user testing.

M2.2. Core Implementation. This milestone fo-

cuses mainly on the process of integrating the features

designed in M1 and implemented in M2.1, where a

continuous debugging process is essential to guaran-

tee a robust application. This particular milestone

will be developed in proximity with the next mile-

stone (M2.3), which will continuously report bugs

and problems to be ﬁxed.

M2.3. Testing and Piloting. This milestone con-

sists of two different phases of quality assurance of

the application: testing and piloting. The goal of

testing is to discover potential software breaking in-

stances. Piloting will consist of assessing the appli-

cation with the purpose of determining (a) the usabil-

ity of the application from an experimenter and par-

ticipant point of view, and, (b) how the application

performs on-site once deployed. This will provide an

overall view of the application’s reliability and collect

data from participants to improve its usability and po-

tentially optimise the experiment protocol.

Expected Outputs

Several increasingly stable versions of the applica-

tion, followed through an internal Git repository and

Modelling Physiological Sensor Noise to Movement-Based Virtual Reality Activities

781

weekly feature and bug reports, are expected during

this milestone. Simple prototypes are initially ex-

pected for observing and validating if the core move-

ments deﬁned in M1 are being effectively achieved.

Alpha versions of the system for iteratively imple-

menting and testing features then follow. A subse-

quent beta version of the feature complete system

will be extensively tested for debugging and usabil-

ity. Lastly, a stable build of the application is the ﬁnal

expected output of this milestone.

Potential Pitfalls

Unexpected issues need to be considered, especially

considering that the project uses several technologies

that work in tandem for its effectiveness (i.e., game

engine, VR devices, and physiological recorder).

Thus, it is important to be aware of hardware lim-

itations, especially once deployed. The interactions

must also have a degree of recording effectiveness to

maintain a coherent noise to signal ratio and avoid

ﬂat signals (e.g., knocking off sensors during inter-

action, or impossible movements due to cable man-

agement limitations). Thus, an insistence on an initial

prototyping phase with rudimentary 3D assets, such

as testing the reliability of the chosen movements, is

crucial. If some movements are found to be unrealis-

tic to perform, others will then be chosen from the list

compiled in M1.

M3. Motion Capture Integration

This core milestone deals with the integration of

the selected mocap system within the Lab Streaming

Layer middleware suite

. Furthermore, to leverage

the granularity of movement capture by such a device,

additional data visualisation software needs to be de-

veloped, allowing the overlap of an ambiguous rep-

resentation of a participant’s avatar with the captured

physiological signals. This would allow the observa-

tion of the small variations of body movement over

time and how it inﬂuences noise on physiological sig-

nals. M3 is split into two additional milestones:

M3.1. Middleware Integration. This milestone

exclusively addresses the integration of a mocap data

stream, with the purpose of outputting the captured

data into a centralised synchronisation middleware

system. This will simplify data processing and analy-

sis in M5, as all time-series (i.e., physiology, mocap,

and game metrics and triggers) will be effectively syn-

chronised. It also offers a way of increasing the accu-

racy of automated triggers sent by the game, allowing

https://labstreaminglayer.org/

automated parsing scripts to cut the data according

to each activity or other requirements (e.g., collecting

baseline).

M3.2. Body Motion Visualisation System. This

milestone focuses on developing a simple body ren-

dering system capable of extrapolating the mocap

data by generating a 3D visualisation of the partici-

pant’s body movement. Avatars will consist of face-

less representations of participants and will accom-

pany the physiological signals. This allows experi-

menters to visualise a side-by-side comparison of spe-

ciﬁc events with the different collected physiological

channels (e.g., ECG, EDA, EMG and Respiration).

Such a system can be built using a game engine and

various assets that use mocap data to create 3D ani-

mations. Considering this has no dependencies with

the experimental application itself, it can be built in

parallel with M2.

Expected Outputs

1. Update of the internal data collection middleware

integrating the selected mocap system.

2. Mocap data visualisation software.

3. Independent validation of the ﬁdelity of data col-

lected through the data collection middleware.

M4. Data Collection

This core milestone consists primarily of the deploy-

ment phase, where the experimenter will collect the

necessary data for statistical analysis. This consists of

conducting several trials of the different movements

deﬁned in M1. This milestone is dependent on both

M2 and M3, as for this process to take place the ap-

plication must be fully integrated and functioning.

During this milestone experimenters need to fol-

low a strict experimental protocol, programmed

within the application, which will present all the

movements the participants are required to make. Ide-

ally the application will be sufﬁciently autonomous to

(a) minimise experimenter mistakes, (b) start collect-

ing data as soon as the experiment begins, and, (c)

guide participants effectively over the course of the

experiment. The experimenter will have their own in-

terface to visualise (a) the physiological data in real-

time (in case any adjustments are necessary during the

experiment), and, (b) what the participant is doing in-

game.

BIOSIGNALS 2024 - 17th International Conference on Bio-inspired Systems and Signal Processing

782

Expected Outputs

1. An experimental dataset, including all expected

data types: physiology, mocap vectors, camera

skeletal features, in-game markers, participant

questionnaires, and demographics.

2. Dataset manual, including instructions for parsing

and description of all data types in the dataset.

3. Automatic parser script providing an initial treat-

ment of the data, which will be included with the

dataset.

Potential Pitfalls

The most relevant risk with this milestone is signal

degradation due to sensors being knocked off by par-

ticipants or hardware failures. The extensive testing

in M2 attempts to mitigate this problem by offering

the experimenter a set of monitoring features that will

help assess the quality of the signal during experimen-

tation. This means experimenters will have real-time

information about the signals being recorded and after

each trial may pause the experiment and re-adjust the

electrodes on participants for more efﬁcient capture.

Although the objective of the experiment is to cap-

ture signal noise, it has to be able to do so in the ideal

capturing conditions, or more precisely, some trace of

the original signal must still be present in the record-

ing for a model to be able to adjust and effectively

remove the noise.

As an additional contingency, a small visualisa-

tion software should be made available for the ex-

perimenter, so that they may parse and visualise all

captured signals from that session and access its qual-

ity and conduct a superﬁcial analysis. If the signal

is deemed inadequate it is tagged as “damaged” and

placed in the dataset with this condition. A replace-

ment participant will then be recruited to replace these

trials and obtain the expected number of trials as de-

ﬁned in M1.

M5. Pre-Processing, Feature

Decomposition, Data Analysis and

Modelling

This milestone will consolidate all activities related

to the treatment, cleaning, pruning, parsing, analy-

sis, and modelling of the data collected in M4. It

is divided into three sub-milestones, which will con-

sider the observed results and recent state-of-the-art

approaches that may beneﬁt the study.

M5.1. Cleaning and Parsing Trials. The ﬁrst sub-

milestone is dedicated to cleaning, analysing, and

parsing the raw signal values. This will include de-

veloping several data visualisation scripts. Despite

the core objective of this proposal, some form of data

pruning will be considered, such as “dead” signals

caused directly by the disconnection of sensors.

M5.2. Time-Series Analysis. Signals are analysed

with respect to their baseline value, and all move-

ments in each category will be compared to this base-

line through polynomial regression. This will allow

researchers to understand the delta ﬂuctuations be-

tween baseline and movement trials. This will also

be done for trials within the same movement category

with the purpose of learning if a large distance ex-

ists between these trials. Furthermore, the correlation

between signals within the same movement category

will also be investigated; the goal is to determine if

a high correlation is still present within a signal de-

spite artefacts being present during this movement.

This will allow reseachers to observe if a natural ten-

dency exists between the actual body movement and

the artefacts that appear in the signal.

M5.3. Categorising and Machine Learning Meth-

ods. One of the main objectives of this experiment

is to learn if certain artefacts can be categorised based

on the movement type, i.e., if we can categorise noise

artefacts based on the movement of an individual. As

a ﬁrst methodology, the intent is to use a technique

such as PCA or UMAP (Fachada et al., 2016; Du

et al., 2023) to reduce the dimensionality of the signal

and apply an unsupervised learning approach such as

k-means clustering. This will provide an initial ob-

servation of the data patterns to test the hypothesis

that certain noise artefacts correlated with a speciﬁc

movement create speciﬁc clusters. It is important to

consider that this proposal is not limited to this type

of methodology. Given its exploratory nature, tech-

niques such as supervised learning, forecasting, and

others can be also be explored.

Expected Outputs

1. Anonymized and clean dataset usable for dissem-

ination purposes (including the raw unprocessed

data).

2. Detailed report on the ﬁndings and observations

related to the questions asked in M1.

3. Detailed report on good and optimal practises of

physiological data-collection in VR applications

with a high-degree of movement.

4. Statistical models capable of categorising noise

based on movement types or common features it

presents.

Modelling Physiological Sensor Noise to Movement-Based Virtual Reality Activities

783

Potential Pitfalls

The knowledge obtained through this work will re-

sult in several studies and methods about the effects

of movement on physiological signals. Thus, even if

models prove difﬁcult to train, many of the method-

ologies can be used to improve how this data can be

collected, in turn enhancing the overall quality of data

collection in studies involving VR.

4 CONCLUSIONS

This position paper proposed a step-by-step research

plan to study the inﬂuence of movement on the qual-

ity of physiological data captured during VR sessions,

putting forth the hypothesis that the noise artefacts

can be classiﬁed based on the type of movements per-

formed by participants. The main outcome of such

study would be a model capable of classifying noisy

physiological signal features, providing insights into

movement’s inﬂuence on artifacts and contributing to

the development of movement-based ﬁlters and best

practices for using the associated technologies.

Although the purpose of this research is to specif-

ically study VR movement, the application of all the

knowledge assembled throughout its implementation

could be applied towards a wide range of healthcare

based applications. As discussed, VR has been ex-

tensively used for psychological treatment, rehabilita-

tion, and even distracting patients from painful proce-

dures. Observing the relationship between gross mo-

tor activity and the noise captured from sensors could

foster the development of new methods for increasing

sensor robustness, while avoiding an increase in cost.

In conclusion, by offering a better understanding of

the movement-signal relationship—therefore allow-

ing higher-quality data collection in VR sessions—

this research proposal can potentially lead to cheaper

and more effective VR treatments for improving the

health and well-being of patients.

ACKNOWLEDGEMENTS

This work was funded by Fundac¸

ao para a

encia e Tecnologia under HEI-Lab R&D Unit

(UIDB/05380/2020: ht t p s : / / d oi . o r g / 1 0 .5 4 4 9 9

/ U I D B / 0 5 3 8 0 / 2 0 20), COPELABS R&D Unit

(UIDB/04111/2020: ht t p s : / / d oi . o r g / 1 0 .5 4 4 9 9

/ U I D B / 0 4 1 1 1 / 2 0 20) and LIBPhys R&D Unit

(UIDB/FIS/04559/2020: https://doi.org/10.54499/U

IDB/04559/2020 and UIDP/FIS/04559/2020: https:

//doi.org/10.54499/UIDP/04559/2020).

REFERENCES

Bavelier, D., Green, C. S., Pouget, A., and Schrater, P.

(2012). Brain plasticity through the life span: learn-

ing to learn and action video games. Annual Review

of Neuroscience, 35(1):391–416.

Bian, D., Wade, J., Swanson, A., Weitlauf, A., Warren, Z.,

and Sarkar, N. (2019). Design of a physiology-based

adaptive virtual reality driving platform for individu-

als with asd. ACM Transactions on Accessible Com-

puting (TACCESS), 12(1):1–24.

Bohil, C. J., Alicea, B., and Biocca, F. A. (2011). Virtual

reality in neuroscience research and therapy. Nature

Reviews Neuroscience, 12:752–762.

Borrego, A., Latorre, J., Alca

niz, M., and Llorens, R.

(2019). Reliability of the Empatica E4 wristband to

measure electrodermal activity to emotional stimuli.

In 2019 International Conference on Virtual Rehabil-

itation (ICVR), pages 1–2. IEEE.

Chavarriaga, R., Bayati, H., and Mill

an, J. d. R. (2013). Un-

supervised adaptation for acceleration-based activity

recognition: robustness to sensor displacement and ro-

tation. Personal and Ubiquitous Computing, 17:479–

490.

Chowdhury, R. H., Reaz, M. B., Ali, M. A. B. M., Bakar,

A. A., Chellappan, K., and Chang, T. G. (2013). Sur-

face electromyography signal processing and classiﬁ-

cation techniques. Sensors, 13(9):12431–12466.

Cosi

c, K., Popovi

c, S., Kukolja, D., Horvat, M., and Drop-

ulji

c, B. (2010). Physiology-driven adaptive virtual re-

ality stimulation for prevention and treatment of stress

related disorders. CyberPsychology, Behavior, and

Social Networking, 13(1):73–78.

Dey, A., Barde, A., Yuan, B., Sareen, E., Dobbins, C.,

Goh, A., Gupta, G., Gupta, A., and Billinghurst, M.

(2022). Effects of interacting with facial expressions

and controllers in different virtual environments on

presence, usability, affect, and neurophysiological sig-

nals. International Journal of Human-Computer Stud-

ies, 160:102762.

Du, Y., Sui, J., Wang, S., Fu, R., and Jia, C. (2023). Motor

intent recognition of multi-feature fusion EEG signals

by UMAP algorithm. Medical & Biological Engineer-

ing & Computing, 61(10):2665–2676.

Fachada, N., Rodrigues, J., Lopes, V. V., Martins, R. C.,

and Rosa, A. C. (2016). micompr: An R package for

multivariate independent comparison of observations.

The R Journal, 8(2):405–420.

Higuera-Trujillo, J. L., Maldonado, J. L.-T., and Mill

an,

C. L. (2017). Psychological and physiological hu-

man responses to simulated and real environments:

A comparison between photographs, 360 panoramas,

and virtual reality. Applied ergonomics, 65:398–409.

Hoffman, H. G., Chambers, G. T., Meyer III, W. J., Arce-

neaux, L. L., Russell, W. J., Seibel, E. J., Richards,

T. L., Sharar, S. R., and Patterson, D. R. (2011). Vir-

tual reality as an adjunctive non-pharmacologic anal-

gesic for acute burn pain during medical procedures.

Annals of Behavioral Medicine, 41(2):183–191.

BIOSIGNALS 2024 - 17th International Conference on Bio-inspired Systems and Signal Processing

784

Howard, M. C. (2017). A meta-analysis and systematic

literature review of virtual reality rehabilitation pro-

grams. Computers in Human Behavior, 70:317–327.

arvel

a, S., Ekman, I., Kivikangas, J. M., and Ravaja, N.

(2014). A practical guide to using digital games as

an experiment stimulus. Transactions of the Digital

Games Research Association, 1(2).

Jawed, Y. T., Golovyan, D., Lopez, D., Khan, S. H., Wang,

S., Freund, C., Imran, S., Hameed, U. B., Smith, J. P.,

Kok, L., et al. (2021). Feasibility of a virtual reality

intervention in the intensive care unit. Heart & Lung,

50(6):748–753.

Kilteni, K., Groten, R., and Slater, M. (2012). The sense of

embodiment in virtual reality. Presence: Teleopera-

tors and Virtual Environments, 21(4):373–387.

Kritikos, J., Tzannetos, G., Zoitaki, C., Poulopoulou, S.,

and Koutsouris, D. (2019). Anxiety detection from

electrodermal activity sensor with movement & inter-

action during virtual reality simulation. In 2019 9th

International IEEE/EMBS Conference on Neural En-

gineering (NER), pages 571–576. IEEE.

Kunze, K. and Lukowicz, P. (2008). Dealing with sensor

displacement in motion-based onbody activity recog-

nition systems. In Proceedings of the 10th Inter-

national Conference on Ubiquitous Computing, Ubi-

Comp ’08, pages 20–29. ACM.

ecuyer, A., Lotte, F., Reilly, R. B., Leeb, R., Hirose, M.,

and Slater, M. (2008). Brain-computer interfaces, vir-

tual reality, and videogames. Computer, 41(10):66–

72.

Liapis, A., Katsanos, C., Sotiropoulos, D., Xenos, M., and

Karousos, N. (2015). Stress recognition in human-

computer interaction using physiological and self-

reported data: a study of gender differences. In Pro-

ceedings of the 19th Panhellenic Conference on Infor-

matics, PCI ’15, pages 323–328. ACM.

Lopes, P. and Boulic, R. (2020). Towards designing games

for experimental protocols investigating human-based

phenomena. In Proceedings of the 15th International

Conference on the Foundations of Digital Games,

FDG ’20, pages 1–11. ACM.

Lopes, P., Liapis, A., and Yannakakis, G. N. (2017). Mod-

elling affect for horror soundscapes. IEEE Transac-

tions on Affective Computing, 10(2):209–222.

Montana, J. I., Matamala-Gomez, M., Maisto, M.,

Mavrodiev, P. A., Cavalera, C. M., Diana, B., Manto-

vani, F., and Realdon, O. (2020). The beneﬁts of emo-

tion regulation interventions in virtual reality for the

improvement of wellbeing in adults and older adults:

a systematic review. Journal of Clinical Medicine,

9(2):500.

Norrholm, S. D., Jovanovic, T., Gerardi, M., Breazeale,

K. G., Price, M., Davis, M., Duncan, E., Ressler, K. J.,

Bradley, B., Rizzo, A., et al. (2016). Baseline psy-

chophysiological and cortisol reactivity as a predictor

of ptsd treatment outcome in virtual reality exposure

therapy. Behaviour Research and Therapy, 82:28–37.

Park, G. and Lee, S. (2020). Environmental noise classi-

ﬁcation using convolutional neural networks with in-

put transform for hearing aids. International Jour-

nal of Environmental Research and Public Health,

17(7):2270.

Petrescu, L., Petrescu, C., Mitrut

, O., Moise, G.,

Moldoveanu, A., Moldoveanu, F., and Leordeanu, M.

(2020). Integrating biosignals measurement in virtual

reality environments for anxiety detection. Sensors,

20(24):7088.

Slater, M. (2017). Implicit learning through embodiment in

immersive virtual reality. In Liu, D., Dede, C., Huang,

R., and Richards, J., editors, Virtual, Augmented, and

Mixed Realities in Education, pages 19–33. Springer.

Solbiati, S., Buffoli, A., Megale, V., Damato, G., Lenzi,

B., Langfelder, G., and Caiani, E. (2021). Detecting

heart rate from virtual reality headset-embedded iner-

tial sensors: a kinetic energy approach. In 2021 Inter-

national Conference on e-Health and Bioengineering

(EHB), pages 1–4. IEEE.

Sra, M., Jain, A., and Maes, P. (2019). Adding proprio-

ceptive feedback to virtual reality experiences using

galvanic vestibular stimulation. In Proceedings of the

2019 CHI Conference on Human Factors in Comput-

ing Systems, CHI ’19, pages 1–14. ACM.

Sweeney, K. T., Leamy, D. J., Ward, T. E., and McLoone,

S. (2010). Intelligent artifact classiﬁcation for am-

bulatory physiological signals. In 2010 Annual In-

ternational Conference of the IEEE Engineering in

Medicine and Biology, pages 6349–6352. IEEE.

Yang, X., Yeh, S.-C., Niu, J., Gong, Y., and Yang, G. (2017).

Hand rehabilitation using virtual reality electromyog-

raphy signals. In 2017 5th International Conference

on Enterprise Systems (ES), pages 125–131. IEEE.

Yuan, Y., Huang, J., and Yan, K. (2019). Virtual reality

therapy and machine learning techniques in drug ad-

diction treatment. In 2019 10th International Con-

ference on Information Technology in Medicine and

Education (ITME), pages 241–245. IEEE.

Modelling Physiological Sensor Noise to Movement-Based Virtual Reality Activities

785