Preliminary Results on the Evaluation of Different Feedback

Methods for the Operation of a Muscle-Controlled Serious Game

Julia Habenicht

and Elsa Andrea Kirchner

1,2

University of Duisburg-Essen, Bismarckstraße 81 47057 Duisburg, Germany

Robotics Innovation Center, German Research Center for Artificial Intelligence, Robert-Hooke Straße 1,

28359 Bremen, Germany

Keywords: Muscle-Controlled Serious Game, Motor Learning, Auditory Feedback, Visual Feedback, Electromyogram.

Abstract: Muscle-controlled serious games can improve the ability of a targeted muscle control. This aspect is important

for controlling muscle-controlled prostheses or for (re-)learning motor movements. Although there are more

options, all muscle-controlled serious games are using visual feedback for providing information of the

current muscle activity. The aim of this study is to compare the feedback methods visual, auditory and haptic

feedback for motor learning with a muscle-controlled serious game. Due to the current status of the study, in

this paper only the results of visual and auditory feedback will be analysed. Subjects were divided into two

groups – visual or auditory feedback. A muscle-controlled serious game was played on three days in a row

by three subjects in each group. For the visual group the game provided only visual and for the auditory group

it provided only auditory feedback. At the end of each session one set without any feedback was played to

control the learning status. Preliminary results show a slightly better performance of the auditory group. As

the results aren’t significant, more subjects are needed to get further information about the most promising

feedback method for motor learning with muscle-controlled serious games.

1 INTRODUCTION

Serious games are games, which aren’t there just for

fun but also to have positive effects on the player

(Olgers, de Weg, & Ter Maaten, 2021). In muscle-

controlled serious games the game is used to provide

biofeedback (in this case the muscle activity) in form

of a game. With those games the targeted control of

muscle activity can be practiced (Ghassemi, et al.,

2019).

There are two primary target groups for muscle-

controlled serious games. One target group consists

of people with muscle-controlled prostheses, for

whom a better prothesis control is aimed. Studies

investigated the effect of practicing with muscle-

controlled serious games with the aim to reach a

better control of muscle-controlled prostheses. The

results show a significant positive effect on fine

motor movements with prostheses (Radhakrishnan,

Smailagic, French, Siewiorek, & Balan, 2019, van

Dijk et al. 2016) and also for the targeted control of

individual muscles (Winslow, Ruble, & Huber, 2018).

The other target group consists of stroke patients.

The results of studies show that playing muscle-

controlled serious games can help the patients re-

learning fine motor movements (Hung, et al., 2021)

and can also help to build muscle mass, which got lost

due to the stroke (Garcia-Hernandez, Garza-Martinez,

& Parra-Vega, 2018).

The results of these studies show consistently

positive effects on learning the control of muscle

activity. According to our research, all of these games

work with visual feedback. Serious games which are

controlled by movements in general not only muscle

activity often use different types of feedbacks. A study

of Schättin et al. (2022) investigated the likability of

visual, auditory and haptic feedback while playing a

serious game. The results of the interviews revealed

that the subjects liked the haptic feedback as well as the

combination of haptic feedback and auditory feedback

the most (Schättin, et al., 2022).

When it comes to the correct execution of

movements, auditory feedback in the form of alarms

has also proven to be a useful feedback method.

(Riskowski, Mikesky, Bahamonde, & Burr, 2009;

Underwood, 2009; Clarkson, James, Watkins, & Foley,

2013 ;Riskowski, Mikesky, Bahamonde, & Burr,

2009; Baudry, Leroy, Thouvarecq, & Choller, 2006).

Habenicht, J. and Kirchner, E.

Preliminary Results on the Evaluation of Different Feedback Methods for the Operation of a Muscle-Controlled Serious Game.

DOI: 10.5220/0012309600003657

Paper published under CC license (CC BY-NC-ND 4.0)

In Proceedings of the 17th International Joint Conference on Biomedical Engineering Systems and Technologies (BIOSTEC 2024) - Volume 1, pages 721-725

ISBN: 978-989-758-688-0; ISSN: 2184-4305

721

The goal of studies that used auditory alarms was to

signal the subject when they were not performing a

movement correctly. These results showed an

immediate positive effect (Riskowski, Mikesky,

Bahamonde, & Burr, 2009) but also effects up to two

weeks (Baudry, Leroy, Thouvarecq, & Choller, 2006).

Haptic feedback just like vibration alone isn’t very

common in serious games. It is mainly added as a

supporting feedback method (Kaul, Meier, & Rohs,

2017). Outside serious games it is often used to

improve orientation in order to reduce the workload of

the visual and auditory system (van Erp, Saturday, &

Jansen, 2006).

As auditory and haptic feedback got positive

results regarding serious games in general, they could

also be a possible feedback method in muscle-

controlled serious games. The aim of this study is to

evaluate the feedback methods of visual, auditory and

haptic feedback while playing a muscle-controlled

serious game. Due to the current status of the study

only the first results of the visual and auditory

modality will be presented.

2 METHODS

2.1 Participants

In the experiment 6 healthy subjects (3 male and 3

female; average age 24.5 ±2.9) voluntarily partici-

pated. All subjects were righthanded in accordance to

the Edinburgh handedness inventory (Oldfield, 1971).

They gave their written informed consent to the

experiment and were told that they could stop the

experiment at any time without any consequences.

Information about the ethic vote can be found in the

section Ethics Statement at the end of the paper. The

six subjects were divided into either the visual group or

the auditory group resulting in two groups of three

subjects. The group selection was randomized.

2.2 Data Acquisition

2.2.1 Electromyography

Before the experiment started subjects were prepared

with a surface EMG electrode to measure the muscle

activity of the right m. flexor digitorum (Figure 1.).

The bipolar 16 channel waveplus pico blue EMG

system from Cometa was used

to measure the muscle

activity. Skin preparation included the cleaning with

alcohol. The electrode was placed in accordance to

the SENIAM guidelines (Hermens, Freriks,

Disselhorst-Klug, & Rau, 2000).

Figure 1: Location of the m. flexor digitorum

(alamy, n.d).

2.2.2 Serious Game

For the visual group the game consists of a column

divided into four areas (Figure 2.). These areas can be

reached by a bar, which is controlled by the

contraction of the muscle. Next to the areas the digits

0, 1, 5, 10 are shown. The goal of the game is to reach

the areas of the digit displayed in randomized order

as accurately as possible with the muscle-controlled

bar. The digit to be reached is clearly visible above

the column. The bar must be in the area of the

displayed digit for at least three seconds. If the bar is

in the correct range, a countdown of three seconds

appears. After the three seconds, the target is reached

and the bar must be steered into the zero range by

relaxing the muscle. After another three seconds in

this area, the next target appears.

The auditory group was placed behind a movable

wall so they weren’t able to see the screen. For this

group the goal was to hear the sound of music

instruments which are connected to a certain muscle

activity. The ranges for the muscle activity of the

instruments correspond to the ranges of the numbers

from the visual group. The subjects were told an

instrument and the task was to find the muscle

contraction which is connected to the before heard

instrument. When they could hear the sound of the

instrument, they knew they were in the right range.

Otherwise, there was no sound. When they lasted in

the right range for at least three seconds a bubble

sound was played and they had to relax. When the

muscle activity was under a pre-defined value the

next instrument was announced.

Before playing the game, it had to be calibrated.

For this, the maximum muscle activity and the

relaxed muscle activity was measured. Based on these

values the ranges for the game were calculated. A

detailed description of which number/ instrument

correspond to which muscle activity can be seen in

Table 1.

BIOSIGNALS 2024 - 17th International Conference on Bio-inspired Systems and Signal Processing

722

Figure 2: Muscle-controlled Serious Game for visual

feedback.

Table 1: Explanation of the game calibration. Goals with

the corresponding muscle activity of the maximal voluntary

contraction (MVC).

Goals Muscle activity

Goal 1

Number 1/

Instrument 1

20%-40% of MVC

Goal 2

Number 5/

Instrument 2

40%-60% of MVC

Goal 3

Number 10/

Instrument 3

60%-80% of MVC

2.2.3 Experimental Design

Seven sets of the game were played with a following

learning control set on three days in a row. In each set

every number/ instrument had to be reached three

times. The order was randomized. The learning

control set consists of a set in which the subjects

didn’t receive any feedback of the game. The game

just showed/told the target and the subjects had to

contract the muscle the right way out of their

memory. After they thought they were in the right

range for at least three seconds they had to relax the

muscle and the next target was shown/told. A detailed

description of the experimental design is depicted in

Figure 3.

To evaluate the different feedback methods, the

learning control set data was used to determine the

percentage of the time the muscle activity was in the

right range for each of the goals. The analysis was

made for each goal in both groups. For the

comparison between the feedback methods a

Wilcoxon Test was performed between the visual and

auditory data of all results of all subjects (Figure 4).

The significance value was set at p< 0.05.

Figure 3: Experimental Design.

Figure 4: Description of statistical analysis.

3 RESULTS

Over all, the auditory group achieved a higher

performance than the visual group although this

difference was not significant. Table 2. shows the

results of each subject for each goal. The values in the

table represent the mean results of all three days for

Table 2: Percentage of time the muscle activity was in the

right range per goal. The results of the learning control set

of all three days are depicted.

Visual Feedback

Subject 1 Subject 2 Subject 3

Goal 1 40.64% 32.03% 27.58%

Goal 2 22.04% 27.47% 14.96%

Goal 3 27.43% 27.68% 56.34%

Auditory Feedback

Subject 1 Subject 2 Subject 3

Goal 1 42.23% 47.57% 24.35%

Goal 2 23.40% 27.31% 27.57%

Goal 3 39.92% 40.32% 55.57%

Muscle-controlled

Shown goal

Numbers to

reach

Goal X

Preliminary Results on the Evaluation of Different Feedback Methods for the Operation of a Muscle-Controlled Serious Game

723

each goal. In average both groups showed the lowest

performance reaching goal 2 and the highest

performance in reaching goal 3. Figure 5 represents

the combined results of each subject per group of all

three days for the three goals. Results show a higher

performance in reaching each goal which leads to a

higher performance in general for the auditory group

(Figure 6)

Figure 5: Achieved goals in the learning control sets of the

visual and auditory group in percent. Each block combines

the results of all three days of each subject.

Figure 6: Comparison between visual and auditory group.

Each block combines all achieved results of each subject for

each goal of the learning control set on all three days.

4 DISCUSSION

The aim of the study was the evaluation of different

feedback methods for a muscle-controlled serious

game. In general, the results show a tendency for

better performance of the auditory group in each goal

which can be interpreted as a higher learning effect

for the auditory group. Since the results are not

significant, more subjects need to be studied.

An explanation for the better results of the

auditory group could be the guidance hypothesis

which states that the permanent feedback during

acquisition leads to a dependency on the feedback

(Salamin, Tadi, Blanke, Vexo, & Thalmann, 2010;

Schmidt, Frequent Augmented Feedback Can

Degrade Learning: Evidence and Interpretations,

1991; Schmidt, Young, Swinnen, & Shapiro, 1989).

As the auditory group only received feedback when

they where in the right range, no dependency on the

feedback could be developed. That could be an

explanation for the higher performance in the learning

control set.

For the investigated subjects it is striking that for

both groups the lowest performance was at goal 2. A

reason could be that for goal 1 and goal 3 one could

figure out that either the muscle activity should be

very high or very low. To find the range in between

is therefore more difficult.

Nevertheless, more subjects are needed to verify

the presented results. For a right statement the

experiments for the haptic feedback have to be

performed and analysed as well.

5 CONCLUSIONS

This study should provide further insights into

possible feedback methods in muscle-controlled

serious games. However, individual adaptations may

be necessary for the application in rehabilitation

practice.

These first results show a tendency towards a

higher learning effect in the auditory group. As these

data are not statistically significant more subjects are

needed for verification. Considering the current

results so far, auditory feedback should be taken into

account as a method for motor learning with muscle-

controlled serious games.

ETHICS STATEMENT

The studies involving human participants were

approved by the local Ethical Committee of the

University of Duisburg-Essen, Germany. The

participants provided their written informed consent

to participate in this study.

BIOSIGNALS 2024 - 17th International Conference on Bio-inspired Systems and Signal Processing

724

REFERENCES

alamy. (n.d.). alamy. Retrieved from https://www.alamy.de/

fotos-bilder/flexor-digitorum-superficialis-muscle.html?

sortBy=relevant

Baudry, L., Leroy, D., Thouvarecq, R., & Choller, D.

(2006, February 24). Auditory concurrent feedback

benefits on the circle performed in gymnastics . Journal

of Sports Science.

Clarkson, P. M., James, R., Watkins, A., & Foley, P.

(February 2013). The Effect of Augmented Feedback

on Foot Pronation during Barre Exercise in Dance.

Research Quarterly for Exercise and Sport.

Garcia-Hernandez, N., Garza-Martinez, K., & Parra-Vega,

V. (2018). Electromyography Biofeedback Exergames

to Enhance Grip Strength and Motivation. Games

Health Journal.

Ghassemi, M., Triandafilou, K., Barry, A., Stoykov, M. E.,

Roth, E., Mussa-Ivaldi, F. A., . . . Ranganathan, R. (27.

February 2019). Development of an EMG-Controlled

Serious Game for Rehabilitation. IEEE Engineering in

Medicine and Biology Society .

Hermens, H. J., Freriks, B., Disselhorst-Klug, C., & Rau,

G. (2000, October). Development of recommendations

for SEMG sensors and sensor placement procedures .

Journal of Electromyography and Kinesiology.

Hung, N. T., Paul, V., Prakash, P., Kovach, T., Tacy, G.,

Tomic, G., Slutzky, M. W. (2021, August 21).

Wearable myoelectric interface enables high-dose,

home-based training in severely impaired chronic

stroke survivors. Annals of Clinical and Translational

Neurology.

Kaul, O. B., Meier, K., & Rohs, M. (2017, September 21).

Increasing Presence in Virtual Reality with a

Vibrotactile Grid Around the Head. Lecture Notes in

Computer Science.

Oldfield, R. C. (1971, March). The assessment and analysis

of handedness: the Edinburgh inventory.

Neuropsychologie.

Olgers, T. J., de Weg, A. A., & Ter Maaten, J. C. (2021,

January 25). Serious Games for Improving Technical

Skills in Medicine: Scoping Review. JMIR Serious

Games.

Radhakrishnan, M., Smailagic, A., French, B., Siewiorek,

D. P., & Balan, R. K. (2019). Design and Assessment

of Myoelectric Games for Prosthesis Training of Upper

Limb Amputees.

Riskowski, J. L., Mikesky, A. E., Bahamonde, R. E., &

Burr, D. B. (2009, September). Design and Validation

of a Knee Brace With Feedback to Reduce the Rate of

Loading. Journal of Biomechanics.

Salamin, P., Tadi, T., Blanke, O., Vexo, F., & Thalmann,

D. (2010, October). Quantifying Effects of Exposure to

the Third and First-Person Perspectives in Virtual-

Reality-Based Training. IEEE Transactions on

Learning Technologies.

Schättin, A., Pickles, J., Flagmeier, D., Schärer, B.,

Riederer, Y., Niederecken, S., . . . Martin_Niederecken,

A. L. (2922, December). Development of a Novel

Home-Based Exergame With On-Body Feedback:

Usability Study. JMIR Serious Games.

Schmidt, R. A. (1991). Frequent Augmented Feedback Can

Degrade Learning: Evidence and Interpretations.

Tutorials in Motor Neuroscience

Schmidt, R. A., Young, G. E., Swinnen, S. E., & Shapiro,

D. C. (1989). Summary Knowledge of Results for Skill

Acquisition: Support for the Guidance Hypothesis.

Journal of Experimental Psychology Learning Memory

and Cognition.

Underwood, S. M. (2009). Effects of Augmented Real-

Time Auditory Feedback on Top-Level Precision

Shooting Performance.

van Erp, J. B., Saturday, I., & Jansen, C. (2006).

Application of tactile displays in sports: where to, how

and when to move.

Winslow, B. D., Ruble, M., & Huber, Z. (11. July 2018).

Mobile, Game-Based Training for Myoelectric

Prosthesis Control. Frontiers Bioengeneering and

Biotechnology.

Preliminary Results on the Evaluation of Different Feedback Methods for the Operation of a Muscle-Controlled Serious Game

725