Real-time Evaluation System for Top Taekwondo Athletes:

Project Overview

Pedro Cunha

1,2 a

, Paulo Barbosa

, Fábio Ferreira

, Carlos Fitas

, Vítor Carvalho

1,2 b

and

Filomena Soares

2Ai - School of Technology, IPCA, Barcelos, Portugal

Algoritmi Research Centre, University of Minho, Guimarães, Portugal

Keywords: Deep Learning, Motion Analysis, Neural Networks, Wearable, Computer Vision, Taekwondo.

Abstract: Assessing athletes' performance is a constant challenge for coaches, whatever the sport is. In some sports

there are no technological solutions to assist coaches in this task. This is the case of Taekwondo, where

currently the methods used are mainly manual. Following this trend, this article presents the work developed

in a PhD project whose main objective is the development of a friendly and low-cost system for assessing the

performance of Taekwondo athletes in real time. Thus, the system uses a 3D camera with depth sensor

(Orbbec Astra), a computer and software developed for data collection and processing. The system also

provides the inclusion of Inertial Measurement Units (IMUs). The system allows an accurate feedback for the

correction or improvement of the athlete's techniques, enabling an increase in the athlete's performance in a

shorter period of time. In all, the project contributes to the evolution of the techniques used during Taekwondo

training, as well as to the technological development in the practice of Taekwondo.

1 INTRODUCTION

Technology plays a major role in everyday life across

the spectrum of society. Its use has been increasing,

as an integral part of the daily routine, becoming

something natural and often imperceptible. In

different areas of society and science, the use of

technology is possible through the combination of

information processing and the continuous use of

tools, extended through computing devices, known as

ubiquitous computing (Baca, Dabnichki, Heller, &

Kornfeind, 2009).

One of the areas in which research and

development of technological solutions has obtained

an active participation from the scientific community

is motion analysis. It is currently possible to analyse

the movements of humans in a natural environment

of activity, without the need for markers on the

human body. The result of these new methods of

motion analysis contributes to the creation and

availability of accessible and easy-to-use motion

capture solutions (King & Paulson, 2007).

https://orcid.org/0000-0001-6170-1626

https://orcid.org/0000-0003-4658-5844

https://orcid.org/0000-0002-4438-6713

In sport, the evolution in movement analysis has

allowed the development of technological solutions

that help athletes, coaches and referees in certain

tasks (Thomas, Gade, Moeslund, Carr, & Hilton,

2017). These technological solutions are divided in

two main groups. The non-optical systems, which

uses sensors placed on the athlete's body to obtain

movement information. And the optical systems, with

or without markers. The optical systems with markers

show better results combined with a higher cost and

complexity of implementation, as well as higher

intrusiveness, and systems without markers are easier

to implement, being less intrusive (Pueo & Jimenez-

Olmedo, 2017).

Assessing the performance of athletes is a

complex and difficult task in any sport. The inclusion

of motion analysis in the practice of sport, through the

systems developed, came to assist in this task. Some

of the systems developed to perform the evaluation

allows to obtain relevant information of the athlete’s

performance like velocity, acceleration, force,

displacement, among other characteristics (Arastey,

Cunha, P., Barbosa, P., Ferreira, F., Fitas, C., Carvalho, V. and Soares, F.

Real-time Evaluation System for Top Taekwondo Athletes: Project Overview.

DOI: 10.5220/0010414202090216

In Proceedings of the 14th International Joint Conference on Biomedical Engineering Systems and Technologies (BIOSTEC 2021) - Volume 1: BIODEVICES, pages 209-216

ISBN: 978-989-758-490-9

209

2020) (Cunha, Carvalho, & Soares, 2018) (Nadig &

Kumar, 2015).

In Taekwondo martial art, the evaluation athletes’

performance is still carried out by traditional

methods, that is, by viewing videos of the training

sessions or in loco of the athletes' movements in real

time. This is time consuming for the coach and delays

the feedback for improvements to the athlete (Pinto,

et al., 2017).

Figure 1: Paper flowchart.

This paper intends to present a project whose

objective is to assess the performance of Taekwondo

athletes in real time. The study performed during the

project aims to contribute with a new method of

identifying and quantifying the movements

performed by the taekwondo athlete during training

sessions using deep learning methodologies applied

to the data collected from the taekwondo athletes'

movements in real time.

This paper is organized into four chapters as

described by the flowchart presented in figure 1. The

second chapter presents the state of the art; in the third

chapter, the work and study performed on the tasks

that make up the main project are described; and in

the fourth chapter are presented the final remarks

along with the future work in the project

development.

2 STATE OF ART

Along the development of technology, several

devices have emerged that enable the monitoring and

analysis of movements performed by the human

body. Taking advantage of this, in Sports area has

been conducted a large number of researches aiming

to contribute to the improvement of athletes'

performance and help in the prevention of injuries.

For trainers and athletes, motion analysis is of

great importance when applied in the training because

it allows to provide technical skills improvement by

correcting the trainee’s body motion in order to

perform correct and most efficient movements in any

sports.

Regarding motion analysis some of the research

carried out aims to study the hands movements and

localization. As the case of the study that presents a

survey where they summarized the techniques used

for hand localization and gesture classification

(Suarez & Murphy, 2012).

Despite a growing increase in image resolution,

traditional video cameras are conditioned by the

luminosity and colours present in the environment

making it difficult to obtain the correct digital

analysis of the image. Depth cameras appear as the

suitable alternative for image collection when these

situations occur, collecting depth images, not

dependable on orientation or intensity of illumination,

or from the colour scheme of the environment. One

of the most used depth sensors in scientific research

regarding gesture classification and hand localization

is the Microsoft Kinect (Microsoft, 2020); mainly due

to its low acquisition value, portability, not

requirement of markers, ease to set up, and creation

of 3D images. It allows to develop affordable 3D

video motion systems that are used to human

movement kinematics analysis of body joints and

segments in several areas.

BIODEVICES 2021 - 14th International Conference on Biomedical Electronics and Devices

210

A Microsoft Kinect camera used to collect data

from whole human body movements was presented in

a study for human gesture recognition using data

mining classification methods in video streaming of

twenty body-joints positions of the human body

(Patsadu, Nukoolkit, & Watanapa, 2012). The study

recognized gesture patterns as stand, sit down, and lie

down. The classification methods adopted for

comparison were backpropagation neural networks,

decision trees, support vector machines and naive

Bayes. The results obtained allowed to conclude that

backpropagation neural networks show superior

performance compared to the other classification

methods, recognizing human gestures with 100%

accuracy. The average accuracy of all classification

methods was 93.72%, confirming the efficiency of

the Kinect camera when used in human body

recognition applications.

Also, with Microsoft Kinect, a study was carried

out where compared the Microsoft Kinect

displacement measures with the Peak Motus marker-

based system displacement measures. The results

allowed to conclude that the Microsoft Kinect, for

being a mark less system, was more favourable during

the setup, data collection and analysis stages as

compared to the Peak Motus (Zerpa, Lees, Patel,

Pryzsucha, & Patel, 2015).

A real-time evaluation approach that uses a

Microsoft Kinect and image processing techniques to

recognize and record the number of occurrences of a

movement in Taekwondo training environment is

presented by Pinto, et al (2017). The recognition of

the movements was obtained through the calculation

of the angles between human body joints and

comparing them with the correct values of each

movement previously saved in a database.

In (Cunha, Carvalho, & Soares, Development of a

Real-Time Evaluation System For Top Taekwondo

Athletes SPERTA, 2018) it is presented a system to

evaluate the performance of Taekwondo athletes

during training sessions. The 3D camera used is the

Orbbec Astra that comparing with Microsoft Kinect 2

is smaller, has less weight, with a higher maximum

reach distance. The system allows to save information

about the athletes and about the movements.

Regarding movements the data obtained are relative

to Cartesian coordinates in the real world of the

human body joints. Providing athlete hands and feet

joints movements data in Cartesian coordinates

numeric values, Cartesian coordinates line chart and

speed line chart, all in real-time.

Still further studies are needed, so that it can be

confirmed the reliability and validity of the Microsoft

Kinect for human movement kinematics analysis

(Polak, Kulasa, VencesBrito, Castro, & Fernandes,

2016). However, some studies verified and agreed

that marker less systems would be a major revolution

in the analysis of human motion making possible the

application of human motion capture studies

(Robertson, Caldwell, Hamill, Kamen, & Whittlesey,

2004) (Corazza, Mundermann, Gambaretto, Ferrigno,

& Andriacchi, 2010).

Other studies follow a different approach, where

movement data is collected using motion sensors used

by athletes. These wearable devices are valuable

instruments for the improvement of sports

performance. However, the existing systems are still

limited (Li, et al., 2016).

In (Camomilla, Bergamini, Fantozzi, & Vannozzi,

2018) the authors agreed that magneto-inertial

technology is a reliable tool to improve athlete’s

performance, the training specificity and to prevent

injuries. These sensors measurements can be used to

estimate temporal, dynamic and kinematic

parameters.

Smart sensors and sensor fusion allow to study the

impact suffered by the athlete. In (Mendes Jr, Vieira,

Pires, & Stevan Jr, 2016), the authors demonstrated

the use of smart sensors and sensor fusion in

biomedical applications and sports areas, promoting

a reflection about techniques and applications to

process physical variables associated with the human

body. The application can be used in areas related to

rehabilitation, the athlete’s performance

development, among others.

In (Amaro, et al., 2017), the authors agreed that

the impact signals combined with IMU may be a

reliable way of scoring, whilst heart rate

measurement enables monitoring of the athlete’s

physical state. The technique used consists in

integrating a “non-invasive” sensor system into

Taekwondo clothes. The impact is measured using

pressure sensors, thin film piezo resistive force and

accelerometers. The communication between the

sensor and the computer is based on Bluetooth and it

was discovered a limitation of bandwidth using this

transmission protocol.

3 PROJECT DEVELOPMENT

The project from which this study derives aims to

contribute with a technological solution that allows

the assessment of the performance of Taekwondo

athletes in real time during training sessions. In order

to achieve the objective, several tasks have to be

completed, each one contributing with elements

necessary for the development of the system.

Real-time Evaluation System for Top Taekwondo Athletes: Project Overview

211

Thus, the main outputs of system will be statistics,

biomechanics and motion analysis. The statistical

analysis, will be made with the results obtained

through the identification and quantification of the

movements performed by the athlete, allowing to

assess the evolution over time of the training sessions.

And the biomechanics and motion analysis, will

allow to calculate acceleration, velocity and the

applied force of the athlete’s movements.

In this chapter these tasks will be presented,

referring to their role in the functioning of the final

system.

3.1 Framework

The developed system is composed by a 3D camera

Orbbec Astra, a computer and by the software, as

presented in figure 2. The option for the 3D Camera

Orbbec Astra is due to the fact that it is smaller in size,

weighs less, does not require external power and has

a higher maximum range system, compared to

Microsoft Kinect 2, the most used depth sensor in

research (Cunha, Carvalho, & Soares, Development

of a Real-Time Evaluation System For Top

Taekwondo Athletes SPERTA, 2018).

Figure 2: Framework system architecture (Cunha,

Carvalho, & Soares, 2018).

The system allows to collect data on movements

performed by taekwondo athletes during training

sessions, calculating and presenting the values of

speed, acceleration and applied force of the athlete's

hand and feet in real time (Cunha, Carvalho, &

Soares, Development of a Real-Time Evaluation

System For Top Taekwondo Athletes SPERTA,

2018).

3.1.1 Software

The software of the framework was developed in C#

with Visual Studio 2017 IDE (Integrated

Development Environment) for build up the interface

and the main features, along with Structure Query

Language (SQL) to add the database where the data

is stored. In order to obtain athletes movements data,

the Nuitrack™ SDK was integrated. Nuitrack™ is a

3D skeleton tracking gesture recognition solution

middleware that allows to obtain the athletes joints

cartesian coordinates relative to the range of the 3D

camera Orbbec Astra.

Figure 3: Raw data from Left Ankle Joint.

The development of the software had as initial

objective to create a dataset with the data of the

movements that the athletes perform during the

practice of Taekwondo. The dataset consists of

several classes, each referring to a movement, in

which the cartesian coordinates of the athlete joints

for each class are stored. The values of each joint refer

to the X, Y, and Z coordinates in a tri-dimensional

environment.

Figure 4: Depth sensor, cartesian coordinates and speed

data output in real-time (Cunha, Carvalho, & Soares, 2018).

It is intended to integrate the software with the

functionality of identifying and quantifying the

movements performed by Taekwondo athletes during

training sessions. For that, motion analysis will be

performed according to skeleton-based action

recognition, through deep learning methodologies.

BIODEVICES 2021 - 14th International Conference on Biomedical Electronics and Devices

212

The main propose of the developed dataset is to

gather information about the taekwondo athletes’

movements aiming to use on training of deep learning

classification methods. The dataset obtained consists

of four classes (front leg bandal, rear leg bandal,

jirugui and front leg miro), in which each class

represents a movement performed by the taekwondo

athlete, with about 200 samples per class. The data

acquisition and treatment were performed to identify

and quantify the athlete techniques.

In order to select the deep learning method that

best fits the type of data, several deep learning were

studied and tested, such as, Convolutional Neural

Networks (CNN), Long Short-Term Memory

(LSTM), Graph Convolution Networks (GCN),

CNN+LSTM and a LSTM with embedded

Convolution (ConvLSTM), figure 5.

Figure 5: Deep learning methods testing system diagram.

During the tests, the classification validation

accuracy was 80% for the CNN model, 88% for the

LSTM model, 93% for the CNN+LSTM model, and

92% for the ConvLSTM model. The results obtained

allow to realize that convolutional layers models

achieve the best results. Both CNN+LSTM (figure 6)

and ConvLSTM, managed to get results above 90%

on accuracy validation, placing these models as the

ones that best fit the characteristics of the Taekwondo

athlete’s performance evaluation system.

Figure 6: CNN+LSTM Network Architecture.

3.2 Inertial Measurement Units (IMUs)

For the movements data acquisition, as presented

above, was used a system data extract the body joints

coordinates in a tri-dimensional environment.

Although this method allows data to be collected

efficiently, sometimes, due to a rotation of the athlete

or overlapping of a limb, there is occlusion. In order

to overcome these occlusions, the addition of motion

sensors, more specifically the inertial measurements

units, was foreseen. They will be positioned on the

extremities of the upper and lower limbs, hands and

feet. For this purpose, custom-made containers were

designed to accommodate the various hardware

components, which will be fixed to the athletes

through a velcro system.

As Taekwondo is a sport in which the athlete

performs fast and wide movements, with a greater

incidence of the lower limbs, the size and weight of

the motion sensors must be considered. The solution

developed should be the least intrusive as possible,

not interfering with the athlete's movements and with

adequate comfort for use during training sessions.

Along with the characteristics related to the

intrusiveness of motion sensors, the value and ease of

acquisition of the components were also considered.

Figure 7: Wemos D1 mini Wi-Fi board (a) and the GY 521

MPU 6050 (b) connection diagram.

Thus, for data processing and transmission it was

selected the Wemos D1 mini a Wi-Fi board based on

ESP-8266, with 11 digital input/output pins and 1

analogue input, as seen in figure 7 a) (LOLIN D1

mini, 2020).

The sensor chosen to obtain the acceleration and

gyroscope data was the GY 521 MPU 6050 (figure 7

b)), which is a three-axis gyroscope and acceleration

module, with standard communication I2C. The

acceleration range is between ±2 and ±16 g and the

gyroscope range is between +250 to +2000 ̊/s

(MPU6050 - Accelerometer and Gyroscope Module,

2020). The motion sensors system also includes a

battery shield (Battery Shield, 2020) which allows to

choose between powering the system through a

battery or through a USB charger, as well as enable

to charge the battery when the system is being

The system presented in figure 8 measures the

displacement and acceleration of the upper and lower

limbs, transmitting this data through a Wi-Fi

communication, stored in the computer belonging to

the system.

For this, a program was developed that allows the

reading of the IMUs and the transmission of data

through the UDP protocol.

Real-time Evaluation System for Top Taekwondo Athletes: Project Overview

213

Figure 8: Motion sensors system architecture diagram.

3.3 Mobile APP

Mobile devices are a constant presence in everyday

life, and some devices currently have processing

capacity comparable to computers. Therefore, they

are also used as a tool to perform various tasks.

Taking this into account, together with the

development of the framework presented above in

which the data processing fell on a portable computer

with Windows operating system, an app was

developed for Android and IOS devices.

The app is intended to be a friendly tool that can

be used by the trainer during training sessions. It will

allow to manually enter the movements performed by

the athlete, with the objective of providing a fast

feedback to the coach and athletes so that they can

analyze, correct and adapt the training method to

improve their performance.

The app makes it possible to enter athletes' data,

such as name, weight category, step and club, saving

this information in a database to add to training

sessions and consultations.

After filling in the training data, when selecting

"Start training", the menu displayed in figure 9

appears, where: in a) it is possible to select the

movements to be performed. A maximum of 10 types

of movements will be allowed per training session,

which can be leg and/or arm movements; in b) the

type of movement (arms or legs) can be selected; in

c) the duration of the training is presented; in d) the

buttons to start or end the training are available; in e)

four additional buttons are available for each

movement where the user (trainer) can specify the

type of movement: right front (1st column), right back

(2nd column), left front (3rd column), left back (4th

column) ); in f) there is a button available to indicate

whether the movement was successful.

Figure 9: App training session interface.

After the training is finished, a small summary of

the training is presented, indicating when the training

started and ended and its duration, the number and

type of movements performed during the session,

which movements were performed with or without

success.

4 FINAL REMARKS

The purpose of this paper is to present the project

under development, whose main objective is to

design and implement a friendly and low-cost system

for assessing the performance of Taekwondo athletes

in real time. The system should have the lowest level

of intrusion to athletes during the practice of

Taekwondo.

A framework for the acquisition of movement

data was developed, which allowed the creation of a

dataset with data on the movements of Taekwondo

BIODEVICES 2021 - 14th International Conference on Biomedical Electronics and Devices

214

athletes. With this dataset it was possible to conduct

a study on deep learning methodologies in order to

define which method has the best performance in

identifying athletes' movements.

With the intention of promoting the inclusion of

technological tools in Taekwondo training sessions

and taking advantage of the easy access to mobile

devices, a mobile App for Android and IOS was

developed. This App allows the trainees to add

information about the athletes and manually collect

data about the movements performed by the athlete in

a training session, saving them in the application's

database. Afterwards, the saved sessions can be

consulted, allowing to analyse the evolution of the

athlete's performance.

As future work, we intend to include the deep

learning model to identify the athlete's movements in

the developed framework and the integration of the

data acquisition system through motion sensors based

on IMUs in the framework.

Then, tests of the overall system will be

performed with athletes in a training environment, in

order to assess the impact of the system developed in

the practice of Taekwondo training and its

contribution to the assessment of the performance of

Taekwondo athletes.

ACKNOWLEDGEMENTS

This work has been supported by COMPETE: POCI-

01-0145-FEDER-007043 and by FCT – Fundação

para a Ciência e Tecnologia within the R&D Units

Project Scope: UIDB/00319/2020

. Pedro Cunha

thanks FCT for the PhD scholarship

SFRH/BD/121994/2016.

REFERENCES

Amaro, B., Antunes, J., Cunha, P., Soares, F., Carvalho, V.,

& Carvalho, H. (2017). Monitoring of Bioelectrical and

Biomechanical Signals in Taekwondo Training: First

Insights. Em European Congress on Computational

Methods in Applied Sciences and Engineering (pp. 417-

426). Springer.

Arastey, G. M. (1 de 12 de 2020). Computer Vision In

Sport. Obtido de SPORT PERFORMANCE

ANALYSIS:

https://www.sportperformanceanalysis.com/article/co

mputer-vision-in-sport

Baca, A., Dabnichki, P., Heller, M., & Kornfeind, P.

(2009). Ubiquitous computing in sports: A review and

analysis. Journal of Sports Sciences, 27, pp. 1335-1346.

Battery Shield. (28 de 11 de 2020). Obtido em 28 de 11 de

2020, de WEMOS:

https://www.wemos.cc/en/latest/d1_mini_shield/batter

y.html

Camomilla, V., Bergamini, E., Fantozzi, S., & Vannozzi,

G. (2018). Trends supporting the in-field use of

wearable inertial sensors for sport performance

evaluation: A systematic review. Sensors, 18(3), 873.

Corazza, S., Mundermann, L., Gambaretto, E., Ferrigno,

G., & Andriacchi, T. P. (2010). Markerless motion

capture through visual hull, articulated icp and subject

specific model generation. International journal of

computer vision, 87(1-2), 156-169.

Cunha, P., Carvalho, V., & Soares, F. (2018). Development

of a Real-Time Evaluation System For Top Taekwondo

Athletes SPERTA. SENSORDEVICES 2018, The Ninth

International Conference on Sensor Device

Technologies and Applications, pp. 140-145.

Cunha, P., Carvalho, V., & Soares, F. (2018). Real-Time

Data Movements Acquisition of Taekwondo Athletes:

First Insights. International Conference on Innovation,

Engineering and Entrepreneurship, pp. 251-258.

King, B. A., & Paulson, L. D. (2007). Motion capture

moves into new realms . Computer, 40(9), pp. 13-16.

Li, R. T., Kling, S. R., Salata, M. J., Cupp, S. A., Sheehan,

J., & Voos, J. E. (2016). Wearable performance devices

in sports medicine. Sports health, 8(1), 74-78.

LOLIN D1 mini. (28 de 11 de 2020). Obtido em 28 de 11 de

2020, de WEMOS:

https://www.wemos.cc/en/latest/d1/d1_mini.html

Mendes Jr, J. J., Vieira, M. E., Pires, M. B., & Stevan Jr, S.

L. (2016). Sensor Fusion and Smart Sensor in Sports

and Biomedical Applications. Sensors, 16(10), 1569.

Microsoft. (30 de 11 de 2020). Kinect para Windows.

Obtido de Microsoft:

https://developer.microsoft.com/pt-pt/windows/kinect/

MPU6050 - Accelerometer and Gyroscope Module. (28 de

11 de 2020). Obtido em 28 de 11 de 2020, de

Components101:

https://components101.com/sensors/mpu6050-module

Nadig, I. S., & Kumar, N. (2015). Measurement of velocity

and acceleration of human movement for analysis of

body dynamics. Measurement, 3(3).

Patsadu, O., Nukoolkit, C., & Watanapa, B. (2012). Human

gesture recognition using Kinect camera. (IEEE, Ed.)

ninth international conference on computer science and

software engineering (JCSSE), pp. 28-32.

Pinto, T., Faria, E., Cunha, P., Soares, F., Carvalho, V., &

Carvalho, H. (2017). Recording of occurrences through

image processing in Taekwondo training: First insights.

Lect. Notes Comput. Vis. Biomech, pp. 427-436.

Polak, E., Kulasa, J., VencesBrito, A., Castro, M. A., &

Fernandes, O. (2016). Motion analysis systems as

optimization training tools in combat sports and martial

arts. Revista de Artes Marciales Asiáticas, 10(2), 105-

123.

Pueo, B., & Jimenez-Olmedo, J. M. (2017). Application of

motion capture technology for sport performance

analysis. Retos, 32, pp. 241-247.

Real-time Evaluation System for Top Taekwondo Athletes: Project Overview

215

Robertson, D. G., Caldwell, G. E., Hamill, J., Kamen, G.,

& Whittlesey, S. (2004). Research methods in

biomechanics. USA: Human Kinetics.

Suarez, J., & Murphy, R. R. (2012). Hand gesture

recognition with depth images: A review. Em ROMAN:

The 21st IEEE International Symposium on Robot and

Human Interactive Communication (pp. 411-417).

IEEE.

Thomas, G., Gade, R., Moeslund, T. B., Carr, P., & Hilton,

A. (2017). Computer vision for sports: Current

applications and research topics . Computer Vision and

Image Understanding, 159, pp. 3-18.

Zerpa, C., Lees, C., Patel, P., Pryzsucha, E., & Patel, P.

(2015). The use of microsoft Kinect for human

movement analysis. International journal of sports

science, 5(4), 120-127.

BIODEVICES 2021 - 14th International Conference on Biomedical Electronics and Devices

216