Data Acquisition from the Integration of Kinect Quaternions and

Myo Armband EMG Sensors to Aid Equinus Foot Treatment

Francisco M. A. Araújo

, N. M. Fonseca Ferreira

2,3 b

, Salviano F. S. P. Soares

4,5 c

António Valente

and Gilson L. S. Junior

Federal Institute of Education, Science, and Technology of Piauí, Teresina, Brazil

Institute of Engineering of Coimbra Polytechnic of Coimbra, Coimbra, Portugal

INESC TEC-INESC Technology and Science and Knowledge Engineering and Decision-Support Research Center

(GECAD) of the Institute of Engineering, Polytechnic Institute of Porto, Portugal

IEETA – UA, Aveiro, Portugal

University of Trás-os-Montes and Alto Douro, Vila Real, Portugal

INESC TEC-INESC Technology and Science and University of Trás-os-Montes and Alto Douro, Vila Real, Portugal

Keywords: Kinect, Quaternions, Myo, EMG, Equinus Foot.

Abstract: This paper shows the advantage of using different sensors such the Microsoft Kinect and Myo Armband to

acquire movement description of the plantarflexion and dorsiflexion of the foot with the help of the

quaternions and the EMG Myo sensor. For the integration of these devices, it was chosen Python to develop

the algorithm and create an interface to aid the signal acquisition. This integration, enabling an accurate

motion description as well as a scale of EMG signal, allow the possibility of quantifying the treatment of the

people with equinus foot.

1 INTRODUCTION

Equinus foot is a condition that is characterized by the

limitation of the dorsiflexion movement of the foot,

in normal circumstances, the foot has the ROM

(Range of motion) of 36º to the action of

plantarflexion and 7º to dorsiflexion, according to

(NASA, 2014). Because of the lack of necessary

flexibility thus leaving the foot in an extended

position. This condition may be either congenital or

acquired, caused by tensions in the Achilles tendon or

the calf muscles (Soleus, lateral gastrocnemius and

medial gastrocnemius), and according to (Schmid et

al., 2016), equinus foot is the most common issue

regarding the human gait motility affecting patients

with hemiplegic cerebral palsy, causing deviation of

the pelvis, hence creating inconsistency in the gait.

The Equinus foot implies the lack of mobility of

the patient. It is of extreme importance, to improve

https://orcid.org/0000-0001-8928-0077

https://orcid.org/0000-0002-2204-6339

https://orcid.org/0000-0001-5862-5706

https://orcid.org/0000-0002-5798-1298

https://orcid.org/0000-0002-5177-2065

methods of treatment, considering gains in movement

is linked to reduction of the incidence of adjacent

problems, such as infections and osteoporosis,

improvement of cardiac functions and even reducing

the dependency of an accompanying person, besides

positive psychological impacts in the process of

rehabilitation (Costa et al., 2005) and (Vital et al.,

2003; 2017 and 2018).

Figure 1: Movements of the foot - Rage of motion in the

Dorsiflexion 7º and Plantarflexion 36º.

This paper proposes to create a way of quantifying

the status of the patient as well as their improvement

along the treatment.

Araújo, F., Ferreira, N., Soares, S., Valente, A. and S. Junior, G.

Data Acquisition from the Integration of Kinect Quaternions and Myo Armband EMG Sensors to Aid Equinus Foot Treatment.

DOI: 10.5220/0007565902350240

In Proceedings of the 12th International Joint Conference on Biomedical Engineering Systems and Technologies (BIOSTEC 2019), pages 235-240

ISBN: 978-989-758-353-7

 2019 by SCITEPRESS – Science and Technology Publications, Lda. All r ights reserved

235

1.1 Rotation and Translation Methods

There are many methods of describing the translation

and rotation of a rigid body, among those, we shall

briefly explain some of them as well as the reason we

chose quaternions over the others. According to

(Kenwright, 2012), the most used method of

representations is the Euler-Angles, used to

representing the orientation and translation through

three angels possessing orthogonal axes (x, y, and z)

we can achieve twelve combinations of sets of angles,

and to work with angles we have to convert them into

matrices. And while they are of easy comprehension,

they suffer from the disadvantage of the angles

changing by up to 2π radians, another problem is the

Euler coordinates are not precise when interpolating

near the Gimbal’s Lock, which occurs when two sets

of axes are aligned, turning them, in that moment in

the same axis.

In the case of matrices, even though it is a more

straightforward subject that students have early

contact with, we have as the main problem the time

to processes the amount of data, and the difficulty in

visualizing in which axis, the rotation, and the angle.

1.2 Quaternions

Quaternions are hypercomplex numbers belonging to

the numerical set



, isomorphic to numbers in the

numerical set





. Such as complex number

quaternions are defined by possessing both real and

imaginary part, wherein real coefficients multiplied

by components form the imaginary part: ̂,̂,



. They

are represented by the equation:









̂





̂







(1)

Another representation is 





, where S

represents the real part and V represents the

imaginary part, when 



equals to zero we will call it

pure quaternion, since it has just imaginary parts.

The operations of quaternions are responsible for

rotation and translation. To rotate a pure quaternion

to an also pure quaternion  that have to use

operator ||. And we also have to know the concept of

a conjugate of a quaternion (Adorno, 2017):

Conjugate of a quaternion is defined as its real

part minus the imaginary part.





Re







Im

(2)

So, the rotation will be represented by:

T







(3)







, 







̂











1



(4)

















∗









̂



















̂



(5)

Figure 2: Quaternion rotation, rotation the quaternion v to

the quaternion w.

2 MATERIALS AND METHODS

2.1 The Myo Armband

The Myo armband is a wearables gesture and motion

control (Żorski and Pałys, 2016) that enables the user

to control device such as computer, tablets and

phones and it composed by nice axis-IMU with three-

axis gyroscope, three-axis accelerometer, three-axis

magnetometer, and eight EMG sensors, those sensor

allow the Myo Armband to track gestures of the user

through electrical signal.

Figure 3: Myo armband - EMG sensors numbered from 1

to 8.

BIODEVICES 2019 - 12th International Conference on Biomedical Electronics and Devices

236

2.1.1 EMG Signals

In this experiment, the Myo is used in the leg, and we

are using the EMG sensors to record data from the

muscles responsible for the plantar and dorsiflexion

movement.

The EMG sensors measure electromyographic

signals, and this method consists of a non-evasive

technique that measures the electrical signals

produced by the muscles during their contraction and

relaxation, this technique was chosen because it

performs the measurement directly in the muscles to

be studied, both voluntary and will be recognized.

Figure 4: Optimal placement of the EMG sensors – 36-

medial gastrocnemius, 37-lateral gastrocnemius, 38-Soleus

and 39-Tibialis anterior.

Regarding the placement of the Myo Armband,

according to (Florimond, 2010) the best position to

place the sensors are in the muscles tibialis anterior,

soleus, medial gastrocnemius and lateral

gastrocnemius (Figure 4).

To control the test using Myo, we created an

interface to simplify the acquisition of data stipulation

a time of 1 minute, 30 seconds or 5 seconds, in the

analysis done in this paper we used the time of 5

seconds. (Micael et al., 2013) and (Dias et al., 2014).

2.2 Microsoft Kinect

The Microsoft Kinect was firstly launched in 2011,

and it brought a new way of playing video games, the

user is now able to play without having to hold any

peripherical device. According to (Pagliari et al.,

2014), Kinect is a device that enables motion capture,

among one of the advantages that Kinect brigs are

official libraries and SDKs (Software Development

Kit) which allow researchers to contribute to this

technology. In this paper, we are using the SDK 2.0

published at 10/21/2014.

Figure 5: Representation of the interface Myo-Kinect.

The motion analysis nowadays is a field of interest

among a variety of areas, and for medical application

as well. The Microsoft Kinect, according to

(Kalkbrenner et al., 2014) can be used in the

evaluation of patient activity, posture as well as

gesture recognition. Including 3D depth sensor and an

optical camera, can be used to analyse angles and

movement, enabling the acquisition of depth and even

of quaternions in any given joint, the sequences of

joints captured by the Kinect are shown in Figure 5.

Initially, we researched ways to extract data from

Kinect and found some applications developed in

Python, C # and C ++ languages. It was decided to

Data Acquisition from the Integration of Kinect Quaternions and Myo Armband EMG Sensors to Aid Equinus Foot Treatment

237

use python using the library PyKinect2 due to its easy

manipulation and quick configuration.

Figure 6: Sequences of joints captured by the Kinect.

After choosing the language and the application

for Kinect, it was studied its operation of to extract

data of position and rotation of each joint of the body

read by Kinect along with the results of Myo (Figure

6).

3 RESULTS AND DISCUSSIONS

The capture of the quaternions is obtained by the

motion capture of the Microsoft Kinect, while the

subject, starting of the total plantarflexion (Figure 6)

going to the total dorsiflexion., it is important to

emphasize that the foot must be positioned as

perpendicular as possible to the Kinect, the

quaternions response is a 3D graphic and the unit used

by the library we are using is in meters. And aiming

integration with Myo armband, it is essential to set a

correct position of the EMG sensor, sensor

enumerated from 1 to 8.

Figure 7: Motion capture of Kinect.

Figure 8: Representation of data – 3: soleus, 1 and 2: lateral

gastrocnemius and 4: Tibialis anterior.

The EMG sensor 4 is representing data from the

muscle tibialis anterior, sensor 5 serves the muscle

soleus, sensor 3 and 4 are representing the medial

gastrocnemius while the sensor 1 and 2 the lateral

gastrocnemius. And for the results of the Kinect, we

BIODEVICES 2019 - 12th International Conference on Biomedical Electronics and Devices

238

find this- graphic (Figure 10) representing the

movement of the right foot joint, with the frequency

of 30 Hz and 30fps and full HD camera. And we have

the graphic of the motion in the axes X, Y, and Z

showing in Figure 8.

Figure 9: Representation of data –7 and 8 medial

gastrocnemius.

Since quaternions can be generated both using

kinect or using third part sensor such as IMU sensors,

Żorski and Pałys, (2016) prove that the quaternions

generate by the Kinect and IMU sensors in the Myo

armband are similar, however they notice limitation

in the IMU sensor resulting in a range of 70 – 80% of

recognition of the gestures.

The IMU sensor were not used in this work

because the gesture recognition it is built just for

arms, however the EMG signal is possible to read

even if it is not assigned to any gesture, and since the

equinus foot has the characteristic of decreasing the

intensity of the electromyographic signal, the

improvement of the patient can be quantified by the

increase of the read signal.

Figure 10: The trajectory of the right foot using the motion

capture features of the Microsoft Kinect.

4 CONCLUSION

This paper proposed to integrate the motion capture

features of Microsoft Kinect and the EMG sensors of

the Myo Armband, and even though the Myo was not

designed to be used in the leg and nonetheless this

device is not primarily used to capture motion of the

leg, the EMG sensor proved useful to bring a

satisfactory accuracy, but in one hand there is the

quality of the signal of the Myo EMG in the other

hand there is the difficulty to apply any unit to its

meseuramnets, so even though they show what occurs

during the movement and it is possible to compere the

results among them it was not possible to assign any

unit.

Regarding the treatment of equinus foot with the

integration of this two devices, thanks to the

smoothness of the results of quaternions and the scale

that we can acquire from the EMG sensor we can both

describe the movement with accuracy and quantify

the improvement of the patient.

Data Acquisition from the Integration of Kinect Quaternions and Myo Armband EMG Sensors to Aid Equinus Foot Treatment

239

In order to improve the data acquisition and

aplication this process can be enhanced by using

precise EMG sensors that allow unit readings and

studying a more precise way of image caputure.

REFERENCES

Adorno, B.V. (2017), “Robot Kinematic Modeling and

Control Based on Dual Quaternion Algebra — Part I :

Fundamentals . Robot Kinematic Modeling and Control

Based on Dual Quaternion Algebra Part I :

Fundamentals Bruno Vilhena Adorno”.

Costa, N., Brown, M., Hutchins, S., Caldwell, D.G. and

Caldwellsalfordacuk, E.D.G. (2005), “Design of

Human-Friendly Powered Lower Limb Rehabilitation

Orthosis”, TDU COE-UK EPSRC Workshop on Human

Adaptive Mechatronics (HAM), No. September.

Jessica Vital, Micael S. Couceiro, Gonçalo Dias, N. M.

Fonseca Ferreira, (2015). “Tecnologias para a Análise

do Movimento Humano”, Livro de Atas do 6º

Congresso Nacional de Biomecânica, CNB2015, 6-7

Fev, Monte Real, Leiria, Portugal, ISBN 978-972-

8793-74-6.

Jessica P. M. Vital, Diego R. Faria, Gonçalo Dias, Micael

S. Couceiro, Fernanda Coutinho, N. M. Fonseca

Ferreira, (2017). “Combining discriminative

spatiotemporal features for daily life activity

recognition using wearable motion sensing suit”,

Pattern Analysis and Applications, vol 20, issue 4, pp

1179-1194. DOI: 10.1007/s10044-016-0558-7

Jessica P. M. Vital, N. M. Fonseca Ferreira, Salviano F.S.P.

Soares, António Valente, João Manuel Pereira Barroso,

(2018). “FatoXtract a new technology in

rehabilitation”, Proceedings of TISHW 2018, 2nd

International Conference on Technology and

Innovation in Sports, Health and Wellbeing, 20-22

June, Thessaloniki, Greece.

Micael S Couceiro, David Portugal, Nuno Gonçalves, Rui

Rocha, J. M. A Luz, Carlos M Figueiredo, Gonçalo

Dias, N.M. Fonseca Ferreira, (2013). "A methodology

for detection and estimation in the analysis of golf

putting", Pattern Analysis and Applications 16, 3: pp.

459 – 474, Springer London ISSN 1433-7541, 2013.

DOI: 10.1007/s10044-012-0276_8

G. Dias, JMA Luz, MS Couceiro, CM Figueiredo, NM

Fonseca Ferreira, P Iglésias, R Mendes, M Castro, O

Fernandes, (2014). “A Proposal for Detection and

Estimation of Golf Putting”, in Fonseca Ferreira N.,

Tenreiro Machado J. (eds) Mathematical Methods in

Engineering, pp. 293-303, ISBN 978-94-007-7182-6,

Springer, Dordrecht. DOI: 10.1007/978-94-007-7183-

3_27

Florimond, V. (2010), “Basics of Surface

Electromyography Applied to Physical Rehabilitation

and”, Thought Technology Ltd, Vol. 1 No. March, pp.

1–50.

Kalkbrenner, C., Hacker, S., Algorri, M. and Blechschmidt-

trapp, R. (2014), “Motion Capturing with Inertial

Measurement Units and Kinect - Tracking of Limb

Movement using Optical and Orientation Information”,

Proceedings of the International Conference on

Biomedical Electronics and Devices, pp. 120–126.

Kenwright, B. (2012), “A Beginners Guide to Dual-

Quaternions: What They Are, How They Work, and

How to Use Them for 3D Character Hierarchies”, The

20th International Conference on Computer Graphics,

Visualization and Computer Vision, pp. 1–13.

NASA. (2014), National Aeronautics and Space

Administration - HUMAN INTEGRATION DESIGN

HANDBOOK.

Pagliari, D., Menna, F., Roncella, R., Remondino, F. and

Pinto, L. (2014), “Kinect fusion improvement using

Depth Camera calibration”, International Archives of

the Photogrammetry, Remote Sensing and Spatial

Information Sciences - ISPRS Archives, Vol. 40 No. 5,

pp. 479–485.

Schmid, S., Romkes, J., Taylor, W.R., Lorenzetti, S. and

Brunner, R. (2016), “Orthotic correction of lower limb

function during gait does not immediately influence

spinal kinematics in spastic hemiplegic cerebral palsy”,

Gait and Posture, Elsevier B.V., Vol. 49, pp. 457–462.

Żorski, W. and Pałys, T. (2016), “Human motion capture

using Kinect and third party sensors”, No. 1, pp. 17–38.

BIODEVICES 2019 - 12th International Conference on Biomedical Electronics and Devices

240