Four Gesture Recognization of a Robotic Hand using EMG

Maryam Arshad

, Nimra Iftikhar

and Noman Naseer

Department of Electrical and Computer Engineering, Air University, E9, Islamabad, Pakistan

Department of Mechatronics Engineering and BioMedical, Air University, Islamabad, Pakistan

Keywords: EMG, Indirect BCI, Myo Armband.

Abstract: Electromyography (EMG) measures muscle response when nerves are stimulated. The objective of this

research paper is to present a work on control of a robotic hand using Electromyography. CAD model was

selected using various open sources. The structure is printed with poly laic acid (PLA) material with the help

of a 3D printer. EMG signals were acquired by wearing the eight channel Myo Armband, placed on the

forearm muscles of 10 subjects. Then, these signals were filtered to remove noise. Different features are

applied on noise free acquired signal and KNN is used for classification. From the KNN classifier, we

achieved 98.9% accuracy.

Gestures exhibited were Victory, Thumbs Up, Open Hand and Grasp. The

classified signals are used to control the robotic hand.

1 INTRODUCTION

A physical disability is a condition that affects a

person's mobility or physical capacity. Therefore we

uses prosthesis which is an artificial device,

developed to replace the function of a lost limb.

Prosthesis are classified as exoskeleton and

endoskeleton. Exoskeleton prosthesis gain its

structural strength from outer laminated strength

whereas, endoskeleton prosthesis gain its structural

integrity from the inner endoskeleton.

There are two major types of exoskeleton: Upper-

extremity prostheses and Lower-extremity

prostheses. Upper-extremity prostheses further have

following types transradial amputation, trans-

humeral amputation, wrist dis-articulation, elbow dis-

articulation and shoulder dis-articulation. Whereas,

lower-extremity prosthesis includes prostheses for

transfemoral amputation, knee disarticulation, hip

disarticulation, ankle disarticulation, transtibial

amputation and partial foot amputation.

Depending on the use of external power they are

divided into Active and Passive techniques (Windrich

et al, 2016). An active prosthesis powered devices

externally and consist of sensors in contact with the

skin, which then pick up the signals from the arm and

sequentially operate the actuators, which in order

controls the movement(Windrich et al, 2016).

Different techniques are used to developed control

for brain controlled interface (BCI) (Mattia, 2016) for

example Surface Electromyography (Anil and

Sreeletha, 2019), Electroencephalography (EEG),

Functional Magnetic Resonance Imaging (FMRI),

(Bright et al, 2016), Force myography (FMG) (Cho et

al, 2016) and Targeted Muscle Reinnervation (TMR)

(Cheesborough et al, 2015). From which sEMG,

EEG, FMG and FMRI are non-invasive techniques

whereas, TMR are invasive techniques.

Electromyography (EMG) measures muscle

response when nerves are stimulated (Anis, 2019).

Electromyography (EMG) is a symptomatic program

used to survey the wellbeing of muscles and the nerve

cells that control them. These nerve cells are called

engine neurons ( Shi, 2018). A brief description of the

different EMG strategies signal separation and

comparison of different strategies by analyzing the

EMG signals, in relation to their execution was

proposed (Raez, 2016).

EMG is performed by an instrument called an

electromyograph, which recognizes electrical signals

generated by muscles. Electromyography (EMG) can

be assessed utilizing conductive parts or anodes on

the outside of the skin, or it tends to be evaluated

utilizing an obtrusive technique (coordinating the

sensor into the muscle) (M. Ali, 2020). Surface EMG

is the most common techniques (Del Vecchio et al,

2017).

EMG is a non-invasive method of indirect BCI

which gradually grows in number of applications,

such as EEG (Mattia, 2016) (Gandhi, 2010).

Arshad, M., Iftikhar, N. and Naseer, N.

Four Gesture Recognization of a Robotic Hand using EMG.

DOI: 10.5220/0010289100800087

In Proceedings of the International Conference on Health Informatics, Medical, Biological Engineering, and Pharmaceutical (HIMBEP 2020), pages 80-87

ISBN: 978-989-758-500-5

Including biomedical, a prosthesis, a combination of

a human machine. In any case, there is noise EMG

symptoms are important barriers to overcoming them

achieves the best performance of any based EMG

application known as Human Machine Identifier

(HMI)( Zimenko, 2013).

A robotic hand is an electro-mechanical system. It

is composed of different parts. The main parts are the

electrical components and mechanical structures that

allow movement. In many such configurations, the

kinematics of the robot are the same as those of a

human hand, and the joints of each finger can be

controlled independently (Naseer, 2018).

Virtual prototyping technology is used to create

designs for 3D printed Myo robotic hands. CAD

models were selected using a variety of open source.

It can be easily accessed on the Internet. This hand

uses a small motor to control the movement of the

fingers of the hand. Use a 3D printer to print small

parts of the hand, and finally assemble all the parts

into a robot. The design is tightly based on the

function of the tendon, which is how the hand works.

By using servo motors to help open and close the

clamps, threads are used to manipulate the structure.

Print structures with polyacrylic material (PLA) using

a 3D printer.

Our robot-based manipulator is controlled by the

movement of the hand. Its working principle is EMG

sensor based. Sensor record hand movements and

then transmit the information to the encoder, which

prepares the encoder for receiver-side transmission.

This information is received via Bluetooth and then

transmitted to the Arduino. The micro-controller

makes various decisions based on the information

received. These decisions are passed to the motor

driver, which triggers the motor in different

configurations, causing the robotic hand to move the

Myo Robotic Hand to a certain direction. Depending

on the application, the robotic hand can be designed

to perform all required tasks such as welding,

gripping, rotating, etc.

The task has numerous utilizations in the

territories of wellbeing, apply autonomy and bio-

robots, just as an individual deadened with a couple

of hands. This structure can be utilized to distinguish

high radiation objects. This plan can be utilized for

prostheses with a high number of degrees of

opportunity. With some modifications, it can be used

in heavy industry to collect and position various

objects with real-time signals. The objects that can be

treated can be, for example, toxic chemicals,

radioactive materials. It can also be used on assembly

lines in the automotive industry

This research has several applications in different

field such as Robotics and Bio-medical Industry.

Such as, to help people with wrist disarticulation so

that they become independent and return to normal

life.

2 METHODOLOGY

A methodology was evolved in which research is

divided into different part or stages.

Fig .1 shows the experimental setup of this

research. CAD model was selected using various

open sources. Then the structure is printed with poly

Figure 1: Experimental setup.

Four Gesture Recognization of a Robotic Hand using EMG

laic acid (PLA) material with the help of a 3D

printer. EMG signals were acquired by wearing the

eight channel Myo Armband, placed on the forearm

muscles of 10 subjects. Different features are applied

on noise free acquired signal and KNN is used for

classification. From the KNN classifier, we achieved

98.9% accuracy.

2.1 3D Printing of Robotic Hand

Rather than structuring a CAD model which isn’t our

project goal. We looked through CAD models of

hand on different open sources. After all the

examination and study we ﬁnished a streamlined plan

which comprise of four moveable ﬁngers, a

moveable and contradicting thumb, improved

gripping design, simple and reliable.

The following point must be considered while

choosing the cad model:

 A cover over all the electrical components to

protect them from environment.

 Close resemblance with natural human hand.

 Entire weigh of hand including electronics

not more than 600g

 Fingers actuation using threads attached

through motors.

Fig. 2 shows the 3d print/design of the robotic hand.

Figure 2: 3D hand design

For 3D printing we used Poly Lactic Acid which

is a thermoplastic and is derived from sugar. It is

carbon free material. PLA when heated above 180° C

become mold-able and upon cooling hard. It is used

with 3D Printer because it is reusable, light, has

strength than produces products with high quality.

Another material that can be used is Acrylonitrile-

Butadiene-Styrene (ABS). It is made from monomers

Acrylonitrile, 1,3-Butadiene and Styrene. It is strong

and durable but it need a hot printing bed above 200C.

It also produces poisonous gases.

After receiving 3d printed robotic hand, it was

assembled. Electronic circuits, controller and motors

are placed inside the

3D Printed Myoelectric Hand. 5 Servo Motors

and ﬁshing line are used to actuate the ﬁngers. Device

is attached to the patients disabled with upper limb.

As our project is robotic hand therefore no need for

exact dimension.

The components/hardware used for regulating the

robotic hand are as follow: Actuator, battery,

microcontroller and wires. Actuator is a device due to

which a machine or any other device can operate.

They are characterize on the basis of their rotation and

the power they use. The actuator we are using in our

project is servo motors. Servo Motor is DC engine

with a mechanism known as feed-back mechanism.

Figure 3: Servo Motor Flow Diagram

This motor run on electricity with the help of a

battery and spin at high RPM. Servo Motor (MG995).

Fig.3 shows Servo Motor Flow Diagram.The reason

for selecting MG995 is that they are providing us the

required Torque as well as they are small in size and

HIMBEP 2020 - International Conference on Health Informatics, Medical, Biological Engineering, and Pharmaceutical

inexpensive in comparison to other DC Motor

available in the market.

RPM stands for rotation per minute but put out

very low torque (a rotational effect used to work).

Servo motors have different types.

Their main feature is to control the position of

shaft which is inside it. Servo DC Motor are of small

size, relatively low in price and provide torque from

9.4kg/cm to about 11kg/cm. They are widely used in

robotic applications. Servo Motor attract current

relative to the mechanical burden applied and has

capacity to turn to explicit precise position, which

depend on the model utilized. Fig.4 shows servo

motor.

Figure 4: Servo motor

In this project 4 buck converters (LM2596) are

used. One with each servo motor. The LM2596 is a

series of buck converter basically regulator which

provide us with the functions of a step down (buck)

controller. They are able to drive 3A load easily that

is enough for the project. Some other specification of

this regulator is given below:

Input Voltage (min-max): 4.5V-40V

Input Voltage (min-max): 3.3V-37V

Arduino is an open-source and open-authorized

programming (IDE) which make it easy to form the

code and to move it on to the Arduino Board.

Utilizing Arduino MATLAB library, Arduino board

goes about as the fundamental correspondence load

up between the constant development of client’s

ﬁnger and the mechanical hand giving every actuator

the ideal PWM. An Arduino as a microcontroller was

utilized to process the information gained from the

sensor of Myo Armband. Arduino sheets depend on

equipment and programming. Arduino Uno is used.

After considering all the requirements, Lithium-

ion polymer battery (LiPO) Battery 5000 mAh is

used. Table 1 shows the battery calculation, on the

basis of which, battery is selected.

Table 1: Battery calculation

Battery Calculation

Component

Quanti

Curren

Volta

Power Total

Power

Servo

Mg995

4 0.45A*

6V 2.7W 2.7x4=10.8

Micro

Controller

1 0.05A

7V 0.35W 0.35x1=0.35

PSU

Circuitry

1 1A

10V 10W 10x1=10W

Miscellaneo

1 0.5A

5V 2.5W 2.5x1=2.5W

Servo

Sg90

1 0.36A*

5V 1.8W 1.8x1=1.8W

Total Power = 10.8+0.35+10+2.5+1.8

= 31.65 W (Power)

Assuming operation for 1hr = 31.65Wh (Energy)

Battery Capacity = 2851.35mAh (Wh x1000/V)

Battery Voltage = 11.1 V

Advantages of using lipo battery is as follow

 High Discharge Current

 Light Weight

 More Charge Cycle

 11.1 V

 5000mAh

Step by step procedure to assemble the hand:

Step1: All fingers except thumb are assembled

from fingertip, joint and knuckle. String are used to

joint these parts. Thumb is assembled from thumb

joint, thumb tip and joint.

Step 2: Four Servo motors are placed in the

Forearm region and tightened with a screw provided

with the servo motor. Threads are connected to the

servo on the horns. The infant finger and the ring

finger are connected to a similar servo, as they

provide the same functionality.

Four Gesture Recognization of a Robotic Hand using EMG

Step 3: Cut ten 20 inches fishing line. Two

fishing line per finger are used. Crimp them by

feeding the fishing line from one side. Feed the

fishing line from the finger tips till the fishing line

exists from the palm.

Step 4: Now the fishing line are attached to the

corresponding servo motor. The fishing line should

be tensioned. To check the fishing line is tension’s

move. The motor counter clockwise with hand and

the finger will move inward. By rotating it clockwise

the finger should open.

Step 5: Now wear Myo Armband and then

connect Arduino with motors and place the forearm

cover on forearm body.

EMG sensor is placed on the forearm muscle

using Myo Armband. EMG is powered by a separate

battery providing 18V. Robotic hand was also tested

by using arduino.

As we are done with the assembling of Myo

Robotic hand, after that we integrated Myo Armband

with the arduino and perform the following four

gestures of victory, thumbsup, open and close using

IDE (open licensed software). Figure 5 shows the

gestures performed by a robotic hand.

Figure 5: Gestures performed by robotic hand

2.2 Data Acquisition

For signal acquisition Myo Armband was utilized. It

is a light weighted armband with a heap of just 93g

and expandable between 7.5–13inches. It contains 8

Medical Grade Stainless Steel EMG sensors,

fundamentally delicate, 9 turn Internal Measurement

Unit (IMU), and there is whirligig, accelerometer and

magnetometer all of 3-center point. It involves ARM

Cortex M4 Processor.

It interacts with computer by methods of

Bluetooth Smart Wireless Technology. Fig.6 shows

Myo Armband sensor.

Figure 6: Myo Armband

A subject wore it on the forearm that continuously

read the muscle data and sent it via the in-built

Bluetooth to the laptop present right next to the

subject in form of a vector. Figure 7 shows the

position of sensor.

Figure 7: Location of electrodes on muscles (frontal view.)

The laptop’s Bluetooth received the incoming

data and passed it to the MATLAB.

The experimental setup decided for this project is

as follows:

• Number of classes (actions) = 4

• Number of subjects = 10(4 male and 6 female)

• Number of male subjects = 4

HIMBEP 2020 - International Conference on Health Informatics, Medical, Biological Engineering, and Pharmaceutical

• Number of female subjects = 6

• Age group = 20-25 years

• Number of activities per subject per trial= 10

• Total time for each trial = 40s

• All subjects are healthy.

2.3 Methods of Detecting Waves

Myo Armband is an advance band that read the

electrical activity of muscles and movement of arm

and signals are ﬁltered by the Myo Armband on

sampling frequency of 200Hz, in order to detect wave

two methods are as follow:

• Detecting wave by Peak.

Detecting wave by Peak is one of those best

solution in which the required patch of the signal

can be separated from the whole signal and can be

utilize in the work. This method used the data of

the indices of peaks by separating then-point to

the left of each index and same n-point to the right

of each index.

• Detecting wave by Slope.

Wave detection can also be found by a method in

which the required signal can be separated

through the whole signal. In this technique the

separation of the point begins when slope starts to

increase till the time when it start decreasing.

2.3.1 Features Extraction

Signal provided by using Myo Armband is a filtered

signal, Savitzky Golay filter was applied to smoothen

the signal (Christov, 2018).

The features were extracted from filtered EMG

signal. This features extraction help to reduce the data

in beneficial way.

In order to get higher end result of classiﬁcation,

features vector must be cautiously selected. We

extracted 4 features for each electrodes making a total

of 32 features (4 features * 8 electrodes). The best

results were found to be Standard Deviation (STD),

Waveform Length (WL), Root Mean Square (RMS)

and Mean Absolute Value (MAV) (Hong, Khan and

Hong, 2018). Fig.8 shows unfiltered EMG signal

whereas Fig.9 shows filter EMG signal.

Figure 8: Un-filtered EMG Signal

Figure 9: Filtered EMG Signal

2.4 Classification

Classiﬁcation is the main part of the project and it is

used to classify the signal as it is known as pattern-

recognition in simple language.

We skilled three classiﬁers on extracted features,

the classiﬁers have been Linear Discriminant

Analysis (LDA), K-Nearest Neighbors (KNN),

Support Vector Machine (SVM). The classifiers such

as LDA, SVM and KNN were trained separately on

each of the above feature. Maximum accuracy was

achieved using Root Mean Square as a feature on

KNN.

2.5 Interfacing

Next step of this project is to build the interface. At

first Myo-Armband and laptop was interface,

followed by laptop to robotic hand. Myo Armband can

be also interfaced with any other devices through

Four Gesture Recognization of a Robotic Hand using EMG

Myo-script. To write a script for Myo-armband and to

control any device, myo-script was used. Myo-scripts

coding language is lua, which is commonly used for

coding a lot of games or software. Notepad ++ for lua

is used because Notepad++ supports different

programming languages. It is also a source code for

free.

Myo Armband was interfaced with the laptop,

next step was to again interface the laptop with the

robotic hand (L. F. Sanchez, 2017)

3 EXPERIMENTATION

Using the essential examination approaches for

grouping we utilized three classifier which are as

follow:

• Linear Discriminant Analysis (LDA)

• K-Nearest Neighbors (KNN)

• Support Vector Machine (SVM)

One of the non-parametric method of classification

is K-Nearest Neighbors (KNN) works by using the

fact of ‘feature similarity’. The data will be assigned

in that category whose feature are more likely similar

with the incoming data points (Altın and Er, 2016).

98.9% accuracy was achieved by using KNN.

Another technique for arrangement Support

Vector Machine (SVM) which is a watch over

machine calculation by placing every information

thing in a form of a points in n-dimensional space, n

is known as numbers of features you have. After that

a hyperplane was form between the data which

actually

separates the classes and the new data is

divided into classes on the basis of side gap (Alkan

and Günay, 2012). With the each feature value being

the value of a particular coordinate, giving us the

exactness of 94.7 Percent the element of Mean

Absolute value (MAV). Fig 9 presents the Percentage

accuracy versus features on three different classifiers.

The basic research technique is LDA. It works or

identify different classes by calculating the

probability using the Bayes Theorem (Alam and

Arefin, 2018). 94.5% accuracy was achieved by using

LDA from the feature of Mean Absolute value

(MAV).

4 CONCLUSIONS

The hardware including the mechanical and electrical

parts was completed within the designated timeline.

Final application of LDA, SVM and KNN algorithms

with maximum accuracy are completed.

A classifier model of LDA, SVM and KNN was

trained using 400 movements (100 for each class). As

the resulting data points after feature extraction was

already well separated, the accuracy achieved was

(accuracy of all classifier)

Although final accuracy will be calculated by

taking account some more observation. The classifier

was trained using 4 feature of data

• RMS (accuracy)

• WL (accuracy)

•STD (accuracy)

•Mav (accuracy)

Figure 10: Percentage accuracy versus features on three different classifiers

HIMBEP 2020 - International Conference on Health Informatics, Medical, Biological Engineering, and Pharmaceutical

Real time classification was checked by performing

the gesture on real time and seeing if the predicted

response is same as the performing one. Testing of the

entire system was done step by step.

This project was started with an aim help the

deprived and disabled people with upper limb

disability. We applied our engineering knowledge

and developed a Myo-robotic hand for this noble

cause. The work done so far in Pakistan for the

rehabilitation cause is not such as should have been.

This motivated us to work on Myo-robotic hand.

REFERENCES

Windrich, M. et al. (2016) ‘Active lower limb prosthetics:

A systematic review of design issues and solutions’,

BioMedical Engineering Online. BioMed Central,

15(3), pp. 5–19. doi: 10.1186/s12938-016- 0284-9.

D. Mattia, L. Astolfi, J. Toppi, “Interfacing brain and

computer in neuro-rehabilitation,” 2016 4th

International Winter Conference on Brain-Computer

Interface (BCI) 22-24 Feb. 2016.

Anil, N. and Sreeletha, S. H. (2019) ‘EMG Based Gesture

Recognition Using Machine Learning’, Proceedings of

the 2nd International Conference on Intelligent

Computing and Control Systems, ICICCS 2018. IEEE,

(Iciccs), pp. 1560–1564. doi:

10.1109/ICCONS.2018.8662987.

Del Vecchio, A. et al. (2017) ‘Associations between motor

unit action potential parameters and surface EMG

features’, Journal of Applied Physiology, 123(4), pp.

835–843. doi: 10.1152/japplphysiol.00482.2017.

Bright, D. et al. (2016) ‘EEG-based brain controlled

prosthetic arm’, Conference on Advances in Signal

Processing, CASP 2016, pp. 479–483. doi:

10.1109/CASP.2016.7746219.

L. F. Sanchez, H. Abaunza, and P. Castillo, “Safe

navigation control for a quadcopter using user’s arm

commands,” International Conference on Unmanned

Aircraft Systems (ICUAS) June 13-16, 2017, Miami,

FL, USA, 2017

Cho, E. et al. (2016) ‘Force myography to control robotic

upper extremity prostheses: A feasibility study’,

Frontiers in Bioengineering and Biotechnology,

4(MAR), pp. 1–12. doi: 10.3389/fbioe.2016.00018.

Cheesborough, J. E. et al. (2015) ‘Targeted muscle

reinnervation and advanced prosthetic arms’, Seminars

in Plastic Surgery, 29(1), pp. 62–72. doi: 10.1055/s-

0035-1544166.

M. B. I. Raez, M. S. Hussain and F. M. Yasin, “Techniques

of EMG signal analysis: detection, processing,

classification and applications,” Biological procedures

Online, 2016.

K. Zimenko, A. Margun, and A. Kremlev EMG, “Real-

Time Classification for Robotics and HMI,” 18th

International Conference on Methods & Models in

Automation & Robotics (MMAR) 2013.

Choksawatdikorn, shutterstock, human hand muscles of

education. Large.5168x3448 pixels and

43.8x29.2cm.300DPI.JPEG

Christov, I., Raikova, R. and Angelova, S. (2018)

‘Separation of electrocardiographic from

electromyographic signals using dynamic filtration’,

Medical Engineering and Physics. Elsevier Ltd, 57, pp.

1–10. doi: 10.1016/j.medengphy.2018.04.007.

Hong, K. S., Khan, M. J. and Hong, M. J. (2018) ‘Feature

Extraction and Classification Methods for Hybrid

fNIRS-EEG Brain-Computer Interfaces’, Frontiers in

Human Neuroscience, 12(June), pp. 1– 25. doi:

10.3389/fnhum.2018.00246.

Altın, C. and Er, O. (2016) ‘Comparison of Different Time

and Frequency Domain Feature Extraction Methods on

Elbow Gesture’s EMG’, European Journal of

Interdisciplinary Studies, 5(1), p. 35. doi:

10.26417/ejis.v5i1.p35-44.

M. Ali, A. Riaz, W. U. Usmani and N. Naseer, "EMG Based

Control of a Quadcopter," 2020 3rd International

Conference on Mechanical, Electronics, Computer,

and Industrial Technology (MECnIT), Medan,

Indonesia, 2020, pp. 250-254, doi:

10.1109/MECnIT48290.2020.9166603.

Alkan, A. and Günay, M. (2012) ‘Identification of EMG

signals using discriminant analysis and SVM

classifier’, Expert Systems with Applications. Elsevier

Ltd, 39(1), pp. 44–47. doi:

10.1016/j.eswa.2011.06.043.

Alam, M. S. and Arefin, A. S. (2018) ‘Real-Time

Classification of Multi-Channel Forearm EMG to

Recognize Hand Movements using Effective Feature

Combination and LDA Classifier’, Bangladesh Journal

of Medical Physics, 10(1), pp. 25–39. doi:

10.3329/bjmp.v10i1.39148.

D. Mattia, L. Astolfi, J. Toppi, “Interfacing brain and

computer in neuro-rehabilitation,” 2016 4th

International Winter Conference on Brain-Computer

Interface (BCI) 22-24 Feb. 2016.

J. Shi, Z. Dai, “Research on Gesture Recognition Method

Based on EMG Signal and Design of Rehabilitation

Training System,” IEEE 3rd Advanced Information

Technology, Electronic and Automation Control

Conference (IAEAC) 12-14 Oct. 2018.

N. Naseer, F. Ali, S. Ahmed, S. Iftikhar, R. A. Khan and H.

Nazeer, "EMG Based Control of Individual Fingers of

Robotic Hand," 2018 International Conference on

Sustainable Information Engineering and Technology

(SIET), Malang, Indonesia, 2018, pp. 6-9, doi:

10.1109/SIET.2018.8693177.

T. Gandhi, A. Jena, A. B. Pal, Novel approach for BCI,

2010 First International Conference on Integrated

Intelligent Computing, 5- 7 Aug. 2010.

Anis, A.; Irshad, M.; Hamza, S.; Naseer, N.; Nazeer, H. and

Andrian, . (2019). EMG based Control of Transtibial

Prosthesis.In Proceedings of the International

Conference on Health Informatics and Medical

Application Technology - Volume 1: ICHIMAT, ISBN

978-989-758-460-2, pages 74-81. DOI:

10.5220/0009464200740081

Four Gesture Recognization of a Robotic Hand using EMG