Gesture Recognizing using Naïve Bayes Algorithm and a Leap

Motion

Daniel Sutopo Pamungkas and Sonny Hendratmo

Department of Electrical, Politeknik Negeri Batam, Jl Ahmad Yani, Batam, Indonesia

Keywords: Gesture Recognizes, Leap Motion, Game, Real-time.

Abstract: This article presents the methods to recognize poses of the hand. The Naïve Bayes algorithm is used to

identify four gestures. These gestures are applying in a customized game. The leap motion as a gesture

sensor is utilized in this study. Five subjects are performing the attempt after the training their gestures data.

Roll left, roll right, and palm face down are the gestures to be trained and tested in this study. The

experiment results show that this system enabled to read the poses of the user's hands. To demonstrate the

abilty of the proposed method, a customized game is created. Moreover, the users enable to play a game

with no difficulty.

1 INTRODUCTION

Human Machine Interface (HMI) is a part of the

machine to make it easy to use the equipment. To

control the equipment more convenient for the user,

the gestures of the human are used. Hand and fingers

are the popular human body part which used to

manage the machine using their poses.

Human gestures, especially human hand poses,

can be divided into two groups, static pose and

dynamic movement, respectively. A static pose is a

pose that is in idle condition and almost with a

constant shape. On the other hand, the active

activities are poses with the pattern motion

(Sonkusare, 2015). Comparing both gestures in the

computational term, the dynamic movements are

more complicated than the static ones (Huang,

2015). However, these dynamic gestures have many

applications for HMI.

Robotics (Andrean, 2019), health (Andreoni,

2007), and also playing a game are several examples

of HMI applications using dynamic gestures. To

interface the human gestures with the machine, there

are several instruments to be used, for instance,

using Electromyograph (EMG) or muscle signal

(Andrean, 2019), brain signal, or

Electroencephalography (EEG) (Andreoni, 2007),

and camera (Liu, 2019).

In this article a leap motion which is used for the

instrument to detect the movement of human poses.

Moreover, the Naïve Bayes algorithms are applied to

recognize the movement of the subjects. Also, a

costumed game is establishing to examine the effect

the proposed system.

To deliver a complete discussion, this paper is

organized as follows: the next section purposes of

providing information about the previous researches

to identify human hand gestures. Section III presents

the proposed system. Then proceed with next,

Section IV, which offers the tests on the proposed

method by examining the effectiveness of the

proposed system using a real game. In final part

provides the concluding remarks and the future work

of this investigation.

2 BACKGROUNDS

The hand poses identification has become the

research topic which be applied in the robotic hand

and prostatic hand. Some of the researchers are

studying recognizing the poses of the hand in

dynamic movement. Dynamic hand gesture

prediction from the movement of the hand before

completed. This system is introduced by Chen et al.

(Chen, 2015). They enabled to identify of 36

patterns which comprise 26 letters and ten numbers.

For this task, the SVM algorithm and a leap motion

sensor are used. The others application is to identify

the movement of the user's hand to translate to the

text line. The Hidden Markov Model (HMM) is used

by this system to identify the gestures of the hand.

664

Pamungkas, D. and Hendratmo, S.

Gesture Recognizing using Naïve Bayes Algorithm and a Leap Motion.

DOI: 10.5220/0010950700003260

In Proceedings of the 4th International Conference on Applied Science and Technology on Engineering Science (iCAST-ES 2021), pages 664-667

ISBN: 978-989-758-615-6; ISSN: 2975-8246

 2023 by SCITEPRESS – Science and Technology Publications, Lda. Under CC license (CC BY-NC-ND 4.0)

Moreover, a leap motion is also used to sense the

poses of the hand (Agarwal, 2015).

Several researchers have done studies about the

identification of Sign Language. The American Sign

Language recognition using HMM is done by Fok et

al. (Fok, 2015). In comparison, the Arabic Sign

Languages is studied by Mohandes et al. (Mohandes,

2014). They used a Multi-layer perceptron Neural

Network (MLP). The Euclidean distance and Cosine

similarity are used for Indian Sign Language

(Mapari, 2015).

The robot hand is controlled using the user's

palm, which has been researched by Bassily et al.

(Bassily, 2014). This system uses leap motion to

obtain the position of the palm of the user.

For this study, the Naïve Bayes algorithm is used

for identifying the gestures of the hand. Naive Bayes

is a simple probabilistic classifier that calculates a

set of probabilities by adding up the frequencies and

combinations of values from a given dataset. To

explain the Naive Bayes method, please note that the

classification process requires a number of

instructions to determine what class is suitable for

the analyzed sample. Therefore, the Naive Bayes

method above is adjusted as follows:







𝑥

𝑐









)

𝑃𝑐

│

𝑋  𝑃𝑥



𝑐



x .. 𝑃



𝑥



𝑐



x 𝑃𝑐

)

Where:

P(c|x) is the probability of posterior class c with

attribute x

P(x|c) is the likelihood which the probability of

the predictor given class

P(x) is the probability of class x

P(c) is the probability of class a

X = (x

…., x

)

In this classification, all features/values are

assumed to be independent of each other in each

class. Because of this, even if a value is dependent

on one another, this classification will consider all

data to be independent thus giving a probability

effect. One of the advantages of Naive Bayes is that

only the mean and standard deviation values of the

variables are needed to estimate the parameters of

the classification:

μ

𝑛

 𝑥









)

σ 





∑

𝑥



 𝜇













)

Where:

µ is the mean of the attribute in a class

σ is the variance of the feature in a class

The decision-making rule for classification is to

choose one more possibility based on the results of

the density of probability values, such as the Gauss

Density equation.

𝑓



𝑥





√

2𝜋𝜎

𝑒



 







)

3 METHODS

To examine the system's effectiveness in identifying

the hand's gesture, a system comprised of a hand

tracking sensor and connected to a computer. This

computer is to process the recognize algorithm also

to run the game to test the results. The block

diagram of the system is shown in figure 1.

The hand poses used by this study are the

gestures. The gestures are the hand that rolls to the

left, roll to the right, and hand that face downward.

Figure 2 shows the motions of the hand that were

used in this experiment.

Figure 3 shows the flow chart of the system. In

the Naïve Bayes classification algorithm, the process

is divided into two phases, namely the training and

test phase. In the training phase, the algorithm is

given the parameters of the movements. For this

experiment, yaws, pitch, and roll of the palm of the

hand for each pose are supplied to the algorithm.

After the training phase is finish, the algorithm

will provide the values that used in the test phase. In

this phase, each gesture will determine the success

rate.

Figure 1: Block Diagram of the proposed system.

Figure 2: (a) Roll to the left (b) face downward (c) roll to

the right.

Gesture Recognizing using Naïve Bayes Algorithm and a Leap Motion

665

Figure 3: Flowchart of the system.

4 RESULTS

A male 24years old subject is done the experiments.

He never used the leap motion before. He uses the

dominant hand to do the experiments. Figure 4

shows the right hand of the subject maneuverings his

hand above the leap motion sensor. He performed

the training phase. For each pose, he has done it five

times. To examine the algorithm, the subject has

performed the tests. He was moving his hand five

times for each gesture. Table 1 shows the results of

the test in the confusion matrix. From this table, the

percentage of the successful classification is about

93%.

Figure 4: Subject hand above the sensor.

Table 1: The Confusion matrix.

 Predicted

Actual

 Left Middle Right

Left 5 0 0

middle 0 5 0

Right 0 1 4

To demonstrate the effectiveness and the reaction

of the system, a costumed game is designed. In this

game, the classification system is included. This

game is to control a cylinder to avoid the obstacle.

This cylinder is controlled by the gesture of the

hand. To move to the left, the user has to roll their

hand to the left. Also, to go to the right, the user has

to roll their hand to the right. The subject never

plays the game before. The subject feels that the

system is responsive. The system enables to

identification of the movement of the hand fast. He

is able to control the movement of the cylinder

easily.

(a)

(b)

Figure 5: The screen shoot of the game (a) success to

avoid the obstacle (b) bump the obstacle.

iCAST-ES 2021 - International Conference on Applied Science and Technology on Engineering Science

666

4 CONCLUSIONS

This article shows how the Naïve Bayes theorem to

be applied to identify the hand gestures. The poses

of the hand is sense using a stereo infrared camera.

The experiments show that the system enables to

recognize the movement of user hand. The

successful rate of identify the poses is around 93%.

Moreover, the proposed system is responsive to read

the hand poses. It shows in the application in the

game. In the future this system will be used to be

applied in the advanced games. This system will be

combining with the tactile actuators, to make user

more immersive with the game.

ACKNOWLEDGEMENTS

This research was supported by Kementrian

Pendidikan, Kebudayaan, Riset dan Kebudayaan

Indonesia.

REFERENCES

Agarwal, C., Dogra, D., Saini, R., & Roy,P. (2015).

Segmentation and recognition of text written in 3D

using Leap motion interface. 539-543.

10.1109/ACPR.2015.7486561.

Andrean, D., Pamungkas, D., & Risandriya, S.K. (2019).

Controlling Robot Hand Using FFT as Input to the NN

Algorithm.

Andreoni, G., Parini, S., Maggi, L., Piccini, L., Panfili, G.,

& Torricelli, A. (2007). Human Machine Interface for

Healthcare and Rehabilitation. Advanced

Computational Intelligence Paradigms in Healthcare -

Bassily, D., Georgoulas, C., Guettler, J., Linner, T., &

Bock, T. (2014). Intuitive and Adaptive Robotic Arm

Manipulation using the Leap Motion Controller. ISR

2014.

Chen, Y., Ding, Z., Chen, Y., & Wu, X. (2015). Rapid

recognition of dynamic hand gestures using leap

motion. 2015 IEEE International Conference on

Information and Automation, 1419-1424.

Fok, K., Ganganath, N., Cheng, C., & Tse, C., (2015). A

Real-Time ASL Recognition System Using Leap

Motion Sensors 2015 International Conference on

Cyber-Enabled Distributed Computing and

Knowledge Discovery (CyberC), Xi'an, China, 2015

pp. 411-414.doi: 10.1109/CyberC.2015.81

Huang, J., Zhou, W., Li, H., & Li, W. (2015). Sign

Language Recognition using 3D convolutional neural

networks. 2015 IEEE International Conference on

Multimedia and Expo (ICME), 1-6.

Liu, F., Zeng, W., Yuan, C., Wang, Q., & Wang, Y.

(2019). Kinect-based hand gesture recognition using

trajectory information, hand motion dynamics and

neural networks. Artificial Intelligence Review, 52,

563-583.

Mapari, R.B., & Kharat, G. (2015). Real time human pose

recognition using leap motion sensor. 2015 IEEE

International Conference on Research in

Computational Intelligence and Communication

Networks (ICRCICN), 323-328.

Mohandes, M., Aliyu, S., & Deriche, M. (2014). Arabic

sign language recognition using the leap motion

controller. 2014 IEEE 23rd International Symposium

on Industrial Electronics (ISIE), 960-965.

Sonkusare, J. S., Chopade, N. B., Sor, R., and Tade, S. L.,

(2015). A Review on Hand Gesture Recognition

System, 2015 International Conference on Computing

Communication Control and Automation, 2015, pp.

790-794, doi: 10.1109/ICCUBEA.2015.158.

Gesture Recognizing using Naïve Bayes Algorithm and a Leap Motion

667