Investigation on the Performance of Haar Cascade Classifier to

Classify Images Using OpenCV

Muhammad Bahit

, Nadia Putri Utami

, Heru Kartika Candra

, Hafizi Al Madhani

and Noprianto

Department of Computerized Accounting, Politeknik Negeri Banjarmasin, Banjarmasin, South Kalimantan, Indonesia

Department of Computer Science, Politeknik Negeri Malang, Malang, East Java, Indonesia

D020319035@akuntansipoliban.ac.id, noprianto@polinema.ac.id

Keywords: Face Detection, Haar Cascade Classifier and OpenCV.

Abstract: Face is an essential part of human identity that others can recognize directly. As technology develops, face is

used as a tool for human interaction with computers in various fields. However, face recognition has many

problems such as accuracy, a long process, and recognition errors due to the clothes worn. Therefore, various

methods have been developed to overcome facial recognition problems, including the haar cascade classifier

method. Therefore, this study aims to investigate the performance of the haar cascade classifier method in

classifying images using OpenCV. This study used six datasets with 300 images in each dataset to test the

accuracy in classyifing images. The results found that the performance accuracy of the haar cascade classifier

method in classifying images increased in each epoch, from 0.2211 in the 1st epoch to 0.7755 in the 10th

epoch. In addition, the validation accuracy also had an increase from 0.2492 in the 1st epoch to 0.7803 in the

10th epoch. Thus, this study suggests using the haar cascade classifier method in image classification for

detection face.

1 INTRODUCTION

As technology develops, the face is used as a tool for

human interaction with computers (Cuimei et al.,

2017). Face recognition has been a concern for four

decades in various fields and disciplines such as bank

card identification, access control, image search,

security monitoring, and surveillance systems

(Riyantoko et al., 2021). In addition, face recognition

can be used to identify human behavior in

communicating and interacting with other people

(Sharmila et al., 2019) (Yulianto et al., 2021).

The most widely used facial recognition

algorithm was Eigenface which was considered one

of the basic concepts of face recognition (Choi et al.,

2022). This algorithm applies statistical analysis with

the concept of Principal Component Analysis (PCA)

for face detection, but the Eigenface algorithm is very

sensitive to the shape and size of the face, so it is not

widely used anymore (Rahmad et al., 2020) (Kainz,

2019).

Paul Viola and Michael Jones introduced an

algorithm called Viola-Jones to detect faces in order

to process images with large resolutions (Javed

Mehedi Shamrat et al., 2022). However, this

algorithm produces a lot of errors in the face detection

process. Then (Thapliyal et al., 2022) proposed the

Histogram of Oriented Gradients (HOG) algorithm

for human face detection. This algorithm is more

accurate than the Viola-Jones algorithm, but the HOG

algorithm has a slow computational process on large-

resolution images (Priadana & Habibi, 2019)

(Malhotra et al., 2021).

Furthermore, the Local Binary Patterns (LBP)

algorithm was introduced (Isnanto et al., 2021) with

more accurate face recognition because it can detect

differences in face shape and skin color. Although

this algorithm achieves the highest success than the

previous algorithm, there is an error rate of more than

20% during the face recognition process (Phuc et al.,

2019). Previous studies used the Viola-Jones

algorithm and Histogram of Oriented Gradients

(HOG) for face detection while the Convolutional

Neural Network (NN) algorithm was used for face

recognition (Kainz, 2019). However, previous studies

resulted in errors in the face detection process with

slow computational processes on high-resolution

images (Ahmad et al., 2021).

Another study conducted an experiment on an

Open CV-based face recognition system with the Haar

Bahit, M., Utami, N., Candra, H., Madhani, H. and Noprianto, .

Investigation on the Performance of Haar Cascade Classiﬁer to Classify Images Using OpenCV.

DOI: 10.5220/0011765500003575

In Proceedings of the 5th International Conference on Applied Science and Technology on Engineering Science (iCAST-ES 2022), pages 313-316

ISBN: 978-989-758-619-4; ISSN: 2975-8246

313

Cascade Classifier algorithm for face detection and the

Local Binary Patterns Histograms (LBPH) algorithm

for face recognition. However, this study can only

recognize faces with a maximum distance of 15 CM

from the camera and with the requirement that no

objects cover the face (Mantoro et al., 2018) (Vinh &

Anh, 2020). A study by (Ahmad et al., 2021)

succeeded in detecting faces with a maximum distance

of 20 CM using the Viola-Jones algorithm for face

detection and the K-Nearest Neighbor (KNN)

algorithm for face recognition. On the other hand, this

study has a weakness in the detection process and face

recognition must be perpendicular to the camera.

Previous studies used various algorithms for face

detection and face recognition but did not pay

attention to the effect of light intensity, movement,

and computational processes during face recognition

on large image resolutions, and the distance between

the camera and the face is still too close. This study

aimed to investigate the performance of Haar Cascade

Classifier to classify images using OpenCV with a

distance of 1 meter.

2 METHOD

2.1 Subject

This study was conducted on 6 respondents consisting

of 3 men and 3 women with an average age of 21

years. All respondents are from the Computerized

Accounting Study Program, Accounting Department,

Politeknik Negeri Banjarmasin.

2.2 Tools and Materials

This study used several tools, both in the form of

hardware and software in order to run properly;

2.2.1 HP Pavillion Computer

The computer device used was HP Pavilion Gaming

15-dk1041TX with a

Intel® Core™ i7-10750H (2.6 -

5 GHz, 12 MB L3 cache, 6 cores) processor and 8 GB

DDR4-2933 SDRAM (1 x 8 GB). The operating

system used is Windows 11 Home Single Language

64-bit. Video game graphic uses NVIDIA®

GeForce® GTX 1650 with memory capacity of 4 GB

GDDR6 dedicated. The camera used is Logitech HD

WEBCAM C310 with HD 720pixels with max

resolution at 30fps.

2.2.2 Logitech Camera

The camera used was Logitech Webcam C310 HD

with image and video capture resolution up to 1280 x

720 pixels.

2.2.3 OpenCV

OpenCV is a library used to translate an image in the

detection and face recognition process with the haar

cascade classifier algorithm.

2.2.4 Python

Python is used to display respondent data on the face

recognition system. python is a high-level

programming language so it has several advantages,

such as simple syntax, object-oriented (OOP), and

many libraries that will simplify the program created.

Python also has one multifunctional array variable

namely 'NumPy' to make programming easier.

2.2.5 Tensorflow

Tensorflow is used as a deep learning library

algorithm in the face detection and recognition

process with the haar cascade classifier algorithm.

2.2.6 Keras

Keras is a deep learning interface library in the face

detection and recognition process with the haar

cascade classifier algorithm.

2.3 Procedures

Respondents were asked to face the camera at a

distance of 1 meter and freely move. On average, 300

pictures of each respondent were taken with a

duration of 60-90 seconds. Before the experiment,

each participant was given a letter of consent to

become a respondent in the study.

2.4 Datasets

This study used 6 datasets containing 250 160x160

pixel images of each respondent. The data were used

to investigate the performance of Haar Cascade

Classifier in classifying images.

iCAST-ES 2022 - International Conference on Applied Science and Technology on Engineering Science

314

Figure 1: Respondents’ faces.

3 RESULTS AND DISCUSSIONS

3.1 Training Loss and Accuracy

Figure 2: Training and validation loss.

Figure 3: Training and validation Accuracy.

Figures 2 and 3 above show the results of the data

train and data test with 10 iterations (epoch). Based

on the figure above, it can be seen that iteration

produces accuracy values and loss values for train

data and test data. The accuracy value is used to

determine the model success rate. While the loss

value indicates errors made by the network that must

be minimized. Figures 2 and 3 show that the haar

cascade classifier method does not experience

overfitting. Overfitting is a condition in which the

model fits too well with the training data and does not

have good generalizability, thus providing low

accuracy on new data (Yustiawati et al., 2018).

Overfitting checks can be performed by conducting a

validation process at each iteration of the deep

learning model training process (Indraswari et al.,

2021). The overfitting model has good performance

in classifying training data but has poor performance

in classifying new data, such as validation data.

Figure 2 shows the training and validation loss for

train data and test data for each iteration (epoch).

Based on Figure 3, the blue line shows the training

loss, while the orange line shows the validation loss.

Figure 3 shows that in the train data, the validity loss

value continues to decrease until the 10th epoch.

Figure 3 shows the accuracy of the test data generated

in each iteration (epoch). Based on the figure above,

it can be seen that the accuracy value for the data train

is increased up to the 10th epoch. For the train data,

at the end of the iteration, the accuracy value was

0.8357. While the validation accuracy in the last

epoch was 0.9148. The iteration results for each

epoch can be seen in table 1 below.

Table 1: The iteration result of each epoch.

Iteration Loss Accuracy

Validation

Loss

Validation

Accuracy

Epoch 1 1.0629 0.2211 1.7337 0.2492

Epoch 2 1.6364 0.3550 1.3813 0.4492

Epoch 3 1.3275 0.4807 1.1129 0.6426

Epoch 4 1.0889 0.5835 0.9574 0.7049

Epoch 5 0.9231 0.6504 0.8093 0.7803

Epoch 6 0.8026 0.7052 0.7246 0.8164

Epoch 7 0.6961 0.7755 0.6412 0.8492

Epoch 8 0.6051 0.8303 0.5514 0.8852

Epoch 9 0.5774 0.8256 0.5107 0.8951

Epoch 10 0.5240 0.8357 0.4624 0.9148

Table 1 shows that the 1st epoch of 1.0629

decreased to 0.5240 in the 10th epoch. The results

showed the decreasing error in face classification.

The accuracy for each epoch increased, where the 1st

epoch was 0.2211 and the 10th epoch was 0.835. The

increase in the accuracy value indicates an increase in

the face classification results.

In addition, Table 1 also shows the Validation

Loss for each epoch, where the 1st epoch of 1.7337

decreased to 0.4624 in the 10th epoch. Thus, this

shows the decreasing Validation Loss in performing

face classification. Based on the Validation

Accuracy, the value of each epoch increased, where

the 1st epoch was 0.2492 to 0.9148 in the 10th epoch.

With the increase in the Validation Accuracy, the haar

cascade classifier in classifying images has good

Investigation on the Performance of Haar Cascade Classiﬁer to Classify Images Using OpenCV

315

accuracy. This can be seen from the increase in

Validation Accuracy.

4 CONCLUSION

This study aimed to investigate the performance of

Haar Cascade Classifier to classify images using

OpenCV. The performance accuracy of haar cascade

classifier in classifying increased the accuracy of each

epoch where the 1st epoch was 0.2211 and 0.7755 in

the 10th epoch. Meanwhile, the validation accuracy

increased where the 1st epoch was 0.2492 and 0.7803

in the 10th epoch. Thus, the haar cascade classifier

has good accuracy in classifying images. This can be

seen from the decreasing training and validation loss

from 1st epoch to 10th epoch.

REFERENCES

Ahmad, A. H., Saon, S., Mahamad, A. K., Darujati, C.,

Mudjanarko, S. W., Nugroho, S. M. S., & Hariadi, M.

(2021). Real time face recognition of video surveillance

system using haar cascade classifier. Indonesian

Journal of Electrical Engineering and Computer

Science, 21(3), 1389–1399.

Choi, C.-H., Kim, J., Hyun, J., Kim, Y., & Moon, B. (2022).

Face Detection Using Haar Cascade Classifiers Based

on Vertical Component Calibration. Human-Centric

Computing and Information Sciences, 12(0), 134–150.

Cuimei, L., Zhiliang, Q., Nan, J., & Jianhua, W. (2017).

Human face detection algorithm via Haar cascade

classifier combined with three additional classifiers.

2017 13th IEEE International Conference on Electronic

Measurement & Instruments (ICEMI), 483–487.

Indraswari, R., Rokhana, R., & Herulambang, W. (2021).

Melanoma image classification based on MobileNetV2

network. Procedia Computer Science, 197, 198–207.

Isnanto, R. R., Rochim, A., Eridani, D., & Cahyono, G.

(2021). Multi-Object Face Recognition Using Local

Binary Pattern Histogram and Haar Cascade Classifier

on Low-Resolution Images. International Journal of

Engineering and Technology Innovation, 11(1), 45–58.

Javed Mehedi Shamrat, F. M., Majumder, A., Antu, P. R.,

Barmon, S. K., Nowrin, I., & Ranjan, R. (2022). Human

Face Recognition Applying Haar Cascade Classifier. In

G. Ranganathan, R. Bestak, R. Palanisamy, & Á. Rocha

(Eds.), Pervasive Computing and Social Networking

(Vol. 317, pp. 143–157).

Kainz, O. (2019). Students’ Attendance Monitoring

through the Face Recognition. 2019 17th International

Conference on Emerging ELearning Technologies and

Applications (ICETA), 342–347.

Malhotra, S., Aggarwal, V., Mangal, H., Nagrath, P., &

Jain, R. (2021). Comparison between attendance

system implemented through haar cascade classifier

and face recognition library. IOP Conference Series:

Materials Science and Engineering, 1022(1), 012045.

Mantoro, T., Ayu, M. A., & Suhendi. (2018). Multi-Faces

Recognition Process Using Haar Cascades and

Eigenface Methods. International Conference on

Multimedia Computing and Systems -Proceedings,

2018-May, 1–5.

Phuc, L. T. H., Jeon, H. J., Truong, N. T. N., & Hak, J. J.

(2019). Applying the haar-cascade algorithm for

detecting safety equipment in safety management

systems for multipleworking environments. Electronics

(Switzerland), 8(10).

Priadana, A., & Habibi, M. (2019). Face Detection using

Haar Cascades to Filter Selfie Face Image on Instagram.

2019 International Conference of Artificial Intelligence

and Information Technology (ICAIIT), 6–9.

Rahmad, C., Asmara, R. A., Putra, D. R. H., Dharma, I.,

Darmono, H., & Muhiqqin, I. (2020). Comparison of

Viola-Jones Haar Cascade Classifier and Histogram of

Oriented Gradients (HOG) for face detection. IOP

Conference Series: Materials Science and Engineering,

732(1).

Riyantoko, P. A., Sugiarto, & Hindrayani, K. M. (2021).

Facial Emotion Detection Using Haar-Cascade

Classifier and Convolutional Neural Networks. Journal

of Physics: Conference Series, 1844(1), 012004.

Sharmila, Sharma, R., Kumar, D., Puranik, V., & Gautham,

K. (2019). Performance Analysis of Human Face

Recognition Techniques. Proceedings - 2019 4th

International Conference on Internet of Things: Smart

Innovation and Usages, IoT-SIU 2019, 1–4.

Thapliyal, A., Verma, O. P., & Kumar, A. (2022). Mask

Covered Face Recognition Using Haar Cascade

Classifier and Fuzzy Logic. International Journal of

Emerging Technology and Advanced Engineering,

12(8), 152–166.

Vinh, T. Q., & Anh, N. T. N. (2020). Real-Time Face Mask

Detector Using YOLOv3 Algorithm and Haar Cascade

Classifier. 2020 International Conference on Advanced

Computing and Applications (ACOMP), 146–149.

Yulianto, R. F., Irwansyah, A., & Tamami, N. (2021).

Smart Parking System Based on Haar Cascade

Classifier and SIFT Method. 2021 International

Electronics Symposium (IES), 505–510.

Yustiawati, R., Husni, N. L., Evelina, E., Rasyad, S., Lutfi,

I., Silvia, A., Alfarizal, N., & Rialita, A. (2018).

Analyzing Of Different Features Using Haar Cascade

Classifier. 2018 International Conference on Electrical

Engineering and Computer Science (ICECOS), 129–

134.

iCAST-ES 2022 - International Conference on Applied Science and Technology on Engineering Science

316