Modular Facial Expression Recognition using Double Channel DNNs

Sujata, Suman K. Mitra

2021

Abstract

Recognizing human expressions is an important task for machines to understand emotional changes in humans. However, the the accurate features that are closely linked to changes in expression are difficult to extract due to the influence of individual differences and variations in emotional intensity. The modular approach presented here imitates the human being’s ability to identify a person with a limited facial part. In this article, we demonstrate experimentally that certain parts of the face, such as the eyes, nose, lips and forehead, contribute more to the recognition of expressions. A combination of two deep neural networks is also proposed to extract the characteristics of the facial images provided. Two preprocessing approaches are implemented, Histogram Equalization (to handle illumination) and Data Augmentation (increasing number of facial images), to restrict the regions used for recognition of the facial expression. Two-channel architecture used for implementation, one channel accepts input as a grayscale face image, processed by VGG16_ft (fine-tuned VGG16), and another channel accepts input as histograms face image. the second order gradients (HSOG), processed from the proposed CNN model and extracts the characteristics accordingly. Then concatenate the characteristics from the two channels. The final recognition result is calculated using the SVM and KNN classifiers. Experimental results indicate that the proposed algorithm is able to recognize six basic facial expressions (happiness, sadness, anger, disgust, fear and surprise) with great precision. Fine tuning is effective for FER activities with a well-trained model if there are not enough samples to collected.

Paper Citation

in Harvard Style

Sujata. and Mitra S. (2021). Modular Facial Expression Recognition using Double Channel DNNs. In Proceedings of the 16th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2021) - Volume 4: VISAPP; ISBN 978-989-758-488-6, SciTePress, pages 277-284. DOI: 10.5220/0010201002770284

in Bibtex Style

@conference{visapp21,
author={Sujata and Suman K. Mitra},
title={Modular Facial Expression Recognition using Double Channel DNNs},
booktitle={Proceedings of the 16th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2021) - Volume 4: VISAPP},
year={2021},
pages={277-284},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0010201002770284},
isbn={978-989-758-488-6},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 16th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2021) - Volume 4: VISAPP
TI - Modular Facial Expression Recognition using Double Channel DNNs
SN - 978-989-758-488-6
AU - Sujata.
AU - Mitra S.
PY - 2021
SP - 277
EP - 284
DO - 10.5220/0010201002770284
PB - SciTePress