Facial Expression Recognition Improvement through an Appearance Features Combination

Taoufik Ben Abdallah, Radhouane Guermazi, Mohamed Hammami


This paper suggests an approach to automatic facial expression recognition for images of frontal faces. Two methods of appearance features extraction is combined: Local Binary Pattern (LBP) on the whole face region and Eigenfaces on the eyes-eyebrows and/or on the mouth regions. Support Vector Machines (SVM), K Nearest Neighbors (KNN) and MultiLayer Perceptron (MLP) are applied separately as learning technique to generate classifiers for facial expression recognition. Furthermore, we conduct to the many empirical studies to fix the optimal parameters of the approach. We use three baseline databases to validate our approach in which we record interesting results compared to the related works regardless of using faces under controlled and uncontrolled environment.


  1. Cao, N., Ton-That, A., and Choi, H. (2016). An effective facial expression recognition approach for intelligent game systems. International Journal of Computational Vision and Robotics, 6(3):223-234.
  2. Chakrabartia, D. and Duttab, D. (2013). Facial expression recognition using Eigenspaces. In CIMTA'13: International Conference on Computational Intelligence, Modeling Techniques and Applications, volume 10, pages 755-761. ELSEVIER.
  3. Chao, W., Ding, J., and Liu, J. (2015). Facial expression recognition based on improved Local Binary Pattern and class-regularized locality preserving projection. Journal of Signal Processing, 2:552-561.
  4. Chen, L., Zhoua, C., and Shenb, L. (2012). Facial expression recognition based on SVM in E-learning. In CSEDU'12: International Conference on Future Computer Supported Education, volume 2, pages 781-787.
  5. Deng, H., Jin, L., Zhen, L., and Huang, J. (2005). A new facial expression recognition method based on Local Gabor Filter Bank and PCA plus LDA. International Journal of Information Technology, 11(11):86-96.
  6. Duda, R., Hart, P., and Stork, D. (2000). Pattern classification. Library of Congress Cataloging-in-Publication Data.
  7. Ekman, P. (1972). Universals and cultural differences in facial expressions of emotion. University of Nebraska Press Lincoln, 19.
  8. Happy, S. L. (2015). Automatic Facial Expression Recognition using Features of Salient Facial Patches. In IEEE Transactions on Affective Computing, pages 511-518. IEEE.
  9. Huang, X. (2014). Methods for facial expression recognition with applications in challenging situations. Phd thesis, University of OULU, INFOTECH OULU.
  10. Kanade, T., Cohn, J., and Tian, Y. (2000). Comprehensive database for facial expression analysis. In Fourth IEEE International Conference on Automatic Face and Gesture Recognition, pages 46-53. IEEE.
  11. Khan, R., Meyer, A., Konik, H., and Bouakaz, S. (2013). Framework for reliable, real-time facial expression recognition for low resolution images. Journal of Pattern Recognition Letters, 34:1159-1168.
  12. Lin, D. (2006). Facial expression classification using PCA and Hierarchical Radial Basis Function Network. Journal of Information Science and Engineering, 22:1033-1046.
  13. Malsburg, C. (1961). Frank Rosenblatt: principles of Neurodynamics: perceptrons and the theory of brain mechanisms. Springer Berlin Heidelberg.
  14. Mliki, H., Hammami, M., and Ben-Abdallah, H. (2013). Mutual information-based facial expression recognition. In ICMV'13: ixth International Conference on Machine Vision. Society of Photo-Optical Instrumentation Engineers (SPIE).
  15. Ojala, T., Pietikainen, M., and Harwood, D. (1996). A comparative study of texture measures with classification based on feature distributions. Journal of Pattern Recognition, 29(1):51-59.
  16. Ojala, T., Pietikainen, M., and Maenpaa, T. (2002). Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Transactions on Pattern Analysis and Machine Intelligence, 24(7):971-987.
  17. Ouyang, Y., Sang, N., and Huang, R. (2015). Accurate and robust facial expressions recognition by fusing multiple sparse representation based classifiers. Journal of Neurocomputing, 149:71-78.
  18. Priya, G. and Banu, R. (2012). Person independent facial expression detection using MBWM and multiclass SVM. International Journal of Computer Applications, 55(17):52-58.
  19. Saha, A. and Wu, Q. (2010). Facial expression recognition using curvelet based Local Binary Patterns. In ICASSP'10: International Conference on Acoustics Speech and Signal Processing, pages 2470-2473. IEEE.
  20. Sánchez, A., Ruiz, J., Montemayor, A. M. A., Hernández, J., and Pantrigo, J. (2011). Differential optical flow applied to automatic facial expression recognition. Journal of Neurocomputing, 74(8):1272-1282.
  21. Saragih, J., Lucey, S., and Cohn, J. (2009). Face alignment through Subspace Constrained Mean-Shifts. In ICCV'09: International Conference on Computer Vision.
  22. Shan, C., Gong, S., and McOwan, P. (2009). A facial expression recognition based on Local Binary Patterns: a comprehensive study. Journal of Image and Vision Computing, 27(6):803-816.
  23. Sirovich, L. and Kirby, M. (1978). Low dimensional procedure for characterization of human faces. Journal of the Optical Society of America, 4(3):519-524.
  24. Soyel, H. and Demirel, H. (2010). Facial expression recognition based on discriminative Scale Invariant Feature Transform. IEEE Electronics Letters, 46(5).
  25. S.Zhang, Zhao, X., and Lei, B. (2012). Facial expression recognition based on Local Binary Patterns and Local Fisher Discriminant Analysis. WSEAS TRANSACTIONS on SIGNAL PROCESSING, 8:21-31.
  26. Vapnik, V. (1995). The nature of statistical learning theory. Springer-Verlag New York, Inc. New York, NY, USA.
  27. Viola, P. and Jones, M. (2001). Rapid object detection using a Boosted Cascade of simple features. In CVPR 2001: International Conference on Computer Vision and Pattern Recognition, pages 511-518. IEEE.
  28. Visutsak, P. (2013). Emotion classification through lower facial expressions using Adaptive Support Vector Machines. Journal of Man, Machine and Technology, 2(1):12-20.
  29. Wan, S. and Aggarwal, J. (2014). Spontaneous facial expression recognition: a robust metric learning approach. Journal of Pattern Recognition, 47(5):1859- 1868.
  30. Yang, P., Liu, Q., and Metaxas, D. N. (2010). Exploring facial expressions with compositional features. In CVPR'10: International Conference on Computer Vision and Pattern Recognition, pages 2638-2644. IEEE.
  31. Zhang, L., Tjondronegro, D., and V.Chandran (2014). Facial expression recognition experiments with data from television broadcasts and the Word Wide Web. Journal of Image and vision computing, 32:107-119.

Paper Citation

in Harvard Style

Ben Abdallah T., Guermazi R. and Hammami M. (2017). Facial Expression Recognition Improvement through an Appearance Features Combination . In Proceedings of the 19th International Conference on Enterprise Information Systems - Volume 3: ICEIS, ISBN 978-989-758-249-3, pages 111-118. DOI: 10.5220/0006288301110118

in Bibtex Style

author={Taoufik Ben Abdallah and Radhouane Guermazi and Mohamed Hammami},
title={Facial Expression Recognition Improvement through an Appearance Features Combination},
booktitle={Proceedings of the 19th International Conference on Enterprise Information Systems - Volume 3: ICEIS,},

in EndNote Style

JO - Proceedings of the 19th International Conference on Enterprise Information Systems - Volume 3: ICEIS,
TI - Facial Expression Recognition Improvement through an Appearance Features Combination
SN - 978-989-758-249-3
AU - Ben Abdallah T.
AU - Guermazi R.
AU - Hammami M.
PY - 2017
SP - 111
EP - 118
DO - 10.5220/0006288301110118