Lip Tracking Using Particle Filter and Geometric Model for Visual Speech Recognition

Islem Jarraya, Salah Werda, Walid Mahdi

2014

Abstract

The automatic lip-reading is a technology which helps understanding messages exchanged in the case of a noisy environment or of elderly hearing impairment. To carry out this system, we need to implement three subsystems. There is a locating and tracking lips system, labial descriptors extraction system and a classification and speech recognition system. In this work, we present a spatio-temporal approach to track and characterize lip movements for the automatic recognition of visemes of the French language. First, we segment lips using the color information and a geometric model of lips. Then, we apply a particle filter to track lip movements. Finally, we propose to extract and classify the visual informations to recognize the pronounced viseme. This approach is applied with multiple speakers in natural conditions.

References

  1. Beaumesnil, B., 2006. Real Time Tracking for 3D Realistic Lip Animation. In ICPR, International Conference Pattern Recognition. IEEE.
  2. Bouvier, C., 2010. Segmentation Region-Contour des contours des lèvres. Prepared in the laboratory GIPSA-lab/DIS within the Graduate School Electronics, Electrotechnics, Automation & Signal Processing Laboratory of Computer Vision and Systems Université Laval.
  3. Kalbkhani, H., Amirani, M., 2012. An Efficient Algorithm for Lip Segmentation in Color Face Images Based on Local Information. In JWEET'01, Journal of World's Electrical Engineering and Technology. Science-line.
  4. Liu, X., Cheung Y., 2011. A robust lip tracking algorithm using localized color active contours and deformable models.In ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing.IEEE.
  5. Mahdi, W., Werda, S., Ben Hamadou, A., 2008. A hybrid approach for automatic lip localization and viseme classification to enhance visual speech recognition.In ICAE'03, Integrated Computer-Aided Engineering. ACM.
  6. Majumdar, J., Kiran, S., 2013. Particle Filter Integrating Color Model for Tracking. In IJETAE'07, International Journal of Emerging Technology and Advanced Engineering.
  7. Meng, J., Liu, J., Zhao, J., Wang, J., 2014. Research of Real-time Target Tracking Base on Particle Filter Framework. In Jofcis'06. Journal of Computational Information Systems. Binary Information Press.
  8. Nicolas, E., 2003. Segmentation des lèvres par un modèle déformable analytique. Prepared at the Laboratory of Images and Signals (LIS) within the Doctoral School.
  9. Segura, C., Hernando, J., 2014. 3D Joint Speaker Position and Orientation Tracking with Particle Filters. In Sensors'02. Sensors and Transducers Journal. MDPI.
  10. Shirinzadeh, F., Seyedarabi, H., Aghagolzadeh, A., 2012. Facial Features Tracking Using Auxiliary Particle Filtering and Observation Model Based on Bhattacharyya Distance. In IJCTE'05. International Journal of Computer Theory and Engineering. EBSCO.
  11. Stillittano, S., Girondel, V., Caplier, A., 2013. Lip contour segmentation and tracking compliant with lip-reading application constraints. In Mach. Vis. Appl.7801. Proceedings of Mach. Vis. Appl. Springer-Verlag.
  12. Sunil, M., Patnaik, S., 2013. Automatic Lip Tracking and Extraction of Lip Geometric Features for Lip Reading. In IJMLC'02. International Journal of Machine Learning and Computing. IACSIT.
  13. Sunil, M., Patnaik, S., 2014. Lip reading using DWT and LSDA. In IACC. IEEE International Advance Computing Conference . IEEE.
Download


Paper Citation


in Harvard Style

Jarraya I., Werda S. and Mahdi W. (2014). Lip Tracking Using Particle Filter and Geometric Model for Visual Speech Recognition . In Proceedings of the 11th International Conference on Signal Processing and Multimedia Applications - Volume 1: SIGMAP, (ICETE 2014) ISBN 978-989-758-046-8, pages 172-179. DOI: 10.5220/0005045601720179


in Bibtex Style

@conference{sigmap14,
author={Islem Jarraya and Salah Werda and Walid Mahdi},
title={Lip Tracking Using Particle Filter and Geometric Model for Visual Speech Recognition},
booktitle={Proceedings of the 11th International Conference on Signal Processing and Multimedia Applications - Volume 1: SIGMAP, (ICETE 2014)},
year={2014},
pages={172-179},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0005045601720179},
isbn={978-989-758-046-8},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 11th International Conference on Signal Processing and Multimedia Applications - Volume 1: SIGMAP, (ICETE 2014)
TI - Lip Tracking Using Particle Filter and Geometric Model for Visual Speech Recognition
SN - 978-989-758-046-8
AU - Jarraya I.
AU - Werda S.
AU - Mahdi W.
PY - 2014
SP - 172
EP - 179
DO - 10.5220/0005045601720179