Can Ultrasonic Doppler Help Detecting Nasality for Silent Speech Interfaces? - An Exploratory Analysis based on Alignement of the Doppler Signal with Velum Aperture Information from Real-Time MRI

João Freitas, António Teixeira, Miguel Sales Dias

2014

Abstract

This paper describes an exploratory analysis on the usefulness of the information made available from Ultrasonic Doppler signal data collected from a single speaker, to detect velum movement associated to European Portuguese nasal vowels. This is directly related to the unsolved problem of detecting nasality in silent speech interfaces. The applied procedure uses Real-Time Magnetic Resonance Imaging (RT-MRI), collected from the same speaker providing a method to interpret the reflected ultrasonic data. By ensuring compatible scenario conditions and proper time alignment between the Ultrasonic Doppler signal data and the RT-MRI data, we are able to accurately estimate the time when the velum moves and the type of movement under a nasal vowel occurrence. The combination of these two sources revealed a moderate relation between the average energy of frequency bands around the carrier, indicating a probable presence of velum information in the Ultrasonic Doppler signal.

References

  1. Freitas, J., Teixeira, A., Dias, M. S., 2012b. Towards a Silent Speech Interface for Portuguese: Surface Electromyography and the nasality challenge. In Int. Conf. on Bio-inspired Systems and Signal Processing, Vilamoura, Algarve, Portugal.
  2. Freitas, J., Teixeira, A., Dias, M. S., 2013. Multimodal silent speech interface based on Video, depth, surface electromyography, and ultrasonic Doppler, Workshop on Speech Production in Automatic Speech Recognition, Lyon, France.
  3. Freitas, J., Teixeira, A., Vaz, F., Dias, M. S., 2012a. Automatic Speech Recognition based on Ultrasonic Doppler Sensing for European Portuguese. Advances in Speech and Language Technologies for Iberian Languages, vol. CCIS 328, Springer.
  4. Hyvarinen, A, 1999. Fast and robust fixed-point algorithms for independent component analysis. Neural Networks, IEEE Transactions on Vol. 10, no. 3, pp. 626-634.
  5. Jennings, D. L., Ruck, D. W, 1995. Enhancing automatic speech recognition with an ultrasonic lipmotion detector, In Int. Conf. on Acoustics, Speech, and Signal Processing, Detroit.
  6. Kalgaonkar, K. and Raj, B., 2007. Acoustic Doppler Sonar for Gait Recognition, IEEE International Conference on Advance Video and Signal-based Surveillance (AVSS2007).
  7. Kalgaonkar, K. and Raj, B., 2008. Ultrasonic Doppler Sensor for Speaker Recognition, IEEE Intl. Conf. on Acoustics Speech and Signal Processing 2008.
  8. Kalgaonkar, K. and Raj, B., 2009. One-handed Gesture Recognition using Ultrasonic Doppler Sonar, IEEE Intl. Conf. on Acoustics, Speech and Signal Processing 2009.
  9. Kalgaonkar, K., Raj B., Hu., R., 2007. Ultrasonic doppler for voice activity detection. IEEE Signal Processing Letters, vol.14(10), pp. 754-757.
  10. Lacerda, A. and Head, B. F., 1996. Análise de sons nasais e sons nasalizados do Português. Revista do Laboratório de Fonética Experimental (de Coimbra). VI:5_70.
  11. Livescu, K., Zhu, B. and Glass, J., 2009. On the phonetic information in ultrasonic microphone signals" Intl. Conf. on Acoustics, Speech and Signal Processing 2009.
  12. Martins, P. Carbone, I. Pinto, A. Silva, A. and Teixeira, A., 2008. European Portuguese MRI based speech production studies. Speech Communication. NL: Elsevier, Vol.50, No.11/12, ISSN 0167-6393, pp. 925- 952.
  13. Raj, B., Kalgaonkar, K. Harrison, C. and Dietz, P., 2012. Ultrasonic Doppler Sensing in HCI. Pervasive Computing, IEEE 11, no. 2, pp. 24-29.
  14. Rossato, S. Teixeira, A. and Ferreira, L., 2006. Les Nasales du Portugais et du Français: une étude comparative sur les données EMMA. In XXVI Journées d'Études de la Parole. Dinard, France.
  15. Schwartz, M. F. 1968. The acoustics of normal and nasal vowel production. Cleft Palate Journal 5: 125-40.
  16. Silva, S., Martins, P., Oliveira, C., Silva, A., Teixeira, A., 2012. Segmentation and Analysis of the Oral and Nasal Cavities from MR Time Sequences. Image Analysis and Recognition. Proceedings of ICIAR 2012, LNCS, Springer.
  17. Srinivasan, S., Raj, B., Ezzat., T., 2010. Ultrasonic sensing for robust speech recognition, In Intl. Conf. on Acoustics, Speech, and Signal Processing 2010.
  18. Teixeira, A., Martins, P., Oliveira, C., Ferreira, C., Silva, A., Shosted, R., 2012. Real-time MRI for Portuguese: database, methods and applications, Proceedings of PROPOR 2012, LNCS vol. 7243. pp. 306-317.
  19. Teixeira, J. S., 2000. Síntese Articulatória das Vogais Nasais do Português Europeu. PhD Thesis, Universidade de Aveiro.
  20. Toth, A. R., Kalgaonkar, K., Raj, B., T. Ezzat, 2010. Synthesizing speech from Doppler signals. IEEE International Conference on Acoustics Speech and Signal Processing, pp.4638-4641.
  21. Trigo, R. L., 1993. The inherent structure of nasal segments, In Nasals, Nasalization, and the Velum, Phonetics and Phonology, M. K. Huffman e R. A. Krakow (eds.), Vol. 5, pp.369-400, Academic Press Inc.
  22. Zhu, B., 2008. Multimodal speech recognition with ultrasonic sensors. Master's thesis. Massachusetts Institute of Technology, Cambridge, Massachusetts.
  23. Zhu, B., Hazen, T. and Glass, J. R., 2007. Multimodal speech recognition with ultrasonic sensors. In Eurospeech 2007.
Download


Paper Citation


in Harvard Style

Freitas J., Teixeira A. and Sales Dias M. (2014). Can Ultrasonic Doppler Help Detecting Nasality for Silent Speech Interfaces? - An Exploratory Analysis based on Alignement of the Doppler Signal with Velum Aperture Information from Real-Time MRI . In Proceedings of the International Conference on Physiological Computing Systems - Volume 1: PhyCS, ISBN 978-989-758-006-2, pages 232-239. DOI: 10.5220/0004725902320239


in Bibtex Style

@conference{phycs14,
author={João Freitas and António Teixeira and Miguel Sales Dias},
title={Can Ultrasonic Doppler Help Detecting Nasality for Silent Speech Interfaces? - An Exploratory Analysis based on Alignement of the Doppler Signal with Velum Aperture Information from Real-Time MRI},
booktitle={Proceedings of the International Conference on Physiological Computing Systems - Volume 1: PhyCS,},
year={2014},
pages={232-239},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0004725902320239},
isbn={978-989-758-006-2},
}


in EndNote Style

TY - CONF
JO - Proceedings of the International Conference on Physiological Computing Systems - Volume 1: PhyCS,
TI - Can Ultrasonic Doppler Help Detecting Nasality for Silent Speech Interfaces? - An Exploratory Analysis based on Alignement of the Doppler Signal with Velum Aperture Information from Real-Time MRI
SN - 978-989-758-006-2
AU - Freitas J.
AU - Teixeira A.
AU - Sales Dias M.
PY - 2014
SP - 232
EP - 239
DO - 10.5220/0004725902320239