Authors:
M. Hamed Mozaffari
;
Shuangyue Wen
;
Nan Wang
and
WonSook Lee
Affiliation:
School of Electrical Engineering and Computer Science, University of Ottawa, Ottawa, Ontario and Canada
Keyword(s):
Image Processing with Deep Learning, Ultrasound for Second Language Training, Ultrasound Video Tongue Contour Extraction and Tracking, Convolutional Neural Network, Augmented Reality for Pronunciation Training.
Related
Ontology
Subjects/Areas/Topics:
Animation and Simulation
;
Computer Vision, Visualization and Computer Graphics
;
Computer-Supported Education
;
e-Learning
;
e-Learning Applications and Computer Graphics
;
Graphical Interfaces
;
Interactive Environments
;
Real-Time Visual Simulation
Abstract:
Ultrasound technology is safe, relatively affordable, and capable of real-time performance. Recently, it has been employed to visualize tongue function for second language education, where visual feedback of tongue motion complements conventional audio feedback. It requires expertise for non-expert users to recognize tongue shape in noisy and low-contrast ultrasound images. To alleviate this problem, tongue dorsum can be tracked and visualized automatically. However, the rapidity and complexity of tongue gestures as well as ultrasound low-quality images have made it a challenging task for real-time applications. The progress of deep convolutional neural networks has been successfully exploited in various computer vision applications such that it provides a promising alternative for real-time automatic tongue contour tracking in ultrasound video. In this paper, a guided language training system is proposed which benefits from our automatic segmentation approach to highlight tongue con
tour region on ultrasound images and superimposing them on face profile of a language learner for better tongue localization. Assessments of the system revealed its flexibility and efficiency for training pronunciation of difficult words via tongue function visualization. Moreover, our tongue tracking technique demonstrates that it exceeds other methods in terms of performance and accuracy.
(More)