Authors:
S. K. Al Kork
1
;
D. Uğurca
2
;
C. Şahin
2
;
P. Chawah
3
;
L. Buchman
3
;
M. Adda-Decker
3
;
K. Xu
1
;
B. Denby
1
;
P. Roussel
1
;
B. Picart
4
;
S. Dupont
4
;
F. Tsalakanidou
5
;
A. Kitsikidis
5
;
F. M. Dagnino
6
;
M. Ott
6
;
F. Pozzi
6
;
M. Stone
7
and
E. Yilmaz
2
Affiliations:
1
Universite Pierre Maire Curie and Signal Processing and Machine Learning Lab ESPCI, France
;
2
Argedor Information Technologies, Turkey
;
3
Phonetics and Phonology Laboratory, LPP-CNRS and University Paris 3 Sorbonne Nouvelle, France
;
4
University de Mons, Belgium
;
5
Information Technologies Institute, Greece
;
6
Institute for Educational Technology, Italy
;
7
University of Maryland Dental School, United States
Keyword(s):
Human Computer Interaction, Interactive Game, 3D Model, Ultrasound Sensor, Portable System, Human Beat Box, Game Based Learning
Abstract:
The paper presents an interactive game-like application to learn, perform and evaluate modern contemporary singing. The Human Beat Box (HBB) is being used as a case study. The game consists of two main modules. A sensor module that consists of a portable helmet based system containing an ultrasonic (US) transducer to capture tongue movements, a video camera for the lips, Kinect camera for face gestures, and a microphone for sound. A 3D environment game module is used to visualize a 3D recording studio as game world with all of its unique elements like guitars, mixer, amplifier, speakers and a microphone in front of the 3D avatar to simulate the recording ambience. The game also features a 2D virtual tutor to help the learner by giving oral and written feedback during the game. He also gives feedbacks during the practice session to improve the student’s performance. The game is still at its early stages of development and it is been tested using simple HBB plosive sounds for percussi
on such as “PTK”.
(More)