Serious Games for Music Education

A Mobile Application to Learn Clef Placement on the Stave

Adriano Barat`e, and Luca A. Ludovico

Department of Computer Science, Universit`a degli Studi di Milano, Via Comelico 39, Milan, Italy

Keywords:

Music, Education, Mobile Applications.

Abstract:

According to recent researches, gaming can be used in educational contexts to improve learning processes.

In this work, at ﬁrst serious games are introduced by deﬁning their key features, then their applicability to

the music education ﬁeld will be discussed. Finally, the paper will present a speciﬁc case study focusing on

real-time solf`ege of scores with frequent clef changes. The resulting pedagogical game, called iClef, has been

designed and implemented to run on mobile devices such as the iPhone and iPad.

1 INTRODUCTION

Recent research has shown that gaming can be prof-

itably used in educational contexts, in order to make

the learning process both effective and amusing. Seri-

ous games (Abt, 1970) are designed for purposes that

go beyond mere amusement. Like any game, they

are intended to entertain users, but their main goal is

usually to train and educate them. In other words,

the recreational aspects of serious games act as a sort

of Trojan horse to convey computer-supported educa-

tion. The effectiveness of games as teaching tools is

due to their potential to engage players.

The term “serious game” was actually used long

before the introduction of computer and electronic

devices into entertainment, but our concern in this

article is with the application of information tech-

nology to educational processes. The article (Zyda,

2005) provides an up-to-date deﬁnition for serious

games, depicting them as “mental contests, played

with a computer in accordance with speciﬁc rules”

that use entertainment to further government or cor-

porate training, education, health, public policy, and

strategic communication objectives.

This paper addresses a speciﬁc ﬁeld, namely mu-

sic education. Serious games could be applied to this

domain for a number of different purposes. For in-

stance, it is possible to teach the key concepts of mu-

sic theory and instrumental practice through ad hoc

hardware and software frameworks based on selected

music materials. The huge success of entertainment-

oriented applications like the celebrated Guitar Hero

(Arsenault, 2008), released for all the principal gam-

ing platforms and PC systems from 2005 on, wit-

nesses not only the pedagogical but also a huge com-

mercial interest towards this kind of products.

In order to provide a comprehensive discussion of

the matter, Section 2 presents a brief review of re-

lated works, whereas Section 3 is devoted to ﬁxing

speciﬁc goals for our serious game. Finally, Section 4

describes iClef, namely an example of music serious

games implemented by the authors.

2 RELATED WORKS

The scientiﬁc literature about computer-based serious

games somehow dealing with music is very rich. Re-

searches and implementations range from simulated

instrumental practice to advanced ear training, from

alternative ways for music composition to graphical

representation and analysis of scores, from gesture

recognition to motion tracking techniques aiming at

controlling music and/or audio parameters. More-

over, the iPhone, iPod and other popular mobile de-

vices intrinsically support music digital libraries, thus

their environment encourages applications where mu-

sic can be employed in an unconventional way. Since

the ﬁeld is so wide, we will narrow our discussion

to mobile-oriented applications related to music edu-

tainment and ear training.

A ﬁrst category includes the huge family of ap-

plications either simulating traditional music instru-

ments or implementing brand new ones. For exam-

ple, (Wang, 2009) describes the so-called Smule Oca-

rina, i.e. a wind instrument designed for the iPhone.

234

Baratè A. and A. Ludovico L..

Serious Games for Music Education - A Mobile Application to Learn Clef Placement on the Stave.

DOI: 10.5220/0004364902340237

In Proceedings of the 5th International Conference on Computer Supported Education (CSEDU-2013), pages 234-237

ISBN: 978-989-8565-53-2

 2013 SCITEPRESS (Science and Technology Publications, Lda.)

In this mobile musical artifact, the interactions of the

ancient ﬂute-like instrument are both preserved and

transformed via breath-control and multitouch ﬁnger-

holes. Besides, the onboard global positioning and

persistent data connection allow the users to listen to

one another, thus creating a social experience. Mo-

bile music making is becoming both a relevant ﬁeld

of research from a technological point of view and an

evolved expression tool in an artistic perspective, as

demonstrated in (Tanaka, 2004). Initiatives such as

Momu - a mobile music toolkit (Bryan et al., 2010)

- and MoPhO - the Stanford Mobile Phone Orchestra

(Oh et al., 2010) - witness this new tendence.

In our short review, it is worth citing researches

about gesture, motion capture and physical metaphors

in designing mobile music performances. An exam-

ple is contained in (Dahl and Wang, 2010), based on

the metaphor of a sound as a ball. Exploring the in-

teractions and sound mappings suggested by such a

metaphor leads to the design of a gesture-controlled

instrument that allows players to “bounce” sounds,

“throw” them to other players, and compete in a game

(called SoundBounce) to “knock out” others’ sounds.

The soundscape is designed so that all actions and

changes of state have audible correlates, allowing

both players and audience to perceive what is hap-

pening without any explicit visual information.

In the music education context, there are many in-

teresting initiatives and applications based on mobile

devices. For instance, (Zhou et al., 2010) describes

the experience of MOGCLASS, namely a system of

networked mobile devices to amplify and extend chil-

drens capabilities to perceive, perform and produce

music collaboratively in classroom context. An-

other relevant example is represented by Rhythmati-

cal (Mooreﬁeld-Lang and Evans, 2011), designed to

be an educational application for the iPhone and iPod

Touch that conveys mathematical topics via musical,

rhythmic, or movement interactive techniques.

With respect to the applications and approaches

reviewed in this section, our work is focused on a

very speciﬁc matter, often referred to as clef read-

ing. In this sense, our software takes advantage from

the mentioned experiences as regardthe use of mobile

technology and interface design.

3 HOW TO PRACTICE SIGHT

READING IN DIFFERENT

CLEFS

In modern notation, usually three clefs (namely three

graphical symbols) are used to notate music, i.e. G, F,

and C. Each type of clef assigns a different reference

note to the line on which it is placed. Modern nota-

tion often assigns a standard position to the three cited

clefs; nevertheless, in order to facilitate writing for

different tessituras, any of the clefs may theoretically

be placed on any of the lines of the staff. This prac-

tice was common in renaissance and baroque music,

above all in vocal pieces such as motets and madri-

gals. However, modern instrumentation and orches-

tration texts, such as (Blatter, 1997), suggest their use

for some instrument scoring: e.g. the alto clef is a

common one for viola, as well as the tenor clef is typ-

ical of trombone scores.

Since there are ﬁve lines on the stave, and three

clefs, it might seem that there would be ﬁfteen possi-

ble clefs. Six of these, however, are redundant clefs,

so only nine possible distinct clefs are allowed and

have been historically adopted: the G-clef on the two

bottom lines, the F-clef on the three top lines, and the

C-clef on any line of the stave except the topmost,

which is deprecated. Each of these clefs has a differ-

ent name based on the tessitura for which it is best

suited: Treble, and French violin clef (symbol: G-

clef); Bass, Baritone, and Sub-bass clef (symbol: F-

clef); Alto, Tenor, Baritone, Mezzo-soprano, and So-

prano clef (symbol: C-clef). The keys listed above are

graphically shown in Figure 1.

In Italian conservatories, sight reading of scores

containing frequent clef changes is one of the tests

the ﬁnal exam of Music Theory and Solf

ege is made

of. Figure 2 provides a hand-written example of this

kind of examination. Since for most young music stu-

dents practicing clef reading is an unusual and dreary

activity, our purpose is providing them with a serious

game oriented to this speciﬁc task. The mobile ap-

plication described in the next section, called iClef,

should do the job.

4 CASE STUDY: LEARNING

CLEFS ON A PORTABLE

DEVICE

iClef is a serious game which addresses the problem

of correctly reading note pitches when the clef is not

only a commonly used one, such as a treble or bass

clef. The interface shows notes with no rythmic in-

dication, written on a staff that carries randomly se-

lected clefs, at an increasing metronome rate. In or-

der to help the user, smaller noteheads suggest the

next notes written in the current clef. The user has

to choose the right corresponding key on a piano-

like keyboard as quickly as possible. The interface

SeriousGamesforMusicEducation-AMobileApplicationtoLearnClefPlacementontheStave

235

Figure 1: Clef types and their placement on the stave.

Figure 2: An example of ﬁnal examination from an Italian

conservatory.

Figure 3: The interface of iClef.

is shown in Figure 3.

4.1 Gameplay

The key idea is providing the player with a very sim-

ple graphical interface, where the main game controls

- namely the music keys - are located on the right side

and are easily accessible through the thumb. In fact,

the user must be able to give a prompt response to the

notes graphically shown in the left panel.

The game is organized in a number of levels, each

one presenting an increasing degree of difﬁculty. It is

worth underlining that melodies are not predeﬁned,

but they are computed in real-time by taking into

account both music-related rules and player perfor-

mances. The former aspect is detailed in the following

subsection, where some notation obstacles are listed.

After a given number of mistakes, the player is not

allowed to access the next tier.

4.2 Levels and Scoring

Like any other game, in order to catch the attention

of the player and to stimulate his/her abilities, iClef

presents increasing degrees of difﬁculty.

A ﬁrst aspect is the overall metronome, which es-

tablishes through its beats when a new note should ap-

pear. Needless to say, a slow metronome is adequate

to the very ﬁrst levels, whereas it becomes faster and

faster as the game goes on.

Besides, the number of clef changes can dramati-

cally impact on the degree of difﬁculty, so the rate in-

creases from level to level according to n = (11 − l),

where n is the number of notes occurring between two

clef changes and l ∈ [1..10] is the current level.

Another aspect taken into account is melody com-

plexity. In fact, a step-by-step melody (often known

as conjunct motion) is intrinsically easier to read than

a line with wide intervals. In this case, the number of

scale grades that separate the endpoints of the (either

ascending or descending) interval is i ∈ N : i ≤ l. As

an example, at level l = 1 only 1-grades (ascending

and descending second) intervals are allowed. There

is a relevant exception to this rule: when a clef change

occurs, the random interval is not related to the pre-

vious note in the sequence, but to the base line of the

new clef. Only in this case, unison is allowed.

For

instance, at level l = 1 and under a treble clef, the ﬁrst

pitch is one of F, G, or A, independently of the previ-

ous pitch under the old key.

Since accidentals and ledger lines make the read-

ing exercise harder, they are progressively introduced

in the higher levels. As regards the former aspect, for

l ∈ [1..4] the algorithm chooses only natural notes, for

l ∈ [5..8] single accidentals (i.e. sharp and ﬂat) are in-

troduced, and ﬁnally for l ∈ [9..10] double accidentals

may appear.

During the gameplay, the interface provides a vari-

able number of clues which provide in advance infor-

mation about the next notes (graphically indicated by

small symbols). In the easier levels clues are two, but

they gradually disappear in higher levels.

All these aspects are conveniently mixed during

Please note that in general terms the concept of interval

implies the distance between two pitches, whereas in this

case the “unison” occurs between a real pitch and the virtual

pitch indicated by the clef position.

CSEDU2013-5thInternationalConferenceonComputerSupportedEducation

236

the gameplay, so that the task becomes harder and

harder.

The player’s performances are mainly measured

in terms of right-answers rate, and this is the key as-

pect to extend play to higher levels. However, scoring

is inﬂuenced also by his/her reaction time: a prompt

response results in a higher score for the current note,

and a series of quick responses gives bonus points.

5 CONCLUSIONS AND FUTURE

WORKS

The present paper has addressed the issue of serious

games’ application to a speciﬁc domain of computer-

supported education, namely music training. Unlike

many other approaches, here the professional voca-

tion of the entertainment activities is stressed. In fact,

the target is represented by young students who attend

music conservatoriesrather than video gamers fond of

music.

This serious game allows the improvement of

score following for some instruments - e.g. violas and

trombones - and for speciﬁc repertoires as early vo-

cal music, whose typical ensemble is Cantus, Altus,

Tenor and Bassus (Kurtzman, 1994). The resulting

pedagogical application has been tested on several

classes of music students. In all cases, such an ap-

proach has produced a high level of amusement and

involvement in students, which has been evaluated

through blind questionnaires. This is a relevant re-

sult for our research, since clef reading is usually con-

sidered either an annoying or a challanging activity.

Needless to say, relevant differencesemergeddepend-

ing on the age and the degree of music students.

Future works may take advantage from a wide

user acceptance test, paying special attention to as-

pects such as usability, learning effectiveness and

player amusement. Besides, we are interested in re-

leasing such a game for other platforms, including a

Web version. Finally, other aspects of music theory

(e.g. rhythmic ear training or music dictation) could

be investigated.

ACKNOWLEDGEMENTS

This work is one of the scientiﬁc outcomes of Project

E2, funded by the Education, Audiovisual and Cul-

ture Executive Agency of the European Commission

in the framework of Lifelong Learning Programme

Leonardo Da Vinci - Development of Innovation (E2

- 517964-LLP-2011-IT-LEONARDO-LMP). The au-

thors wish to acknowledge all the researchers and

contributors who made this result possible.

REFERENCES

Abt, C. (1970). Serious games. Viking Press.

Arsenault, D. (2008). Guitar hero: ”not like playing guitar

at all”? Loading..., 2(2).

Blatter, A. (1997). Instrumentation and orchestration.

Schirmer Books.

Bryan, N., Herrera, J., Oh, J., and Wang, G. (2010). Momu:

A mobile music toolkit. In Proceedings of the Interna-

tional Conference on New Interfaces for Musical Ex-

pression (NIME), Sydney, Australia.

Dahl, L. and Wang, G. (2010). Sound bounce: Physical

metaphors in designing mobile music performance. In

Proceedings of the International Conference on New

Interfaces for Musical Expression (NIME), Sydney,

Australia.

Kurtzman, J. (1994). Tones, modes, clefs and pitch in ro-

man cyclic magniﬁcats of the 16th century. Early mu-

sic, 22(4):641–665.

Mooreﬁeld-Lang, H. and Evans, M. (2011). Rhythmatical:

A game to combine music and mathematics for mobile

devices. Music Reference Services Quarterly, 14(1-

2):46–51.

Oh, J., Herrera, J., Bryan, N., Dahl, L., and Wang, G.

(2010). Evolving the mobile phone orchestra. In Pro-

ceedings of the International Conference on New In-

struments for Musical Expression.

Tanaka, A. (2004). Mobile music making. In Proceedings

of the 2004 conference on New interfaces for musi-

cal expression, pages 154–156. National University of

Singapore.

Wang, G. (2009). Designing smules iphone ocarina. In

Proceedings of the International Conference on New

Interfaces for Musical Expression. Pittsburgh.

Zhou, Y., Percival, G., Wang, X., Wang, Y., and Zhao, S.

(2010). Mogclass: a collaborative system of mobile

devices forclassroom music education. In Proceed-

ings of the international conference on Multimedia,

pages 671–674. ACM.

Zyda, M. (2005). From visual simulation to virtual reality

to games. Computer, 38(9):25–32.

SeriousGamesforMusicEducation-AMobileApplicationtoLearnClefPlacementontheStave

237