Understanding Relationships between Reading Behavior and Difficulty
Level of Musical Score based on Cognitive-behavioral Science
Competency Level Evaluation via Musical Score Reading Processes
Katsuko T. Nakahira and Muneo Kitajima
Nagaoka University of Technology, Nagaoka, Japan
Keywords:
Competency Level Evaluation, Reading Musical Score, User Profiling.
Abstract:
This paper proposes a method to understand relationships between reading behavior and the level of difficulty
of musical score. The model of reading behavior is constructed by following the cognitive-behavioral science
approach, which has three components: external objects, and internal perceptual-cognitive-motor processes
and memory. It suggests that the most critical for smooth score reading is that he or she is able to acquire
the right number of objects, k, in a single eye-fixation and to retrieve their relevant information from long-
term memory within the time allowed. The model predicts that differences in smoothness should appear in the
mean, t
Mean
, and/or median, t
Med
, duration times of the field of view, which are strongly related with the degree
of matching of the musical score with the level of competency of the student. It was estimated in the space of
the following five parameters by collecting eye-movement data: the number of notes, rests, accidentals, meter
and velocity, and the other musical symbols. This paper concludes that the estimation procedure should be
effective to evaluate the level of difficulty of musical scores perceived by an individual person.
1 INTRODUCTION
A domain of playing music, “piano singing and play-
ing”, has been studied extensively in Japan because
it is one of the critical skills that pre-school teach-
ers should have in Japan and it has been difficult in
allocating enough time for education at universities.
To overcome this difficulty, a method that implements
the concept of blended learning was proposed for ed-
ucating piano singing and playing skill (for example,
see (Nakahira et al., 2009)). The method consisted
of two components; checking model performance for
each musical compositions, and reflection via video
which recorded students’ performance. A field study
was conducted to find that learners’ skill improved via
blended learning if it was accompanied by video re-
flection. The degrees of improvement, however, were
quite diverse; some learners improved remarkably,
but others modestly. Looking at the findings from the
viewpoint of the two constituent skills, the degrees of
improvement in singing skill were more apparent than
those in playing piano.
The study, such as (Nakahira et al., 2009), sug-
gested two things: a blended-learning-based approach
would be appropriate for skill development education
but at the same time its effectiveness for individual
learners would become diverse. This fact suggests
that integration of “student-centered” approach with
the blended-learning approach should be promising to
develop a method for skill education in the domain of
educating piano singing and playing skill. One direc-
tion towards implementing the student-centered ap-
proach is to adopt an ethnographic approach such as
the CCE methodology (Kitajima et al., 2012). CCE
has been successfully applied to understand the be-
havior of people who intend to accomplish some hap-
piness goal, such as achieving a self-determined goal,
devoting to someone else, etc. A CCE study was
conducted in the domain of educating piano singing
and playing skill to find that self-instruction, which
is the basic mode of blended learning, was more ef-
fective in learning syntactic elements than semantic
elements (Nakahira and Kitajima, 2014).
Among the syntactic elements the most important
was “reading musical score.” This is a subject that has
been studied extensively by means of eye movements
while reading. For example, Kinsler (Kinsler and
Carpenter, 1995) focused on the patterns that char-
acterize saccadic eye movements while reading mu-
sical scores, suggesting that the relationship between
the spatial patterns of the notes and the existence of
fixations. Major fixations appear at notes and bar-
Nakahira K. and Kitajima M.
Understanding Relationships between Reading Behavior and Difficulty Level of Musical Score based on Cognitive-behavioral Science - Competency Level Evaluation via Musical Score
Reading Processes.
DOI: 10.5220/0006129900670074
In Proceedings of the 12th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2017), pages 67-74
ISBN: 978-989-758-229-5
Copyright
c
2017 by SCITEPRESS – Science and Technology Publications, Lda. All rights reserved
67
memory
process
objects
visual
information
musical
score
sensory
memory
working
memory
long-term
memory
selecting action
auditory
information
operate
piano key
vocalized sound
move
line of sight
compare with
real and
ideal sound
process for
the result
action search
A
C
B
F
D
E
Figure 1: Procedure for reading musical score. Processes A and C are transferring symbols or auditory information. Processes
B and D are searching pitch, hand motor to push key, and area what see next. Process E is output of these information
processing. In case of reading musical score, output process is moving area of musical score.
lines. Most significantly, the observed fixation du-
ration times tended to be proportional to the length
of note values. However, the tempo of performance
was inversely related with the average time of sac-
cadic eye movements. On the ground of the afore-
mentioned observations, a number of cognitive model
of reading musical score have been proposed. Among
them, the model proposed by Emond et al. (Emond
and Comeau, 2013) is the one based on the ACT-R
cognitive model and is the most sophisticated one.
Through simulations with the model, they indicated
a possible impact of the different teaching approach
on the acquisition of initial reading skills.
This paper aims at gaining more thorough under-
standing on how people would carry out visual infor-
mation processing while reading musical scores. It
focuses on the relationships between the amount of
“reading span” and the perceived difficulty of the mu-
sical score to be read. “Reading span” is the term to
indicate the degree of proficiency while reading text
such as a printed book. It becomes large when the text
to read is perceived to be easy. On the other hand, it
becomes shorter when it does not match the reading
level of the reader, i.e., the text to read is perceived
to be hard. This construct might be useful to estimate
the degree of matching of the level of the piano player
with the level of the musical scores.
2 MODEL FOR READING
MUSICAL SCORE
This section describes a model of human behavior
while reading musical scores. In the following, the
model is described by incorporating three compo-
nents for reading musical score followed a subsection
describing the procedure that constructs reading mu-
sical score activity.
2.1 Three Components for Reading
Musical Score
Figure 1 shows the basic and general framework of
reading musical score. When a learner moves his or
hey eye-balls to a specific portion of musical score
that exists in the external environment, he or she per-
ceives the objects on the musical score, and then se-
lectively focuses on those where eye-balls are fixated,
i.e., pays attention to selected objects, and recognizes
HUCAPP 2017 - International Conference on Human Computer Interaction Theory and Applications
68
them using knowledge retrieved from his or her long-
term memory, and then comprehends the meaning
the recognized objects considering the context where
they appear as they are. The result of comprehension
should affect to plan how to move the eye-balls in the
future cycles. This basic and general process is sum-
marized as follows:
move eye-balls → perceive objects → pay at-
tention to objects selectively → recognize ob-
jects using long-term memory → comprehend
contextually → move eye-balls → ···
In the following, the components, i.e., objects,
perceptual–cognitive–motor processes, and memory,
that appear in describing the basic and general
processes are further elaborated in the context of
musical score reading activity.
Objects: In case of reading musical score, there are
many symbols for representing musical sounds or
melody (syntax) and indicating how to play with
expression (semantics). Notes, rests, accidental,
meter, velocity, etc., are the symbols for syntax.
Dynamics, articulation, relation, etc., are those
for semantics. Objects are input to the perceptual
process, and output of the motor process, i.e., after
completion of an eye movement process, the resultant
visual field defines the objects to be processed in the
next perceptual process.
Perceptual–Cognitive–Motor Processes: In case of
piano playing, processes which a person carry out
are visual/auditory information processes including
making decision about how to react. For example,
when people receive objects via optic nerves, the
stimuli are carried to perceptual processes. In this
process, objects are the input information and sensory
memory is the output. Perceptual process plays a role
of maintaining the information.
Memory: In the domain of piano singing and
playing, three memory systems are relevant: sensory
memory, working memory, and long-term memory
systems.
Sensory memory systems include iconic, echoic,
and haptic memory systems, each of which is con-
cerned with the process of “reading” musical score,
“hearing” the sound he or she makes by playing the
piano, and “touching” the keyboard of the piano,
respectively. In the case of just reading musical
score this paper is concerned with only the iconic
memory is considered which which creates memory
that represents the type of musical symbols which
is regarded as “objects.” On reading musical score,
a person does not make any real sounds. However,
he or she would activate any relevant portions of
long-term memory that are associated with what he
or she is visually stimulated in any sense: this would
include memory of sound and touch, which may or
may not be included in working memory depending
on the degree of strength of individual memory traces.
Working memory is the place where any cognitive
processes should use for their operations, including
retrieval of relevant knowledge stored in working-
memory to place its result, matching between the
information from sensory memory and long-term
memory to place the result of comparison. This may
include pieces of declarative knowledge to be used to
comprehend what the musical score just having been
read, or pieces of procedural knowledge concerning
how to move his or her fingers in a coordinated way,
i,e, the patterns of muscle movements.
Long-term memory is a storage with infinite capac-
ity for declarative, procedural, and the other types of
knowledge. Reading musical score is not a process
of just reading as reading a printed book but a pro-
cess for associating what is read with how to move
his or her fingers, i.e., motor planning including co-
ordination of five fingers of right and left hands with
the degree of strengths of hitting the keyboards. This
is a very complex association process of visual infor-
mation with motor information, and the rules for as-
sociation must exist in long-term memory, or must be
available, and need to be activated via retrieval pro-
cess, or must be accessible, in order to be smoothly
played on the piano as indicated by the visually indi-
cated on the musical score.
2.2 Musical Score Reading Procedure
This subsection describes Fig.1 in more detail by ex-
tending the explanation given in the previous subsec-
tion.
The first step is to transfer external objects that
physically exist on musical score to internal memo-
ries to be cognitively processed as information for de-
termining next bodily actions to perform. Normally,
a person obtains physical features of symbols on mu-
sical score via his or her eyes, which are transmitted
to sensory memory, i.e., the visual image store, and
then some portion of the contents in the visual im-
age store is transferred to working memory and repre-
sented as symbols corresponding to the physical fea-
tures of the external objects. Symbols are manipu-
lated by the cognitive processes that follow to retrieve
relevant knowledge from long-term memory for the
Understanding Relationships between Reading Behavior and Difficulty Level of Musical Score based on Cognitive-behavioral Science -
Competency Level Evaluation via Musical Score Reading Processes
69
purpose of carrying out a series of transformations of
information stored in working memory to derive next
actions to be realized as a motor movement including
“moving eye-balls to the next appropriate location to
read the musical score given the results of compre-
hension reached so far.”
In the information transformation process, match-
ings between symbols in working memory and infor-
mation in long-term memory are carried out. When
these symbols are represented in working mem-
ory, cognitive process starts searching information in
long-term memory. It is assumed that the degree of
richness of information in long-term memory should
depend on the amount of training of reading and play-
ing musics that a person has experienced in terms of
both quantitatively and qualitatively. The more he or
she has the experience of training, the shorter the time
required of him or her to get access to relevant in-
formation stored in long-term memory and the more
elaborated or richer the information he or she can re-
trieve. In addition, his or her information network in
long-term memory would be robust because he or she
frequently uses the same paths to activate the relevant
information associated with the input symbols which
includes not only the meaning of symbols but also
how he or she should make reaction when he or she
encounters with it. The information stores a number
of association pairs of symbols–reaction or optimized
symbol chunks.
The purpose of the cognitive process is to compre-
hend the meaning of the symbols by associating them
with the procedure of how to play the symbols by his
or her fingers, mental simulation of playing the sym-
bols. If a person gets enough information from sym-
bols having been fixated, he or she does not need to
get another information from another fixation. In this
case, the comprehension process is successfully com-
pleted and the cognitive process initiates a new cycle
starting from acquiring symbols using his or her eyes,
i.e., move eyes to the next location of musical score to
cause the shift of the field of vision. In another case
where a person has failed to get enough information
from the current fixation, the comprehension process
would not be carried out satisfactorily. In this case,
there would be two possibilities of the cognitive pro-
cesses to happen: stay within the same field of vision
and get more information through more fixations, or
give up reaching sufficient comprehension level but
initiate a new cycle.
Through these processes, a person continues to
obtain as much information as possible from the mu-
sical score. It is assumed that it should take a cer-
tain amount of time to comprehend the meaning of
the symbols, on which his or her eyes must be fixated
during a certain amount of time, by activating rele-
vant knowledge stored in long-term memory, and that
there should be interactions between the amount of
experience of playing piano and the level of difficulty
of the musical score to read, which should affect the
observed eye-movement patters while reading musi-
cal scores.
3 DESCRIBING MUSICAL
SCORE READING PROCESS
This section describes the features of musical score,
a framework to represent the behavior of reading mu-
sical score, and a method to estimate the behavior of
reading musical score.
3.1 Features of Musical Score
The following categories of symbols appear in musi-
cal score:
(1) notes consisting of head, stem, flag, and value,
(2) rests,
(3) accidentals,
(4) meter, velocity, and
(5) other musical symbols (dynamics, articulation,
relation, etc.).
The first category 1 is the most important symbol cat-
egory when playing music. This is because it is the
necessities for communicating melody. In playing
music, no melody is produced without recognizing
sound lengths, pitches, and chords. All players are
required to be able to recognize every note. It is as-
sumed that the degree of ability to recognize notes
should be a good predictor for playing skill, and it
might be observed in the patterns of eye movements
while reading a musical score. When a skilled player
reads a musical score, he or she usually searches for a
chunk on the musical score consisting of a number of
notes, which is a unit to be manipulated by cognitive
processes. In some cases it is a group of three notes
that constitutes a single chord, or in other cases it is a
melody that extentds, e.g., over two bars.
The symbols in Categories 2, 3, 4 and 5 are neces-
sary for emotional playing. These symbols are, how-
ever, only effective after recognizing the symbols of
category 1 defining notes and melody . Since these
symbols modify the piano play defined by the notes
and melody, a person who shows fixations on these
symbols is likely to have a higher ability of playing
skill.
HUCAPP 2017 - International Conference on Human Computer Interaction Theory and Applications
70
Table 1: Representation frame of musical score.
bar num. of
ID notes(no chunk) notes(chunk) rest accidental other
1 6 6 0 1 3
2 8 4 0 0 0
·· · ·· ·· ·····
3.2 Reading Musical Score Process
This subsection describes musical score reading
process by using the five categories introduced in the
previous subsection.
Ideal Reading Time “T
ideal
”: There are a number of
relationships among the features and the following is
the first one that defines the relationships among the
meter, M, the velocity, v, and the number of bars, N
b
,
and the ideal playing time for the score T
ideal
(sec):
T
ideal
= N
b
×
60
v
× M
(1)
Features of Musical Score: In order to characterize
a musical score, the following four quantities are in-
troduced for the i-th bar, b
i
:
N
notes
(b
i
) : the number of notes,
N
rest
(b
i
) : the number of rest,
N
acc
(b
i
) : the number of accidental,
N
sym
(b
i
) : the number of other symbols
Reading Musical Score with Chunking: Consider-
ing that T
ideal
can be achieved by reading the musical
score through compressing several notes in chunks.
This compression process is described by introducing
N
T c
(b
i
), the size of chunks in the i-th bar, as follows:
N
~
f
T c
(b
i
) = N
notes
(b
i
) × f (N
notes
)
+ N
rest
(b
i
) × f (N
rest
)
+ N
acc
(b
i
) × f (N
acc
)
+ N
sym
(b
i
) × f (N
sym
), (2)
by using f (x):
f (x) =
1 : can deal with symbol x
0 : otherwise
where f (x) takes the value of 1 or 0 depending on
whether a person has the skill to deal with musical
score in category x as a chunk or not, respectively. For
example, a moderate level player can recognize rest
symbols and accidental symbols as respective chunks
but not for the other symbols; a real beginner cannot
recognize a chord but read individual notes that con-
stitute it, in this case there is no chunk for chord but
a chunk for a note. To represent this situation, Eq. (2)
becomes as follows:
N
~
f =beginner
T c
(b
i
) = N
notes
(b
i
) × 1 + N
rest
(b
i
) × 1
N
acc
(b
i
) × 0 + N
sym
(b
i
) × 0
= N
notes
(b
i
) + N
rest
(b
i
).
Finally, the number of chunks actually processed by a
person for the i-th bar, b
i
, N
s
(b
i
) becomes as follows:
N
s
(b
i
) =
N
~
f =
~
1
T c
k
, (3)
where k represents the average size of chunks.
Creating a Chunk by Compression: The features of
musical score is represented in Table 1. “Compress”
in the table means that the case of packing several
notes when a person reads musical score as a chunk,
which is regarded as an action of recognizing a chord.
Using the framework and the information for meter
and velocity, a more sophisticated chunk will be used
by a piano player while reading a musical score.
3.3 Estimate Reading Musical Score
Behavior
Normally, true values of the average size of chunks,
k, are not known but can be calculated as described
below. The values available for this procedure are,
N
~
f =
~
1
T c
(b
i
), fixation times for a fixation point x, t
x
(sec),
and moving times by saccadic eye-movements to the
next field of vision, t
m
. Given these values, the esti-
mated reading time, T
t
x
, for a given t
x
is represented
as follows:
T
t
x
=
N
b
∑
i
(N
s
(b
i
) ×t
x
+t
m
)
=
N
b
∑
i
(
N
~
f =
~
1
T c
(b
i
)
k
×t
x
+t
m
) (4)
In the case of an expert musical score reader, Eq. (4)
becomes as follows:
T
ideal
= T
t
x
'
t
cog
(k)
k
×
N
b
∑
i
N
~
f =
~
1
T c
(b
i
) (5)
Understanding Relationships between Reading Behavior and Difficulty Level of Musical Score based on Cognitive-behavioral Science -
Competency Level Evaluation via Musical Score Reading Processes
71
num. of symbols per field
average duration time per field(ms)
T
t
Mean
k
1
opt
T
t
Med
k
2
opt
(b)
t
cog
(k)
Figure 2: The relationships between t
cog
(k) and k
(1,2)
opt
. k
(1,2)
opt
correspond to the points where T
t
Med/Mean
intersects with
t
cog
(k), respectively.
where t
cog
(k) is introduced to represent the time nec-
essary for processing a chunk with the size of k.
Eq. (5) indicates the relationship that t
cog
(k) is pro-
portional to k:
t
cog
(k) =
T
ideal
∑
N
b
i
N
~
f =
~
1
T c
(b
i
)
× k (6)
The quantities characterize eye-movement times, t
x
and t
m
, are convoluted in t
cog
(k) in Eq. (6), whose val-
ues are to be determined by fitting experimental data
in due course.
The optimum value of t
x
is obtained by controlling
t
x
for k = 1,2,···, with the condition of minimizing
|T
ideal
− T
t
x
|, which in turn defines the value t
cog
(k).
Using t
cog
(k) and t
Mean or Med
, where t
Mean
is the av-
erage or the median of the length of the observed fix-
ation times, respectively, it is possible to estimate a
reasonable range of k values. Figure 2 shows the solu-
tion. The coordinate system is defined as follows: the
x-axis represents k, the number of symbols per field of
vision, and the y-axis represents T , the average dura-
tion times per the field of vision. On this plane, t
cog
(k)
looks like the one shown in the figure.
In Figure 2, there are two lines that correspond
to T = t
Mean
and T = t
Med
, respectively. The two k
values at which these two lines intersect the curve T =
t
cog
(k), k
1
opt
and k
2
opt
, define a reasonable range of k
values.
The degrees of steepness of the slope of the func-
tion T = t
cog
(k) are dependent on the levels of diffi-
culty of musical scores. The more difficult the musi-
cal score becomes, the steeper the slope of T = t
cog
(k)
becomes. Or for a given k, the longer it takes to read
the content with the fixed chunk size of k, the more
difficulty of musical score
num. of symbols per field of view
easy
difficult
k
1
opt
k
2
opt
(a)
(b)
(c)
(d)
Figure 3: Each bar shows the k values with the range from
k
1
opt
to k
2
opt
. Depending on the levels of difficulty of musical
score, the bars locate at specific regions in the plane and
shows a characteristic distribution pattern on the plane.
difficult the music score become. Figure 3 shows the
expectation relationships of the range k
1
opt
– k
2
opt
and
the score difficulty level. For example, suppose a case
a person feels easy for a given score. In this case he
or she can adopt the optimize vale of k to read the
score. In this case, the difference between k
1
opt
and
k
2
opt
will be come large. As the result, all k
1
opt
− k
2
opt
will be overlapped each other, and it seems the range
of k
1
opt
−k
2
opt
has wide value. Fig. 3 (a) corresponds to
this case. There are many k
1
opt
− k
2
opt
bars per musical
scores, but as a total they seem to become one area.
There is another case where a person feels difficult
for a score. In this case, he or she cannot optimize k
per each field of vision. Some fields have optimized
k values, but not for the others. For this reason, the
differences of k
1
opt
− k
2
opt
will become small and all
ranges k
1
opt
− k
2
opt
will not overlap. Fig. 3 (c) shows
this case.
There are cases in-between, which correspond to
Fig. 3 (b).
4 EXAMINATION OF THE
MODEL
It is assumed that the values of reading span, k, re-
late with the degree of difficulty of the music score
to read and an expert piano player should have large
values of k compared of those that entry level players
would have. This section examines the relationships
between the degree of difficulty of musical score and
the values of k for expert piano players. To do this,
twelve musical scores are used, each of which is as-
sociated with the respective difficulty level, or grade,
defined by the music company. The levels of difficulty
HUCAPP 2017 - International Conference on Human Computer Interaction Theory and Applications
72
Figure 4: The shaded bands in red color shown in the figures from top-left to bottom-right indicate the solutions for k
(1,2)
opt
for
grade 3, 4, 5, and 6, respectively. They were derived by applying the method defined by Fig. 2.
extend from grade-6, the easiest, to grade-3, the most
difficult. Eye movement data were obtained from an
expert piano player while she read each of twelve mu-
sical scores using the Tobii Pro X2-60 and the data
were analyzed by the Tobii Studio software.
The detailed specification of the eye-tracking ex-
periment is as follows:
• gazes are measured with the time resolution of
60Hz and 0.4 degree (∼ 20 pix) for precision,
• the expert has 60 seconds for reading each musical
score, which is conformed to typical sight playing
examination for a grade examination system,
• three musical scores per grade are measured, and
the order of presentation is random,
• after measuring gazes, fixation points are identi-
fied to derive the field of vision, which is con-
verted to the size of reading chunk by counting
the symbols included in the fixated range,
• the mean and median duration times, t
Mean
and
t
Med
, respectively, are extracted for each of the
field of vision.
By applying the the above-mentioned procedure,
t
Mean
and t
Med
are obtained from the eye-tracking data
for each musical score, and the ideal trajectory for
t
cog
(k) is derived. k
1
opt
and k
2
opt
are obtained as the
2
3
4
5
6
7
1 2 3 4 5 6
scoredifficulty(grade)
num.ofk
Figure 5: Distribution of k
(1,2)
opt
for each grade. Dark-gray
bars represent the result of simulation with no-chunk case,
and light-gray bars with chunk-included case.
points that t
cog
(k) intersects with t
Mean
and t
Med
, re-
spectively. In this procedure, it is assumed that the
expert level participant of this experiment should be
able to read read the musical scores with the time de-
fined by T
ideal
.
Figure 4 shows the results, that provide the val-
ues for k
1
opt
and k
2
opt
for each musical score with a
different level of difficulty by using the empirically
estimated values of t
Mean
and t
Med
. By extracting k
1
opt
and k
2
opt
values for each musical score from the data
in Fig. 4 and showing them as a whole, Figure 5 is
generated, which plots the level of difficulty of musi-
Understanding Relationships between Reading Behavior and Difficulty Level of Musical Score based on Cognitive-behavioral Science -
Competency Level Evaluation via Musical Score Reading Processes
73
cal score against the empirically estimated ranges of
k
1
opt
and k
2
opt
values. The dark-gray bars correspond to
the cases with the minimal number of chunks, i.e., it
is impossible for the expert player to create any chunk
due to the layout of the symbols even if he or she is
able to create a chunk with the size of k if the symbols
are appropriately arranged. On the other hand, the
light-gray bars correspond to the cases where there
are a number of chunks with the size of k. Here, the
number of chunks are estimated by using empirical
rules; in case of the same or similar chord patterns,
it is assumed that there is one chunk for several ob-
jects; in case of there being refrain of a similar melody
phrase, one chunk is assumed for each of this phrase.
From Fig. 5, the range of k
1
opt
−k
2
opt
for the easiest
grade appears in a wide and overlapped fashion, and
gradually it narrows its range and becomes less over-
lapped towards the higher grades. Comparing them
with the no-chunk cases (a real beginner who cannot
recognize a chord), chunk-included estimation, which
is for players who can read a chord, has a different
feature. For example, in the grade 3 case, no-chunk
estimation predicts that it needs to include up to 5.4
symbols for each field of vision. But chunk-included
estimation predicts that it needs to include under 4
symbols. Through these features, it is indicated that
if an expert has a method of making effective chunks
for musical symbols, he or she can use more time to
examine the details of the musical score. For exam-
ple, if an expert can make 4 chunks from 12 notes,
he or she can cut it by watching notes in 2/3 times
comparing with 12 chunks from 12 notes. Using the
saved times, the expert makes a time to examine the
other symbols, such as dynamics, articulation, rela-
tion, and so on. The differences in Fig. 5 suggest that
there are possibilities that the lager k values allows a
piano player to read smoothly when provided with the
musical scores at higher grades.
5 CONCLUSION
This paper proposed a method to understand relation-
ships between reading behavior and the level of diffi-
culty of a musical score. The method has a possibility
of training reading musical score – syntactic element
for playing music – a supporting tool with ICT.
First, we set three components to construct the
model for reading musical score. It was based on a
cognitive-behavioral science approach. With the three
components, we described the processes. By examin-
ing the processes, it was found that the features of
eye-movement behaviors such as fixation times and
the areas of fixation trace reflected the performance
of memory searching in musical objects, which ap-
proximated performance of reading musical scores.
On the basis of the above considerations, this pa-
per proposed a method for estimating personal perfor-
mance of reading musical score using the average and
median duration times for each fixation. It was sug-
gested that, along with the observed duration times,
the predicted optimum duration times from each mu-
sical score were related with the estimated total num-
ber of symbols per field of vision, which in turn re-
lated with personal performance of reading musical
score. It was suggested that the method has a possibil-
ity to apply supporting tool for training reading mu-
sical score with ICT. In the future, the observation of
eye-movement behavior while reading musical scores
will be extended to that of beginners to examine the
possibility of applying this method to more broad pi-
ano players.
ACKNOWLEDGEMENTENTS
The authors are deeply grateful to Miss Takako
Nakano for useful comments. This work was sup-
ported by JSPS KAKENHI Grant Number 15H02784.
REFERENCES
Emond, B. and Comeau, G. (2013). Cognitive modelling
of early music reading skill acquisition for piano: A
comparison of the middle-c and intervallic methods.
Cognitive Systems Research, 24:26 – 34. Cognitive
Systems Research:Special Issue on ICCM2012.
Kinsler, V. and Carpenter, R. (1995). Saccadic eye
movements while reading music. Vision Research,
35(10):1447 – 1458.
Kitajima, M., Tahira, H., Takahashi, S., and Midorikawa,
T. (2012). Understanding tourists’ in situ behavior:
A cognitive chrono-ethnography study of visitors to
a hot spring resort. Journal of Quality Assurance in
Hospitality & Tourism, 13:247–270.
Nakahira, K. T., Akahane, M., and Fukami, Y. (2009). Ver-
ification of the effectiveness of blended learning in
teaching performance skills for simultaneous singing
and piano playing. Proceedings of the 17th Interna-
tional Conference on Computers in Education, pages
975–979.
Nakahira, K. T. and Kitajima, M. (2014). Understand-
ing differences of eye movements patterns while read-
ing musical scores between instructors and learners
to design learner - centered teaching strategies. the
Proceedings of The 22nd International Conference on
Computers in Education, pages 101–106.
HUCAPP 2017 - International Conference on Human Computer Interaction Theory and Applications
74
