DANCE EVALUATION SYSTEM BASED ON MOTION

ANALYSIS

Masahiro Tada, Masahide Naemura

ATR Media Information Science Laboratories, 2-2-2 Hikaridai, Keihanna Science City, Japan

Keywords: Motion Analysis, Wavelet Multi-Resolution Correlation Analysis, Edutainment System, Dance.

Abstract: We are conducting research on computer-aided edutainment with a view to creating learning environments

where anybody can acquire advanced skills. In this paper, we focus on dance actions as a part of

edutainment research and propose a method to evaluate dance skills through motion analysis. Our method

consists of wavelet multi-resolution analysis and correlation analysis. Firstly, by using wavelet multi-

resolution analysis, we decompose complex dance motion data acquired from a motion-capture system into

different frequency components. And by applying correlation analysis to the decomposed data, we extract

motion features that play a dominant role in evaluating sense of rhythm and harmony of movement of each

body part. By comparing the extracted features of amateurs to those of experts, we have achieved a

quantitative evaluation method for dance skills. Through experiments, we confirmed that there is a strong

correlation amongst extracted motion features and subjective evaluation results of dance skills. Using the

proposed method, we have developed a computer-aided edutainment system for dance. By mapping motion-

captured dance data and its evaluation results onto the 3-D CG figure, our system enables users to visually

know bad points of their dance and acquire more advanced dance skills.

1 INTRODUCTION

We are conducting research on computer-aided

edutainment with a view to creating learning

environments where anybody can acquire advanced

skills (Naemura, 2005, Oshima, 2004). In order to

make good use of computer technology in

edutainment, it is important to identify the basic

factors that characterize the performance difference

between an amateur and an expert and to

computationally analyse the difference. In this

paper, we focus on dance actions as a part of

edutainment research.

We can roughly classify dance into two

categories: formal dance and rhythmical dance.

Formal dance (e.g. ballet) has precise and highly

formalized set steps and gestures, whereas

rhythmical dance (e.g. jazz dance, hip-hop dance)

emphasizes improvisation.

There are many works that focus on formal

dance; e.g. classic ballet (Soga, 2001), traditional

folk dance (Shiratori, 2004, Hachimura, 2005). Most

aim to digitally archive the dance of experts as

intangible cultural heritage, and do not consider

amateur dance at all.

Compared to formal dance, there are few works

on rhythmical dance. However, in recent years,

popularity of rhythmical dance (especially hip-hop

dance) is rapidly increasing. Therefore, in this paper,

we focus on hip-hop dance.

As a dance analysis method, Laban Movement

Analysis (LMA, Bartenieff, 1980) is widely used

(Naugle, 1999, Camurri, 1999, Hachimura, 2005).

LMA is a methodology classifying dynamical and

geometrical features of body motions in detail.

Nevertheless, LMA does not deal with rhythm that is

an essential factor of hip-hop dance.

Our goal is to develop an evaluation method for

rhythmical dance and help amateurs acquire

advanced dance skills. In this paper, we propose an

evaluation method for rhythmical dance based on

wavelet multi-resolution analysis and motion

correlation analysis.

2 MOTION CAPTURE SYSTEM

A motion capture system is one of the most effective

methods for digitalizing human motions. Therefore,

in order to acquire dance action movements, we use

243

Tada M. and Naemura M. (2006).

DANCE EVALUATION SYSTEM BASED ON MOTION ANALYSIS.

In Proceedings of the First International Conference on Computer Graphics Theory and Applications, pages 243-250

DOI: 10.5220/0001358602430250

Copyright

c

SciTePress

an optical motion capture system (Vicon612).

Vicon612 uses 12 infrared cameras to detect

reflective markers (small balls) attached to a dancer.

Spatial resolution of Vicon612 is about 2mm, and

sampling interval is set at 1/60 second. The number

of markers attached to a dancer is 30. Based on

acquired three-dimensional coordinate positions of

markers, we calculate joints angles shown in Table 1

and Fig.1.

Table 1: Adopted joints angles.

ID Joints ID Joints

0 Right Elbow 5 Left Knee

1 Left Elbow 6

Right

Thigh

2

Right Upper

Arm

7

Left

Thigh

3

Left Upper

Arm

8

Body

Angle

4 Right Knee

Figure 1: Adopted joints angles.

3 MULTI-RESOLUTION

ANALYSIS

In human motions, there are many correlations

among joint actions. Nakata (2005) proposed a

behaviour recognition method based on motion

correlation analysis. However, dance motion is very

complex. It is a mixture of various kinds of motions,

each having a different period. This complexity

would give a negative effect to motion correlation

analysis. Therefore, firstly, by using a multi-

resolution discrete wavelet transform (DWT), we

decompose complex dance motion data acquired

from a motion-capture system into different

frequency components.

3.1 Discrete Wavelet Transform

Multi-resolution DWT can provide information of

signals both in the time domain and in the frequency

domain. A wavelet transform can be obtained by

projecting the signal onto a scaled and translated

version of a basic function. This function is known

as mother wavelet, Ψ(t). A mother wavelet must

satisfy following conditions.

.1)(,0)(

2

∫∫

∞

∞−

∞

∞−

== dttdtt

ψψ

(1)

A scaled and translated mother wavelet Ψ

j,k

(t)

forms basis of functions. By discretizing scaling

parameters and translating parameters, Ψ

j,k

(t) is

represented as

.)2(2)(

2/

,

ktt

jj

kj

−=

−−

ψ

ψ

(2)

The variables j and k are integers that scale and

translate the mother wavelet Ψ(t) to generate

wavelets. The scaling index j indicates the wavelet’s

width, and the translating index k gives its position.

By using (2), wavelet coefficient d

j,k

is represented

as follows.

,)()(

,,

∫

∞

∞−

= dttxtd

kjkj

ψ

(3)

where x(t) is time-series joint-angle data. The

wavelet coefficient d

j,k

represents information at a

particular resolution (2

-j

) at a particular spatial

location (2

j

k) of x(t). Therefore, a frequency

component of x(t) corresponding to resolution 2

-j

can

be represented as follows.

.)()(

,,

∑

=

k

kjkjj

tdtx

ψ

(4)

x

j

(t) is called as level-j wavelet detail. High-level

wavelet details represent low frequency components.

Fig.2 shows time-series right-elbow angle data

acquired from motion capture system and its wavelet

details.

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

t

Original Level5 Level8

Figure 2: Right-elbow angle data of a dancer and its

wavelet details.

GRAPP 2006 - COMPUTER GRAPHICS THEORY AND APPLICATIONS

244

3.2 Energy Analysis

By using multi-resolution DWT, we can decompose

complex dance motion data into different frequency

components (wavelet details). However, not all the

components are important for evaluation of dance

skills; high frequency components (low-level

wavelet details) may contain only noise, whereas

low frequency components (high-level wavelet

details) may contain only too little and coarse

information.

Since a multi-resolution DWT conserves signal

energy (Walker, 1999), by comparing each

frequency component’s energy, we can select

components having enough information for

evaluation of dance skills.

Based on energy analysis, we define the

contribution ratio of the level-j wavelet detail as

,

x

c

j

j

−

=

x

d

(5)

where x is time-series angle data and d

j

is level-j

wavelet coefficient vector of x.

In this paper, wavelet details whose contribution

ratios exceed particular criteria are selected. Using

the selected wavelet details, we evaluate dance skills.

Figure 3: Overview of Multi-Resolution Analysis.

4 EVALUATION METHOD OF

DANCE SKILL

In order to evaluate dance skill, we employ (1) sense

of rhythm and (2) harmony of movement of all body

parts as evaluation criteria.

4.1 Evaluation Method of Sense of

Rhythm

Dancing to the rhythm of music is very important for

rhythmical dance. When a dancer moves his/her

limbs to the rhythm of music, each limb will draw a

periodic (proportion to beat-to-beat period)

trajectory.

In this paper, we propose an evaluation method of

sense of rhythm based on the autocorrelation

function and beat-to-beat interval information of

music.

Autocorrelation function of level-j wavelet detail

is defined as

,)()(

1

lim)(

2/

2/

0

,

∫

−

→

+=

T

T

jj

T

jxx

dttxtx

T

R

ττ

(6)

where x

j

(t) is level-j wavelet detail. R

xx,j

(τ) takes its

maximum at τ=0, and if x

j

(t) is periodic, R

xx,j

(τ)

attains its peak at τ=nT

j

, where T

j

is a period of x

j

(t)

and n is an integer.

Let us assume that beat-to-beat interval of music

is τ

b

, then autocorrelation function of motion data

moving perfectly to the rhythm of music will attain

its peak at τ=n τ

b

, where n is an integer. Therefore,

as a criterion for evaluating dancer’s sense of

rhythm, we employ the peak value of R

xx,j

(τ) and

difference between T

j

and τ

b

.

4.2 Evaluation Method of Harmony

of Movement of Each Body Part

Since dance is a gesture of the whole body, harmony

of movement of all body parts is essential. Therefore,

in this paper, we propose a method to evaluate

harmony of movement of each body part based on

the mutual-correlation function and beat-to-beat

interval information of music.

Let x

j

(t) and y

j

(t) be the level-j wavelet detail of

time-series angle data x(t), y(t) respectively. Then,

mutual-correlation function of x

j

(t) and y

j

(t) is

defined as follows.

.)()(

1

lim)(

2/

2/

0

,

∫

−

→

+=

T

T

jj

T

jxy

dttytx

T

R

ττ

(7)

If x

j

(t) and y

j

(t) are periodic, and move in

harmony with each other, R

xy,j

(τ) attains its peak at

τ=nT

j

, where T

j

is a period of x

j

(t), y

j

(t). Therefore,

as a criterion for evaluating harmony of movement

of body parts, we employ the peak value of R

xy,j

(τ)

and difference between T

j

and τ

b

.

Although there are many combinations of body

parts to calculate mutual-correlation functions, we

DANCE EVALUATION SYSTEM BASED ON MOTION ANALYSIS

245

adopted 24 pairs of body parts shown in Table 2

under consideration of dance motion's characteristics.

Table 2: Pairs of Body Parts for Evaluation of Harmony of

Movements.

Harmony of Arm Parts

ID Body Part A Body Part B

0 Right Elbow Right Upper Arm

1 Left Elbow Left Upper Arm

Harmony of Leg Parts

ID Body Part A Body Part B

2 Right Knee Right Thigh

3 Left Knee Left Thigh

Harmony of top and bottom Parts

ID Body Part A Body Part B

4 Right Upper Arm Right Thigh

5 Left Upper Arm Left Thigh

6 Right Elbow Right Knee

7 Left Elbow Left Knee

Harmony of right and left Parts

ID Body Part A Body Part B

8 Right Upper Arm

Left Upper

Arm

9 Right Elbow Left Elbow

10 Right Thigh Left Thigh

11 Right Knee Left Knee

Harmony of Diagonal Parts

ID Body Part A Body Part B

12 Right Upper Arm Left Thigh

13 Left Upper Arm Right Thigh

14 Right Elbow Left Knee

15 Left Elbow Right Knee

Harmony of Body-Angle and Limbs

ID Body Part A Body Part B

16 Right Elbow

17 Left Elbow

18 Right Upper Arm

19 Left Upper Arm Body Angle

20 Right Thigh

21 Left Thigh

22 Right Knee

23 Left Knee

5 EXPERIMENTS

In our experiment, we let 4 dancers (1 expert and 3

amateurs) dance (7 kinds of hip-hop dance) to the

rhythm of music, and acquired their dance motions

using Vicon612.

Based on the acquired three-dimensional

coordinate positions of markers, we calculate joint

angles (Fig.1, Table 1), and decompose the data into

different frequency components (wavelet details)

using multi-resolution DWT. After selecting

important wavelet details by contribution ratios, we

evaluate each dancer’s skill.

5.1 Subjective Evaluation of Dance

Firstly, in order to confirm the subjective difference

in dance skill between an expert and an amateur, we

carried out a subjective evaluation experiment using

20 evaluators. We showed captured dances to

evaluators at random, and let them select the best

dance. As a result, all evaluators selected the

expert’s dance as the best one. This result shows that

there is a significant difference between the expert’s

dance and amateurs’ dance from the subjective

viewpoint. In the following sections, we try to

quantitatively evaluate this difference.

5.2 Evaluation Result of Sense of

Rhythm

Fig.4, 5 show the expert’s autocorrelation functions

(level-4, 6). X axis represents joints (numbers on x

axis correspond to ID in Table 1) and y axis is τ.

Colour of pixels in the figure represents the value of

autocorrelation functions. White pixels represent

strong positive correlation, black pixels represent

strong negative correlation, and gray pixels represent

week correlation. Dashed lines in the figure

represent beat-to-beat interval of the music.

Level-4 wavelet details capture short-period

motions (i.e. subtle motions), whereas level-6

wavelet details capture long-period motions (i.e.

general motions). As shown in Fig.4, 5, each

autocorrelation function peaks at the same time, and

the period of each joint’s motion is almost equal to

beat-to-beat interval of the music; i.e. the expert

moves her body parts completely to the rhythm of

music. These results show that the expert pays

conscious attention to subtle dance motions as well

as general dance motions.

GRAPP 2006 - COMPUTER GRAPHICS THEORY AND APPLICATIONS

246

Figure 4: Expert’s autocorrelation functions (Level-4).

Figure 5: Expert’s autocorrelation functions (Level-6).

Fig.6, 7 show an amateur’s autocorrelation

function (level-4, 6). Most pixels in Fig.7 are gray;

i.e. most of level-6 wavelet details of amateur’s

dance motion data are not periodic. However, as

shown in Fig.6, when focusing on short-period

motions, most of autocorrelation functions attain

their peaks at the same timing that corresponds to

the beat-to-beat interval. These results show that

whereas the amateur can move her limbs at each

moment, she can't pay conscious attention to long-

period dance motions.

Figure 6: Amateur’s autocorrelation functions (Level-4).

Figure 7: Amateur’s autocorrelation functions (Level-6).

5.3 Evaluation Result of Harmony of

Movement of Each Body Parts

Fig.8, 9 show expert’s mutual-correlation functions

(level-4, 6). X axis represents pairs of body parts

(numbers on x axis correspond to ID in Table 2) and

y axis is τ.

As shown in Fig.8, 9, each mutual-correlation

function peaks (bottoms) at almost the same time; i.e.

in expert’s dance, each body part moves to make a

good harmony with the other body parts.

DANCE EVALUATION SYSTEM BASED ON MOTION ANALYSIS

247

Figure 8: Expert’s mutual-correlation function (Level-4).

Figure 9: Expert’s mutual-correlation functions (Level-6).

Fig.10, 11 show the amateur’s mutual-correlation

functions (level-4, 6). Most pixels in Fig.10, 11 are

gray; i.e. in the amateur’s dance, most of body parts

move separately without considering harmony.

As shown above, the proposed evaluation

method shows that whereas the expert pays

conscious attention to subtle dance motions and to

the harmony of whole body parts, the amateur only

moves her limbs separately as an approximation of

dance.

Figure 10: Amateur’s mutual-correlation functions

(Level4).

Figure 11: Amateur’s mutual-correlation functions

(Level6).

5.4 Comparison: Evaluation Result

WITHOUT Multi-Resolution

DWT

To evaluate efficiency of multi-resolution DWT, we

also evaluate harmony of movement of all body

parts using “normal (without DWT)” mutual-

correlation functions as a comparison.

Fig.12 shows the expert’s mutual-correlation

functions (without DWT), and Fig.13 shows the

amateur’s. Whereas we can see clear difference

between Fig.8, 10 and Fig.9, 11, there is little

difference between Fig.12 and Fig.13. Most pixels in

Fig.12, 13 are gray; without multi-resolution DWT,

no strong motion correlation has appeared on dance

motion data.

GRAPP 2006 - COMPUTER GRAPHICS THEORY AND APPLICATIONS

248

Figure 12: Expert’s mutual-correlation functions

WITHOUT DWT.

Figure 13: Amateur’s mutual-correlation functions

WITHOUT DWT.

These results show that complexity of dance

motion would give a negative effect to motion

correlation analysis. By decomposing complex

dance motions into simple motion components using

multi-resolution DWT, we can cancel the negative

effect to motion correlation analysis.

Fig.14 shows the expert’s mutual-correlation

function with DWT between left-thigh and body

angle, and Fig.15 shows the amateur’s. X axis

represents level of wavelet details, and y axis is τ.

As shown in Fig.14, by decomposing complex dance

motions into simple motions, strong correlations

have appeared on the expert's dance. In contrast, as

shown in Fig.15, no strong correlation has appeared

on the amateur's dance. These results show that

decomposition of complex dance motions by DWT

is indispensable to evaluate dance skills.

Figure 14: Expert’s mutual-correlation functions between

left-thigh and body angle with DWT (Level 4-6).

Figure 15: Amateur’s mutual-correlation functions

between left-thigh and body angle with DWT (Level 4-6).

6 DANCE EVALUATION SYSTEM

As discussed in Sec.5, by using the proposed method,

we can evaluate dance skills. However, it is difficult

to instinctively know bad points of a dance from

patterns like Fig.10. Therefore, in this paper, we

score an amateur’s dance skill by comparing

evaluation results of the amateur (e.g. Fig.10) to

those of the expert (e.g. Fig.8) by DP matching

(Cormen, 2001). By mapping motion-captured dance

data and its scoring result onto the 3-D CG figure,

we have developed a computer-aided edutainment

system for dance (Fig.16). Colour of balls attached

to each joint of 3D CG figure shows evaluation

result; blue balls represent a good score, and red

balls represent a bad score. By using our system,

amateurs are able to visually know the bad point of

DANCE EVALUATION SYSTEM BASED ON MOTION ANALYSIS

249

their dance, and to check their dance from any

viewpoint in a 3D CG space.

7 CONCLUSION

In this paper, we have developed an evaluation

method for rhythmical dance based on wavelet

multi-resolution analysis and motion correlation

analysis. A dance motion is a mixture of various

kinds of motions, each having a different period.

This complexity would give a negative effect to

motion correlation analysis. Therefore, by using

wavelet multi-resolution analysis, we decompose

complex dance motion data acquired from a motion-

capture system into different frequency components.

And by applying correlation analysis to the

decomposed data, we extract motion features that

play a dominant role in evaluating sense of rhythm

and harmony of movement of each body part. By

comparing the extracted features of amateurs to

those of experts, we have achieved a quantitative

evaluation method for dance skills.

Using the proposed method, we have developed a

computer-aided edutainment system for dance. By

mapping motion-captured dance data and its

evaluation results onto the 3-D CG figure, our

system enables users to visually know bad points of

their dance.

Figure 16: Screen shot of computer-aided edutainment

system for dance.

Figure 17: Expert’s Dance and Amateur’s Dance.

ACKNOWLEDGEMENTS

This research was partially supported by Japan

Society for the Promotion of Science, Grant-in-Aid

for Scientific Research (B), 1600038, 2004, and a

grant for “Research on Interaction Media for High-

Speed and Intelligent Networking” from the

National Institute of Information and

Communications Technology, Japan.

REFERENCES

Naemura, M. and Suzuki, M., 2005. Extraction of

Rhythmical Factors on Dance Actions through Motion

Analysis, In Proc. IEEE WACV2005.

Oshima, C., et al., 2004. Family Ensemble: A

Collaborative Musical Edutainment System for

Children and Inexperienced Parents, In Proc. ACM

MM2004.

Soga, A., et al., 2001. Motion Description and Composing

System for Classic Ballet Animation on the Web. In

Proc. 10

th

IEEE Roman.

Shiratori, T., et al., 2004. Detecting Dance Motion

Structure through Music Analysis, In Proc. 6

th

IEEE

International Conference on Automatic Face and

Gesture Recognition.

Hachimura, K., et al., 2005. Analysis and Evaluation of

Dancing Movement Based on LMA, In Proc. IEEE

International Workshop on Robots and Human

Interactive Communication.

Naugle, L. M., 1999. Motion Capture: Re-Collecting the

Dance, In Proc. Twenty-First Biennial Conference,

International Council of Kinetography

Laban/Labanotation.

Laban, R., 1963. Modern Educational Dance, Macdonald

and Evans Ltd.

Bartenieff, I. and Lewis D., 1980. Body Movement -

Coping with the environment, Gordon & Breach

Publishers.

Camurri, A., et al., 1999. KANSEI analysis of dance

performance, In Proc IEEE SMC '99.

Nakata, T., 2005. Behaviour Recognition by Temporal

Segmentation, In Proc. 32nd SICE Symposium on

Intelligent Systems.

Mallat, S.G, 1989. A theory for multiresolution signal

decomposition: the wavelet representation, IEEE

Transactions on Pattern Analysis and Machine

Intelligence, vol. 11, Issue 7.

Walker, J.S, 1999. A Primer on Wavelets and Their

Scientific Applications, CRC Press.

Cormen, T.H., et.al. 2001, Introduction to Algorithms,

MIT Press.

GRAPP 2006 - COMPUTER GRAPHICS THEORY AND APPLICATIONS

250