SKETCHED INTERACTION METAPHORS FOR CHARACTER

ANIMATION

Konstantin Schauwecker, Simon van den Hurk, Wallace Yuen and Burkhard C. W

unsche

Department of Computer Science, University of Auckland, Private Bag 92019, Auckland, New Zealand

Keywords:

Sketch-based interfaces, Character animation, Skeletal animation.

Abstract:

The use of 3D virtual worlds is increasing rapidly and new tools are necessary to enable untrained users to cre-

ate 3D content and interact with it. In this paper we present and evaluate sketch-based interaction metaphors

for specifying complex animations of 3D skeletally animated models. Sketched interactions include bone ro-

tation, motion path translation, sequencing and synchronisation of animations, and an undo-functionality. The

sketches are drawn directly onto a model in a 3D view and are translated into time-dependent joint transforma-

tions. A user study demonstrates that the animation metaphors are intuitive, with the exception of animation

ordering. More work is necessary to synchronise animations better. Overall our research demonstrates that

sketched-based animations can be useful for applications requiring rapid prototyping containing a limited

number of joint animations. Examples are the programming of household robots and the creation of simple

animated scenes in education and social network applications.

1 INTRODUCTION

Animated 3D virtual environments are common in

computer games and movie special effects and are

increasingly used in other applications such as sci-

ence, engineering, training, education and social me-

dia. Traditional modelling and animation tools such

as “Maya” are extremely powerful, but have a steep

learning curve and are not suitable for inexperienced

users. With more and more 3D content now accessi-

ble to and created by general users, new intuitive an-

imation tools are required. Sketch-input is a promis-

ing approach because of its intuitive pen-and-paper

metaphor. Sketching avoids the necessity of 3D input

devices, does usually not require a 3D mental model,

is already supported by many social networking tools

such as MSN Messenger, Google Talk and Yahoo!,

and can be achieved with a wide variety of input de-

vices such as touch screens (Windows 7), interactive

white boards, and sketch pads.

In this paper we present and evaluate sketch-based

interaction metaphors for specifying complex anima-

tions of 3D skeletally animated models, such as char-

acters in computer games and virtual worlds, and real

or simulated robots and machinery. Section 2 reviews

previous work in sketch-based modelling and anima-

tion. Section 3 introduces the design of our animation

framework. Section 4 presents a user study and eval-

uates the effectiveness of our interaction metaphors.

The results are summarised in section 5. We conclude

the paper and suggest future work in section 6.

2 RELATED WORK

Sketch input has been used to animate models in

two ways: to sketch motion paths and to sketch key

poses which are translated into animations of charac-

ter components. Steger represents 2D motions with

directed motion paths. Disparate motions are syn-

chronised using events which are indicated by time

stamps along the motion paths (Steger, 2004). Motion

Doodles (Thorne et al., 2004) allow the user to sketch

a motion path for a sketched character which can con-

sist of up to seven components with predeﬁned func-

tionalities (head, body, arms, etc. ). The system parses

the motion path and maps it to a parameterised set

of 18 different output motions. Motion paths have

also been used for robot navigation (Sakamoto et al.,

2009). Additional control is achieved by stroke ges-

tures and sketching operation areas. A different ap-

proach is used for the “As-Rigid-As-Possible Shape

Manipulation” (Igarashi et al., 2005). The user can

animate a shape by selecting arbitrary points within

it and moving them, i.e., the user is effectively creat-

247

Schauwecker K., van den Hurk S., Yuen W. and C. Wünsche B..

SKETCHED INTERACTION METAPHORS FOR CHARACTER ANIMATION.

DOI: 10.5220/0003375702470252

In Proceedings of the International Conference on Computer Graphics Theory and Applications (GRAPP-2011), pages 247-252

ISBN: 978-989-8425-45-4

 2011 SCITEPRESS (Science and Technology Publications, Lda.)

ing motion paths for key points of the object which

is then deformed subject to an inherent rigidity con-

straint. The second approach for sketch-based ani-

mation is to draw key poses and extract motion from

them. This can be achieved by sketching skeletons (Li

et al., 2006) or body contours (Mao et al., 2007) for

key frames and interpolating them.

3 DESIGN

Our goal is to animate models represented by a joint

hierarchy. A typical example are skeletally animated

objects where surface vertices are deﬁned with re-

spect to rigid bones which are connected by joints.

Note that this also includes rigid objects such as

robots, where each component has a separate surface

ﬁxed to a (virtual) bone. For objects without bones a

skeleton can be generated using force ﬁelds (Liu et al.,

2003; Cornea et al., 2005), a mesh contraction ap-

proach (Au et al., 2008), or by embedding an existing

skeleton into the mesh (Baran and Popovi

c, 2007).

3.1 Sketching System

The interface of our prototype application consists of

two windows. The sketch window shows a static ver-

sion of the model, upon which the user sketches the

desired animations. The display window shows an an-

imated version of the model, obtained by interpret-

ing the user sketches. This leads to a system design

where the user is able to quickly sketch the desired

animation and examine it right away. If the user is

unhappy with the resulting animation, modiﬁcations

to the sketches can be made quickly and the effects

of the modiﬁed animation will be immediately dis-

played.

The ﬁrst step toward enabling a user to specify an-

imations by sketching is the creation of a sketching

system. A user should be able to freely draw any

desired shape with a mouse, a graphics tablet or a

touch screen, using the sketching application. Since

we want to animate a 3D-model, we must be able to

sketch animations in all three dimensions. We achieve

this by allowing the user to rotate the virtual camera

of the 3D environment and thus being able to sketch

from any camera orientation. All 2D-sketches the

user draws are mapped onto a plane in 3D space par-

allel to the view plane. The z-value of the plane is

determined by the model’s component closest to the

user sketch, which we assume is the component the

user wants to modify.

All points drawn by the user for a ﬁxed camera

orientation will have the same z-value. We hence

group all points of a stroke into one sketch object,

which holds the 2D points and additionally stores

the origin and axes of the mapping plane. This ap-

proach simpliﬁes the subsequent processing steps as

the sketches can still be treated as 2-dimensional ob-

jects. As result of deﬁning strokes with respect to

different view planes many existing sketch classiﬁca-

tion tools, such as Microsoft’s InkAnalysis API (Mi-

crosoft, 2010), can not be used directly since they

assume a single 2D canvas. For this reason and for

increased ﬂexibility we deﬁne our own customised

sketch classiﬁcation algorithms.

3.2 Sketch Classiﬁcation

Sketch classiﬁcation is achieved by grouping sketches

according to the current camera view, giving them a

time stamp to check temporal relationships, and by

computing the following geometric attributes: length,

curvature and number of direction changes of a

sketch; position of start and end point; distance be-

tween start and end point; and length, width and as-

pect ratio of the axis aligned bounding box (AABB)

of each sketch. The AABB is obtained by using the

gift-wrapping algorithm described in (Lambert, 2009)

to compute the convex hull of a sketch and from this

its bounding box (Eberly, 2009).

2-segment and 3-segment arrows are recognised

by considering all sketches for a current camera ori-

entation. For 2-segment arrows we ﬁrst search for

a sketch that resembles an arrowhead (sketch with

one direction change and similar segment lengths)

and then look for another curve-like sketch with an

end point close to the arrow head’s corner point.

A 3-segment arrow is recognised by three approxi-

mately straight lines (narrow AABB’s) with similar

end points. The longest sketch is the arrow direction

and the two shorter sketches must have similar lengths

and form appropriate angles with the long sketch.

Enclosures (closed sketches) are used to select ob-

jects and play an important role in animation ordering,

motion path animation, and animation grouping. For

animation ordering, short strokes are drawn in trans-

verse direction on an enclosure sketch. The number

of strokes determines the order at which the animation

should start. For motion path animation, an enclosure

is drawn that includes the entire skeletal structure, and

then an arrow is drawn to indicate the path that the

model should to be moved along. Finally, enclosures

are used for animation grouping, which works in con-

junction with animation ordering. This allows differ-

ent arrows to be grouped into one animation sequence

by encircling them together using the same enclosure,

so animations consisting of multiple sub-animations

GRAPP 2011 - International Conference on Computer Graphics Theory and Applications

248

can be created. Enclosures are characterised by a cir-

cular or oval structure (no inﬂection points) and by

an end point distance which is much smaller than the

sketch length.

A “scribble” sketch is used to delete other

sketches and hence “undo” previously deﬁned in-

structions if the user makes a mistake or wants to

change an animation. The concept extends the sketch-

ing methodology, is intuitive, and avoids the usage of

buttons, keystrokes, or speciﬁc sketch symbols, which

might be more complicated for a user to understand.

A “scribble” sketch is characterised by many direc-

tion changes and a large length compared to the cir-

cumference of its AABB.

3.3 Overlap Detection

The use of enclosures and scribbles requires the de-

tection of the sketches and objects they refer to. The

placement of sketches in three dimensions causes a

scribble or an enclosure to potentially cover sketches

that lie on different planes. In order to determine if a

sketch is overlapped by an enclosure or scribble, we

project the sketch onto the plane of the enclosure or

scribble. The projected points are then subjected to an

inside/outside test with regard to the bounding box of

the enclosure / scribble. If the number of points con-

tained by the enclosure or scribble exceeds a certain

threshold, then the sketch is considered to be over-

lapping. If it overlaps with an enclosure then it can

be modiﬁed with subsequent commands, if it overlaps

with a scribble then it is deleted.

3.4 Character Animation

Our goal is to animate skeletally animated objects,

which are deﬁned by a bone hierarchy connected by

joints. To make rotations easier, a local coordinate

system is deﬁned centered at the joint connecting the

current bone to its parent bone. The x-axis of this co-

ordinate system is aligned with the bone. The orien-

tation of the coordinate system will deﬁne the bone’s

rotation. A 4x4 homogeneous matrix is used to rep-

resent the translation of the coordinate system and its

rotation around the origin. We store one such ma-

trix for each bone and animate it by continuously

multiplying it with a transformation matrix. Rota-

tions around arbitrary axis are implemented using the

method described in (Owen, 2009). For long anima-

tion sequence a representation using quaternions is

preferably in order to avoid artifacts caused by accu-

mulating numerical errors (Kavan et al., 2007). To

map the animations of the skeleton to a 3D-model we

use Vertex Blending, in which every mesh vertex is

assigned to multiple bones, in order to reduce mesh

distortions in the joint regions. In order to ﬁnd the in-

ﬂuencing bones for a vertex, we calculate the shortest

distance away from any bone. If this shortest distance

is less than a threshold, then the bone has some inﬂu-

ence on the mesh vertex. The inﬂuence will reduce

with increasing distance.

In order to map sketches to animations we ﬁrst

ﬁnd the bone closest to an input sketch’s reference

point, which depends on the type of sketch, e.g., start

point of a single headed arrow. This is achieved by

tracing a ray from the sketch plane’s view point and

measuring its distance to each bone (Bourke, 2009).

For more details see (Schauwecker et al., 2011).

With the correct bone being selected, we can now

determine its rotation. The origin of rotation is always

the joint connecting it to its parent bone. The rotation

axis is the normal of the sketching plane. The rotation

angle is the angle between the vector from the joint

to the arrow tip and the vector from the joint to the

point on the bone closest to the reference point of the

sketch. The orientation of the rotation (clockwise or

counterclockwise) is determined from the order of the

joint point, the point on the bone closest to the sketch

reference point, and the arrow tip. If the points form a

clockwise order with respect to the sketch plane then

the rotation must be in clockwise direction.

3.5 Motion Paths

The purpose of motion paths in our application is to

allow the character model to walk arbitrary user de-

ﬁned routes on a terrain. This is achieved by drawing

an enclosure around the object to be animated and by

drawing the motion path as an arbitrarily curved line

with arrow head. The sketched curve is sampled and

approximated with a Catmull-Rom Spline curve p(t).

The character is moved along the curve by using its

parameter as time step, such that at time t the char-

acter will be rendered at position p(t). The character

is aligned with the curve’s tangent p

(t) using a sim-

ple coordinate system transformation. As a result the

character is always facing in “walking” direction. The

animation stops when the character reaches the tip of

the arrow.

3.6 Ordering Animations

More complex animations require simultaneous or se-

quential animations of multiple joints and/or charac-

ters. This means, a speciﬁcation of the ordering of

sketched animations is necessary.

We specify the order of sequential animations

by labelling the enclosures of an animated ob-

SKETCHED INTERACTION METAPHORS FOR CHARACTER ANIMATION

249

Figure 1: Sequential ordering of two motions of a character.

ject/component. The user must draw an enclosure

around a group of arrows sketches. A series of short

strokes intersecting the contour of the enclosure de-

ﬁnes the position of these animations in an animation

sequence as depicted in ﬁgure 1. Animations belong-

ing to the same enclosure are executed in parallel. A

user must be allowed to order arrows after they have

been sketched, and in particular the ordering must be

possible even after the view of the character has been

changed for sketching another animations.

We implement this idea using the overlap detec-

tion discussed in subsection 3.3. Whether or not a

stroke intersects the enclosure can easily be tested by

performing a 2D line intersection test with the stroke

and all the enclosure line segments. A sketch will be

considered to be a potential stroke if the ratio of its

bounding box is inside a deﬁned range and its length

is shorter than a given fraction of the enclosure size.

To allow the user to coordinate the movement

of different body parts of the character model, all

sketches with the same order will be executed at the

same time. This way, a user can synchronise the

movements of limbs by arranging the order of their

sub-animations. For this to work, we need to deﬁne

ﬁxed time slots for animating the sketches of each par-

ticular order. Thus, all animations of the same order

will be required to have the same length, which is cur-

rently 3 seconds. However, this value can be conﬁg-

ured and might have to be changed, depending on the

type of animation a user wants to sketch.

4 USER STUDY

The usability of the sketch-based animation prototype

was evaluated in a user study with 11 participants.

The majority of them were students. Seven partici-

pants were between 20 and 30 years old, and the rest

older. Due to the small sample size, the results de-

duced from this study are only indicative and serve as

a pilot study. For this user study, a list of commands

and instructions were provided to the participants to

gain a basic understanding of the use of the applica-

tion. Each participant had to solve seven tasks:

1. Use a single-headed arrow to deﬁne an arm mo-

tion of the character.

2. Use a double-headed arrow to deﬁne a repeated

arm motion of the character.

3. Use double-headed arrows to deﬁne a walking

motion of the character’s legs. Translation of the

character is not required.

4. Use a motion path to translate the character.

5. Use a circular motion path to move the character

around a circle.

6. Create a walking simulation by using a motion

path and animating the legs of the character.

7. Use animation ordering to put animations of two

different arms in a sequence, with one arm move-

ment performed after the other.

Experiment 1 and 2 were designed to ﬁnd out

whether our basic sketching metaphors are intuitive

and easy to use for animation. We recorded for all

experiments any problems participants had with iden-

tifying and specifying the correct viewpoint (track-

ball rotation) for drawing an animation sketch, and

whether the correct functionality was chosen. Exper-

iment 3 was designed to ﬁnd out whether animations

with multiple arrows and different rotation axes are

more difﬁcult. Experiment 4 and 5 were designed to

assess the ease of use of the motion path metaphor,

and whether specifying a desired conﬁguration causes

problems. Experiment 6 was designed to determine,

whether more strokes and metaphors inﬂuence the

difﬁculty in sketching animations. Finally, Experi-

ment 7 tests the intuitiveness of our tool for sequenc-

ing animations.

For all tasks we measured the time required for

completion and recorded any problems. After the us-

ability test, participants were given a post-study ques-

tionnaire for a qualitative assessment. The question-

naire consisted of free-form questions and statements

with answers on a seven-level Likert scale.

GRAPP 2011 - International Conference on Computer Graphics Theory and Applications

250

Table 1: Average completion time in minutes for each task

in the user study.

Task 1 2 3 4 5 6 7

Mean 2.45 2.77 4.60 2.13 2.27 3.40 3.00

Median 1.50 1.50 3.50 1.50 1.50 3.00 3.50

5 RESULTS

The time each participant required to complete the ex-

periments was recorded in 30 seconds intervals (0.5

minutes). Table 1 shows the mean and median time

we measured for each task.

The measurements for task 1 and 2 suggest that

the arrow metaphor is intuitive and easy to use. No

differences were observed for single and repeated mo-

tions (single-headed vs. double-headed arrows). Both

motion path experiments were also completed in a rel-

atively short time, which implies that it does not mat-

ter whether a user has to draw a random or a speciﬁc

path. Some problems were experienced with selecting

the correct bone of a character, but users usually ﬁg-

ured this out with some trial and error. No quantitative

assessment of the precision of a task was performed

because of the inherent inaccuracies of mouse input

for sketching.

Table 1 shows that Task 3 has the highest mean

and median time. This was partially due to problems

with rotating the character and selecting the correct

limb. However, the main factor was that participants

were unaware that the initial direction of a repeated

motion is determined by which head of the double-

headed arrow is drawn ﬁrst. Thus, several partici-

pants created animations in which both legs moved

synchronously. Task 6 required quite a long time, but

no particular additional problems were observed. The

main reason for the recorded time was the complexity

of the task which required rotations and zooms in or-

der to draw the required arrows for bone animations

and motion paths. The time measurements for Task

7 indicate that participants had problems creating or-

dered animations. This was expected to be the least

intuitive sketching metaphor. A common problem

was that users drew enclosures around bones rather

than around the arrows.

5.1 Questionnaire

We measured the participants’ perception of the intu-

itiveness and effectiveness of different sketched an-

imation controls using statements rated on a seven

level Likert scale ranging from “-3” (strong disagree-

ment) to “3” (strong agreement).

For each of the three animation controls “Ar-

rows for joint rotation”, “Arrows for motion path”,

“Sequencing of animations” responses were recorded

for statements regarding intuitiveness and satisfaction

with the achieved results. For example, in order to

evaluate the intuitiveness of arrows for joint rotation

we used the statement “Arrows were an intuitive way

to specify bone rotations around a joint”.

Table 2: Participant ratings of the intuitiveness and effec-

tiveness of different sketch-based animation controls [“-3”

(lowest) to “3” (highest)].

Intuitiveness Effectiveness

Animation Control Mean σ Mean σ

Joint Rotation 0.27 1.85 0.45 1.69

Sequencing 0.18 1.72 0.45 1.57

Motion Path 0.82 2.36 0.91 1.45

Table 2 indicates that overall the sketch-based an-

imations were regarded only as slightly intuitive and

effective, with the motion path sketch tool achieving

the highest score and the sequencing tool achieving

the lowest score. The results were surprisingly di-

verse, although users with previous experience with

modelling tools seemed to give higher scores (not

enough demographic data was recorded to conﬁrm

this). One surprising result was that the motion path

had the highest standard derivation, i.e., the strongest

difference in ratings. We suspect that this might have

to do with the required change in perspective and

zoom factor for drawing the motion path, and pos-

sibly with the lack of rendering of a ground plane.

Subsequent interviews with participants revealed that

some confusion existed between synchronisations of

motions (e.g., when drawing double-headed arrows)

and sequencing of motions. Also some users re-

marked they would have preferred more control about

synchronisation and sequencing of motions, e.g., by

clicking on a component to temporarily stop its mo-

tion. Overall, the participants’ opinion regarding the

ease-of-use of the application was slightly negative

(Mean -0.64, Standard Deviation 1.86). This seems to

contradict the results of the two previous questions, as

they indicated that the metaphors are moderately intu-

itive, and the tool is moderately effective in translating

sketches into the desire animation.

For a possible explanation consider the follow-

ing user comments (number of responses in brackets).

Positive aspects were: viewport control (3), simple

animation deﬁnition (4), motion path (1), and intuitive

(1). Negative aspects were: unintended animation (3),

poor sketch recognition (3), no undo function (4), no

scroll wheel for zooming (2), lack of animation syn-

chronization (1), need to rotate camera to animate (1).

SKETCHED INTERACTION METAPHORS FOR CHARACTER ANIMATION

251

6 CONCLUSIONS

We have presented a novel tool for deﬁning mod-

erately complex animations using sketch input. For

sketch recognition, our main animation metaphor, ar-

rows, are correctly mapped to the bones that the user

intends to move. In the resulting animation the bone

is rotated until it reaches the tip of the drawn arrow.

However, this approach has some draw-backs, as the

user is required to always move the camera to a per-

pendicular viewing direction, before sketching an an-

imation. Overall the animation controls were per-

ceived as moderately intuitive and effective, but the

perception of the ease-of-use of the application was

slightly negative. The small sample size of the user

study and insufﬁcient demographic data did not allow

us to make statements about how perception of the

tool differs between experienced and inexperienced

computer users. A positive aspect of the application is

that the animation controls are applied to object com-

ponents connected by joints and can hence be used

for a wide range of applications such as computer

generated characters, robots and machinery. A more

detailed discussion is given in (Schauwecker et al.,

2011). Future work includes an improved sketch

recognition, and giving users more control about the

range of motions and their synchronisation and se-

quencing.

REFERENCES

Au, O. K.-C., Tai, C.-L., Chu, H.-K., Cohen-Or, D., and

Lee, T.-Y. (2008). Skeleton extraction by mesh con-

traction. In SIGGRAPH ’08: ACM SIGGRAPH 2008

papers, pages 1–10, New York, NY, USA. ACM.

Baran, I. and Popovi

c, J. (2007). Automatic rigging and

animation of 3d characters. ACM Trans. Graph.,

26(3):72. http:// www.mit.edu/ ∼ibaran/ autorig/.

Bourke, P. (2009). The shortest line between two lines in 3d.

http://local.wasp.uwa.edu.au/∼pbourke/geometry/

lineline3d/.

Cornea, N., Silver, D., Yuan, X., and Balasubramanian, R.

(2005). Computing hierarchical curve-skeletons of 3d

objects. The Visual Computer, 21(11):945–955.

Eberly, D. (2009). Minimum-area rectangle containing

a convex polygon. http:// www.geometrictools.com/

Documentation/ MinimumAreaRectangle.pdf.

Igarashi, T., Moscovich, T., and Hughes, J. F. (2005). As-

rigid-as-possible shape manipulation. ACM Trans.

Graph., 24(3):1134–1141.

Kavan, L., Collins, S.,

ara, J., and O’Sullivan, C. (2007).

Skinning with dual quaternions. In I3D ’07: Proceed-

ings of the 2007 symposium on Interactive 3D graph-

ics and games, pages 39–46, New York, NY, USA.

ACM.

Lambert, T. (2009). Gift wrapping algorithm. http://

www.cse.unsw.edu.au/∼lambert/java/3d/

giftwrap.html.

Li, Q., Geng, W., Yu, T., Shen, X., Lau, N., and Yu, G.

(2006). MotionMaster: authoring and choreographing

Kung-fu motions by sketch drawings. In Proceedings

of the 2006 ACM SIGGRAPH/Eurographics sympo-

sium on Computer animation, page 241. Eurographics

Association.

Liu, P., Wu, F., Ma, W., Liang, R., and Ouhyoung, M.

(2003). Automatic animation skeleton construction

using repulsive force ﬁeld. In Paciﬁc Graphics, pages

409–413. Citeseer.

Mao, C., Qin, S. F., and Wright, D. (2007). Sketch-based

virtual human modelling and animation. In SG ’07:

Proceedings of the 8th international symposium on

Smart Graphics, pages 220–223. Springer-Verlag.

Microsoft (2010). Ink Analysis Overview.

http://msdn.microsoft.com/en-us/library/

ms704040(VS.85).aspx.

Owen, G. S. (2009). 3d rotation. http:// www.siggraph.org/

education/materials/HyperGraph/modeling/

mod

tran/ 3drota.htm.

Sakamoto, D., Honda, K., Inami, M., and Igarashi, T.

(2009). Sketch and run: a stroke-based interface for

home robots. In CHI ’09: Proceedings of the 27th in-

ternational conference on Human factors in comput-

ing systems, pages 197–200. ACM.

Schauwecker, K., van den Hurk, S., Yuen, W., and

unsche, B. C. (2011). Sketched interac-

tion metaphors for character animation. Graph-

ics group technical report #2011-001, Department

of Computer Science, University of Auckland.

http://www.cs.auckland.ac.nz/∼burkhard/Reports/

GraphicsGroupTechnicalReport2011 001.pdf.

Steger, E. (2004). Sketch-based animation language. http://

www.cs.toronto.edu/∼esteger/sketchlang/index.html.

Thorne, M., Burke, D., and van de Panne, M. (2004).

Motion doodles: an interface for sketching charac-

ter motion. ACM Transactions on Graphics (TOG),

23(3):424–431.

GRAPP 2011 - International Conference on Computer Graphics Theory and Applications

252