A Proposal for the Classiﬁcation of Virtual Character

Plessiet C

edric

, Gagner

e Georges

and Sohier Remy

INREV EA4010 AIAC(EA4010), Universit

e Paris 8,2 rue de la libert

e, Saint-Denis, France

enes du monde- cr

eation- savoirs critiqu

es(EA1753), Universit

e Paris 8,2 rue de la libert

e, Saint-Denis, France

Keywords:

Virtual Actor, Virtual Reality, Artiﬁcial Intelligence.

Abstract:

This paper proposes a classiﬁcation of virtual actors based on the movement analysis of their creation medium

that we call matrix sculpture. The analysis is based on two concepts: the origin, either internal or external,

of movement and decision. It leads to deﬁne four categories, the virtual puppet,the virtual golem, the virtual

actor and the virtual mask.

1 INTRODUCTION

Nowadays, it is common to meet ”virtual actors”, ei-

ther in movies, video games or digital worlds. Some

are referred to as avatars, others as bots, others as dig-

ital humans or virtual companions, the terms being

used most often according to the context in a empiri-

cal way.

Would it not be possible to establish a precise

classiﬁcation that would allow to class and compare

these different virtual entities? This article propose

a nomenclature based on two concepts: the origin of

movement and the origin of the decision, that provide

with an effective grid for analyzing these virtual enti-

ties.

2 THE THALMAN

CLASSIFICATION

Nadia and Daniel Thalman had established in the ar-

ticle (Thalmann, 1996) a classiﬁcation of virtual enti-

ties based on 4 categories:

• participants whose gestures are reproduced ”in

the same way” that of the real manipulator thanks

to a motion capture device,

• guided actors whose movements are controlled by

an external manipulator thanks to an input device

(keyboard, mouse, etc.)

• autonomous actors able to interact with their en-

vironments through the use of ”simulated senses”

• interactive perceptive actors able to interact with

other actors, real or virtual

We see a strong link between participants, guided

actors and the notions of mocaptor and manipulactor

described respectively in (Gagner

e et al., 2018a) and

(Gagner

e et al., 2018b) in a research about theater and

avatar. The mocaptor controls a virtual puppet in a

mimetic way through a capture device , while the ma-

nipulactor controls the same virtual puppet through

other devices.

The boundary between autonomous actors and in-

teractive perceptive actors seems more blurred. And,

if this classiﬁcation is effective in describing their

work, it seems to us that it is not necessarily suit-

able for more widespread use. For example, in this

classiﬁcation, what differentiates a virtual entity from

a video game Creatures(Grand and Mindscape, 1996)

whose intelligence interacts with the player and learns

through a artiﬁcial neural network, and a non-player

scripted character that we may encounter in an appli-

cation like Facade(Mateas and Stern, 2004)?

3 MATRIX SCULPTURE

In our classiﬁcation, we assume that all these virtual

entities are above all a geometric shape, which we call

matrix sculpture in order to emphasize that this geo-

metric shape does not express a ﬁxed pose as for a real

sculpture, but a shape that can express multiple poses

by adding controllers capable of manipulating it(see

ﬁgure 1).

The matrix sculpture must therefore be thought

in terms of topology (Raitt and Minter, 2000) with a

view to a future deformation. On one hand, the move-

ment controllers receive data from sources that can be

168

Cédric, P., Georges, G. and Remy, S.

A Proposal for the Classiﬁcation of Virtual Character.

DOI: 10.5220/0007569401680174

In Proceedings of the 14th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2019), pages 168-174

ISBN: 978-989-758-354-4

Figure 1: A matrix sculpture with its animation mechanism.

situated in relation with the sculpture matrix. Either

the source has no relation with it and could be consid-

ered as external, or the source is partially inﬂuenced

or totally produced by process dependent of the ma-

trix sculpture. We speak then of internal (or partially

internal) sources of movement.

On the other hand, the animated matrix sculpture

is placed in a digital world. We can build a mod-

elization of this world and endow the sculpture with

an algorithm able to interpret this modelization in or-

der to adapt its movement. We call decision of the

movement this process of contextualizing the move-

ment according to an interpretation of a modelization.

When the modelization and the interpretation algo-

rithms are ﬁxed and independent of the matrix sculp-

ture, we say that the movement decision is external.

When the matrix sculpture can modify either the in-

terpretation algorithm or/and the world modelization,

we speak about internal decision. Our classiﬁcation

consequently derives from two fundamental dimen-

sions, source and decision of movement, that we map

on two axes as following (see ﬁgure 2):

Figure 2: Mapping the matrix sculpture on movement and

decision axes.

4 VIRTUAL PUPPET

Adding controllers to the matrix sculpture gives ca-

pacity of movement to this formal envelope originally

based on aesthetic criteria and dynamic possibilities.

Controllers need sources of data, that we consider ﬁrst

without any connection with the sculpture matrix. For

instance, the manipulative sources may have both hu-

man and algorithmic origins, external to the envelop :

a synthesized image character animated by keyframe

animators, a digital lining controlled by a mocaptor

wearing a motion capture suit or a video game player

character.

Moreover, the relationship between the matrix

sculpture and the world in which it is moving only

derives from the movement sources. The sculpture

matrix has neither perception of its environment, nor

any mean of inﬂuencing it. In other words, the con-

textualization process that we called decision is exter-

nal to the matrix sculpture. We nevertheless consider

that some external decisions are closer to the point of

view of the matrix sculpture than others. For instance,

the mocaptor who controls a sculpture matrix with his

own body through a motion capture suit approaches

a better embodiment of it that the manipulactor that

controls it with his hands and devices ( see ﬁgure 3).

Figure 3: A mocaptor and its virtual puppet.

We therefore understand that the embodiment will

have a signiﬁcant impact on the control. We deﬁne the

virtual puppet as a entity that has no decision-making

capacity and is manipulated from the outside by an-

other entity (human or algorithmic). There are three

ways to handle it:

• deliberate manipulation: the manipulator is aware

of his manipulative power, he positions himself as

A Proposal for the Classiﬁcation of Virtual Character

169

Figure 4: The virtual puppet: decision and movement of

external origin.

the user of a tool because he knows that his actions

lead to a response from the virtual puppet.

• unintentional manipulation: the manipulator does

not know that he animates the virtual puppet, and

therefore does not have a grid of correspondences

between his and its gestures.

• tamed manipulation: the manipulator becomes

aware of his manipulative power and understands

the relationships between his manipulations and

the movements of the puppet as he interacts with

The puppeteer is therefore closely linked to his

virtual puppet, which is incapable of movement initia-

tive or decision-making initiative. And the manipula-

tion requires an empathic attention because he has to

”put himself in the place of his virtual puppet” (Gag-

ner

e et al., 2018).

5 VIRTUAL GOLEM

The virtual puppet needed an external entity as a

source of movement and process of decision. The

concept of virtual golem is based on a fundamental

difference: it can ﬁgure out its environment and re-

act in consequence following external orders. This

makes it able to produce partially internal sources for

its movements inside the environment (see ﬁgure 5).

Figure 5: The virtual golem: decision of external origin,

movement of internal origin.

The virtual puppet has a system for receiving

movement input sources from external manipulators,

an output actuation mechanism, and a system for

matching inputs and outputs. The architecture of a

virtual golem is more complex and consists in :

• A mechanism for listening to an environment

modelization

• A mechanism for listening to the manipulator

• A mechanism for interpreting its environment

• An order execution mechanism.

• An actuation mechanism

The virtual golem is equipped with a modelization

of its environment and with algorithms able to receive

external instructions and produce internal movement

answers in relation with the modelization. Even if

these internal answers are generally mixed with ex-

ternal sources, it makes the virtual golem able to par-

tially move by itself, that was impossible for a virtual

puppet. In any case, the environment modelization

and the interpreting and instructing algorithms remain

under the external control of the manipulator, depriv-

ing it of any decision process. As the mythological

ﬁgure, the virtual golem only follows orders without

free will.

Figure 6: The three rules of cohesion separation and align-

ment that boids follow(Reynolds, 1987).

Thus we can store video game boxes based on a

number of predeﬁned algorithms (path ﬁnding, be-

havior tree), as well as swarm systems. Sometimes,

even if the virtual golem only executes more or less

complex orders after analyzing its environment, we

HUCAPP 2019 - 3rd International Conference on Human Computer Interaction Theory and Applications

170

observe behavioral phenomenon, as for boids (see

6), due to the multitude of entities (Reynolds, 1987),

or, as for chat-bots, due to the multitude of imple-

mented rules in syntax analyses such as AIML (Wal-

lace, 2003) or Eliza (Weizenbaum, 1966). Indeed,

even simple virtual golems like the entities of the

game of life, create a chaotic system that makes them

appear as impredictable complex structures.

6 VIRTUAL ACTOR

The virtual golem obeys orders, obediently without

questioning them. It only unrolls an algorithm, pro-

cessed in an modelized environment, without any pos-

sibility to change it neither the environment. An entity

able to add its own rules and change the environment

modelization would become able to take decisions.

We are talking here about learning in the sense of the

memorization processes used by animals or humans

to develop or modify speciﬁc behavioral patterns un-

der the inﬂuence of their environment and experience.

We realize that there is indeed a memorization

process but that there is also the inﬂuence of the en-

vironment and the experience. This entity must be

able to conduct an experiment, that is to test, evaluate

if there is success or failure, and from there draw its

own conclusions. True learning requires some form

of understanding and assimilation, or even question-

ing through experimentation, in order to transform a

belief into an ontology.

Figure 7: The virtual actor: decision and movement of in-

ternal origin.

A virtual actor would therefore be an entity able

to generate its movement like a virtual golem. But

when the latter can’t take any decision, the virtual ac-

tor can change the algorithms modelizing the environ-

ment and giving instructions for movement (see ﬁgure

7). Therefore the virtual actor becomes his own arbi-

trator. He is able to generate his decision that triggers

his movement and reciprocally a movement will make

him generate a decision in an enacted step. The vir-

tual actor gains in complexity, but also in autonomy.

In a certain way, a complete internal decision process

endows the matrix sculpture with a free will.

Figure 8: Karl Sims’ creatures(Sims, 1994) and Michel

Bret’s dancer(Thirioux et al., 2009).

Thus Karl Sims’ creatures (Sims, 1994), or Michel

Bret’s characters (Thirioux et al., 2009) or more re-

cent behavioral animation works based on genetic al-

gorithms, or artiﬁcial neural networks fall into this

category because they develop their own responses

and strategies in interaction with their environment.

7 VIRTUAL MASK

A fourth entity capable of decision-making but no

movement could be ﬁgured out when we study the

concept of autonomy by necessity that Jacques Tis-

seau uses in his description of different forms of au-

tonomy (Tisseau, 2004) :

• Autonomy in essence associated with living organ-

isms and animates

• Autonomy by necessity which is based on a model

that constrains movement, and adapts to changes

in the system.

A Proposal for the Classiﬁcation of Virtual Character

171

• Autonomy by ignorance whose behavior is not

predictable and tends to organize by itself towards

a goal (teleological behavior).

In autonomy by necessity, the ”model”, external to

the entity, manipulates the entity based on the infor-

mation contained in the model. We propose to call

this last category virtual mask, noting that in a num-

ber of cultures the mask represents a superior entity

(god, spirit...) that takes possession of its carrier (see

ﬁgure 9).

Figure 9: The virtual mask: decision of internal origin,

movement of external origin.

The gravity could be considered as a virtual mask

controlling a rag-doll, that is a matrix sculpture

equipped with actuators organized with external rules.

In this case, an untouchable entity manipulates the

matrix sculpture as a dynamic motor. The virtual

mask could also interact with the three other entities,

either virtual actor, virtual puppet or virtual golem.

For example, the sims video game is based on the

joint use of these two categories, the golem for the

different bots encountered, and the masks for intelli-

gent objects that control the different characters.

8 DISCUSSION

This classiﬁcation made it easier to deﬁne the er-

gonomic, structural and conceptual issues that we

encountered in various projects. We point out that

we could have entities capable of oscillating between

puppet, golem and actor. The Cigale project (see ﬁg-

ure 10) (Batras et al., 2016) entails an entity that is:

• a puppet, when controlled by a kinect capture de-

vice,

• a golem when it replays pre-recorded animations

• an actor when it generates new populations of

movements by genetic algorithm.

Should we consider these three states as different

entities or as an entity which plays different charac-

ters? Indeed an virtual actor can play the role of a

Figure 10: Interacting with the Cigale project avatar.

puppet or a golem, but the converse is not true : a

puppet cannot play the role of a golem, and a golem

cannot play the role of an actor (see ﬁgure 11).

Figure 11: Interweaving virtual puppet, golem and actor.

We have to quote Norbert Wiener(Wiener, 1964)

and Marvin Minsky(Minsky, 1988), pioneers of the

ﬁeld who codiﬁed forms of learning, which are cur-

rently questioned by deep neural network approaches.

The notion of virtual actor could be further broken

down by taking into account new paradigms leading

to a category called ”virtual being”, which we still

need to reﬂect on. This notion would perhaps make it

possible to better deﬁne virtual entities closer to arti-

ﬁcial life.

The notion of a mask is also subject to further in-

vestigation. Indeed, if in the case of dynamic systems

we are in the presence of a pure algorithmic model,

but for example, in the case of genetic algorithms we

can say that the mask is the ﬁtness function and the se-

lection method used to choose and construct the new

generation.

HUCAPP 2019 - 3rd International Conference on Human Computer Interaction Theory and Applications

172

9 APPLICATION OF THE MODEL

It is now possible for us to differentiate between

the entities of creature(Grand and Mindscape, 1996)

game and facade game. Creatures’s Norns(see ﬁgure

12) are therefore learning entities that evolve as they

interact with the environment and the player. These

entities are at the origin of both their decision and

their movement, so they are virtual actors (Taylor,

1997).

Figure 12: the Norn, characters of the game crea-

tures(Grand and Mindscape, 1996).

On the other hand, the characters of Fa-

cade(Mateas and Stern, 2004) follow an established

program based on behaviour trees and natural lan-

guage recognition, so they evolve within a precise

framework and obey their program, they are there-

fore generators of their movement, but the decision-

making process is the result of orders transcribed

in the form of an algorithm by the creators of the

game(see ﬁgure 13).

Figure 13: The two main character of Facade(Mateas and

Stern, 2004).

If we have to analyze the game the sims(Wright

and Maxis, 2000), we discover a more complex struc-

ture. The sims themselves are golems that follow and

evolve as part of their program. But the objects them-

selves act as principals: in fact they are the ones that

contain the information of animations and decisions

(Forbus and Wright, 2001). in this case we can see

Figure 14: The Sims(Wright and Maxis, 2000).

them as masks, because they have a decision-making

mechanism, but must use an external entity to gener-

ate the movement (see ﬁgure 14).

Figure 15: Siren from Epic Game (Cowley, 2018).

We can similarly describe the Siren digital dou-

ble of Epic Game(Cowley, 2018) (see ﬁgure 15) or

Senua from Ninja theory studio (Theory, 2016) as

virtual puppets, because the decision and movement

come from the actor who animates the virtual char-

acter. But we can also identify as a puppet an avatar

of the second-life virtual world or the character of the

pacman video games.

10 CONCLUSION

The reﬂection started with two questions:

• is the movement of the virtual entity controlled by

itself, or by an external input, human or not?

• does the virtual entity have free will, i.e. take its

own decisions, or does it obey to external orders?

The answers lead to a classiﬁcation of a virtual enti-

ties, allowing to describe their nature (see ﬁgure 16).

The classiﬁcation aims to provide both a tool and a

basis to deﬁne virtual entities. From our experience, it

helps to more effectively describe their relationship to

their virtual environment, their manipulator and their

programming rules.

Movement analysis of the matrix sculpture, a

model with control mechanisms and movement capa-

bilities, brings up four categories:

A Proposal for the Classiﬁcation of Virtual Character

173

Figure 16: virtual entities classiﬁcation.

• Virtual puppet whose decision to move and the

origin of the movement comes from an external

entity.

• Virtual golem capable of moving by itself but

without free will.

• Virtual actor who has a freedom of thought and a

capacity for movement by its own will.

• Virtual mask which is an external entity capable of

making decisions, but which manipulates a third

person (such as a puppet, golem or other actor).

This classiﬁcation helps to differentiate an avatar

(virtual puppet), a bot (virtual golem), or a creature

from artiﬁcial life (virtual actor).

REFERENCES

Batras, D., Guez, J., J

ego, J.-F., and Tramus, M.-H. (2016).

A virtual reality agent-based platform for improvisa-

tion between real and virtual actors using gestures. In

Proceedings of the 2016 Virtual Reality International

Conference on - VRIC ’16, pages 1–4.

Cowley, D. (2018). Siren at FMX 2018: Crossing the Un-

canny Valley in Real Time.

Forbus, K. D. and Wright, W. (2001). Some notes on pro-

gramming objects in The Sims. Northwestern Univer-

sity.

Gagner

e, G., Plessiet, C., Lavender, A., and White, T.

(2018a). The challenges of movement quality using

motion capture in real-time theatrical performance.

In Proceedings of the 5th International Conference

on Movement and Computin, pages 44–53, Genoa

(Italia). ACM.

Gagner

e, G., Plessiet, C., and Sohier, R. (2018b). Avatar

Staging : an evolution of a real time framework for

theater based on an on-set previz technology . In

Proceedings of the 2018 Virtual Reality International

Conference on - VRIC ’18, page 4. ACM.

Gagner

e, G., Plessiet, C., and Sohier, R. (2018). Intercon-

nected virtual space and theater: A research–creation

project on theatrical performance space in the network

era. In Challenges of the Internet of Things: Tech-

nology, Use, Ethics, volume 7, pages 165–190. Wiley

Online Library.

Grand, S. and Mindscape (1996). Creatures.

Mateas, M. and Stern, A. (2004). Natural language under-

standing in fac¸ade: Surface-text processing. In Inter-

national Conference on Technologies for Interactive

Digital Storytelling and Entertainment, pages 3–13.

Springer.

Minsky, M. (1988). Society of mind. Simon and Schuster.

Raitt, B. and Minter, G. (2000). Digital sculpture tech-

niques. URL, www. izware. com/news/indexa446.

html.

Reynolds, C. W. (1987). Flocks, herds and schools: A dis-

tributed behavioral model. ACM SIGGRAPH Com-

puter Graphics, 21(4):25–34.

Sims, K. (1994). Evolving virtual creatures. In Proceedings

of the 21st annual conference on Computer graphics

and interactive techniques - SIGGRAPH ’94, pages

15–22. ACM.

Taylor, D. (1997). Three ways to get a-life. IEEE Expert,

12(4):25–30.

Thalmann, D. (1996). A New Generation of Synthetic Ac-

tors: the Real-time and Interactive Perceptive Actors.

In Proc. Paciﬁc Graphics, volume 96, pages 200–219.

Citeseer.

Theory, N. (2016). From Previs to Final in Five Minutes —

Siggraph 2016 — Full Presentation.

Thirioux, B., Jorland, G., Bret, M., Tramus, M.-H., and

Berthoz, A. (2009). Walking on a line: a motor

paradigm using rotation and reﬂection symmetry to

study mental body transformations. Brain and cog-

nition, 70(2):191–200.

Tisseau, J. (2004). Virtual reality and complex-

ity. Manifeste scientiﬁque de l’Ecole Nationale

d’Ing

enieurs . . . , pages 1–26. http://enib.fr/ tis-

seau/pdf/paper/manifeste.uk.pdf.

Wallace, R. (2003). The elements of AIML style. Alice AI

Foundation.

Weizenbaum, J. (1966). ELIZA- a computer program for

the study of natural language communication between

man and machine. Communications of the ACM,

9(1):36–45.

Wiener, N. (1964). God and Golem, Inc: A Comment on

Certain Points where Cybernetics Impinges on Reli-

gion. MIT press.

Wright, W. and Maxis (2000). the sims.

HUCAPP 2019 - 3rd International Conference on Human Computer Interaction Theory and Applications

174