A FRAMEWORK FOR TELEPRESENT GAME-PLAY

IN LARGE VIRTUAL ENVIRONMENTS

Patrick R

oßler, Frederik Beutler and Uwe D. Hanebeck

Intelligent Sensor-Actuator-Systems Laboratory

Institute of Computer Science and Engineering

Universit

at Karlsruhe (TH)

Karlsruhe, Germany

Keywords:

Extended Range Telepresence, Motion Compression, Virtual Reality.

Abstract:

In this paper we present a framework that provides a novel interface to avatar control in immersive computer

games. The user’s motion is tracked and transferred to to the game environment. This motion data is used as

control input for the avatar. The game graphics are rendered according to the avatar’s motion and presented to

the user on a head-mounted display. As a result, the user immerses into the game environment and identiﬁes

with the avatar. However, without further processing of the motion data, the virtual environment would be

limited to the size of the user’s real environment, which is not desirable. By using Motion Compression, the

framework allows exploring an arbitrarily large virtual environment while the user is actually moving in an

environment of limited size. Based on the proposed framework, two game applications were implemented, a

modiﬁcation of a commercially available game and a custom designed game. These two applications prove,

that a telepresence system using Motion Compression is a highly intuitive interface to game control.

1 INTRODUCTION

Telepresence gives a human user the impression

of being present in another environment. This is

achieved by having a robot gather visual data of the

remote environment and present it to the user, who is

wearing a head-mounted display. The user thus per-

ceives the remote environment through the eyes of the

robot. In order to extend telepresence to an intuitive

user interface, the motion of the user’s head and hands

is tracked and transferred to the robot that replicates

this motion. As a result, the user identiﬁes with the ro-

bot, i.e., he is telepresent in the remote environment.

Of course, this technique is also applicable to vir-

tual reality games, where the user controls an avatar

instead of a robot. Using telepresence as an input to

the computer, the user experiences a high degree of

immersion into the game’s virtual environment and

identiﬁes fully with the avatar. Thus, telepresence

techniques provide an appropriate interface for intu-

itive avatar control.

Common input devices for avatar control like key-

boards, mice, joysticks, and game pads, all lack the

ability of controlling the avatar intuitively. A possible

approach for controlling an avatar in a game environ-

ment by means of immersive interfaces is Augmented

Reality. This approach is applied in the ARQuake

project (Piekarski and Thomas, 2002), where virtual

objects from the game are superimposed onto a real

environment. This system, however, only allows vir-

tual environments that feature the same layout as the

real environment.

CAVE Quake II (Rajlich, 2001) uses the CAVE En-

vironment (Cruz-Neira et al., 1993), where images are

projected onto walls of a box surrounding the user, in

order to provide a realistic impression of the ﬁrst per-

son game Quake II to the user. The motion of a tool

called wand is tracked for avatar movement. Inter-

faces like this, however, are known for producing an

impression that resembles ﬂying, rather than walking.

Other approaches use walking-in-place metaphors

(Slater et al., 1994) or complex mechanical setups

(Iwata, 1999), (Iwata et al., 2005) to allow free nat-

ural locomotion in virtual environments. However, it

is not known, that these systems feature the typical

user motion of computer games.

This paper presents a framework that combines im-

mersive computer games and extended range telep-

resence by means of Motion Compression (Nitzsche

et al., 2004). Motion Compression allows the user in

a conﬁned user environment to control the avatar in

an arbitrarily large virtual world by natural walking.

150

Rößler P., Beutler F. and D. Hanebeck U. (2005).

A FRAMEWORK FOR TELEPRESENT GAME-PLAY IN LARGE VIRTUAL ENVIRONMENTS.

In Proceedings of the Second International Conference on Informatics in Control, Automation and Robotics - Robotics and Automation, pages 150-155

DOI: 10.5220/0001165901500155

 SciTePress

Fig. 1 shows a user wearing a non-transparent head-

mounted display while playing an immersive game.

According to several sources, (Peterson et al.,

1998), (Darken et al., 1999), and (Tarr and Warren,

2002), there is evidence, that using full body motion,

e.g. normal walking, results in better navigation in

the virtual environment than using common input de-

vices. By using the approach presented in this paper,

the user is expected to feel present in the virtual world

and identify well with the avatar under control.

Figure 1: User playing an immersive game with a telepres-

ence interface.

The remainder of this paper is structured as fol-

lows. Section 2 reviews Motion Compression, as it is

a major part of the proposed framework. An overview

on the framework is given in section 3. Section 4 de-

scribes the tracking system and section 5 presents two

different games that use this framework. An experi-

mental evaluation of the suitability for intuitive avatar

control is given in section 6. Finally, conclusions are

drawn in section 7.

2 MOTION COMPRESSION

The Motion Compression algorithm transforms the

user’s position and orientation in the user environ-

ment, i. e., the physical world surrounding the user,

into the target environment, which in this applica-

tion is the virtual environment of the game (Fig. 2).

The target environment is perceived visually by the

user wearing a non-transparent head-mounted dis-

play, which makes the physical world invisible for

him. The effect is that he moves in the user envi-

ronment but feels present in the virtual environment

instead.

The Motion Compression algorithm is partitioned

into three functional modules: path prediction, path

transformation, and user guidance.

motion

data

visual

perception

user

Motion

Compression

target

environment

user

environment

Figure 2: Overview of the different Motion Compression

environments.

The path prediction unit predicts the path the user

wants the avatar to follow in the target environment.

This prediction is based on the user’s view direction

and, if available, additional information on the target

environment. The resulting path is called target path.

Path transformation maps the target path onto the

user path in the user environment. Since the user en-

vironment is in most cases smaller than the target en-

vironment, the target path cannot always be mapped

directly. Motion Compression aims at giving users

a realistic impression of the virtual environment by

keeping distances and turning angles in the user envi-

ronment and target environment locally equal. Thus,

only the curvature of the path is changed. A possible

target path and the corresponding user path is illus-

trated in Fig. 3. To give the user a realistic impression

of controlling the avatar, the resulting curvature devi-

ation is kept at a minimum.

4 m

8 m

7 m

(a) (b)

user environment target environment

Figure 3: User path (a) and corresponding target path (b).

When walking, humans continuously check if they

are on the direct way to their desired target and adjust

their direction accordingly. This behavior is exploited

in user guidance. While moving along, the avatar’s

orientation in the target environment is changed in

such a way, that the user follows the path by correct-

ing the perceived deviations.

As a result of the three processing steps Motion

Compression provides a linear, but location-variant

transformation from the user’s position to the avatar’s

position in the target environment.

A FRAMEWORK FOR TELEPRESENT GAME-PLAY IN LARGE VIRTUAL ENVIRONMENTS

151

3 SOFTWARE FRAMEWORK

We designed a software framework, which allows

connecting arbitrary game environments to Motion

Compression control. This framework, however, is

not limited to game applications, but can also be used

for controlling teleoperators in telepresence scenar-

ios (R

oßler et al., 2005). In order to provide an ex-

tensible interface that may be adapted to future ap-

plications, we decided to base the framework on the

well known CORBA middleware standard. Another

advantage of CORBA is, that it is platform indepen-

dent and available for virtually any programming lan-

guage.

As shown in Fig. 4, the core of the setup is a

CORBA server, the MC Server, which contains an im-

plementation of the Motion Compression algorithm.

In order to provide an up-to-date target position, the

server runs asynchronously and constantly accepts

updates of the user position from the tracking sub-

system, which acts as a CORBA client. Based on

these position updates MC Server calculates the cur-

rent transformation and target position. The target

position is made available to be fetched by the game

client.

HMD

tracking

software

tracking

hardware

user interface

avatar

motion

video

rendering

game applicatio

game-

play

user

guidance

path

trans-

formation

path

prediction

MC Server

user

Figure 4: Data ﬂow in the software framework.

The communication between the game application,

which is a CORBA client, and the MC Server requires

a data connection. This connection is established

when the game is started. During the game, the con-

nection is used to continuously refresh the target po-

sition and the avatar is moved accordingly. The con-

nection is maintained until the game is quit. The co-

operation of the game application and the MC Server

is illustrated in detail in Fig. 5. The game is also re-

sponsible for game-play and rendering the ﬁrst per-

son view of the game-environment. These rendered

images are displayed to the user on a head-mounted

display.

Game Client MC Server

connect

target position

get position

acknowledge

start

stop

loop

disconnect

initalize

Figure 5: Collaboration of the game client and the MC

Server module.

4 TRACKING SYSTEM

For the estimation of the user’s posture, i. e., trans-

lation and orientation, an acoustic tracking system

is used. This system consists of four loudspeakers,

which are placed in the corners of the user environ-

ment, emitting distinct acoustic signals.

These signals are received by four microphones at-

tached to the head-mounted display. In order to es-

timate the time delay between sending and receiving

the signal, the cross correlation between the ﬁltered

signal and the transmitted signal is calculated. The

estimated time delay is converted to the range based

on the velocity of sound.

Based on the arrangement of four loudspeakers and

four microphones 16 estimated ranges are available.

These range estimates are used in a gradient descent

algorithm to estimate the posture of the user’s head.

The initial values for the gradient descent algorithm

are calculated by means of a closed form solution pre-

sented in (Beutler and Hanebeck, 2005). The tracker

data is fused with information from a gyroscope cube

by means of a Kalman Filter, resulting in more accu-

rate estimates for the orientation. Fig. 6 shows the

setup of the tracking system attached to the head-

mounted display.

ICINCO 2005 - ROBOTICS AND AUTOMATION

152

Head-Mounted Display

Microphone

Gyroscope Cube

Gyroscope

Microcontroller

Figure 6: Top view on the hardware setup consisting of

four microphones and a gyroscope cube mounted on a head-

mounted display.

5 GAME APPLICATIONS

In order to prove the applicability of the proposed

software framework, two game applications were im-

plemented.

5.1 Quake

The ﬁrst game application is a modiﬁcation of the

commercially available ﬁrst person game Quake III

Arena (id Software, 2001). The resulting modiﬁca-

tion is called MCQuake.

In MCQuake the modiﬁcations include a CORBA

client that implements the interface required by the

software framework. In order to control the avatar,

MCQuake simulates normal user input from the po-

sitions received from MC Server. Based on the the

last position of the avatar and on the commanded po-

sition a motion vector is calculated, which is handed

to the game engine. The game engine now moves the

avatar accordingly. By doing a collision detection, it

prevents the avatar from moving through walls.

The height of the target position has also to be

mapped onto the avatar’s position in the virtual envi-

ronment. The virtual environment supports two kinds

of height information, which are mapped differently

as described below.

The ﬁrst kind of height information is the absolute

height of the avatar. This height is unrestricted al-

lowing the avatar to climb stairs. Absolute height is

handled by the game engine itself. If, for example,

the user maneuvers the avatar over a set of stairs, the

avatar’s absolute height changes with the height of the

ﬂoor beneath him.

The second kind of height information, called view

height, is relative to the ﬂoor the avatar moves on. In

the game, however, it is restricted to only two differ-

ent values used for crouched and normal movement.

A threshold applied to the user’s view height in the

user environment, determines which kind of move-

ment is to be used.

Given these mappings, the user is now able to con-

trol the avatar in an intuitive way in arbitrarily large

virtual environments by normal walking. Of course,

MCQuake also supports other kinds of motion, like

running and straﬁng, which are very common in ﬁrst

person games. For Motion Compression, there is no

difference between those and normal walking as mo-

tion is always handled as a sequence of position up-

dates.

5.2 PacMan

A second game application is a custom build ﬁrst

person telepresence version of the arcade game clas-

sic PacMan, called paMCan (Pa

cMan with a Motion

ompression driven artiﬁcial environment) as shown

ﬁn Fig. 7. While MCQuake is written in C, paMCan

is written completely in python. In order to obtain

high quality graphics output, cgkit (Baas, 2005) was

used as graphics back end. For paMCan the path pre-

diction module was modiﬁed in such a way, that it

uses not only view direction, but also the virtual map

layout.

Figure 7: An impression of the paMCan-game.

Motion commands are handled similar to MC-

Quake. The game application computes a motion vec-

tor and moves the avatar accordingly if no obstacles

block the way. However, if there are obstacles, the

motion vector is modiﬁed in accordance with the map

information. This prevents the avatar from moving

through walls. Although modifying the motion vec-

tor leads to a displacement of the commanded avatar

positions and the actual avatar position, this has no ef-

fect on user and game. In paMCan view height is the

only height information and is mapped directly.

A FRAMEWORK FOR TELEPRESENT GAME-PLAY IN LARGE VIRTUAL ENVIRONMENTS

153

6 EXPERIMENTAL EVALUATION

In order to gain a high degree of realism the setup

uses a high quality head-mounted display with a res-

olution of 1280 × 1024 Pixels per eye and a ﬁeld

of view of 60

◦

. Both the game engine and the MC

Server, run on a multimedia Linux-PC, which allows

a frame rate of approximately 80 images/sec for MC-

Quake and 60 images/sec for paMCan.

The acoustic tracking system currently provides 15

estimates per second for the position and 50 estimates

per second for the orientation, which is enough for

the given application. Fig. 8 illustrates a motion tra-

jectory recorded by the tracking system. The vec-

tors are directed in view directions. It can be ob-

served, that the tracking system has a good relative

accuracy, which is very important for the given appli-

cation to avoid shaking images. Absolute accuracy, or

ground truth, is of less importance. If the tracking sys-

x-axis in meters

Start position End position

Discarded

measurements

y-axis in meters

-2

1.5

-1.5

-1

0.5

-0.5

2.5

-2.5

Figure 8: The estimated translation sequences in a test run

with a predeﬁned motion trajectory.

tem detects outliers, it discards the measurement and

provides no estimate.The tracking system again pro-

vides reasonable estimates after some acoustic mea-

surements.

In order to test the users’ ability to navigate prop-

erly in the virtual environment, an environment well-

known to the users was chosen. Hence the map from

MensaQuake (The MensaQuake Project, 2002) was

loaded into MCQuake. This map is a realistic model

of the student cafeteria of the University of Karlsruhe

(Fig. 9).

The experiment compares the time a user needs to

navigate his avatar along a speciﬁed path in the virtual

cafeteria using MCQuake and Quake with keyboard

and mouse as inputs. In addition the user was asked

to walk the same path in the real cafeteria. In order to

avoid effects of user adaptation, a user was chosen for

Figure 9: Impression of the student cafeteria in MC-

Quake. (The MensaQuake Project, 2002)

the experiment, who was experienced in both, using

Motion Compression and Quake with standard input

devices. He was also familiar with the cafeteria. The

path was partitioned into three parts (a), (b), and (c),

as shown in Fig. 10.

5 m

(a)

(b)

(c)

down

Figure 10: Actual user path from the completion time ex-

periment in the virtual cafeteria.

Table 1 gives a comparison of the completion times

gathered in this experiment. When using Quake with

Table 1: Average time for a speciﬁed path from three runs

in the virtual and real cafeteria, respectively.

(a) (b) (c) total

Quake 4.8 s 4.7s 4.1 s 13.4s

real 8.9s 14.0 s 15.4s 38.2 s

MCQuake 15.0s 15.1 s 14.4s 44.5 s

standard inputs the avatar reaches his goal much faster

than in MCQuake. This is a result of unrealistically

high walking speed in standard Quake, even when

running mode is turned off. A comparison with walk-

ing in the real cafeteria shows, that MCQuake pro-

vides a more realistic motion. Thus, the gaming expe-

rience is more realistic than with common game con-

trol.

It can be observed, that users using Motion Com-

pression for the ﬁrst time start with a few cautious

ICINCO 2005 - ROBOTICS AND AUTOMATION

154

steps. After several minutes of adjustment, however,

they adapt to the system and are able to navigate intu-

itively through the target environment.

When playing paMCan the users adapted to the

system even faster. In fact, all three testers stated, that

they did not notice the inﬂuence of Motion Compres-

sion at all. This fact may be a result of paMCan’s dy-

namic environment, which provides much more dis-

tractions to the user than most other target environ-

ments. In paMCan the user has to collect pills, escape

from ghosts, and navigate through a narrow maze,

leaving him less time to focus on the inconsistency

of visual and proprioceptive feedback.

7 CONCLUSIONS

Telepresence techniques were designed for control-

ling robots remotely. Since the remote environment

can easily be replaced by a virtual environment, telep-

resence techniques can also be used to control an

avatar in a ﬁrst person game.

This paper presented a CORBA-based framework

for telepresent game-play in large virtual environ-

ments using Motion Compression. The algorithm al-

lows a user to control the avatar intuitively through

large virtual environments, by actual locomotion in a

limited user environment. This framework was tested

with two different game applications, MCQuake and

paMCan. Motion Compression proved to be very in-

tuitive as an input for the virtual reality games. As a

result, users had a realistic impression of the virtual

environment and, thus, experienced a high degree of

presence.

In order to give the users the possibility to experi-

ence the virtual environment with all senses, we will

implement a haptic feedback device, which allows to

feel obstacles and weapon recoil. This will lead to an

even higher degree of immersion.

The authors believe, that this new kind of gam-

ing experience will lead to a revolution in how peo-

ple experience computer games. We expect systems

like this to become omnipresent in gaming halls in

the next couple of years. As soon as the hardware

is affordable, people might even start installing these

systems in their homes.

ACKNOWLEDGEMENTS

The authors thank two teams of students for the im-

plementation of the games. Henning Groenda and

Fabian Nowak implemented MCQuake. paMCan was

written by Jens K

ubler, Jan Wassenberg, and Lutz

Winkler.

REFERENCES

Baas, M. (2005). cgkit – The Python Computer Graphics

Kit. http://cgkit.sourceforge.net/.

Beutler, F. and Hanebeck, U. D. (2005). Closed-Form

Range-Based Posture Estimation Based on Decou-

pling Translation and Orientation. In Proceedings of

IEEE Intl. Conference on Acoustics, Speech, and Sig-

nal Processing (ICASSP05), pages 989–992, Pennsyl-

vania, PA, USA.

Cruz-Neira, C., Sandin, D. J., and DeFanti, T. A. (1993).

Surround-Screen Projection-Based Virtual Reality:

The Design and Implementation of the CAVE. In

Proceedings of the 20th ACM Annual Conference on

Computer Graphics and Interactive Techniques (SIG-

GRAPH 1993), pages 135–142, Anaheim, CA, USA.

Darken, R. P., Allard, T., and Achille, L. B. (1999). Spa-

tial Orientation and Wayﬁnding in Large-Scale Virtual

Spaces ii. Presence, 8(6):iii–vi.

id Software (2001). Quake III Arena. http:

//www.idsoftware.com/games/quake/

quake3-arena.

Iwata, H. (1999). The Torus Treadmill: Realizing Locomo-

tion in VEs. IEEE Computer Graphics and Applica-

tions, 19(6):30–35.

Iwata, H., Yano, H., Fukushima, H., and Noma, H. (2005).

CirculaFloor. IEEE Computer Graphics and Applica-

tions, 25(1):64–67.

Nitzsche, N., Hanebeck, U. D., and Schmidt, G. (2004).

Motion Compression for Telepresent Walking in

Large Target Environments. Presence, 13(1):44–60.

Peterson, B., Wells, M., Furness III, T. A., and Hunt, E.

(1998). The Effects of the Interface on Navigation in

Virtual Environments. In Proceedings of Human Fac-

tors and Ergonomics Society 1998 Annual Meeting,

volume 5, pages 1496–1505.

Piekarski, W. and Thomas, B. (2002). ARQuake: The Out-

door Augmented Reality Gaming System. ACM Com-

munications, 45(1):36–38.

Rajlich, P. (2001). CAVE Quake II. http://brighton.

ncsa.uiuc.edu/

∼

prajlich/caveQuake.

oßler, P., Beutler, F., Hanebeck, U. D., and Nitzsche, N.

(2005). Motion Compression Applied to Guidance of

a Mobile Teleoperator. In Proceedings of the IEEE

Intl. Conference on Intelligent Robots and Systems

(IROS’05), Edmonton, AB, Canada.

Slater, M., Usoh, M., and Steed, A. (1994). Steps and Lad-

ders in Virtual Reality. In ACM Proceedings of VRST

’94 - Virtual Reality Software and Technology, pages

45–54.

Tarr, M. J. and Warren, W. H. (2002). Virtual Reality in

Behavioral Neuroscience and Beyond. Nature Neuro-

science Supplement, 5:1089–1092.

The MensaQuake Project (2002). MensaQuake. http:

//mensaquake.sourceforge.net.

A FRAMEWORK FOR TELEPRESENT GAME-PLAY IN LARGE VIRTUAL ENVIRONMENTS

155