ALUATION OF VISUALIZATION FEATURES IN

THREE-DIMENSIONAL LOCATION-BASED MOBILE SERVICES

ario Freitas

, A. Augusto Sousa

1,2

and Ant

onio Coelho

1,2

FEUP, Rua Dr. Roberto Frias, s/n 4200-465 Porto, Portugal

INESC Porto, Campus da FEUP, Rua Dr. Roberto Frias, s/n 4200-465 Porto, Portugal

Keywords:

Computer graphics, Visualisation paradigms, Location-based mobile services, Visual perception, Navigation

systems, Mobile 3D city maps, User interaction.

Abstract:

Nowadays, there is a wide range of commercial LBMS (Location-Based Mobile Services) available in the

market, mainly in the form of GPS-based navigation solutions, and a trend towards the display of 3D maps

can be clearly observed. Given the complete disparity of ideas and a visible commercial orientation in the

industry, the study of the visualisation aspects that inﬂuence user performance and experience in the explo-

ration of urban environments using 3D maps becomes an important issue. In this work, a generic conceptual

framework is proposed whose main purpose is to objectively evaluate the impact and contribution of the major

visualisation elements involved (henceforth mentioned as feature vectors). With this framework in mind, an

online questionnaire was developed and administered to 149 test subjects in order to measure the real impact of

feature vectors. The results clearly demonstrated that certain features have clear impact on user performance,

and should be taken in account in LBMS development. As an example, just by displaying buildings with a

3D appearance, subjects were able to match more accurately the real environment with the one presented on a

mobile device. In general, users were able to perform the tasks entrusted to them faster, if they were provided

more realistic imagery.

1 INTRODUCTION

The LBMS technology, namely in the form of GPS-

based navigation systems, has just recently reached a

state of technological maturity, enabling the develop-

ment of 3D map-based graphical interfaces. Nowa-

days, there is a wide offer of LBMS solutions in the

market, especially in the form of automotive navi-

gation systems. Motivated by commercial interests,

many of these products promise to offer the “best vi-

sualisation experience ever”, in search for a differ-

entiating factor from the competition. By looking at

the variety of visualisation paradigms being proposed,

one can clearly notice a great disparity of ideas with-

out a clear notion of its usefulness.

Provided the non-existence of an objective state-

of-the-art generalising theory capable of unifying and

evaluating all the visualisation elements and proper-

ties, the main motivation of this work is to study the

most relevant of these features and how to adjust them

appropriately, in order to maximise the usability of

mobile maps and to improve the navigation experi-

ence, in accordance with the following objectives:

1. Elicit and assess the state-of-the-art contributions

on visualisation paradigms of 3D maps, with par-

ticular interest on mobile services and devices;

2. Develop a methodology for evaluating the differ-

ent issues that inﬂuence user experience and per-

formance when exploring an urban environment

with mobile maps.

2 STATE OF THE ART

2.1 Visual Perception of Realism

The variety of free and commercial products featuring

three-dimensional map-based mobile services avail-

able to the masses, usually ranges from very abstract

to reasonably realistic and immersive visualisation

paradigms. However, there is a common misconcep-

tion on what is Image Realism, how is it visually per-

ceived, and how can it be effectively measured.

In (Rademacher et al., 2001), a scientiﬁc exper-

iment was conducted to understand what aspects of

328

Freitas M., de Sousa A. and Coelho A.

EVALUATION OF VISUALIZATION FEATURES IN THREE-DIMENSIONAL LOCATION-BASED MOBILE SERVICES.

DOI: 10.5220/0001805903280336

In Proceedings of the Fourth International Conference on Computer Graphics Theory and Applications (VISIGRAPP 2009), page

ISBN: 978-989-8111-67-8

an image can make it look “real” or “not real”, i.e.,

whether it is perceptually indistinguishable or not

from the corresponding photographs. The results

showed that subjects were not convinced by the in-

creasing number of light sources and shadows nor the

variety or number of shapes. The same could be said

for “perfectly sharp” shadows or “perfectly polished”

surfaces.

In (Lange and Ch, 2003), an experiment was car-

ried out with 75 test subjects to classify 90 images of

the virtual landscape of Brunnen / Schwyz (Switzer-

land) from three different viewpoints in a degree of

realism from 1 (very low) to 5 (very high). The

results generally demonstrated that the variable that

most contributed to the sense of realism was – by far

– the high-resolution orthophotographic imagery, and

the second most important being texture-mapping.

In other works like (McNamara et al., 2000), the

importance of perception-based image quality metrics

is studied, such as the ones given by the VDP (Visible

Differences Predictor) and the VDM (Visual Discrim-

ination Metric). These two metrics aim to analytically

predict the differences between a computer-generated

image and the photograph it depicts, taking into ac-

count the limitations of the human eye described by

the HVS (Human Visual System). The VDP quality

metric takes the two images as input and generates a

difference map that predicts the probability of the hu-

man eye ﬁnding differences between the two pictures,

as demonstrated in (Bolin and Meyer, 1999) (see Fig-

ure 1).

Figure 1: Difference map in the VDP quality metric (ob-

tained from the previously mentioned work).

A simpliﬁcation of the VDM quality metric was

provided by following a similar approach (Bolin and

Meyer, 1999): instead of ﬁnding a difference map, a

just noticeable difference map was proposed which

corresponds to a 75% probability of a person detect-

ing a difference between the two images (McNamara

et al., 2000).

Because of some controversy and no agreed-

upon standards for measuring realism in computer-

generated imagery, a conceptual framework for mea-

suring image realism and evaluating its usefulness

was proposed in (Ferwerda, 2003). The frame-

work distinguishes three different varieties of real-

ism: physical realism, photo-realism and functional

realism. However, this framework does not seem to

be enough to encompass the extents to which real-

ity or virtuality can be “augmented”. Accounting for

such circumstances, the concept of Virtuality Contin-

uum was introduced in (Milgram and Kishino, 1994)

as represented in Figure 2.

Figure 2: The Virtuality Continuum (obtained from the pre-

viously mentioned work).

At the left end, we have the “completely real”

Real Environment, which is made up of “real” objects:

“any objects that have an actual objective existence”.

At the right end, we have the “completely computer-

simulated” Virtual Environment, which is made up of

“virtual” objects: “objects that exist in essence or ef-

fect, but not formally or actually”.

2.2 User Tasks

The underlying basic equation that can help us ﬁnd

the “perfect” balance in map-based mobile services is

what could be called of Mobility Equation. This equa-

tion was ﬁrst formulated by Leonard and Durrant-

Whyte for mobile robot navigation (Borenstein et al.,

1996) but can be equally extended to human naviga-

tion. The equation is made up of the following three

questions:

• ‘Where am I?’

• ‘Where am I going?’

• ‘How do I get there?’

In (Hunolstein and Zipf, 2003), the tasks are clas-

siﬁed into 4 different groups of high-level user tasks

that have a strong relationship with these questions,

as described in table 1.

2.3 Location-based Mobile Services

In this work we have analysed and studied several

state-of-the-art contributions on LBMS which pro-

vide a wide variety of visualisation paradigms, in or-

der to understand the current tendencies in the indus-

try and to formulate hypothesis regarding their valid-

ity and usefulness. The contributions range from pilot

studies to commercial products, within the scope of

road and pedestrian maps, as follows: TellMaris, m-

LOMA, LAMP3D, TomTom, Navigon, NDrive, iGO,

Google Earth, INSTAR, Virtual Cable

, and Enkin.

EVALUATION OF VISUALIZATION FEATURES IN THREE-DIMENSIONAL LOCATION-BASED MOBILE

SERVICES

329

Table 1: The primary tasks that 3D maps are used for.

Task Description

Locator Identiﬁcation of the user’s own position

and other objects. Answers ‘Where am

I?’ questions.

Proximity Inform the users of nearby facilities. Im-

plied by ‘Where am I going?’ questions.

Navigation The most tangible example is routing

from one location to another. Answers

‘How do I get there?’ questions.

Event Time/Location dependent objects, allow-

ing the users to know what is happen-

ing and when/where. Answers ‘And now

what?’ questions.

3 CONCEPTUAL FRAMEWORK

In this section, a generic evaluation framework is pro-

posed which can be used as the main methodology for

the speciﬁcation, development and evaluation of new

or existing solutions in the visualisation problem do-

main. This framework is proposed in order to simplify

the evaluation process to the most relevant features,

to the detriment of other classical analysis methods

that can be used to obtain a more thorough evaluation.

This framework deﬁnes the concept of feature vectors

comprising orientations and magnitudes. The orien-

tation deﬁnes the idea or concept the visualisation

paradigm represents, and magnitude the degree/level

to which the paradigm “ampliﬁes” the vector. An ex-

ample can be seen in Figure 3 to describe a possible

feature vector for transportation. An orientation of

this feature vector is the mode of transport, while pol-

lution, cost and speed are magnitudes.

Figure 3: A possible feature vector for “Transportation”.

The framework is composed by six feature vectors

as shown in Figure 4 and described below. These fea-

ture vectors are not intended to characterise the com-

plete set of visualisation features, but the most rele-

vant ones observed from the current state of the art

described in section 2.3.

Figure 4: Evaluation Framework through feature vectors.

3.1 Image Realism

Image Realism is the feature vector that is concerned

with how real, i.e., free from any idealisations or

abstractions, is the image of the map presented to

the user. Taken into account what was previously

mentioned on this matter (see Section 2.1), the sug-

gested magnitudes for this vector will be based on the

framework proposed in (Ferwerda, 2003) and the con-

cepts on virtuality continuum deﬁned in (Milgram and

Kishino, 1994), with a few modiﬁcations. Firstly, a

“relaxed” version of physical realism will be adopted,

i.e., it is assumed that current displays are consid-

ered perfect in the sense that they can emit the ac-

tual energy we want them to reproduce. Secondly,

this framework will be incorporated into the virtuality

continuum as illustrated in Figure 5, adapted from the

above work.

Figure 5: An illustration of the proposed framework com-

bining the Virtuality Continuum spectrum with varieties

of image realism (adapted from the previously mentioned

work).

Photo-Realism is located to the left of Functional

Realism, not because it is considered “less virtual”

than Functional Realism but because it is closer to the

Physical Realism, and consequently providing a more

“realistic” environment.

In terms of orientations, this vector includes the

visualisation elements that represent the real world

visual information, namely 3D Buildings (city build-

ings, landmarks), Map Vectors (roads and polygons),

and Surface Model (ground surface elevations).

3.2 Object Labelling

Object Labelling encompasses the kind of visual tech-

niques and strategies that are followed to label map

elements such as rivers, streets, cities, and so on.

GRAPP 2009 - International Conference on Computer Graphics Theory and Applications

330

In (Been et al., 2006) and other studies, the impor-

tance of two types of labelling, namely static labelling

and dynamic labelling, is discussed. This is relevant

to distinguish since, depending on the case, we might

be dealing with dynamic maps, i.e., maps that support

continuous zoom (changing the scale) and continuous

panning (usually by dragging the map). Based on the

framework proposed in the previous study, the mag-

nitudes for this vector will include the concepts of

Static/Dynamic Selection (visibility) and Placement

(size, position and orientation) of labels.

One of the possible approaches when labelling ob-

jects is to project the labels oriented towards the cur-

rent perspective, analogous to a billboard in Computer

Graphics. This approach is followed by all the con-

tributions except Google Earth where labels are ﬂat-

tened and laid down on the maps surface.

Based on the works of (Wolff, 1999; van Dijk

et al., 1999) and the previous discussion on adaptive-

ness to the current perspective, the proposed orien-

tations for Object Labelling are Perspective-Adaptive

(oriented towards the current perspective), Point Posi-

tioning (point symbols), Line Positioning (polygonal

chains, such as rivers), Area Positioning (areal fea-

tures such as countries), and General Positioning (a

combination of the three previous methods).

3.3 Visual-Spatial Abstraction

Visual-Spatial Abstraction measures the complexity

of mental operations that are required to perform the

visual matching of the real environment that can be

observed and the one on the screen. This vector is

speciﬁcally focused on the mental viewing transfor-

mation that is required in order to have a perfect cor-

respondence between both images: the reality and

the screen. The proposed orientations for this vector

are presented, regardless of the elevation angle of the

“camera”, namely Ground Level (when it is only pos-

sible to observe the current street and its junctions),

Local-Area Level (when streets that may not even be

part of the route can be observed), and Wide-Area

Level (when municipalities and an overview of the

route are visible). The proposed magnitudes reﬂect

the adaptiveness of the camera to the users’ behaviour.

We deﬁne Adaptive Level and Adaptive Orientation

when the camera adapts to the user’s movement (ac-

cording to some variable like speed), and whether it

adapts to his looking direction, respectively.

3.4 Route Indication

Route Indication provides a classiﬁcation of the visual

techniques and strategies for showing the itinerary

path in the road maps, and the kind of manoeuvre in-

dicators or way points that are presented in the dis-

play. The proposed orientations for this vector, can

be regarded as the visual indicators that are gener-

ally used by the majority of the contributions to dis-

play the route, namely Arrows, Cords, Way Points and

Carpet-like shapes to indicate the route. These in-

dicators can be used with different “immersion” lev-

els which are considered the proposed magnitudes for

Route Indication, namely Instructive (when indicators

are merely instructive) and Simulative (when they re-

semble real world indicators).

3.5 Landmark Symbology

Landmark Symbology evaluates the cartographic sym-

bology that is used to portray the world using a pic-

torial language, represented by “map symbols”, often

accompanied by a legend. This vector is also related

to Image Realism, in the way that both should be com-

plementary, i.e., excessive realism may distract the

users, but a great lack of symbology may completely

blur their sense of orientation.

New concepts and design guidelines for the carto-

graphic visualisation of landmarks in mobile maps are

proposed in (Elias et al., 2005). Based on these con-

cepts, the orientations for this vector will reﬂect the

kind of buildings represented by symbols, speciﬁcally

Shops referenced by name (e.g., KFC, McDonalds),

Shops referenced by type (e.g., hotel, pharmacy),

Buildings with unique name / function (e.g., Tokyo

Tower, Statue of Liberty), and Buildings with unique

visual properties (e.g., “the large yellow house”). Ad-

ditionally, the ﬁrst proposed magnitude for this vec-

tor will deﬁne in itself, the concept of levels of ab-

stractions for landmarks, according to a scale (from

the most abstract, to the most concrete): Words, Sign,

Icon, Sketch, Drawing, and Image, as deﬁned in the

previous study.

There are other parameters that inﬂuence the de-

cision of whether an abstraction level should be used

in a mobile map for a given situation. For in-

stance, some cartographic generalisation procedures

(like scaling down a landmark object to an appropri-

ate size suited for its representation in a map) might

raise some problems such as congestion, coalescence,

and imperceptibility (Elias et al., 2005). To account

for these restrictions, the proposed magnitudes con-

sist of Adaptive Zoom and Adaptive Complexity, re-

spectively, whether the abstraction level of landmarks

adapts to the current zoom level, and whether they

change with the varying complexity of features.

EVALUATION OF VISUALIZATION FEATURES IN THREE-DIMENSIONAL LOCATION-BASED MOBILE

SERVICES

331

3.6 Contextual Awareness

Contextual Awareness measures the extent to which

a visualisation paradigm is applied to get additional

information on a contextual or situational basis.

It is important to distinguish the three groups of

application areas in which virtual urban environments

can be valuable, according to the spatio-temporal na-

ture. These groups constitute the proposed orienta-

tions for this vector, depending on whether they focus

on the past, present or ﬁction, according to (Coelho,

2006): Reconstructional (reconstruction of urban en-

vironments that were totally or partially lost), Recre-

ational (urban design, urban planning, etc.), and Fic-

tional (creation of imaginary realities).

Levels of awareness regarding the current loca-

tion, time, and situation can vary from contribution

to contribution. In (Burigat and Chittaro, 2005), it is

claimed that a passive contextual-awareness approach

is generally more ﬂexible than an active approach. In

the latter case, if the user is constantly presented with

unwanted information it can become “too obtrusive”.

Contrarily, in most automotive navigation systems,

direction instructions or location-based information

such as nearby points of interest are automatically

presented, i.e., without the need of the user’s interven-

tion. For these reasons, the proposed magnitudes for

this vector will reﬂect the different autonomy levels

of “contextual awareness” an application can demon-

strate in different contexts and tasks, as previously

denoted by (Chen and Kotz, 2000), speciﬁcally Ac-

tive Awareness (without the need of user interven-

tion), and Passive Awareness (when the user shows

interest for getting context-based information).

Table 2 summarises the evaluation framework, ac-

cording to the proposed magnitudes and orientations.

4 METHODOLOGY

An interactive online questionnaire was developed

and several hypothesises were formulated, in order

to assess the real impact of each visualisation feature

described in the conceptual framework. Since avail-

able free online questionnaires are generally limited

to allow users to set their preferences, an interactive

online questionnaire was developed speciﬁcally for

this study, enabling the measuring of time for each

answer and a more adequate visual aspect deﬁnition.

However, due to the intrinsic limitations of the pro-

posed questionnaire, and in order not to make it per-

ceived by potential participants as “too exhaustive”,

only the features for which there are no signiﬁcant

indications from the state-of-the-art (regarding their

impact and relevance) were evaluated with the ques-

tionnaire. Moreover, there are some components that

were not possible to evaluate, and therefore were not

included in this study, given the limitations imposed

by this kind of questionnaire.

The questionnaire was divided into 3 parts. In

the ﬁrst part, the exercises were mainly based on the

pointing task paradigm as previously performed in

other studies (Nurminen, 2006). In the second part,

a similar approach was followed, but instead of eval-

uating the matching of the two realities, the main ob-

jective was to measure how well users perform a given

task (see Section 2.2). In the last part, users were

asked about their preferences regarding the visualisa-

tion of map elements such as landmarks.

4.1 Image Realism

All Image Realism orientations were tested along

with the various degrees of magnitudes, in accordance

with the vector instances (orientations and magni-

tudes combined) found in the state-of-the-art contri-

butions. These instances were considered eligible for

the evaluation through the questionnaire, since there

are few or no indications, with regards to their impact:

• Simple Textured Buildings and Photo Textured

Buildings

• Coloured Map and Orthophotomap

• Flat Model and Terrain Model

It was hypothesised that, in the absence of Sim-

ple Textured Buildings, test subjects will have to rely

on their ability to match the 3D geometry of the real

building with the geometry of the 2D polygon repre-

sentation on the map. At the same time, it is supposed

that by providing the three-dimensional (yet simple)

geometry of the whole building, in the presence of

this component, test subjects will make fewer mis-

takes and, as a consequence, will require less time

matching both realities (see Figure 6).

Figure 6: The 2 images supporting the questions that evalu-

ate the impact of Simple Textured Buildings.

In the case of the Photo Textured Buildings com-

ponent (see Figure 7), it was hypothesised that, by

GRAPP 2009 - International Conference on Computer Graphics Theory and Applications

332

Table 2: Structure of the proposed evaluation framework.

Feature Vector Orientations Magnitudes

Image Realism 3D Buildings, Map Vectors, Surface Model Physical Realism, Mixed Realism, Photo-

Realism, Functional Realism

Object Labelling Perspective-Adaptive, Positioning (Point,

Line, Area, General)

Static / Dynamic Selection / Placement

Visual-Spatial Abstraction Ground Level, Local-Area Level, Wide-

Area Level

Adaptive Level, Adaptive Orientation

Route Indication Arrows, Cords, Way points, Carpet Instructive, Simulative

Landmark Symbology Shops (referenced by name), Shops (ref-

erenced by type), Buildings (with unique

name / function), Buildings (with unique vi-

sual properties)

Abstractness (Words, Sign, Icon, Sketch,

Drawing, Image), Adaptive Zoom, Adaptive

Complexity

Contextual Awareness Reconstructional, Recreational, Fictional Active Awareness, Passive Awareness

simultaneously providing the 3D geometry of a build-

ing along with photographic fac¸ades, test subjects

will be able to detect features (e.g. windows, doors,

unique wall patterns, etc.) more accurately and faster

than in the case of Simple Textured Buildings.

Figure 7: The 2 images supporting the questions that evalu-

ate the impact of Photo Textured Buildings.

Regarding Map Vectors, it is assumed that an Or-

thophotomap can provide subjects a much more en-

riching visualisation experience than the one provided

by a Coloured Map (see Figure 8). The hypothesis

rests on the belief that an Orthophotomap component

can make easier for users to discern the true features

of the map’s surface, by giving a realistic view rather

than a rough generalisation. There are many situa-

tions were coloured vector polygons are not enough

to represent features like a tiled pavement; a group of

trees arranged in a special and unique way; and sev-

eral “static” features like public benches, zebra cross-

ings, and many others that are impossible to ﬁnd in a

coloured vector map.

Regarding the Surface Model, it was hypothesised

that by using a Terrain Model rather than a Flat Model

component, users will be able to perform the spa-

tial matching of both reality and virtuality in a much

more immersive and natural way (see Figure 9). It

is expected that by providing the Terrain Model com-

ponent, users will be able to use elevated reference

points, and to understand and visualise occlusions

caused by the varying landscape elevation.

In the end, it is expected that users will be able to

perform their tasks in less time, since they just need to

Figure 8: The 2 images supporting the questions that evalu-

ate the impact of Coloured Map and Orthophotomap.

Figure 9: The 2 images supporting the questions that evalu-

ate the impact of Flat Model and Terrain Model.

think “outside the box”. On the other hand, by using a

Flat Model, users would understand that the image on

the screen does not account for occlusions, and there-

fore, they would have to do that job themselves.

4.2 Object Labelling

With respect to Object Labelling, it was hypothesised

that, when users are analysing labels (e.g. of streets,

rivers, cities, and so on) which are not oriented to-

wards the current viewing direction depicted in the

device, they will feel much more difﬁculty reading

the words, due to the decreased visibility, especially

when looking in a direction which is parallel to the

map’s surface (see Figure 10).

In such case, users will not be able to read labels

as faster, and will pan the map closer to the camera so

it becomes easier to read. Particularly in the case of

EVALUATION OF VISUALIZATION FEATURES IN THREE-DIMENSIONAL LOCATION-BASED MOBILE

SERVICES

333

Figure 10: The 2 tasks that evaluate the impact of

Perspective-Adaptive Labelling (close-up).

labels which are almost parallel to the camera’s view-

ing direction, some users will wish to skip words, if

they ﬁnd them “too difﬁcult” to read.

4.3 Route Indication

It was hypothesised that, when a user is presented

with an image which looks more familiar to him,

given the current context, the user will be able to per-

form his task with lesser effort (see Figure 11). It is

assumed that users won’t make more mistakes using

one approach or the other, but that a signiﬁcant dif-

ference in the time they require to complete their task

may arise, i.e., that a Simulative component will result

in faster responsiveness than an Instructive approach.

Figure 11: 2 of the tasks that evaluate the impact of Instruc-

tive and Simulative route indications.

4.4 Landmark Symbology

For this feature vector, it was hypothesised that users

will require Adaptive Zoom functionality, i.e., that the

majority of them will choose an abstract landmark

representation of a given building, when a map which

is zoomed out far from the ground is used, but a more

concrete representation when at close range (see Fig-

ure 12).

The basis of such hypothesis rests on the various

issues raised by the cartographic generalisation pro-

cedures, as previously explained in Section 3.5. For

instance, even if a concrete landmark is used rather

than an abstract representation, there are certain zoom

levels of a map which do not allow users to perceive

enough features of that landmark, in order to identify

it with a signiﬁcant conﬁdence level.

Figure 12: The preferences that evaluates the users’ need for

an Adaptive Zoom approach, when a map which is zoomed

out far / zoomed in close to the ground is used.

5 RESULTS

In total, 149 test subjects answered the questionnaire,

mostly from a student population in Computer Sci-

ence and Informatics: 89% were male, and 78% were

in the 18 to 25 age group. In general, prior to an-

swering the questionnaire, subjects considered them-

selves fairly capable of using both maps and GPS nav-

igators, given the approximate 50-50 ratio shared be-

tween “average” and “experienced” users. Only 3%

of the participants reported they were unfamiliar with

either maps or GPS navigators.

5.1 Image Realism

Regarding the impact of the presence and absence of

Simple Textured Buildings, there were 91% and 77%

correct answers, respectively, in both situations. Al-

though slight, the difference between the two cases

shows the advantage of the presence of Simple Tex-

tured Buildings over its absence. Test subjects re-

quired, in average, 11s (7.3s standard deviation) to an-

swer when buildings were shown, and 15s (6.4s s.d.)

using a classic 2.5D map. While in this case there

was just a 14% difference in the number of correct

answers, in the case of Photo Textured Buildings com-

ponent there were 88% and 30% correct answers re-

spectively. Despite this difference between both ques-

tions, the number of correct answers in the presence

of Photo Textured Buildings was almost the same as

in the case of Simple Textured Buildings. In terms of

answers times, 95% of the subjects had already an-

swered before the ﬁrst 21s in the presence of Photo

Textured Buildings, about 4.4s less than in the pres-

ence of Simple Textured Buildings. When the build-

ings were all removed from the exercise with Photo

Textured Buildings (i.e., in its absence), 95% of test

GRAPP 2009 - International Conference on Computer Graphics Theory and Applications

334

subjects answered before the ﬁrst 42.7s (avg. 17.8s,

s.d. 14.4s) against 25.4s (avg. 15s, s.d. 6.4s). This

clearly demonstrates that the results with Photo Tex-

tured Buildings are more stable, considering the in-

crease in difﬁculty of the exercise.

In the presence of a Coloured Map, the num-

ber of participants who were unable to answer the

question was quite high (14%). The same happened

with the number of wrong answers being quite differ-

ent from the Orthophotomap (67% and 7%, respec-

tively). Nevertheless, subjects had no apparent dif-

ﬁculty in ﬁnding the correct answer, in the presence

of the Orthophotomap component, as 92% chose the

correct answer in similar conditions (as shown in Fig-

ure 13). Besides being more effective, the Orthopho-

tomap proved also to be more efﬁcient as subjects

took an average time of 9.3s (s.d. 18.4s) to answer the

question, considerably faster compared to the 23.5s

(s.d. 16.8s) in the case of the Coloured Map.

Figure 13: Answers in the presence of a Coloured Map and

an equivalent Orthophotomap.

In terms of Surface Model, there was just a 5%

difference in the number of correct answers between

both cases, with advantage to the Terrain Model.

However, the Terrain Model was much more efﬁcient,

as the average response time was 7.5s (s.d. 5s), com-

pared to the 15.3s (s.d. 13.8s) obtained with the Flat

Model.

These results point out that image realism can im-

prove the task of matching the 3D map with reality,

both maximising effectiveness (lesser mistakes) and

effectiveness (lesser time).

5.2 Object Labelling

With respect to Object Labelling, when labels were

oriented towards the camera, the subjects took lesser

time to perform the task (avg. 11.8s, s.d. 5.2s) than

when labels were not oriented according to the cam-

era (avg. 15s, s.d. 6.4s), as shown in Figure 14.

From these results a conclusion can be made that

Perspective-Adaptive Labelling can increase readabil-

ity of labels in 3D maps.

Figure 14: Answer times in the presence and absence of

Perspective-Adaptive Labelling component.

5.3 Route Indication

With respect to the Route Indication there was no rel-

evant difference in terms of answer correctness be-

tween Instructive or Simulative components. In terms

of average answer time, the Instructive component re-

sulted in 11.8s (s.d. 7.9s) against 8.6s (s.d. 5.9s).

Provided with the Simulative approach, 95% of the

participants had already found the matching route in-

dication, after about 19.8s, compared to 26.6s in the

opposite case.

Although both techniques can achieve similar lev-

els of correctness, the Simulative approach can speed-

up the task of matching reality with the 3D map. This

can be of great importance when supporting activities

that demand short response times, such as driving.

5.4 Landmark Symbology

A vast majority of participants (87%) answered they

would more easily identify and recognise the presence

of a given distant landmark, when an abstract repre-

sentation of that landmark was used. Approximately

86% of them indicated their preference towards the

use of concrete landmarks at close range.

Different zoom levels over 3D maps will encom-

pass also different levels of visual complexity, and as

such, Adaptive Zoom functionality is of great impor-

tance for maximising readability.

6 CONCLUSIONS AND FUTURE

WORK

In this study, a generic Evaluation Framework was

proposed as the main methodology for the speciﬁca-

tion, development and evaluation of new or existing

solutions in the 3D map visualisation problem domain

for LBMS. Feature Vectors can individually describe

a set of choices (orientations) and degrees of appli-

EVALUATION OF VISUALIZATION FEATURES IN THREE-DIMENSIONAL LOCATION-BASED MOBILE

SERVICES

335

cability (magnitudes). The proposed framework fo-

cuses on 6 feature vectors namely, Image Realism,

Object Labelling, Visual-Spatial Abstraction, Route

Indication, Landmark Symbology, and ﬁnally Con-

textual Awareness. These feature vectors encompass

the most relevant visualisation issues in 3D maps on

LBMS, but there was no intent to cover them com-

pletely. A future line of research would consist in

analysing the totality of features that address visuali-

sation aspects, in the context of exploration of urban

environments, using 3D LBMS as guidance.

Although the state of the art contemplates some

of the issues involved, the questionnaire gave a much

more clear insight on them. In general, it is ob-

served a greater tendency towards the need of Image

Realism rather than Image Functionalism. In terms

of Perspective-Adaptive Labelling, it was proved that

users are at disadvantage, if they are given the task to

read labels of a map, when these labels are not ori-

ented towards the camera’s viewing direction. The

results also demonstrated that users can more easily

identify the presence of a distant landmark with an ab-

stract representation, and a close landmark with a con-

crete representation, which is indicative of the need of

an Adaptive-Zoom behaviour.

Since there are several limitations on the kind of

measurements that can be performed with the pro-

posed questionnaire in order to evaluate feature vec-

tors, it would be interesting to perform other kinds of

tests, with particular focus on dynamic experiments,

to get more information about other vectors such as

Visual-Spatial Abstraction and Contextual Awareness

which were not evaluated. An example of these exper-

iments would include using a driving simulator to test

the participants’ reﬂexes, given a situation where they

are approaching a manoeuvre, and deciding which

way to go.

From the results obtained from this work, and fu-

ture lines of research, we expect the deﬁnition of new

paradigms of visualisation for 3D map visualisation

on LBMS that maximise usability and improve user

experience and performance.

ACKNOWLEDGEMENTS

We would like to thank NDrive Navigation Systems,

S.A. for the support provided for this research project.

REFERENCES

Been, K., Daiches, E., and Yap, C. (2006). Dynamic map la-

beling. IEEE Transactions on Visualization and Com-

puter Graphics, 12(5):773–780.

Bolin, M. R. and Meyer, G. W. (1999). A visual difference

metric for realistic image synthesis. In Proc. SPIE,

pages 106–120.

Borenstein, J., Everett, H. R., and Feng, L. (1996). ”Where

am I?” – Sensors and Methods for Mobile Robot Po-

sitioning. The University of Michigan.

Burigat, S. and Chittaro, L. (2005). Location-aware visual-

ization of vrml models in gps-based mobile guides. In

John, N. W., Ressler, S., Chittaro, L., and Duce, D. A.,

editors, Web3D, pages 57–64. ACM.

Chen, G. and Kotz, D. (2000). A survey of context-

aware mobile computing research. Technical Report

TR2000-381, Dept. of Computer Science, Dartmouth

College.

Coelho, A. (2006). Expeditious Modelling of Virtual Urban

Environments based on Interoperability and Geospa-

tial Awareness (in Portuguese). PhD thesis, Faculdade

de Engenharia da Universidade do Porto.

Elias, B., Paelke, V., and Kuhnt, S. (2005). Concepts for the

cartographic visualization of landmarks. In Proceed-

ings of Symposium 2005 Location Based Services &

TeleCartography, page 11.

Ferwerda, J. A. (2003). Three varieties of realism in com-

puter graphics. In In Proceedings SPIE Human Vision

and Electronic Imaging ’03, pages 290–297.

Hunolstein, S. V. and Zipf, A. (2003). Towards task oriented

map-based mobile guides. In Workshop ”HCI in Mo-

bile Guides” at Mobile HCI 2003. 5th International

Symp. on HCI with Mobile Devices and Services.

Lange, E. and Ch, Z. R. (2003). The degree of realism of

GIS-based virtual landscapes: Implications for spa-

tial planning. In D. Fritsch and R. Spiller (eds) Pho-

togrammetric Week ’99, pages 367–374.

McNamara, A., Chalmers, A., and Trocianko, T. (2000).

Visual perception in realistic image synthesis. In Co-

quillart, S. and Duke, D., editors, STAR Proceedings

of Eurographics 2000, Interlaken, Switzerland. Euro-

graphics Association.

Milgram, P. and Kishino, F. (1994). A taxonomy of mixed

reality visual displays. IEICE Transactions on Infor-

mation Systems, E77-D(12):1321–1329.

Nurminen, A. (2006). The m-loma mobile 3d

map project website. http://www.init.hut.ﬁ/research

%26projects/m-loma/. Last Checked: November,

2008.

Rademacher, P., Lengyel, J., Cutrell, E., and Whitted, T.

(2001). Measuring the perception of visual realism

in images. In Proceedings of the 12th Eurographics

Workshop on Rendering Techniques, pages 235–248,

London, UK. Springer-Verlag.

van Dijk, S., van Kreveld, M., Strijk, T., and Wolff, A.

(1999). Towards an evaluation of quality for label

placement methods. In Proc. 19th Internat. Car-

tographic Conf. (ICC’99), pages 905–913, Ottawa,

Canada. Internat. Cartographic Association.

Wolff, A. (1999). Automated label placement in theory and

practice. In PhD thesis, Freie Universit

at.

GRAPP 2009 - International Conference on Computer Graphics Theory and Applications

336