PHOTOTIMA: VISUAL EXPLORATION OF PHOTOS

WITH SPATIOTEMPORAL REFERENCES

Dinh Quyen Nguyen and Heidrun Schumann

Institute of Computer Science, University of Rostock, Rostock, Germany

Keywords:

Spatiotemporal Photos, Photo Exploration, Photo Browsing.

Abstract:

Nowadays, photos associated with spatiotemporal references are available widely on computers as well as on

the Internet. There exist several tools for the exploration of photos with regard to their geographical and/or

temporal structures. However, communicating spatial and temporal contents of photos remains a challenging

task. In this paper, we follow Peuquet’s triad framework in visually combining space, time, and descriptive

contents of photos to deal with that task. We address the visual exploration of spatiotemporal photos to be

communicated by an integrated design of different abstraction levels in space, time, and content displaying.

The tool PhotoTima is introduced for the exploration of Flickr photos in connection with their timestamps,

geo-coordinates, and user-generated tags.

1 INTRODUCTION

Today, photos are created, stored, and shared widely

on smart devices, computers, and on the Internet. An

interesting issue about those photos is that many of

them are referenced with geographical coordinates

and timestamps (i.e., places and times that the pho-

tos were taken or uploaded). Time and geo-space

therefore can be seen as interesting aspects for the ex-

ploration and analysis of photos. Nonetheless, due to

the fact that many photos are stored in folders or col-

lections regarding user-deﬁned topics or events, they

are typically presented in a straightforward manner of

slide-show or grid-based view tools. This is a conve-

nient way for the exploration of photos. However, it

is not feasible for the analysis or examination of spa-

tiotemporal dependencies. For example, in the case

that users want to see how their photos are geograph-

ically distributed, it is much more intuitive when the

photos are displayed on geographical maps (as seen

on e.g., Google Panoramio - www.panoramio.com).

Generally speaking, by visually combining photos

with their referenced features of geo-space, time, and

other aspects, we could better understand and be able

to analyze contents of the photos.

In (Peuquet, 1994), Peuquet indicates that when

examining spatiotemporal data, one can get not only

thematic contents of the data, but also insights with

patterns, information, and knowledge from their inter-

related spatiotemporal combinations. She shows that

with a triplet of time (when), space (where), and con-

tent (what) of any spatiotemporal data, one can come

up with three general combinatory situations: what +

when → where, what + where → when, and when +

where → what. In the area of visualization, those sit-

uations are reviewed by Andrienko, Andrienko, and

Gatalsky for numerical and statistical spatiotemporal

data (Andrienko et al., 2003). In this work, we further

concentrate on the visual exploration of spatiotempo-

ral photos. In that regard, we would have the follow-

ing contextual situations:

S1: What + When → Where: Suppose that users

browse speciﬁc photos (what) with speciﬁc times-

tamps (when), and want to know where on the earth

those photos exist. It would be useful if there are vi-

sual hints to show photos with patterns in time so that

users can conveniently browse for photos of interest

over geospace. This can be considered as an exten-

sion compared to the well-known Google Panoramio

(as time-referenced searching has not been provided

in that tool).

S2: What + Where → When: Another situa-

tion, suppose that users are interested in photos with

speciﬁc thematic contents (what) on a geographical

map (where). In other saying, the photos exist and

the places are given. Because those photos were

taken and distributed over time, exploring their his-

tory (when) is another interesting task.

S3: When + Where → What: Finally, sup-

pose that users are interested in some selected time

332

Quyen Nguyen D. and Schumann H..

PHOTOTIMA: VISUAL EXPLORATION OF PHOTOS WITH SPATIOTEMPORAL REFERENCES.

DOI: 10.5220/0003930803320341

In Proceedings of the 8th International Conference on Web Information Systems and Technologies (WEBIST-2012), pages 332-341

ISBN: 978-989-8565-08-2

 2012 SCITEPRESS (Science and Technology Publications, Lda.)

points or patterns (when) in association with geo-

space (where), and they want to examine relevant pho-

tos of interest (what). This would also be helpful to

provide means to highlight those photos to users.

Though the three above situations are somewhat

interrelated, developers can come up with very spe-

ciﬁc designs for speciﬁc purposes (see Section 2).

However, most of existing techniques do not explic-

itly support a comprehensive exploration of photos

with regard to all three aspects: time, space, and de-

scriptive contents of photos. Therefore, we investi-

gate an approach that simultaneously addresses the

three aspects by an appropriate interplay of show-

ing different levels of abstractions and by allowing

the browsing and detailing information with regard to

those three data aspects.

Our basic design consists of two parts: a tags

view and a map representation. The tags view looks

like a parallel coordinates view, each coordinate dis-

plays a time-referenced tag set (to support S1). The

map display shows photo data through sunburst icons

or thumbnails on geographical maps, allowing inter-

active browsing for photos with timestamps on geo-

space (S2 & S3). In doing so, instead of just showing

photos through maps and a simple tag cloud (e.g., as

in Google Panoramio), our visualization design sup-

ports users to ﬁnd tag patterns, time patterns, and pho-

tos of interest (satisfying situations S1-S3). For this

purpose, we provide the tool PhotoTima for the explo-

ration of Flickr photos displayed with Google Maps.

This paper is organized as follows. In Section 2,

related work concerning the visualization of photos

with a focus on techniques dealing with time and geo-

space is given. Section 3 presents the general view on

our visualization design. Section 4 speciﬁcally de-

scribes the time-referenced parallel tag clouds view.

Section 5 presents how sunburst icons and photo

thumbnails are visualized on geographical maps. Sec-

tion 6 presents the implementation and application of

PhotoTima using Flickr and Google Maps APIs. And

Section 7 concludes the paper.

2 RELATED WORK

This section presents related work about photo visu-

alization techniques with regard to the exploration of

the what, the when, and the where of photos.

2.1 Imagery Photo Exploration

Presenting photos (and images, in general) has been

considered from the ﬁrst days of GUI designs. Thus,

there are now many developments in presenting and

visualizing photos on screens. There are two main

trends corresponding to the tasks that photos are ex-

plored: browsing or searching. Browsing means that

users navigate with visual hints to interplay and enjoy

the photos. Searching means that users provide visual

queries and the systems respond with the photos. In

any case, goal of a photo visualization technique is to

support users to get information from the photos. In

this context, such information can be speciﬁed by the

what, the where, and the when of the photos.

A very popular photo visualization technique is

showing photos for browsing in forms of slide-shows

or grid-based views, where users are assisted to eas-

ily ﬁnd photos in collections. Windows Photo Viewer

and Google Picasa (picasa.google.com) are examples

of this approach. Users scroll a view or navigate

forth and back a photo list to examine each photo.

The main goal of those applications is to show im-

agery photos (the what), while their geo-references

(the where) and time-references (the when) are just

speciﬁcally supplemented if needed. One can get that

information from the descriptive title (if existing) or

the detailed properties of each photo. In general, they

are the very loose situations of what (+ slightly where

/ when) → what and what → what (+ slightly where /

when).

There are also many other speciﬁc designs. Porta

in (Porta, 2006) developed some particular forms to

arrange photos as “cylinder”, “rotor”, “tornado”, or

other views. Bederson created a hierarchical visual-

ization structure that highlights the relations of photos

in collections (Bederson, 2001). In PhotoLand (Ryu

et al., 2010), Ryu, Chung, and Cho suggested another

way to arrange photos on screen: the photos similar

to each other (with pre-deﬁned content criteria) are

placed close together to form spatial clusters, and the

clusters are in turn forming a land-based presentation.

By doing so, they showed that users can more ﬂexibly

ﬁnd photos of interest compared to traditional grid-

based views.

To interact with those visualizations, users can se-

lect an area of interest, change a zooming level (typ-

ically for a set of photos), and navigate through the

photoset (i.e., interactive browsing). A photo can be

shown in a separated view, highlighted with visual at-

tributes (e.g., size, border color), magniﬁed to be dis-

tinguishable with other photos (e.g., through a ﬁsheye

presentation (Liu et al., 2004)), or linked with other

descriptive data (e.g., in (Kristensson et al., 2008), a

tag cloud is used where each tag in the cloud can be

connected with a slide-show collection of photos).

So far, we have seen techniques for the visual ex-

ploration of photos in form of what → what. In the

next subsection, we will see how time and geo-space

PHOTOTIMA:VISUALEXPLORATIONOFPHOTOSWITHSPATIOTEMPORALREFERENCES

333

are combined for other situations, which are when →

what, where → what, and vice versa.

2.2 Photo Visualization with Time and

Geo-space

We ﬁrst consider the time aspect. Yahoo! Taglines

(Dubinko et al., 2006) is an example for the visual

exploration of Flickr photos, where users are sup-

ported to select linear time points in a timeline slide-

bar, and then relevant photos are presented on screen.

Huynh et al., in another way, used not only the time-

line but also a set of graphical charts to connect photo

thumbnails with time (Huynh et al., 2005). Photohe-

lix (Hilliges et al., 2007), with spiral-based time vi-

sualization, is another example for visualizing photos

linking with time. Those are techniques that present

a general kind of when → what (or what → when)

representation.

To present photos with regard to their geo-

dependencies, geographical maps are usually

used. Commercial tools currently provided on

the Internet such as Google Panoramio, Flickr

Map (www.ﬂickr.com/map), or iMapFlickr

(imapﬂickr.com) are typical applications for the

exploration of photos based on geographical maps.

Those applications support showing photos as

thumbnails (Google Panoramio, iMapFlickr) or

placemarks (Flickr Map) on the maps. A list of

photos is optionally connected with the map on a

separated view for referencing. Those tools support

the task of browsing photos over geo-space, while

temporal dependencies of the photos are neglected.

In other words, those tools express the situations of

what → where and where → what.

WWMX (Toyama et al., 2003) can be seen as

one of the ﬁrst known applications that support well

the visual exploration of photos connecting with both

geo-space and time. It is a multiple-views design,

with a view representing the geographical map, a

view supporting time selection (dots with weights on

a timeline presentation), and a view that shows an ex-

plored photo linking with a list of thumbnails. Users

can select a dot which represents a set of photos on

geographical map, and the list of thumbnails, each

linked with a timeline view, is updated. However,

visual patterns or analytical information from spa-

tiotemporal photos are not considered. This leads to

recent research in visual analysis and exploration of

spatiotemporally referenced photos, such as those in

(Gomi and Itoh, 2010) or (Peca et al., 2011).

In conclusion, although there are useful and well-

established techniques for the exploration of photos

in space and/or time, many techniques focus only on

one or two aspects rather than supporting the whole

triplet of space, time, and thematic contents of pho-

tos; or they need investigations for the visual explo-

ration of photos through more analytical patterns. In

that regard, we follow Peuquet’s triad framework in

visually combining the triplet for the examination of

spatiotemporally referenced photos.

3 GENERAL DESIGN ISSUES

This section presents general design considerations

that will be addressed in detail in Section 4 and 5.

The considerations are twofold with regard to (1) the

presentation of the three aspects of photo data, and (2)

the communication of insights in terms of combining

the different photo aspects.

3.1 Presentation of Photo Aspects

To support the visual exploration of photos with their

spatiotemporal references, we ﬁrst consider the pre-

sentation of three data aspects of photos in different

abstraction levels.

In information visualization, temporal data can

be represented through time plots, cyclic patterns,

branch views, and so on (Aigner et al., 2008). This

provides ways for users to discover temporal patterns

as well as relations amongst the explored data. In this

work, we visualize photo tags as time plots (see Sec-

tion 4), and use cyclic patterns to indicate the number

of photos on geo-space (see Section 5).

To present geospatial data, usually geographical

maps are used because they are effective means for

geo-data communication (as seen in cartography and

geovisualization). Therefore, we will also use maps

to show the spatial context of photos (see Section 5).

In this regard, photo data can be visually represented

on maps, linked with maps, and interactively explored

as maps are zoomed and panned.

To present photo content, its image has to be dis-

played. We show photos both as thumbnails or full-

size views as presented in Section 5. Besides, as pho-

tos are also describable through textual tags, the tags

are used for the exploration of photos as well. In this

work, tags are abstractly presented with various visual

encodings on time plots (see Section 4).

3.2 Visual Combination and Linking

To provide further insights, the three aspects are visu-

ally combined.

For the representation of time-referenced photo

content (i.e., when + what), we show numerous tag

WEBIST2012-8thInternationalConferenceonWebInformationSystemsandTechnologies

334

sets, each is associated with a speciﬁc time point, to

form a parallel tag clouds view (see Section 4). To

present the combination of where + what or where +

when, we show photo thumbnails and particular sun-

burst icons for time visualization on maps (see Sec-

tion 5).

Those visual combinations provide ways for the

examination of patterns based on two aspects. Fol-

lowing that, we link each visual combination with the

remaining aspect to resolve the situations S1-S3 (Sec-

tion 1).

For situation S1, visual patterns of tags over time

in the form of parallel tags view are visually linked

with their geo-references on geographical maps. In

doing so, user can easily explore photos in form of

what + when → where. To deal with situation S2

(what + where → when), with the selected tags from

the parallel tags view, users zoom and pan on maps

to ﬁnd out temporal patterns of photos over space

by sunburst icons. Finally, with temporal patterns of

photos on maps (i.e., through sunburst icons), we sup-

port users to ﬁlter time data to get detailed photos. In

this way, situation S3 (when + where → what) is an-

swered.

4 VISUALIZING TAGS WITH

TEMPORAL REFERENCES

In this section, we introduce our solution to cope with

situation S1 (what + when → where), where photo

tags are visualized in the context of their temporal ref-

erences.

With Flickr, Google Picasa, and many web-based

photo applications, photos are typically tagged with

textual tags. Normally a set of photos is linked with

a set of tags. In that regard, if examining photos with

time references, we would have different tag sets for

different time points. This leads to the expectation for

the presentation of time-referenced patterns of photos

within tag sets.

To deal with tags over time, there are recent devel-

opments such as time-referenced Taggram (Nguyen

et al., 2011), SparkClouds (Lee et al., 2010), or Paral-

lel Tag Clouds (Collins et al., 2009). Each of them has

advantages and disadvantages if employed for photo

visualization (due to the scope of this paper, we will

not discuss them in detail). In this work, we introduce

the enhancements of the Parallel Tag Clouds for the

visualization of time-referenced photo tags.

4.1 Time-referenced Tag Clouds

Derived from the ideas of tag clouds and parallel co-

ordinates plot, Parallel Tag Clouds is a technique de-

veloped for the exploration and analysis of tag clouds

over axes of tags (Collins et al., 2009). This approach

is applicable for the exploration of time-referenced

photo tags: each axis in the parallel tag clouds shows

a set of photo tags at a time point, and the whole tag

cloud shows all tags over time. However, we have to

enhance the basic approach to allow large volumes of

tags to be examined.

Our idea is to apply a ﬁsheye lens to focus on

axes of interest and tags of interest. It means: (i) tag

clouds’ axes are visually abstracted in different ways

to emphasize different selected time points, and (ii)

tags within each axis are displayed in different sizes

and positions with regard to their levels of interest.

4.1.1 Showing Axes of Interest

An axis of interest is displayed in the size that its tags

are readable, while for other axes, tags are resized

much smaller to ﬁt the display screen (see Figure 1).

(a) One axis is selected. (b) Two axes are selected. (c) Three axes are selected.

Figure 1: Parallel tag clouds with selected axes of interest and tags of interest.

PHOTOTIMA:VISUALEXPLORATIONOFPHOTOSWITHSPATIOTEMPORALREFERENCES

335

Users are supposed to choose particular axes of inter-

est, for example in order to compare tags over a set

of particular time points. Anyway, due to the lim-

ited display area, just a limited number of axes can be

shown with tags in readable sizes (e.g., in Figure 1,

we show at most three selected axes). In this regard,

the axes are updated through user interactions: when

users select an axis, its tags are readable and can be

further examined (see Section 4.1.2); when users de-

select the axis, the tags are minimized. If users se-

lect too many axes, the older axes will be marked as

deselected and temporarily minimized. They will be

shown again when users deselect other axes.

4.1.2 Showing Tags of Interest

The tags shown on each axis are displayed in alpha-

betical order for easy navigating. The heights of the

tags indicate their weights (i.e., the number of related

photos). Each tag can be highlighted as hovered or

selected through mouse interactions. Color and back-

ground of the tags indicate their selected or hovered

states. We show the hovered tags in red with pink bor-

der, while the selected tags are colorized with orange

background (In Figure 2(a), tag “nature” is hovered

and tags “myla” and “nederland” are selected).

Because there could be much more tags than we

can show, on each selected axis just a subset of tags

is displayed. Again, the idea of ﬁsheye menu (Beder-

son, 2000) is integrated. The tags in a selected range

are displayed in full size (according to their weights),

while those which are out of that range are minimized

or removed from the axis’ view (in this case, digits at

the top and bottom of the axis indicate the numbers of

unshown tags). The shown tags are updated through

user interactions (i.e., by scrolling or paging up and

down) on that axis. We also visually link tags over

axes (see Figure 2(b)). Through user interaction, tags

which are identical to the examined tag on the hovered

axis (i.e., tag “nature”) are highlighted with connected

pink lines and red dots on other axes. The size of the

dots are relative to weights of the tags through axes.

If a linked tag is out of range on another selected axis,

its range is updated (in Figure 2(b), tag “nature” is

shifted to the bottom of axis 4 and to the top of axis 9).

Users can select or deselect tags identical to the hov-

ered one on all axes as well (selected tags on the min-

imized axes are encoded in orange color). Lastly, we

support ﬁltering tags in terms of their weights. Fig-

ure 2(c) shows tags with the updated weight range of

[3, 16].

In doing so, tags with different characteristics in

weights, colors, and positions on parallel time plots

of tag clouds are visualized to represent temporal pat-

terns of photos (in form of what + when). In Section

6, we will see how this design is used for the explo-

ration of photos on maps to satisfy situation S1.

5 VISUALIZING PHOTOS WITH

SPATIAL REFERENCES

This section presents the design to cope with situa-

tions S2 (what + where → when) and S3 (when +

where → what). Temporal patterns of photos are

shown on maps as sunburst icons (to support situa-

tion S2), and pop-up window is used to show detailed

photos for situation S3.

5.1 Photo Thumbnails and Sunburst

Icons

Combining the what (or the when) with the where as-

pects of photos are carried out through different ab-

straction levels of photo aspects in connection with

geographical maps. As thumbnails (the reduced-size

versions for images representation) are typically used

(a) Selected and hovered tags. (b) Examining tags over axes. (c) Tags ﬁltered with weights.

Figure 2: Presenting hovered and selected tags over axes.

WEBIST2012-8thInternationalConferenceonWebInformationSystemsandTechnologies

336

(a) Showing 100 photo thumbnails on maps. (b) Those thumbnails are shown in smaller sizes.

Figure 3: Showing photos as thumbnails or sunburst icons on geographical maps.

for recognizing and organizing photos in collections

(e.g., as in the grid-based view of Google Picasa),

they can be used to provide a glance view of photo

distribution over geographical maps (see Figure 3).

However, if we have many photos close together,

showing all thumbnails would cause perceptual prob-

lems. In Figure 3(a), when thumbnails are shown in

size of 40x40 pixels, they overlap each other. If they

are shown in smaller sizes (e.g., by 14x14 pixels as in

Figure 3(b)), the overlapping problem reduces, but the

thumbnails are too small to comprehend. Therefore,

alternative visual abstractions are needed.

Firstly, we suggest to selectively show a subset of

thumbnails in comprehensible sizes (i.e., 40x40 pix-

els). For this purpose, we cluster photos based on

their geo-coordinates at each zooming level of the

map. A cluster is composed of a list of photos within

a local region around a centroid photo. Each centroid

(and then a new cluster) is created when a photo does

not fall into any existing cluster. In Figure 3(c), pho-

tos from Figure 3(a) are clustered with radius = 40

pixels. In this manner, the clusters form an orthogo-

nal distribution of photos over geo-space. The clus-

ters will be redistributed when the map changes its

zooming level. With each cluster, we show the thumb-

nail for a photo of particular interest (e.g., the photo

with speciﬁc tags). For clusters that contain more

than one photo, a “stack”-background is added to the

thumbnail. However, since photos are not evenly dis-

tributed on maps (i.e., they are condensed at some

places and sparser at other places), thumbnails with

stack-background do not differentiate such informa-

tion. Therefore, additional visual cues are added. For

instance, numbers are added on the thumbnails to in-

dicate how many photos are examined with that clus-

ter (see Section 6). This design allows for the repre-

sentation of what + where aspects.

Secondly, we further show temporal patterns of

photos in form of visual icons on maps. In doing so,

we can provide insights about where + when patterns.

Within each geo-referenced cluster of photos, there

could be interesting information about their tempo-

ral references. For instance, some photos co-occur

at speciﬁc days in week. Thus, for each cluster of

photos, we design a sunburst icon with three main

parts: (1) a center circle with a number indicating the

number of photos in the cluster, (2) a ring with labels

for cyclic temporal patterns (we developed 3 patterns:

PHOTOTIMA:VISUALEXPLORATIONOFPHOTOSWITHSPATIOTEMPORALREFERENCES

337

hours-in-day, days-in-week, and months-in-year), and

(3) arcs linked with the ring indicating the numbers

of photos falling into the associated time slots. There,

the length and lightness of an arc represent its number

of photos. Figure 3(d) illustrates our design.

In our implementation (see Section 6), photos are

speciﬁcally selected for examination, we therefore

colorize the arcs in orange for selected photos, and

in blue to indicate the unselected ones. If an arc is too

long (i.e., exceeding the space reserved for the sun-

burst icon), it is shortened and colorized darker (in

Figure 3(d), the cluster in Europe is darker than those

at the other places).

5.2 Browsing Full-size Photos

With photo thumbnails or sunburst icons, we get the

overview of photos distributed over geospace and

time. However, for photo browsing, users need to see

the photos in detail as well (this is a necessary task in

any photo viewer tool). In that regard, we do the same

as Google Panoramio in showing photos on demand.

Users click on a cluster’s thumbnail or choose an arc

on a sunburst icon, and a window is popped-up with

detailed information about relevant photos. We show

on the pop-up window: (1) title of the photo under

examination, (2) its imagery content in size of max

240x240 pixels (Flickr photos’ small size), (3) links

to other photos (if existing) in the cluster or the ex-

amined arc, and (4) navigator link to the source photo

(e.g., the photo on Flickr website with full descrip-

tions, comments, etc.) (see Figure 4(c)).

6 THE TOOL PHOTOTIMA

In this section, we describe the implementation and

demonstrate the tool PhotoTima for the exploration

of spatiotemporal Flickr photos in the context of situ-

ations S1-S3.

6.1 Implementation

We implemented a web-based visualization applica-

tion in Flash, built on Adobe Flash Builder 4.5. The

application PhotoTima is used for the visual explo-

ration of spatiotemporal Flickr photos. Flickr pho-

tos are retrieved directly from Flickr servers through

its APIs (www.ﬂickr.com/services/api), while Google

Maps API (code.google.com/apis/maps/ documenta-

tion/ﬂash) is employed for the manipulation of geo-

graphical maps.

The application consists of three components: a

main toolbar on the top, a viewport for geographi-

cal maps on the left, and a view for time-referenced

parallel tag clouds on the right (Figure 4). We al-

low users to toggle the tags view, select a period

of time in the view, provide some initial tags (if

needed), and then load Flickr data. Through Flickr

API ﬂickr.photos.search, PhotoTima loads a list of

photos, each contains a set of tags, a taken time, and a

geographical coordinate. As Flickr photos are too nu-

merous (e.g., in 2011, millions of photos are uploaded

to Flickr everyday), by default we iteratively load 10

photos per query and refresh the interface. The photos

are loaded from the most interesting ones (criterion

supported by Flickr). Then, tags are grouped in vari-

ous ways: per day, week, month, or year (options are

provided on the tags view, e.g., “months” is selected

in Figure 4). Based on the selection, tags are accumu-

lated for relevant time points (e.g., months) and then

consequently visualized on the plots of the parallel tag

clouds.

With options on the main toolbar, users choose

whether to show photos as thumbnails or sunburst

icons on maps. The thumbnails or sunburst icons

are implemented as overlay objects added on Google

Maps.

6.2 Application

To explore Flickr photos in the context of situation

S1 (what + when → where), users are supposed to

specify photo tags and time points on the tags view

and examine them in correlation with geographical

regions of interest. To demonstrate the procedure, we

loaded 200 most interesting Flickr photos taken in the

year 2010 (from Jan 01 to Dec 31) in terms of the tag

“poor”. The tool PhotoTima showed clusters with 51

photos (the most) in the area of Europe, then 40 in In-

dia, 37 around South East Asia, 30 in Central Africa,

and 17 in Central America, while in other areas there

were very few photos (15 photos in the US, and less

than 5 photos on all other areas). However, when we

selected an additional tag “homeless” (which is se-

mantically related to tag “poor” - a criterion provided

by Flickr) just 6 of 51, 5/40, 1/37, 0/30, and 3/17 pho-

tos were counted for those “poorest” areas, respec-

tively, but up to 8 of 15 photos were highlighted at the

area of US (numbers in pink in Figure 4(a)).

Here, users might wonder about the visual pat-

terns. Thus, PhotoTima provides an interlinking

mechanism for further exploration. For example,

users can explore the sunburst icons for temporal pat-

terns of photos on maps (situation S2: what + where

→ when) or click the thumbnails to see the detailed

photos on maps for situation S3 (when + where →

what).

WEBIST2012-8thInternationalConferenceonWebInformationSystemsandTechnologies

338

(a) Searching Flickr photos with preliminary tag “poor” and highlighting patterns with tag “homeless”.

(b) With preliminary tag “football”, sunburst icons show peaks for “July” in South Africa and Europe.

Figure 4: Screenshots of the tool PhotoTima.

PHOTOTIMA:VISUALEXPLORATIONOFPHOTOSWITHSPATIOTEMPORALREFERENCES

339

We investigated another example for 200 most in-

teresting photos taken in 2010 with initial tag “foot-

ball”. The visualization showed that there were 102

photos in Europe, 28 around North America, 17 in

South America, 15 in South East Asia, and 14 in

South Africa, and so on (Figure 4(b)). What were the

reasons for this distribution?

We found the answers by examining time-

referenced tags of “football” photos. The visualiza-

tion showed that the highest number of tags is on July

(552 tags) and the fewest one is on December (142

tags). It means that Flickr users seem to consider

photos taken in July most, but least in December. In

that regard, we examined how photos were distributed

with time pattern of months over geographical maps.

As seen in Figure 4(b), almost all photos taken in De-

cember were accumulated only for the region of Eu-

rope, while the photos in July appeared as peaks in

South Africa and Europe.

What was that special pattern of July? Why most

of photos in South Africa (12/14 photos) appeared

in July? By clicking on the sunburst piece indicat-

ing July at South Africa, the pop-up window showed

photos with titles such as “World Cup 2010 South

Africa: Spain v Netherlands” or “World Cup 2010

South Africa: Spain v Germany”. The answer was

clear: the particular pattern about “football” photos in

2010 was about World Cup 2010 taken place in July

in South Africa.

Alternatively, we examined how tag patterns in

parallel tag clouds visualization could be used to help

us in ﬁnding out that information. One of the related

tags of the tag “football” is tag “stadium”. Interest-

ingly, tag “stadium” appeared in all columns of the

parallel tag clouds. By selecting it on all columns

(i.e., for all photos), the two remaining photos (in-

dicated by February and May) in South Africa’s sun-

burst icon were highlighted. Why those two photos

were about stadium? Could we ﬁnd any relation be-

tween those photos and the others in July? By click-

ing on them, e.g., the photo in May, we found out that

the description was about the Green Point Stadium I

in Cape Town, and when we navigated to Flickr web-

site, we saw descriptions and comments about this

stadium in its preparation for World Cup 2010 (Fig-

ure 4(c)). From that, we thought that users can also

guess what the photos in July were about.

6.3 Preliminary Feedback

We conducted informal interviews to get a prelimi-

nary feedback for our tool. Six users (two visualiza-

tion experts, and four users from other domains) have

been invited to use PhotoTima to explore Flickr pho-

tos. We ﬁrst explained the tool and the provided op-

tions (interface components, mouse options, and hot

keys). Afterwards, the users applied PhotoTima to ex-

plore Flickr photos. After having used the tool, they

were asked to give an informal feedback.

In general, all users gave positive feedbacks. They

particularly pointed out that PhotoTima presents use-

ful temporal patterns of Flickr photos on maps. They

said that the tool is easy to use as all of them are fa-

miliar with photo thumbnails, tags, and geographical

maps. In addition, they said that our tool is very com-

prehensible because it smoothly updates the interface.

However, although the tag clouds design was ex-

pressive, more explanations about it were requested.

Besides, the users also expected the tool to be im-

proved with additional supports: (1) providing hints

so that users can select related tags of any tag, (2)

showing all selected tags in a separated view as they

are important information, (3) providing more hints

for the photos from clusters without clicking on the

icons, and (4) automatically updating photos and tags

when the map-view is updated.

7 CONCLUSIONS

In this paper, we have presented an approach for the

visual exploration of Flickr photos with regard to their

spatiotemporal references. We addressed the visual-

ization of complex photo data communicated through

different visual encodings in space, time, and descrip-

tive contents. Geographical maps were employed

to communicate geospatial references, sunburst icons

and time plots were applied for temporal patterns, and

image displaying and tags were used for the presenta-

tion of thematic contents of photos.

The visual representations were linked to provide

insights for spatiotemporal photos exploration. We

combined what + when aspects of photos in the form

of parallel tag clouds (focused tags are shown-in-

context by a ﬁsheye lens). To communicate infor-

mation with regard to where + what and where +

when combinations, we created thumbnails and sun-

burst icons on geographical maps.

By brushing-and-linking the tag clouds view and

the maps view, we allowed users to explore photos

with regard to all three photo aspects. In doing so,

our approach can simultaneously communicate infor-

mation in terms of the what, the where, and the when

of photos, and thus supporting all situations S1-S3.

The tool PhotoTima was developed for the explo-

ration of Flickr photos. Today, there are a lot of sites

for other media sharing, such as video with Youtube -

www.youtube.com, music with Last.fm - www.last.

WEBIST2012-8thInternationalConferenceonWebInformationSystemsandTechnologies

340

fm, and many more. Most of media sharing sites pro-

vide APIs for data access and manipulation. Thus,

exploring and entertaining media with regard to their

spatial, temporal, and descriptive contents (e.g., tags)

are of interest. The prototype approach of PhotoTima

can absolutely be applied for the development of simi-

lar tools for other media. Anyway, how relevant a tool

is depends on the APIs that a media site supports, as

currently most of sharing sites provide APIs suitably

for contents retrieving, instead of time and geoloca-

tion fetching (e.g., even with powerful systems such

as Google Panoramio or Last.fm).

For future work, we will improve the tool Pho-

toTima following the issues raised from users’ feed-

backs. In details, we will show related tags on the

parallel tag clouds, create a magic lens (Bier et al.,

1993) that provides hints for the photos at the exam-

ined cluster, and update the Flickr photos and tags as

the map view is changed. In addition, the tool Photo-

Tima will be examined in the case of using multitouch

tablet computers and smartphones.

REFERENCES

Aigner, W., Miksch, S., Muller, W., Schumann, H., and

Tominski, C. (2008). Visual methods for analyz-

ing time-oriented data. Visualization and Computer

Graphics, IEEE Transactions on, 14(1):47 –60.

Andrienko, N., Andrienko, G., and Gatalsky, P. (2003). Ex-

ploratory spatio-temporal visualization: an analytical

review. Journal of Visual Languages & Computing,

14(6):503–541.

Bederson, B. B. (2000). Fisheye menus. In Proceedings

of the 13th annual ACM symposium on User interface

software and technology, UIST ’00, pages 217–225,

New York, NY, USA. ACM.

Bederson, B. B. (2001). Photomesa: a zoomable image

browser using quantum treemaps and bubblemaps. In

Proceedings of the 14th annual ACM symposium on

User interface software and technology, UIST ’01,

pages 71–80, New York, NY, USA. ACM.

Bier, E. A., Stone, M. C., Pier, K., Buxton, W., and DeRose,

T. D. (1993). Toolglass and magic lenses: the see-

through interface. In Proceedings of the 20th an-

nual conference on Computer graphics and interac-

tive techniques, SIGGRAPH ’93, pages 73–80, New

York, NY, USA. ACM.

Collins, C., Viegas, F., and Wattenberg, M. (2009). Par-

allel tag clouds to explore and analyze faceted text

corpora. In Visual Analytics Science and Technology,

2009. VAST 2009. IEEE Symposium on, pages 91 –98.

Dubinko, M., Kumar, R., Magnani, J., Novak, J., Ragha-

van, P., and Tomkins, A. (2006). Visualizing tags over

time. In Proceedings of the 15th international confer-

ence on World Wide Web, WWW ’06, pages 193–202,

New York, NY, USA. ACM.

Gomi, A. and Itoh, T. (2010). Miaow: a 3d image browser

applying a location- and time-based hierarchical data

visualization technique. In Proceedings of the Inter-

national Conference on Advanced Visual Interfaces,

AVI ’10, pages 225–232, New York, NY, USA. ACM.

Hilliges, O., Baur, D., and Butz, A. (2007). Photohe-

lix: Browsing, sorting and sharing digital photo col-

lections. In Horizontal Interactive Human-Computer

Systems, 2007. TABLETOP ’07. Second Annual IEEE

International Workshop on, pages 87 –94.

Huynh, D. F., Drucker, S. M., Baudisch, P., and Wong,

C. (2005). Time quilt: scaling up zoomable photo

browsers for large, unstructured photo collections. In

CHI ’05 extended abstracts on Human factors in com-

puting systems, CHI EA ’05, pages 1937–1940, New

York, NY, USA. ACM.

Kristensson, P. O., Arnell, O., Bj

ork, A., Dahlb

ack, N.,

Pennerup, J., Prytz, E., Wikman, J., and

Astr

om, N.

(2008). Infotouch: an explorative multi-touch visu-

alization interface for tagged photo collections. In

Proceedings of the 5th Nordic conference on Human-

computer interaction: building bridges, NordiCHI

’08, pages 491–494, New York, NY, USA. ACM.

Lee, B., Riche, N., Karlson, A., and Carpendale, S. (2010).

Sparkclouds: Visualizing trends in tag clouds. Visu-

alization and Computer Graphics, IEEE Transactions

on, 16(6):1182 –1189.

Liu, H., Xie, X., Tang, X., Li, Z.-W., and Ma, W.-Y. (2004).

Effective browsing of web image search results. In

Proceedings of the 6th ACM SIGMM international

workshop on Multimedia information retrieval, MIR

’04, pages 84–90, New York, NY, USA. ACM.

Nguyen, D. Q., Tominski, C., Schumann, H., and Ta, T. A.

(2011). Visualizing tags with spatiotemporal refer-

ences. In Information Visualisation (IV), 2011 15th

International Conference on, pages 32 –39.

Peca, I., Zhi, H., Vrotsou, K., Andrienko, N., and An-

drienko, G. (2011). Kd-photomap: Exploring pho-

tographs in space and time. In Visual Analytics Sci-

ence and Technology (VAST), 2011 IEEE Conference

on, pages 291 –292.

Peuquet, D. J. (1994). It’s about time: A conceptual frame-

work for the representation of temporal dynamics in

geographic information systems. Annals of the Asso-

ciation of American Geographers, 84(3):pp. 441–461.

Porta, M. (2006). Browsing large collections of images

through unconventional visualization techniques. In

Proceedings of the working conference on Advanced

visual interfaces, AVI ’06, pages 440–444, New York,

NY, USA. ACM.

Ryu, D.-S., Chung, W.-K., and Cho, H.-G. (2010). Pho-

toland: a new image layout system using spatio-

temporal information in digital photos. In Proceed-

ings of the 2010 ACM Symposium on Applied Com-

puting, SAC ’10, pages 1884–1891, New York, NY,

USA. ACM.

Toyama, K., Logan, R., and Roseway, A. (2003). Geo-

graphic location tags on digital images. In Proceed-

ings of the eleventh ACM international conference

on Multimedia, MULTIMEDIA ’03, pages 156–166,

New York, NY, USA. ACM.

PHOTOTIMA:VISUALEXPLORATIONOFPHOTOSWITHSPATIOTEMPORALREFERENCES

341