PHOTOTIMA: VISUAL EXPLORATION OF PHOTOS
WITH SPATIOTEMPORAL REFERENCES
Dinh Quyen Nguyen and Heidrun Schumann
Institute of Computer Science, University of Rostock, Rostock, Germany
Keywords:
Spatiotemporal Photos, Photo Exploration, Photo Browsing.
Abstract:
Nowadays, photos associated with spatiotemporal references are available widely on computers as well as on
the Internet. There exist several tools for the exploration of photos with regard to their geographical and/or
temporal structures. However, communicating spatial and temporal contents of photos remains a challenging
task. In this paper, we follow Peuquet’s triad framework in visually combining space, time, and descriptive
contents of photos to deal with that task. We address the visual exploration of spatiotemporal photos to be
communicated by an integrated design of different abstraction levels in space, time, and content displaying.
The tool PhotoTima is introduced for the exploration of Flickr photos in connection with their timestamps,
geo-coordinates, and user-generated tags.
1 INTRODUCTION
Today, photos are created, stored, and shared widely
on smart devices, computers, and on the Internet. An
interesting issue about those photos is that many of
them are referenced with geographical coordinates
and timestamps (i.e., places and times that the pho-
tos were taken or uploaded). Time and geo-space
therefore can be seen as interesting aspects for the ex-
ploration and analysis of photos. Nonetheless, due to
the fact that many photos are stored in folders or col-
lections regarding user-defined topics or events, they
are typically presented in a straightforward manner of
slide-show or grid-based view tools. This is a conve-
nient way for the exploration of photos. However, it
is not feasible for the analysis or examination of spa-
tiotemporal dependencies. For example, in the case
that users want to see how their photos are geograph-
ically distributed, it is much more intuitive when the
photos are displayed on geographical maps (as seen
on e.g., Google Panoramio - www.panoramio.com).
Generally speaking, by visually combining photos
with their referenced features of geo-space, time, and
other aspects, we could better understand and be able
to analyze contents of the photos.
In (Peuquet, 1994), Peuquet indicates that when
examining spatiotemporal data, one can get not only
thematic contents of the data, but also insights with
patterns, information, and knowledge from their inter-
related spatiotemporal combinations. She shows that
with a triplet of time (when), space (where), and con-
tent (what) of any spatiotemporal data, one can come
up with three general combinatory situations: what +
when where, what + where when, and when +
where what. In the area of visualization, those sit-
uations are reviewed by Andrienko, Andrienko, and
Gatalsky for numerical and statistical spatiotemporal
data (Andrienko et al., 2003). In this work, we further
concentrate on the visual exploration of spatiotempo-
ral photos. In that regard, we would have the follow-
ing contextual situations:
S1: What + When Where: Suppose that users
browse specific photos (what) with specific times-
tamps (when), and want to know where on the earth
those photos exist. It would be useful if there are vi-
sual hints to show photos with patterns in time so that
users can conveniently browse for photos of interest
over geospace. This can be considered as an exten-
sion compared to the well-known Google Panoramio
(as time-referenced searching has not been provided
in that tool).
S2: What + Where When: Another situa-
tion, suppose that users are interested in photos with
specific thematic contents (what) on a geographical
map (where). In other saying, the photos exist and
the places are given. Because those photos were
taken and distributed over time, exploring their his-
tory (when) is another interesting task.
S3: When + Where What: Finally, sup-
pose that users are interested in some selected time
332
Quyen Nguyen D. and Schumann H..
PHOTOTIMA: VISUAL EXPLORATION OF PHOTOS WITH SPATIOTEMPORAL REFERENCES.
DOI: 10.5220/0003930803320341
In Proceedings of the 8th International Conference on Web Information Systems and Technologies (WEBIST-2012), pages 332-341
ISBN: 978-989-8565-08-2
Copyright
c
2012 SCITEPRESS (Science and Technology Publications, Lda.)
points or patterns (when) in association with geo-
space (where), and they want to examine relevant pho-
tos of interest (what). This would also be helpful to
provide means to highlight those photos to users.
Though the three above situations are somewhat
interrelated, developers can come up with very spe-
cific designs for specific purposes (see Section 2).
However, most of existing techniques do not explic-
itly support a comprehensive exploration of photos
with regard to all three aspects: time, space, and de-
scriptive contents of photos. Therefore, we investi-
gate an approach that simultaneously addresses the
three aspects by an appropriate interplay of show-
ing different levels of abstractions and by allowing
the browsing and detailing information with regard to
those three data aspects.
Our basic design consists of two parts: a tags
view and a map representation. The tags view looks
like a parallel coordinates view, each coordinate dis-
plays a time-referenced tag set (to support S1). The
map display shows photo data through sunburst icons
or thumbnails on geographical maps, allowing inter-
active browsing for photos with timestamps on geo-
space (S2 & S3). In doing so, instead of just showing
photos through maps and a simple tag cloud (e.g., as
in Google Panoramio), our visualization design sup-
ports users to find tag patterns, time patterns, and pho-
tos of interest (satisfying situations S1-S3). For this
purpose, we provide the tool PhotoTima for the explo-
ration of Flickr photos displayed with Google Maps.
This paper is organized as follows. In Section 2,
related work concerning the visualization of photos
with a focus on techniques dealing with time and geo-
space is given. Section 3 presents the general view on
our visualization design. Section 4 specifically de-
scribes the time-referenced parallel tag clouds view.
Section 5 presents how sunburst icons and photo
thumbnails are visualized on geographical maps. Sec-
tion 6 presents the implementation and application of
PhotoTima using Flickr and Google Maps APIs. And
Section 7 concludes the paper.
2 RELATED WORK
This section presents related work about photo visu-
alization techniques with regard to the exploration of
the what, the when, and the where of photos.
2.1 Imagery Photo Exploration
Presenting photos (and images, in general) has been
considered from the first days of GUI designs. Thus,
there are now many developments in presenting and
visualizing photos on screens. There are two main
trends corresponding to the tasks that photos are ex-
plored: browsing or searching. Browsing means that
users navigate with visual hints to interplay and enjoy
the photos. Searching means that users provide visual
queries and the systems respond with the photos. In
any case, goal of a photo visualization technique is to
support users to get information from the photos. In
this context, such information can be specified by the
what, the where, and the when of the photos.
A very popular photo visualization technique is
showing photos for browsing in forms of slide-shows
or grid-based views, where users are assisted to eas-
ily find photos in collections. Windows Photo Viewer
and Google Picasa (picasa.google.com) are examples
of this approach. Users scroll a view or navigate
forth and back a photo list to examine each photo.
The main goal of those applications is to show im-
agery photos (the what), while their geo-references
(the where) and time-references (the when) are just
specifically supplemented if needed. One can get that
information from the descriptive title (if existing) or
the detailed properties of each photo. In general, they
are the very loose situations of what (+ slightly where
/ when) what and what what (+ slightly where /
when).
There are also many other specific designs. Porta
in (Porta, 2006) developed some particular forms to
arrange photos as “cylinder”, “rotor”, “tornado”, or
other views. Bederson created a hierarchical visual-
ization structure that highlights the relations of photos
in collections (Bederson, 2001). In PhotoLand (Ryu
et al., 2010), Ryu, Chung, and Cho suggested another
way to arrange photos on screen: the photos similar
to each other (with pre-defined content criteria) are
placed close together to form spatial clusters, and the
clusters are in turn forming a land-based presentation.
By doing so, they showed that users can more flexibly
find photos of interest compared to traditional grid-
based views.
To interact with those visualizations, users can se-
lect an area of interest, change a zooming level (typ-
ically for a set of photos), and navigate through the
photoset (i.e., interactive browsing). A photo can be
shown in a separated view, highlighted with visual at-
tributes (e.g., size, border color), magnified to be dis-
tinguishable with other photos (e.g., through a fisheye
presentation (Liu et al., 2004)), or linked with other
descriptive data (e.g., in (Kristensson et al., 2008), a
tag cloud is used where each tag in the cloud can be
connected with a slide-show collection of photos).
So far, we have seen techniques for the visual ex-
ploration of photos in form of what what. In the
next subsection, we will see how time and geo-space
PHOTOTIMA:VISUALEXPLORATIONOFPHOTOSWITHSPATIOTEMPORALREFERENCES
333
are combined for other situations, which are when
what, where what, and vice versa.
2.2 Photo Visualization with Time and
Geo-space
We first consider the time aspect. Yahoo! Taglines
(Dubinko et al., 2006) is an example for the visual
exploration of Flickr photos, where users are sup-
ported to select linear time points in a timeline slide-
bar, and then relevant photos are presented on screen.
Huynh et al., in another way, used not only the time-
line but also a set of graphical charts to connect photo
thumbnails with time (Huynh et al., 2005). Photohe-
lix (Hilliges et al., 2007), with spiral-based time vi-
sualization, is another example for visualizing photos
linking with time. Those are techniques that present
a general kind of when what (or what when)
representation.
To present photos with regard to their geo-
dependencies, geographical maps are usually
used. Commercial tools currently provided on
the Internet such as Google Panoramio, Flickr
Map (www.flickr.com/map), or iMapFlickr
(imapflickr.com) are typical applications for the
exploration of photos based on geographical maps.
Those applications support showing photos as
thumbnails (Google Panoramio, iMapFlickr) or
placemarks (Flickr Map) on the maps. A list of
photos is optionally connected with the map on a
separated view for referencing. Those tools support
the task of browsing photos over geo-space, while
temporal dependencies of the photos are neglected.
In other words, those tools express the situations of
what where and where what.
WWMX (Toyama et al., 2003) can be seen as
one of the first known applications that support well
the visual exploration of photos connecting with both
geo-space and time. It is a multiple-views design,
with a view representing the geographical map, a
view supporting time selection (dots with weights on
a timeline presentation), and a view that shows an ex-
plored photo linking with a list of thumbnails. Users
can select a dot which represents a set of photos on
geographical map, and the list of thumbnails, each
linked with a timeline view, is updated. However,
visual patterns or analytical information from spa-
tiotemporal photos are not considered. This leads to
recent research in visual analysis and exploration of
spatiotemporally referenced photos, such as those in
(Gomi and Itoh, 2010) or (Peca et al., 2011).
In conclusion, although there are useful and well-
established techniques for the exploration of photos
in space and/or time, many techniques focus only on
one or two aspects rather than supporting the whole
triplet of space, time, and thematic contents of pho-
tos; or they need investigations for the visual explo-
ration of photos through more analytical patterns. In
that regard, we follow Peuquet’s triad framework in
visually combining the triplet for the examination of
spatiotemporally referenced photos.
3 GENERAL DESIGN ISSUES
This section presents general design considerations
that will be addressed in detail in Section 4 and 5.
The considerations are twofold with regard to (1) the
presentation of the three aspects of photo data, and (2)
the communication of insights in terms of combining
the different photo aspects.
3.1 Presentation of Photo Aspects
To support the visual exploration of photos with their
spatiotemporal references, we first consider the pre-
sentation of three data aspects of photos in different
abstraction levels.
In information visualization, temporal data can
be represented through time plots, cyclic patterns,
branch views, and so on (Aigner et al., 2008). This
provides ways for users to discover temporal patterns
as well as relations amongst the explored data. In this
work, we visualize photo tags as time plots (see Sec-
tion 4), and use cyclic patterns to indicate the number
of photos on geo-space (see Section 5).
To present geospatial data, usually geographical
maps are used because they are effective means for
geo-data communication (as seen in cartography and
geovisualization). Therefore, we will also use maps
to show the spatial context of photos (see Section 5).
In this regard, photo data can be visually represented
on maps, linked with maps, and interactively explored
as maps are zoomed and panned.
To present photo content, its image has to be dis-
played. We show photos both as thumbnails or full-
size views as presented in Section 5. Besides, as pho-
tos are also describable through textual tags, the tags
are used for the exploration of photos as well. In this
work, tags are abstractly presented with various visual
encodings on time plots (see Section 4).
3.2 Visual Combination and Linking
To provide further insights, the three aspects are visu-
ally combined.
For the representation of time-referenced photo
content (i.e., when + what), we show numerous tag
WEBIST2012-8thInternationalConferenceonWebInformationSystemsandTechnologies
334
sets, each is associated with a specific time point, to
form a parallel tag clouds view (see Section 4). To
present the combination of where + what or where +
when, we show photo thumbnails and particular sun-
burst icons for time visualization on maps (see Sec-
tion 5).
Those visual combinations provide ways for the
examination of patterns based on two aspects. Fol-
lowing that, we link each visual combination with the
remaining aspect to resolve the situations S1-S3 (Sec-
tion 1).
For situation S1, visual patterns of tags over time
in the form of parallel tags view are visually linked
with their geo-references on geographical maps. In
doing so, user can easily explore photos in form of
what + when where. To deal with situation S2
(what + where when), with the selected tags from
the parallel tags view, users zoom and pan on maps
to find out temporal patterns of photos over space
by sunburst icons. Finally, with temporal patterns of
photos on maps (i.e., through sunburst icons), we sup-
port users to filter time data to get detailed photos. In
this way, situation S3 (when + where what) is an-
swered.
4 VISUALIZING TAGS WITH
TEMPORAL REFERENCES
In this section, we introduce our solution to cope with
situation S1 (what + when where), where photo
tags are visualized in the context of their temporal ref-
erences.
With Flickr, Google Picasa, and many web-based
photo applications, photos are typically tagged with
textual tags. Normally a set of photos is linked with
a set of tags. In that regard, if examining photos with
time references, we would have different tag sets for
different time points. This leads to the expectation for
the presentation of time-referenced patterns of photos
within tag sets.
To deal with tags over time, there are recent devel-
opments such as time-referenced Taggram (Nguyen
et al., 2011), SparkClouds (Lee et al., 2010), or Paral-
lel Tag Clouds (Collins et al., 2009). Each of them has
advantages and disadvantages if employed for photo
visualization (due to the scope of this paper, we will
not discuss them in detail). In this work, we introduce
the enhancements of the Parallel Tag Clouds for the
visualization of time-referenced photo tags.
4.1 Time-referenced Tag Clouds
Derived from the ideas of tag clouds and parallel co-
ordinates plot, Parallel Tag Clouds is a technique de-
veloped for the exploration and analysis of tag clouds
over axes of tags (Collins et al., 2009). This approach
is applicable for the exploration of time-referenced
photo tags: each axis in the parallel tag clouds shows
a set of photo tags at a time point, and the whole tag
cloud shows all tags over time. However, we have to
enhance the basic approach to allow large volumes of
tags to be examined.
Our idea is to apply a fisheye lens to focus on
axes of interest and tags of interest. It means: (i) tag
clouds’ axes are visually abstracted in different ways
to emphasize different selected time points, and (ii)
tags within each axis are displayed in different sizes
and positions with regard to their levels of interest.
4.1.1 Showing Axes of Interest
An axis of interest is displayed in the size that its tags
are readable, while for other axes, tags are resized
much smaller to fit the display screen (see Figure 1).
(a) One axis is selected. (b) Two axes are selected. (c) Three axes are selected.
Figure 1: Parallel tag clouds with selected axes of interest and tags of interest.
PHOTOTIMA:VISUALEXPLORATIONOFPHOTOSWITHSPATIOTEMPORALREFERENCES
335
Users are supposed to choose particular axes of inter-
est, for example in order to compare tags over a set
of particular time points. Anyway, due to the lim-
ited display area, just a limited number of axes can be
shown with tags in readable sizes (e.g., in Figure 1,
we show at most three selected axes). In this regard,
the axes are updated through user interactions: when
users select an axis, its tags are readable and can be
further examined (see Section 4.1.2); when users de-
select the axis, the tags are minimized. If users se-
lect too many axes, the older axes will be marked as
deselected and temporarily minimized. They will be
shown again when users deselect other axes.
4.1.2 Showing Tags of Interest
The tags shown on each axis are displayed in alpha-
betical order for easy navigating. The heights of the
tags indicate their weights (i.e., the number of related
photos). Each tag can be highlighted as hovered or
selected through mouse interactions. Color and back-
ground of the tags indicate their selected or hovered
states. We show the hovered tags in red with pink bor-
der, while the selected tags are colorized with orange
background (In Figure 2(a), tag “nature” is hovered
and tags “myla” and “nederland” are selected).
Because there could be much more tags than we
can show, on each selected axis just a subset of tags
is displayed. Again, the idea of fisheye menu (Beder-
son, 2000) is integrated. The tags in a selected range
are displayed in full size (according to their weights),
while those which are out of that range are minimized
or removed from the axis’ view (in this case, digits at
the top and bottom of the axis indicate the numbers of
unshown tags). The shown tags are updated through
user interactions (i.e., by scrolling or paging up and
down) on that axis. We also visually link tags over
axes (see Figure 2(b)). Through user interaction, tags
which are identical to the examined tag on the hovered
axis (i.e., tag “nature”) are highlighted with connected
pink lines and red dots on other axes. The size of the
dots are relative to weights of the tags through axes.
If a linked tag is out of range on another selected axis,
its range is updated (in Figure 2(b), tag “nature” is
shifted to the bottom of axis 4 and to the top of axis 9).
Users can select or deselect tags identical to the hov-
ered one on all axes as well (selected tags on the min-
imized axes are encoded in orange color). Lastly, we
support filtering tags in terms of their weights. Fig-
ure 2(c) shows tags with the updated weight range of
[3, 16].
In doing so, tags with different characteristics in
weights, colors, and positions on parallel time plots
of tag clouds are visualized to represent temporal pat-
terns of photos (in form of what + when). In Section
6, we will see how this design is used for the explo-
ration of photos on maps to satisfy situation S1.
5 VISUALIZING PHOTOS WITH
SPATIAL REFERENCES
This section presents the design to cope with situa-
tions S2 (what + where when) and S3 (when +
where what). Temporal patterns of photos are
shown on maps as sunburst icons (to support situa-
tion S2), and pop-up window is used to show detailed
photos for situation S3.
5.1 Photo Thumbnails and Sunburst
Icons
Combining the what (or the when) with the where as-
pects of photos are carried out through different ab-
straction levels of photo aspects in connection with
geographical maps. As thumbnails (the reduced-size
versions for images representation) are typically used
(a) Selected and hovered tags. (b) Examining tags over axes. (c) Tags filtered with weights.
Figure 2: Presenting hovered and selected tags over axes.
WEBIST2012-8thInternationalConferenceonWebInformationSystemsandTechnologies
336
(a) Showing 100 photo thumbnails on maps. (b) Those thumbnails are shown in smaller sizes.
(c) Thumbnails for clusters (radius = 40 pixels). (d) Sunburst icons for clusters (radius = 40 pixels).
Figure 3: Showing photos as thumbnails or sunburst icons on geographical maps.
for recognizing and organizing photos in collections
(e.g., as in the grid-based view of Google Picasa),
they can be used to provide a glance view of photo
distribution over geographical maps (see Figure 3).
However, if we have many photos close together,
showing all thumbnails would cause perceptual prob-
lems. In Figure 3(a), when thumbnails are shown in
size of 40x40 pixels, they overlap each other. If they
are shown in smaller sizes (e.g., by 14x14 pixels as in
Figure 3(b)), the overlapping problem reduces, but the
thumbnails are too small to comprehend. Therefore,
alternative visual abstractions are needed.
Firstly, we suggest to selectively show a subset of
thumbnails in comprehensible sizes (i.e., 40x40 pix-
els). For this purpose, we cluster photos based on
their geo-coordinates at each zooming level of the
map. A cluster is composed of a list of photos within
a local region around a centroid photo. Each centroid
(and then a new cluster) is created when a photo does
not fall into any existing cluster. In Figure 3(c), pho-
tos from Figure 3(a) are clustered with radius = 40
pixels. In this manner, the clusters form an orthogo-
nal distribution of photos over geo-space. The clus-
ters will be redistributed when the map changes its
zooming level. With each cluster, we show the thumb-
nail for a photo of particular interest (e.g., the photo
with specific tags). For clusters that contain more
than one photo, a “stack”-background is added to the
thumbnail. However, since photos are not evenly dis-
tributed on maps (i.e., they are condensed at some
places and sparser at other places), thumbnails with
stack-background do not differentiate such informa-
tion. Therefore, additional visual cues are added. For
instance, numbers are added on the thumbnails to in-
dicate how many photos are examined with that clus-
ter (see Section 6). This design allows for the repre-
sentation of what + where aspects.
Secondly, we further show temporal patterns of
photos in form of visual icons on maps. In doing so,
we can provide insights about where + when patterns.
Within each geo-referenced cluster of photos, there
could be interesting information about their tempo-
ral references. For instance, some photos co-occur
at specific days in week. Thus, for each cluster of
photos, we design a sunburst icon with three main
parts: (1) a center circle with a number indicating the
number of photos in the cluster, (2) a ring with labels
for cyclic temporal patterns (we developed 3 patterns:
PHOTOTIMA:VISUALEXPLORATIONOFPHOTOSWITHSPATIOTEMPORALREFERENCES
337
hours-in-day, days-in-week, and months-in-year), and
(3) arcs linked with the ring indicating the numbers
of photos falling into the associated time slots. There,
the length and lightness of an arc represent its number
of photos. Figure 3(d) illustrates our design.
In our implementation (see Section 6), photos are
specifically selected for examination, we therefore
colorize the arcs in orange for selected photos, and
in blue to indicate the unselected ones. If an arc is too
long (i.e., exceeding the space reserved for the sun-
burst icon), it is shortened and colorized darker (in
Figure 3(d), the cluster in Europe is darker than those
at the other places).
5.2 Browsing Full-size Photos
With photo thumbnails or sunburst icons, we get the
overview of photos distributed over geospace and
time. However, for photo browsing, users need to see
the photos in detail as well (this is a necessary task in
any photo viewer tool). In that regard, we do the same
as Google Panoramio in showing photos on demand.
Users click on a cluster’s thumbnail or choose an arc
on a sunburst icon, and a window is popped-up with
detailed information about relevant photos. We show
on the pop-up window: (1) title of the photo under
examination, (2) its imagery content in size of max
240x240 pixels (Flickr photos’ small size), (3) links
to other photos (if existing) in the cluster or the ex-
amined arc, and (4) navigator link to the source photo
(e.g., the photo on Flickr website with full descrip-
tions, comments, etc.) (see Figure 4(c)).
6 THE TOOL PHOTOTIMA
In this section, we describe the implementation and
demonstrate the tool PhotoTima for the exploration
of spatiotemporal Flickr photos in the context of situ-
ations S1-S3.
6.1 Implementation
We implemented a web-based visualization applica-
tion in Flash, built on Adobe Flash Builder 4.5. The
application PhotoTima is used for the visual explo-
ration of spatiotemporal Flickr photos. Flickr pho-
tos are retrieved directly from Flickr servers through
its APIs (www.flickr.com/services/api), while Google
Maps API (code.google.com/apis/maps/ documenta-
tion/flash) is employed for the manipulation of geo-
graphical maps.
The application consists of three components: a
main toolbar on the top, a viewport for geographi-
cal maps on the left, and a view for time-referenced
parallel tag clouds on the right (Figure 4). We al-
low users to toggle the tags view, select a period
of time in the view, provide some initial tags (if
needed), and then load Flickr data. Through Flickr
API flickr.photos.search, PhotoTima loads a list of
photos, each contains a set of tags, a taken time, and a
geographical coordinate. As Flickr photos are too nu-
merous (e.g., in 2011, millions of photos are uploaded
to Flickr everyday), by default we iteratively load 10
photos per query and refresh the interface. The photos
are loaded from the most interesting ones (criterion
supported by Flickr). Then, tags are grouped in vari-
ous ways: per day, week, month, or year (options are
provided on the tags view, e.g., “months” is selected
in Figure 4). Based on the selection, tags are accumu-
lated for relevant time points (e.g., months) and then
consequently visualized on the plots of the parallel tag
clouds.
With options on the main toolbar, users choose
whether to show photos as thumbnails or sunburst
icons on maps. The thumbnails or sunburst icons
are implemented as overlay objects added on Google
Maps.
6.2 Application
To explore Flickr photos in the context of situation
S1 (what + when where), users are supposed to
specify photo tags and time points on the tags view
and examine them in correlation with geographical
regions of interest. To demonstrate the procedure, we
loaded 200 most interesting Flickr photos taken in the
year 2010 (from Jan 01 to Dec 31) in terms of the tag
“poor”. The tool PhotoTima showed clusters with 51
photos (the most) in the area of Europe, then 40 in In-
dia, 37 around South East Asia, 30 in Central Africa,
and 17 in Central America, while in other areas there
were very few photos (15 photos in the US, and less
than 5 photos on all other areas). However, when we
selected an additional tag “homeless” (which is se-
mantically related to tag “poor” - a criterion provided
by Flickr) just 6 of 51, 5/40, 1/37, 0/30, and 3/17 pho-
tos were counted for those “poorest” areas, respec-
tively, but up to 8 of 15 photos were highlighted at the
area of US (numbers in pink in Figure 4(a)).
Here, users might wonder about the visual pat-
terns. Thus, PhotoTima provides an interlinking
mechanism for further exploration. For example,
users can explore the sunburst icons for temporal pat-
terns of photos on maps (situation S2: what + where
when) or click the thumbnails to see the detailed
photos on maps for situation S3 (when + where
what).
WEBIST2012-8thInternationalConferenceonWebInformationSystemsandTechnologies
338
(a) Searching Flickr photos with preliminary tag “poor” and highlighting patterns with tag “homeless”.
(b) With preliminary tag “football”, sunburst icons show peaks for “July” in South Africa and Europe.
(c) Pop-up window for detailed information of the photo tagged with “stadium” in South Africa.
Figure 4: Screenshots of the tool PhotoTima.
PHOTOTIMA:VISUALEXPLORATIONOFPHOTOSWITHSPATIOTEMPORALREFERENCES
339
We investigated another example for 200 most in-
teresting photos taken in 2010 with initial tag “foot-
ball”. The visualization showed that there were 102
photos in Europe, 28 around North America, 17 in
South America, 15 in South East Asia, and 14 in
South Africa, and so on (Figure 4(b)). What were the
reasons for this distribution?
We found the answers by examining time-
referenced tags of “football” photos. The visualiza-
tion showed that the highest number of tags is on July
(552 tags) and the fewest one is on December (142
tags). It means that Flickr users seem to consider
photos taken in July most, but least in December. In
that regard, we examined how photos were distributed
with time pattern of months over geographical maps.
As seen in Figure 4(b), almost all photos taken in De-
cember were accumulated only for the region of Eu-
rope, while the photos in July appeared as peaks in
South Africa and Europe.
What was that special pattern of July? Why most
of photos in South Africa (12/14 photos) appeared
in July? By clicking on the sunburst piece indicat-
ing July at South Africa, the pop-up window showed
photos with titles such as “World Cup 2010 South
Africa: Spain v Netherlands” or “World Cup 2010
South Africa: Spain v Germany”. The answer was
clear: the particular pattern about “football” photos in
2010 was about World Cup 2010 taken place in July
in South Africa.
Alternatively, we examined how tag patterns in
parallel tag clouds visualization could be used to help
us in finding out that information. One of the related
tags of the tag “football” is tag “stadium”. Interest-
ingly, tag “stadium” appeared in all columns of the
parallel tag clouds. By selecting it on all columns
(i.e., for all photos), the two remaining photos (in-
dicated by February and May) in South Africa’s sun-
burst icon were highlighted. Why those two photos
were about stadium? Could we find any relation be-
tween those photos and the others in July? By click-
ing on them, e.g., the photo in May, we found out that
the description was about the Green Point Stadium I
in Cape Town, and when we navigated to Flickr web-
site, we saw descriptions and comments about this
stadium in its preparation for World Cup 2010 (Fig-
ure 4(c)). From that, we thought that users can also
guess what the photos in July were about.
6.3 Preliminary Feedback
We conducted informal interviews to get a prelimi-
nary feedback for our tool. Six users (two visualiza-
tion experts, and four users from other domains) have
been invited to use PhotoTima to explore Flickr pho-
tos. We first explained the tool and the provided op-
tions (interface components, mouse options, and hot
keys). Afterwards, the users applied PhotoTima to ex-
plore Flickr photos. After having used the tool, they
were asked to give an informal feedback.
In general, all users gave positive feedbacks. They
particularly pointed out that PhotoTima presents use-
ful temporal patterns of Flickr photos on maps. They
said that the tool is easy to use as all of them are fa-
miliar with photo thumbnails, tags, and geographical
maps. In addition, they said that our tool is very com-
prehensible because it smoothly updates the interface.
However, although the tag clouds design was ex-
pressive, more explanations about it were requested.
Besides, the users also expected the tool to be im-
proved with additional supports: (1) providing hints
so that users can select related tags of any tag, (2)
showing all selected tags in a separated view as they
are important information, (3) providing more hints
for the photos from clusters without clicking on the
icons, and (4) automatically updating photos and tags
when the map-view is updated.
7 CONCLUSIONS
In this paper, we have presented an approach for the
visual exploration of Flickr photos with regard to their
spatiotemporal references. We addressed the visual-
ization of complex photo data communicated through
different visual encodings in space, time, and descrip-
tive contents. Geographical maps were employed
to communicate geospatial references, sunburst icons
and time plots were applied for temporal patterns, and
image displaying and tags were used for the presenta-
tion of thematic contents of photos.
The visual representations were linked to provide
insights for spatiotemporal photos exploration. We
combined what + when aspects of photos in the form
of parallel tag clouds (focused tags are shown-in-
context by a fisheye lens). To communicate infor-
mation with regard to where + what and where +
when combinations, we created thumbnails and sun-
burst icons on geographical maps.
By brushing-and-linking the tag clouds view and
the maps view, we allowed users to explore photos
with regard to all three photo aspects. In doing so,
our approach can simultaneously communicate infor-
mation in terms of the what, the where, and the when
of photos, and thus supporting all situations S1-S3.
The tool PhotoTima was developed for the explo-
ration of Flickr photos. Today, there are a lot of sites
for other media sharing, such as video with Youtube -
www.youtube.com, music with Last.fm - www.last.
WEBIST2012-8thInternationalConferenceonWebInformationSystemsandTechnologies
340
fm, and many more. Most of media sharing sites pro-
vide APIs for data access and manipulation. Thus,
exploring and entertaining media with regard to their
spatial, temporal, and descriptive contents (e.g., tags)
are of interest. The prototype approach of PhotoTima
can absolutely be applied for the development of simi-
lar tools for other media. Anyway, how relevant a tool
is depends on the APIs that a media site supports, as
currently most of sharing sites provide APIs suitably
for contents retrieving, instead of time and geoloca-
tion fetching (e.g., even with powerful systems such
as Google Panoramio or Last.fm).
For future work, we will improve the tool Pho-
toTima following the issues raised from users’ feed-
backs. In details, we will show related tags on the
parallel tag clouds, create a magic lens (Bier et al.,
1993) that provides hints for the photos at the exam-
ined cluster, and update the Flickr photos and tags as
the map view is changed. In addition, the tool Photo-
Tima will be examined in the case of using multitouch
tablet computers and smartphones.
REFERENCES
Aigner, W., Miksch, S., Muller, W., Schumann, H., and
Tominski, C. (2008). Visual methods for analyz-
ing time-oriented data. Visualization and Computer
Graphics, IEEE Transactions on, 14(1):47 –60.
Andrienko, N., Andrienko, G., and Gatalsky, P. (2003). Ex-
ploratory spatio-temporal visualization: an analytical
review. Journal of Visual Languages & Computing,
14(6):503–541.
Bederson, B. B. (2000). Fisheye menus. In Proceedings
of the 13th annual ACM symposium on User interface
software and technology, UIST ’00, pages 217–225,
New York, NY, USA. ACM.
Bederson, B. B. (2001). Photomesa: a zoomable image
browser using quantum treemaps and bubblemaps. In
Proceedings of the 14th annual ACM symposium on
User interface software and technology, UIST ’01,
pages 71–80, New York, NY, USA. ACM.
Bier, E. A., Stone, M. C., Pier, K., Buxton, W., and DeRose,
T. D. (1993). Toolglass and magic lenses: the see-
through interface. In Proceedings of the 20th an-
nual conference on Computer graphics and interac-
tive techniques, SIGGRAPH ’93, pages 73–80, New
York, NY, USA. ACM.
Collins, C., Viegas, F., and Wattenberg, M. (2009). Par-
allel tag clouds to explore and analyze faceted text
corpora. In Visual Analytics Science and Technology,
2009. VAST 2009. IEEE Symposium on, pages 91 –98.
Dubinko, M., Kumar, R., Magnani, J., Novak, J., Ragha-
van, P., and Tomkins, A. (2006). Visualizing tags over
time. In Proceedings of the 15th international confer-
ence on World Wide Web, WWW ’06, pages 193–202,
New York, NY, USA. ACM.
Gomi, A. and Itoh, T. (2010). Miaow: a 3d image browser
applying a location- and time-based hierarchical data
visualization technique. In Proceedings of the Inter-
national Conference on Advanced Visual Interfaces,
AVI ’10, pages 225–232, New York, NY, USA. ACM.
Hilliges, O., Baur, D., and Butz, A. (2007). Photohe-
lix: Browsing, sorting and sharing digital photo col-
lections. In Horizontal Interactive Human-Computer
Systems, 2007. TABLETOP ’07. Second Annual IEEE
International Workshop on, pages 87 –94.
Huynh, D. F., Drucker, S. M., Baudisch, P., and Wong,
C. (2005). Time quilt: scaling up zoomable photo
browsers for large, unstructured photo collections. In
CHI ’05 extended abstracts on Human factors in com-
puting systems, CHI EA ’05, pages 1937–1940, New
York, NY, USA. ACM.
Kristensson, P. O., Arnell, O., Bj
¨
ork, A., Dahlb
¨
ack, N.,
Pennerup, J., Prytz, E., Wikman, J., and
˚
Astr
¨
om, N.
(2008). Infotouch: an explorative multi-touch visu-
alization interface for tagged photo collections. In
Proceedings of the 5th Nordic conference on Human-
computer interaction: building bridges, NordiCHI
’08, pages 491–494, New York, NY, USA. ACM.
Lee, B., Riche, N., Karlson, A., and Carpendale, S. (2010).
Sparkclouds: Visualizing trends in tag clouds. Visu-
alization and Computer Graphics, IEEE Transactions
on, 16(6):1182 –1189.
Liu, H., Xie, X., Tang, X., Li, Z.-W., and Ma, W.-Y. (2004).
Effective browsing of web image search results. In
Proceedings of the 6th ACM SIGMM international
workshop on Multimedia information retrieval, MIR
’04, pages 84–90, New York, NY, USA. ACM.
Nguyen, D. Q., Tominski, C., Schumann, H., and Ta, T. A.
(2011). Visualizing tags with spatiotemporal refer-
ences. In Information Visualisation (IV), 2011 15th
International Conference on, pages 32 –39.
Peca, I., Zhi, H., Vrotsou, K., Andrienko, N., and An-
drienko, G. (2011). Kd-photomap: Exploring pho-
tographs in space and time. In Visual Analytics Sci-
ence and Technology (VAST), 2011 IEEE Conference
on, pages 291 –292.
Peuquet, D. J. (1994). It’s about time: A conceptual frame-
work for the representation of temporal dynamics in
geographic information systems. Annals of the Asso-
ciation of American Geographers, 84(3):pp. 441–461.
Porta, M. (2006). Browsing large collections of images
through unconventional visualization techniques. In
Proceedings of the working conference on Advanced
visual interfaces, AVI ’06, pages 440–444, New York,
NY, USA. ACM.
Ryu, D.-S., Chung, W.-K., and Cho, H.-G. (2010). Pho-
toland: a new image layout system using spatio-
temporal information in digital photos. In Proceed-
ings of the 2010 ACM Symposium on Applied Com-
puting, SAC ’10, pages 1884–1891, New York, NY,
USA. ACM.
Toyama, K., Logan, R., and Roseway, A. (2003). Geo-
graphic location tags on digital images. In Proceed-
ings of the eleventh ACM international conference
on Multimedia, MULTIMEDIA ’03, pages 156–166,
New York, NY, USA. ACM.
PHOTOTIMA:VISUALEXPLORATIONOFPHOTOSWITHSPATIOTEMPORALREFERENCES
341