PLOG:

Easily Create Digital Picture Stories

through Cell Phone Cameras

Rich Gossweiler, Joshua Tyler

HP Labs, Palo Alto, California, U.S.A. 94304

Abstract. With a cell phone camera a person can take pictures and imme di-

ately transmit them to larger repositories using telecommunication channels.

This provides the potential for anywhere, anytime picture sharing and story

telling. It can also increase awareness among peers within a social network.

But while we enjoy telli ng stories around this informal content, the necessary

act of downloading, organizing and then presenting the digital media is tedious

and can discourage frequent use. We present PLOG, an application that

stresses ease-of-use story telling through cell pho ne photography. The user

simply takes pictures and performs simple click -and-send uploads using their

phone. From a local machine, PLOG automatically downloads the images;

time clusters the pictures together and then presents them as story vignettes.

These vignettes can be viewed different ways and at home, shared with friends

or posted in public places. We describe the implementation of PLOG and di s-

cuss issues related to this light -weight method of story telling and awareness.

1 Introduction

People enjoy telling stories through photographs. [1] references [2], stating "one of

the most common and enjoyable uses for photographs is to share stories about e x-

periences, travels, friends and family." By using cell phones equipped with digital

camera technologies, it is easy and convenient for people already carrying phones to

snap informal, interesting pictures and immediately transmit these pictures. People

do not have to carry a camera as another device along with the phone, people do not

have to deal with cables and downloading images to computers and people can d i-

rectly target members of their social network with interesting pictures.

At the other end of this digital photography channel we are seeing an increasing

number of appliances that allow people to rec eive and view these pictures. There

exists everything from commercially available digital picture frames [ 3] to exciting

research in new devices for storytelling [ 1], [4]. Cell phone cameras present the

opportunity to not only take and transmit pictures e asily, but a deeper, richer oppo r-

tunity to tell stories and pass on interesting vignettes that strengthen social ne tworks.

But while we can imagine taking a short series of pictures on the ubiquitous cell

phone (e.g. "My day at the beach" or "funny thing a t the park") and then beaming

Gossweiler R. and Tyler J. (2004).

PLOG: Easily Create Digital Picture Stories through Cell Phone Cameras.

In Proceedings of the 1st International Workshop on Ubiquitous Computing, pages 94-103

DOI: 10.5220/0002669300940103

 SciTePress

the set to a friend, the actual process can be tedious and discouraging. It is not u n-

common that I have to:

1. Take the picture and preview it to make sure it is reasonable

2. Type an email address on a cell phone keypad (or naviga te contact list menus)

and attach the picture

3. Press send and wait for the transmission to complete

4. Repeat for each picture

This becomes more difficult when I want to string together several pictures as a

group, want to show them to several people and/or w ant to send them to myself for

future storytelling. If the cognitive load for these tasks is sufficiently high, then

pictures don't get taken, pictures don't get sent, or pictures are sent and shown once

and then rarely viewed again. More generally, this is a systemic problem that we

often observe in many distributed systems: Getting the Data In, Information Ma n-

agement, getting the Data Out.

Our solution, PLOG, is based on the belief that we want a simple, end -to-end

mechanism that allows us to easily cre ate short stories (picture vignettes) while we

are going about our daily lives. These vignettes should be presented to interested

viewers with minimal effort on either the creator's or viewer's part. PLOG allows the

photographer/story teller to click and o ne-button-send pictures (a shoot-and-forget

model) to an internet repository. PLOG employs back end software to automatically

and continuously gather new pictures from the repository onto the interested viewer's

local machine, time-cluster them into vignettes and present them as story sequences.

Figure 1: PLOG uses simple click -and-send, then clusters and shows vignettes locally

For the creators, they simply take and send pictures, using the internet repository

as their infinite "film roll" without worrying about running out of memory or what

email addresses to type in. For the people viewing on the other end, they can simply

watch or be aware of new vignettes as they come in. These vignettes can be saved,

distributed and shown again as small stories at later times.

Since the presentation uses templates, different templates can control how the

story is revealed (for example, in our implementation we have both a slideshow pre s-

entation and a comic strip presentation). Not only does this model support various

creative presentations, but it also allows devices with different capabilities to present

the story optimized to the device’s capabilities. For example, viewing the images

from another cell phone with a web browser might use a sl ideshow model due to the

small screen real estate, while a large plasma display might present several pictures

as a comic strip.

Figure 2: PLOG from picture capture to presentation

PLOG makes it easy for people to tell digital p icture stories, from the process of

getting the picture into the system (data in), to low -maintenance organizing via i m-

plicit time-clustering (data organization) to presenting and potentially printing the

story (data out).

2 Related Work

In [5], Frohlich et al outlined requirements for Photoware based on ethnographic

studies. In their discussion they make recommendations, including the ability to

perform instant photo sharing. While not instant, PLOG significantly shortens the

gap. Based on the speed of the digital repository, viewers may be able to see updates

within a minute. Frohlich also suggests the requirement of "sending miscellaneous

collections of reprints to particular people with a personalized message" which

PLOG attempts to do. The sender could personalize a story, perhaps by typing in a

message on the phone. PLOG allows, but does not require people to annotate their

photos. Another button option opens up a text box to allow photo annotation.

FotoFile [6] is a consumer multimedia organization and retrieval system. It sup-

ports "scraplets" which are small groupings of photos which can be arranged chron o-

logically. PLOG and FotoFile both share the capability to create small grou pings for

story telling. FotoFile however was developed to improve the org anization process of

existing photos while PLOG attempts to create a simple story telling pro cess.

KAN-G [7] is much closer to PLOG, informing recipients when new photos are

available. KAN-G was described as a framework with emphasis on awareness

through digital pictures in the home and shares many of the high level goals that

PLOG has (e.g. ease-of-use, implicit interaction and awareness). PLOG could be

categorized as a specific implementation within the spirit of the KAN -G framework.

PLOG specifically differs from KAN-G by focusing on the story telling aspect, time

clustering and use of a camera cell phone.

Another area of related work deals with the process of auto -albuming [8] [9].

PLOG uses a simple time-clustering algorithm. This allows PLOG to create vignettes

based on the premise that people take pictures in tight groups around a particular

story. Other systems also use various clustering and content -based information to

group digital images. For example, Graham et al [ 10] compares time-clustered

photo browsers to measure how temporal grouping improves finding images. [ 8] uses

time-clustering and histogram (image comparison) to create a software system for

automatic albuming. Others have commented [ 10] that k-means clustering, which

pre-defines the number of slots, is questionable, since knowing that number is diff i-

cult. Using histogram or image -content information may not be acc urate, since a

given vignette may have very different pictures in it (e.g. a picture of a cake followed

by a picture of a child at a dinner table). Another example of time -clustering is [11].

In addition to automatic organization, there has been ample work in presentation,

not only in albums and on the web, but with new devices that incorporate the story

telling process into helping organize the photos [ 4], [1].

We use Text America (www.text\america.com

) to store our pictures. It is a photo

weblog site. Weblogs are an example of presenting lists of annotated links, logs and

stories using the internet [12]. Unlike typical web pages that require an editing tool

and a publishing process to upload pages, blogs attempt to make it easy to edit and

link information together. Novel “trackback” links allow people to see who has

linked to their weblog links.

3 Implementation

PLOG uses two pieces of software, one that is installed on your phone to make the

picture sending simpler, and one that is installed on your local machine to receive,

time-cluster and present the vignettes. In between w e use a commercial World Wide

Web service (www.textamerica.com) to store the photography and associated meta -

data. Since the software is modular, you can replace and configure different elements

(such as the time clustering, the web service, etc.) based on your needs. The follow-

ing describes the click-and-send phone application, the internet repository and the

PLOG application on your local machine. The PLOG application is broken down

into initialization, time-clustering and presentation.

3.1 Click-And-Send: Cell Phone Software

While cell phone cameras let you take pictures, often you need to type in an email

address or navigate through an options menu and contact list to deliver each picture.

On a cell phone this is tedious and takes away from the otherwis e fluid process of

taking the picture. It potentially limits the number of photos one takes, and therefore

the depth of the experience captured) To reduce this cognitive load, we wrote a small

application that let's you set up the email address initially. From then on, it is only

two or three clicks to send the image. Picture taking and sending moves to a click -

and-send model where you can just quickly send the picture off to a repository and

then go take another picture. Our implementation was developed fo r and runs on

Nokia Series 60 Symbian OS phones, which include the Nokia 3650, 7650, and 6600

models.

Figure 3: click-and-send simplicity

3.2 Internet Repository

The primary role of the internet repository is to hold the images so that we can de-

liver them from anywhere and also receive them from anywhere. Since the cell

phone uses email over the telecommunication channel (either SMTP or MMS over

GPRS), the web hosting site needs to be able to receive this protocol. In our impl e-

mentation, we used TextAmerica ( www.textamerica.com

), a photo weblog site that

provides an email upload interface for photos. One advantage to this site was that it

creates a Really Simple Syndication (RSS) feed [13]. RSS is an XML protocol that is

growing in popularity as a delivery mechanism for information. This RSS allows us

to write specific viewers based on a well -known and defined XML syntax instead of

trying to "scrape" HTML syntax. Generally, one can use any internet hosting site

that supports an email server mechanism. Generating well -formed RSS XML from

the set of pictures is relatively easy.

For our implementation, we used the met a-data: unique-id (guid), title, pubDate,

imageURL. An example of the RSS is:

<item>

<title>My Lunch</title>

<link>

http://someone.textamerica.com/default.asp?r=215513

</link>

<textamerica:fullimg >

http://mlog.textamerica.com/IMG_341879/IMG_someone_0_12

05223126140.jpg

</textamerica:fullimg>

<textamerica:thumbimg>

http://mlog.textamerica.com/IMG_341879/Thumb/IMG_someon

e_0_1205223126140.jpg

</textamerica:thumbimg>

<textamerica:replycount>0</textamerica:replycount>

</item>

3.3 PLOGViewer

PLOGViewer is the local "browser" or player for the people viewing the stories on

their local machines/display devices. It is a single software application that period i-

cally checks for new content fr om the website using the RSS feed, downloads the

new images, time clusters them into vignettes and then displays them (e.g. slid e-

show or comic book).

3.3.1 Initialization XML

PLOGViewer uses an initialization XML file to configure where to go for the RSS

feed and how frequently to check for updates to the RSS XML file. This initialization

file also describes a mapping from the RSS tags to the fields PLOG uses. This is

necessary because there is no <IMAGE> tag in RSS, so different feeds provide i m-

ages different ways. For example, textamerica uses <textamerica:fullimage> while

another feed might simply put the image URL in the <DESCRIPTION> field.

Once started, PLOGViewer periodically downloads the RSS, checks for new i m-

age URLs not already locally stored (ba sed on the unique-id or GUID) and

downloads the new pictures. The complete set of images is then time -clustered into

vignettes and presented.

3.3.2 Time-Clustering

We use clustering algorithms based on the premise that people take closely related

pictures together in time. Clustering is the process of trying to identify groups of data

within a larger set. This seems relatively easy when the clusters are well defined, but

there can be many cases where the clustering is ambiguous. Some clustering alg o-

rithms define a fixed number of clusters (e.g. k -means) while others create clusters

as needed (e.g. some heirarchical clustering). We refer you to [ 14] for more informa-

tion on clustering.

Time-Clustering is an ease-of-use example for the PLOG viewer. We use it to al-

low the system to automatically create vignettes based on usage patterns (we assume

a property of temporal locality for pictures). The trade -off is that the clustering may

be inaccurate. One can imagine using different clustering algorithms, learning he u-

ristics, etc. to improve the accuracy of the stories the creator intended to make.

3.3.3 Presentation

Once clustered, the PLOG viewer rotates through each cluster (a cluster is a photo

vignette). Each cluster can then be presented in a variety of ways an d we have im-

plemented two. The first is as a slide -show. Each image is saved out with an assoc i-

ated HTML file. The PLOG viewer uses an IE COM object to show the HTML page,

then fade to the next HTML-enveloped image.

In addition to PLOG being a self-contained presenter, this presentation model a l-

lows the creator to bundle the HTML together and present it anywhere an HTML

browser exists. The second template lays out the images of a cluster into a "comic

strip." and then animates through each frame of the comi c (highlighting one frame

while leaving the other cells in the background). Again, this is accomplished using a

series of HTML files for each animation section. One can imagien creating other

presentation templates (e.g. a collage).

In both cases, since HTML is used, it is relatively easy to create a "Print" button

that will print out the slide set or comic strip for offline use (e.g. a storybook). Once

through the complete set of vignettes and after a configurable time has passed, the

PLOG viewer rescans for new images.

100

Figure 4: screenshots of slideshow and comic presentation

4 Discussion

PLOG makes it easy to click -and-send pictures on the front -end and automatically

cluster them into stories or vignettes on the back end. But PLOG is not without its

issues and future work.

4.1 Issues

PLOG relies on the telecommunication channel to send the pictures. If the channel is

not available, then the pictures need to be batched up and trickled down (or

downloaded through another channel , such as bluetooth or a cable connection).

PLOG uses a very simple time-clustering algorithm. We are considering more s o-

phisticated alternatives (another topic of research). Even the best automatic time -

clustering algorithm cannot predict the story you we re trying to tell in your head

(much less, deal with interleaved stories). Mechanisms for fluidly tuning and post -

production editing might further improve the stories and vignette value. For example

as the author sits down and later tells the story to a fr iend, he or she might adjust the

vignette.

4.2 Future Work

PLOG uses the cell-phone for data in, but other sources can also feed into PLOG

(such as batch downloads from Bluetooth, from web pages). Advanced story telling

might allow the author to tell the story and tune the vignettes. Other meta -data such

as location and environmental conditions might enhance the story.

101

We would also like to support categorization on at least two levels. One is to cr e-

ate friend categories -- send these to my friends and my family, or send these only to

my private space. Second, stories themselves may be categorized (work -related, vaca-

tion-related).

While PLOG focused on story telling, other derivatives may be useful. For exa m-

ple, medical examiners, police investigators, anyo ne dealing with field operations

and temporally related pictures might want to cluster a collection of data together.

Frohlich [5] has several suggestions that would be interesting to implement to e x-

tend PLOG. They suggest photo -conferencing tools. PLOG mi ght be able to support

this, not only phone-to-machine, but phone-to-phone. They also suggest multi -user

album software. Extending PLOG to allow users to re -weave the vignettes or com-

bine them might be fluid if done as part of the story telling process. Th ey also sug-

gest feedback mechanisms and audio -annotation. It would be interesting research to

see if people take more or different pictures based on social network effects and

whether they would annotate the pictures with voice.

4.3 Conclusion

We believe that we have implemented a very easy-to-use system for creating and

distributing stories through cell phones. We hope that this will improve the process

of cell phone camera story telling and strengthen social networks. We also believe

that there is an opportunity for continued research and exploration in this area of

anywhere, anytime story telling.

Acknowledgements

We would like to thank Gene Becker and Eytan Adar at HP Labs, Patrick Goddi’s

Contextual Media Systems (CMS) group at HP Labs, Stephen Sorkin at Berkeley and

Text America (www.textamerica.com

References

[1] Balabanovi•, M., Chu, L., Wolff G., Storytelling with Digital Photographs , proceedings

of SIGCHI’00 , pp. 564-571, April 2000.

[2] Chalfen, R. Snapshot Versions of Life . Bowling Green State Unive rsity Press, Bowling

Green OH, 1987.

[3] http://www.pacificdigitalcorp.com/products/memoryframe.asp

[4] Shen, C., Lesh, N., Verni er, F., Forlines, C., Frost, J. Sharing and Building Digital Group

Histories, proceedings of CSCW'02 , pp. 324-333, November 16-20, 2002.

[5] Frohlich, D., Kuchinsky A., , Pering, C., Don, A., Ariss, S., Requirements for Phot oware,

Proceedings of. CSCW 20 02, pp. 166-175, November 16-20, 2002.

102

[6] Kuchinsky, A., Pering, C., Creech, M., Freeze, D., Serra, B., Gwizdka, J., FotoFile: A

Consumer Multimedia Organization and Retrieval Sy stem, proceedings of SIGCHI’99 ,

pp. 496-503, 1999.

[7] Liechti, O., Ichikawa, T., A Digital Photography Framework Enabling Affective Awar e-

ness in Home Communication , Personal Technologies , 4(1), pp. 6-24, 2000.

[8] Loui, A., Wood, M., A Software System for Automatic Albuming of Consumer Pi ctures,

ACM Multimedia’99 , pp. 159-162, 1999.

[9] Rodden, K., Wood, K., How do People Manage Their Digital Photographs? , proceedings

of SIGCHI’03 , pp. 409-416, April 5-10, 2003.

[10] Graham, A., Garcia -Molina, H., Paepcke, A., Winograd, T., Time as Essence for Photo

Browsing Through Personal Digit al Libraries, proceedings of JCDL’02 , pp. 326-335,

July 13-17, 2002.

[11] Cooper, M., Foote, J., Girgensohn, A., Wilcox, L., Temporal Event Clustering for Dig ital

Photo Collections , Multimedia’03 , pp. 364-373, November 2-8, 2003.

[12] Blood, R., The Weblog Handbook: Practical Advice on Creating and Maintaining Your

Blog, Perseus Publishing , Cambridge, Massachusetts, 2002.

[13] http://blogs.law.harvard.edu/tech/rss

[14] Manning, C., Schütze, H. , Foundations of Statistical Natural Language Proces sing,

MIT Press, Cambridge, Massachusetts, 1999.

103