SPONTANEOUS AND PERSONALIZED ADVERTISING THROUGH

MPEG-7 MARKUP AND SEMANTIC REASONING

Exploring New Ways for Publicity and Marketing over Interactive Digital TV

Martín López-Nores, José J. Pazos-Arias, Jorge García-Duque

Yolanda Blanco-Fernández, Marta Rey-López and Esther Casquero-Villacorta

Department of Telematics Engineering, University of Vigo, Spain

Keywords:

Interactive Digital Television, spontaneous advertising, personalization, t-commerce.

Abstract:

Publicity is one of the sustaining pillars of the television industry. In an increasingly competitive market, the

involved agents are striving to exploit all the possibilities to get revenues from advertising, but their techniques

lack targeting and are usually at odds with the comfort of the TV viewers. In response to those problems, this

paper introduces a new advertising model that aims at harnessing the interactive capabilities of the modern

TV receivers (either domestic or mobile ones). The approach is based on automatically identifying products

which are semantically related to the things on screen that catch the viewer’s attention, and then assembling

interactive services that provide him/her with personalized commercial functionalities.

1 INTRODUCTION

Numerous market studies point out a signiﬁcant drop

in the effectiveness of the advertising techniques cur-

rently employed on TV, suggesting that publicity

needs to be reinvented in the years to come (Kim,

2006). In order to preserve such a fundamental source

of income, it is necessary to address the limitations of

those techniques, which stem mostly from presenting

the same products to all the TV viewers in a way that

interferes (temporally or spatially) with their enjoy-

ment of the audiovisual contents.

Our vision in this paper is that the solution to the

aforementioned problems will come from the grow-

ing development of the Interactive Digital Television

(IDTV) technologies, due to the possibility of trans-

mitting interactive software applications jointly with

the audiovisual contents. Speciﬁcally, we present here

a system that implements a new advertising model

following three basic steps:

• Broadcasting the audiovisual contents along with

metadata characterizing the things that will appear

on screen in the different scenes.

• Identifying products which are semantically re-

lated to the things that draw the viewer’s attention

on the screen.

• Assembling interactive services (hereafter, i-

spots) that provide the viewer with commer-

cial functionalities suited to his/her interests and

needs.

We refer to this model as spontaneous and person-

alized advertising, because it does not require insert-

ing publicity in the audiovisual contents, and it adapts

to the peculiarities and circumstances of each indi-

vidual viewer. The following section describes the

main design features of the implementing system; af-

ter that, Section 3 discusses a sample application sce-

nario, and the paper ﬁnishes with a summary of con-

clusions in Section 4.

2 SYSTEM OVERVIEW

Our advertising system assumes a receiver architec-

ture following the GEM standard (DVB consortium,

2006), with a Java-based middleware deﬁning the

mechanisms available to access the broadcast net-

works and the return channels, to construct the inter-

active applications, etc. This architecture is now com-

mon for the ﬁxed IDTV receivers at homes, and is also

being considered for the incipient developments over

mobile receivers (DVB consortium, 2004).

217

López-Nores M., J. Pazos-Arias J., García-Duque J., Blanco-Fernández Y., Rey-López M. and Casquero-Villacorta E. (2007).

SPONTANEOUS AND PERSONALIZED ADVERTISING THROUGH MPEG-7 MARKUP AND SEMANTIC REASONING - Exploring New Ways for

Publicity and Marketing over Interactive Digital TV.

In Proceedings of the Second International Conference on Signal Processing and Multimedia Applications, pages 213-217

DOI: 10.5220/0002131402130217

 SciTePress

Figure 1: The operational scheme of the spontaneous and personalized advertising model.

The operational scheme of our system, shown in

Figure 1, includes four core elements, to be described

in the following subsections. Since the personaliza-

tion tasks exceed (by far) the memory and comput-

ing capabilities of an IDTV receiver, the elements in-

volved are primarily placed in remote personalization

servers; nonetheless, there are also local instances in

the receivers to enable sub-optimal operation in cases

of intermittent, sporadic or null access to a return

channel.

2.1 The Interaction Engine

The interaction engine is the element in charge of de-

tecting the viewer’s interest for the things appearing

on screen. To this aim, it introduces different inter-

active elements depending on the input capabilities of

the receiver:

• When using a remote control or the keypad of a

mobile phone, a certain button opens a menu list-

ing the things that have appeared on the current

scene thus far. The viewer can then navigate to

select the particular thing he/she is interested in.

• When using a device with a touch screen (for in-

stance, a PDA), the viewer can click directly on

the things he/she wants with a ﬁnger or using a

stylus.

Obviously, the system allows interaction only with

the things characterized in the metadata linked to

the audiovisual contents, which are expressed follow-

ing the MPEG-7 standard (Manjunath et al., 2002).

Whichever the interaction method, the viewer can

choose to launch i-spots immediately, or later (most

commonly, when the current program ﬁnishes). In

the former case, if the IDTV receiver allows opera-

tion as a PVR (Personal Video Recorder), it is even

possible to do time shift, i.e. to pause the programs

while the viewer explores the i-spot, and then resume

the viewing.

2.2 The AVATAR Recommender

The AVATAR recommender is responsible for identi-

fying products related to the things that have caught

the viewer’s attention, and then selecting those which

best match his/her preferences, interests and needs.

This is done by applying the semantic reasoning al-

gorithms presented in (Blanco-Fernández et al., 2006)

over the following sources of information:

• The metadata describing the things and the action

on screen.

• Ontologies that characterize and interrelate audio-

visual contents, commercial products and adver-

tising material —using metadata ﬁelds from the

TV-Anytime speciﬁcations (TV-Anytime forum,

SIGMAP 2007 - International Conference on Signal Processing and Multimedia Applications

218

2003), MPEG-7 and several ad hoc syntaxes.

• A proﬁle containing demographical data of the

viewer (age, gender, job, ...), together with point-

ers to contents/products that he/she has evaluated

in the past (each one labeled with a degree of in-

terest, DOI) and products that he/she owns (la-

beled with a degree of satisfaction, DOS).

The ﬁltering is triggered by context information

indicating what the viewer is watching. The person-

alization servers proceed over complete ontologies

and viewer proﬁles, whereas the local instances of

AVATAR handle partial versions due to the limited

resources. In the latter case, the quality of the rea-

soning can be enhanced using ontological information

and stereotypical proﬁles received through the broad-

cast networks. This supporting material, inserted in

the IDTV head-end, can be tailored to the programs

and the advertisements which are broadcast at any

moment.

2.3 The I-Spot Composer

The product selections made by AVATAR are the en-

try for the i-spot composer to assemble interactive ser-

vices that let the viewers navigate for detailed infor-

mation about the products, search for the closest es-

tablishment where they can be bought, purchase on-

line, subscribe to the notiﬁcation of novelties, etc.

It is foreseeable that many i-spots will provide es-

sentially the same functionality. In our system, there-

fore, they are assembled from template classes, and

it is up to the i-spot composer to decide which tem-

plates to use (considering hardware requirements, re-

turn channel availability, ...) and which contents to

lay over them (received through broadcast or down-

loaded from the Internet). The outcome of those de-

cisions is later used in the receivers to start the ser-

vices, retrieving the corresponding templates from the

broadcast stream and setting them up using the Java

reﬂection mechanisms. This approach enables good

use of bandwidth while not increasing the computa-

tional cost for the receivers, as it is not necessary to

modify or recompile any source code.

Remarkably, the selection of contents in the i-spot

composer can be driven by the same semantic reason-

ing algorithms of AVATAR, to increase the odds that

the viewer will appreciate the information included in

the i-spots.

2.4 The Feedback Agent

Just because the interests, preferences and needs of

the viewers may vary with time, the model of spon-

taneous and personalized advertising requires mech-

anisms to update their proﬁles by capturing new data

and discarding obsolete knowledge. In literature, one

can ﬁnd proposals to do this implicitly, by inferring

information from the viewers’ interaction with a sys-

tem, or explicitly, by asking the viewers to enter some

information from time to time.

Our prototype system supports both forms of feed-

back to recompute the DOI values stored in the

proﬁles, for which it applies the same functions of

gradual forgetting and relevance feedback deﬁned

in (Montaner et al., 2002). The implicit form gath-

ers information by monitoring how long the viewer

takes to learn about the different products, whether

he/she decides to buy, hire or subscribe, and how

much money he/she spends. On the other hand, the

explicit form relies on interactive questionnaires, con-

structed just the same way as the i-spots. We also

provide the viewers with questionnaires to enter DOS

values for the products they own, and to update their

demographical data.

3 A SAMPLE SCENARIO

For a brief example, consider the case of a TV viewer

whose proﬁle indicates, among other facts, that she

has two children and a middle-income economy (de-

mographical data), that she is a loyal viewer of pro-

grams devoted to travel (viewing history), and that she

is subscribed to an oenology magazine (consumption

data). The viewer is watching a documentary about

Switzerland on her PDA, with the current scene de-

scribing a town in the Geneva canton. As shown in

Figure 2, the metadata linked to this part of the audio-

visual contents identify four things on screen, namely

the village, the sky, the lake and the vineyard. These

entities are related to some concepts in the system on-

tologies, represented by the gray and dashed elements

of the ﬁgure.

If the viewer clicked over the village, our sys-

tem would be able to locally assemble one i-spot de-

scribing affordable trips and tourist attractions around

Geneva, getting the contents from broadcast. Alter-

natively, if bi-directional communication were avail-

able, another i-spot could be designed that described

rural tourism houses in the viewer’s area, emphasiz-

ing those which offer facilities for children, and giv-

ing the viewer the possibility to book a room for her

holidays (see Figure 3).

On the other hand, clicking on the vineyard, the

viewer’s expertise in wine would take our system to

display a list of remarkable vintages (with i-spots al-

lowing her to purchase some bottles), trips to famous

wineries, etc. Other viewers would be offered wines

SPONTANEOUS AND PERSONALIZED ADVERTISING THROUGH MPEG-7 MARKUP AND SEMANTIC

REASONING - Exploring New Ways for Publicity and Marketing over Interactive Digital TV

219

Figure 2: The image on screen, with sample MPEG-7 markup and links to the system ontologies.

Figure 3: A personalized i-spot.

for everyday consumption, guides for cultivating, or

simply indications to get to the nearest bar.

4 CONCLUSIONS

In this paper, we have outlined the implementation of

a new advertising model for Interactive Digital TV,

that exploits the markup of audiovisual contents to

face the viewers with products semantically related to

the things that catch their attention on the TV screen.

This approach is intuitive and pleasant for the view-

ers, because it does not interfere with their perception

of the TV programs and, besides, it removes the need

to populate the screen with advertising material.

In what concerns commercial exploitation, our

proposal can achieve much better targeting than the

current advertising techniques, inasmuch as the pub-

licity is selected by taking into account the prefer-

ences, interests and needs of each individual viewer.

In this regard, to the best of our knowledge, there is

only one precedent of personalized advertising for TV

in (Lekakos and Giaglis, 2004), but that work was

limited to selecting commercial breaks.

Finally, we want to emphasize that the automatic

aggregation of interactive commercials is certainly

one step forward to harness the potential of the IDTV

technologies to support t-commerce and m-commerce

platforms —as argued in the survey of (Myers Group,

2006), that potential is enormous, but remains heav-

ily underexploited. Our current work is precisely de-

voted to this part of our system, in an attempt to port

our solutions to the standard technologies used for au-

tomatic service composition in the ﬁeld of Web Ser-

vices (Milanovic and Malek, 2004).

SIGMAP 2007 - International Conference on Signal Processing and Multimedia Applications

220

ACKNOWLEDGEMENTS

This work has been supported by the Spanish Min-

istry of Education and Science (Project TSI2004-

03677).

REFERENCES

Blanco-Fernández, Y., Pazos-Arias, J., López-Nores,

M., Gil-Solla, A., and Ramos-Cabrer, M. (2006).

AVATAR: An improved solution for personalized TV

based on semantic inference. IEEE Transactions on

Consumer Electronics, 52(1):223–231.

DVB consortium (2004). Transmission system for handheld

terminals. ETSI standard EN 302 304.

DVB consortium (2006). Globally Executable MHP. ETSI

standard TS 102 819.

Kim, P. (2006). Advertisers face TV reality. Business View

Trends, Forrester Research.

Lekakos, G. and Giaglis, G. (2004). A lifestyle-based ap-

proach for delivering personalised advertisements in

digital interactive television. Journal of Computer-

Mediated Communications, 9(2).

Manjunath, B. S., Salembier, P., and Sikora, T. (2002). In-

troduction to MPEG-7: Multimedia content descrip-

tion language. John Wiley & Sons.

Milanovic, N. and Malek, M. (2004). Current solutions for

Web Service composition. IEEE Internet Computing,

8:51–59.

Montaner, M., López, B., and de la Rosa, J. (2002). Improv-

ing case representation and case base maintenance in

recommender agents. Lecture Notes in Computer Sci-

ence, 2416:234–248.

Myers Group (2006). T-commerce case studies.

http://www.researchandmarkets.com.

TV-Anytime forum (2003). TV-Anytime speciﬁcation se-

ries. ETSI standard TS 102 822.

SPONTANEOUS AND PERSONALIZED ADVERTISING THROUGH MPEG-7 MARKUP AND SEMANTIC

REASONING - Exploring New Ways for Publicity and Marketing over Interactive Digital TV

221