SPONTANEOUS AND PERSONALIZED ADVERTISING THROUGH
MPEG-7 MARKUP AND SEMANTIC REASONING
Exploring New Ways for Publicity and Marketing over Interactive Digital TV
Martín López-Nores, José J. Pazos-Arias, Jorge García-Duque
Yolanda Blanco-Fernández, Marta Rey-López and Esther Casquero-Villacorta
Department of Telematics Engineering, University of Vigo, Spain
Keywords:
Interactive Digital Television, spontaneous advertising, personalization, t-commerce.
Abstract:
Publicity is one of the sustaining pillars of the television industry. In an increasingly competitive market, the
involved agents are striving to exploit all the possibilities to get revenues from advertising, but their techniques
lack targeting and are usually at odds with the comfort of the TV viewers. In response to those problems, this
paper introduces a new advertising model that aims at harnessing the interactive capabilities of the modern
TV receivers (either domestic or mobile ones). The approach is based on automatically identifying products
which are semantically related to the things on screen that catch the viewer’s attention, and then assembling
interactive services that provide him/her with personalized commercial functionalities.
1 INTRODUCTION
Numerous market studies point out a significant drop
in the effectiveness of the advertising techniques cur-
rently employed on TV, suggesting that publicity
needs to be reinvented in the years to come (Kim,
2006). In order to preserve such a fundamental source
of income, it is necessary to address the limitations of
those techniques, which stem mostly from presenting
the same products to all the TV viewers in a way that
interferes (temporally or spatially) with their enjoy-
ment of the audiovisual contents.
Our vision in this paper is that the solution to the
aforementioned problems will come from the grow-
ing development of the Interactive Digital Television
(IDTV) technologies, due to the possibility of trans-
mitting interactive software applications jointly with
the audiovisual contents. Specifically, we present here
a system that implements a new advertising model
following three basic steps:
Broadcasting the audiovisual contents along with
metadata characterizing the things that will appear
on screen in the different scenes.
Identifying products which are semantically re-
lated to the things that draw the viewer’s attention
on the screen.
Assembling interactive services (hereafter, i-
spots) that provide the viewer with commer-
cial functionalities suited to his/her interests and
needs.
We refer to this model as spontaneous and person-
alized advertising, because it does not require insert-
ing publicity in the audiovisual contents, and it adapts
to the peculiarities and circumstances of each indi-
vidual viewer. The following section describes the
main design features of the implementing system; af-
ter that, Section 3 discusses a sample application sce-
nario, and the paper finishes with a summary of con-
clusions in Section 4.
2 SYSTEM OVERVIEW
Our advertising system assumes a receiver architec-
ture following the GEM standard (DVB consortium,
2006), with a Java-based middleware defining the
mechanisms available to access the broadcast net-
works and the return channels, to construct the inter-
active applications, etc. This architecture is now com-
mon for the fixed IDTV receivers at homes, and is also
being considered for the incipient developments over
mobile receivers (DVB consortium, 2004).
217
López-Nores M., J. Pazos-Arias J., García-Duque J., Blanco-Fernández Y., Rey-López M. and Casquero-Villacorta E. (2007).
SPONTANEOUS AND PERSONALIZED ADVERTISING THROUGH MPEG-7 MARKUP AND SEMANTIC REASONING - Exploring New Ways for
Publicity and Marketing over Interactive Digital TV.
In Proceedings of the Second International Conference on Signal Processing and Multimedia Applications, pages 213-217
DOI: 10.5220/0002131402130217
Copyright
c
SciTePress
Figure 1: The operational scheme of the spontaneous and personalized advertising model.
The operational scheme of our system, shown in
Figure 1, includes four core elements, to be described
in the following subsections. Since the personaliza-
tion tasks exceed (by far) the memory and comput-
ing capabilities of an IDTV receiver, the elements in-
volved are primarily placed in remote personalization
servers; nonetheless, there are also local instances in
the receivers to enable sub-optimal operation in cases
of intermittent, sporadic or null access to a return
channel.
2.1 The Interaction Engine
The interaction engine is the element in charge of de-
tecting the viewer’s interest for the things appearing
on screen. To this aim, it introduces different inter-
active elements depending on the input capabilities of
the receiver:
When using a remote control or the keypad of a
mobile phone, a certain button opens a menu list-
ing the things that have appeared on the current
scene thus far. The viewer can then navigate to
select the particular thing he/she is interested in.
When using a device with a touch screen (for in-
stance, a PDA), the viewer can click directly on
the things he/she wants with a finger or using a
stylus.
Obviously, the system allows interaction only with
the things characterized in the metadata linked to
the audiovisual contents, which are expressed follow-
ing the MPEG-7 standard (Manjunath et al., 2002).
Whichever the interaction method, the viewer can
choose to launch i-spots immediately, or later (most
commonly, when the current program finishes). In
the former case, if the IDTV receiver allows opera-
tion as a PVR (Personal Video Recorder), it is even
possible to do time shift, i.e. to pause the programs
while the viewer explores the i-spot, and then resume
the viewing.
2.2 The AVATAR Recommender
The AVATAR recommender is responsible for identi-
fying products related to the things that have caught
the viewer’s attention, and then selecting those which
best match his/her preferences, interests and needs.
This is done by applying the semantic reasoning al-
gorithms presented in (Blanco-Fernández et al., 2006)
over the following sources of information:
The metadata describing the things and the action
on screen.
Ontologies that characterize and interrelate audio-
visual contents, commercial products and adver-
tising material —using metadata fields from the
TV-Anytime specifications (TV-Anytime forum,
SIGMAP 2007 - International Conference on Signal Processing and Multimedia Applications
218
2003), MPEG-7 and several ad hoc syntaxes.
A profile containing demographical data of the
viewer (age, gender, job, ...), together with point-
ers to contents/products that he/she has evaluated
in the past (each one labeled with a degree of in-
terest, DOI) and products that he/she owns (la-
beled with a degree of satisfaction, DOS).
The filtering is triggered by context information
indicating what the viewer is watching. The person-
alization servers proceed over complete ontologies
and viewer profiles, whereas the local instances of
AVATAR handle partial versions due to the limited
resources. In the latter case, the quality of the rea-
soning can be enhanced using ontological information
and stereotypical profiles received through the broad-
cast networks. This supporting material, inserted in
the IDTV head-end, can be tailored to the programs
and the advertisements which are broadcast at any
moment.
2.3 The I-Spot Composer
The product selections made by AVATAR are the en-
try for the i-spot composer to assemble interactive ser-
vices that let the viewers navigate for detailed infor-
mation about the products, search for the closest es-
tablishment where they can be bought, purchase on-
line, subscribe to the notification of novelties, etc.
It is foreseeable that many i-spots will provide es-
sentially the same functionality. In our system, there-
fore, they are assembled from template classes, and
it is up to the i-spot composer to decide which tem-
plates to use (considering hardware requirements, re-
turn channel availability, ...) and which contents to
lay over them (received through broadcast or down-
loaded from the Internet). The outcome of those de-
cisions is later used in the receivers to start the ser-
vices, retrieving the corresponding templates from the
broadcast stream and setting them up using the Java
reflection mechanisms. This approach enables good
use of bandwidth while not increasing the computa-
tional cost for the receivers, as it is not necessary to
modify or recompile any source code.
Remarkably, the selection of contents in the i-spot
composer can be driven by the same semantic reason-
ing algorithms of AVATAR, to increase the odds that
the viewer will appreciate the information included in
the i-spots.
2.4 The Feedback Agent
Just because the interests, preferences and needs of
the viewers may vary with time, the model of spon-
taneous and personalized advertising requires mech-
anisms to update their profiles by capturing new data
and discarding obsolete knowledge. In literature, one
can find proposals to do this implicitly, by inferring
information from the viewers’ interaction with a sys-
tem, or explicitly, by asking the viewers to enter some
information from time to time.
Our prototype system supports both forms of feed-
back to recompute the DOI values stored in the
profiles, for which it applies the same functions of
gradual forgetting and relevance feedback defined
in (Montaner et al., 2002). The implicit form gath-
ers information by monitoring how long the viewer
takes to learn about the different products, whether
he/she decides to buy, hire or subscribe, and how
much money he/she spends. On the other hand, the
explicit form relies on interactive questionnaires, con-
structed just the same way as the i-spots. We also
provide the viewers with questionnaires to enter DOS
values for the products they own, and to update their
demographical data.
3 A SAMPLE SCENARIO
For a brief example, consider the case of a TV viewer
whose profile indicates, among other facts, that she
has two children and a middle-income economy (de-
mographical data), that she is a loyal viewer of pro-
grams devoted to travel (viewing history), and that she
is subscribed to an oenology magazine (consumption
data). The viewer is watching a documentary about
Switzerland on her PDA, with the current scene de-
scribing a town in the Geneva canton. As shown in
Figure 2, the metadata linked to this part of the audio-
visual contents identify four things on screen, namely
the village, the sky, the lake and the vineyard. These
entities are related to some concepts in the system on-
tologies, represented by the gray and dashed elements
of the figure.
If the viewer clicked over the village, our sys-
tem would be able to locally assemble one i-spot de-
scribing affordable trips and tourist attractions around
Geneva, getting the contents from broadcast. Alter-
natively, if bi-directional communication were avail-
able, another i-spot could be designed that described
rural tourism houses in the viewer’s area, emphasiz-
ing those which offer facilities for children, and giv-
ing the viewer the possibility to book a room for her
holidays (see Figure 3).
On the other hand, clicking on the vineyard, the
viewer’s expertise in wine would take our system to
display a list of remarkable vintages (with i-spots al-
lowing her to purchase some bottles), trips to famous
wineries, etc. Other viewers would be offered wines
SPONTANEOUS AND PERSONALIZED ADVERTISING THROUGH MPEG-7 MARKUP AND SEMANTIC
REASONING - Exploring New Ways for Publicity and Marketing over Interactive Digital TV
219
Figure 2: The image on screen, with sample MPEG-7 markup and links to the system ontologies.
Figure 3: A personalized i-spot.
for everyday consumption, guides for cultivating, or
simply indications to get to the nearest bar.
4 CONCLUSIONS
In this paper, we have outlined the implementation of
a new advertising model for Interactive Digital TV,
that exploits the markup of audiovisual contents to
face the viewers with products semantically related to
the things that catch their attention on the TV screen.
This approach is intuitive and pleasant for the view-
ers, because it does not interfere with their perception
of the TV programs and, besides, it removes the need
to populate the screen with advertising material.
In what concerns commercial exploitation, our
proposal can achieve much better targeting than the
current advertising techniques, inasmuch as the pub-
licity is selected by taking into account the prefer-
ences, interests and needs of each individual viewer.
In this regard, to the best of our knowledge, there is
only one precedent of personalized advertising for TV
in (Lekakos and Giaglis, 2004), but that work was
limited to selecting commercial breaks.
Finally, we want to emphasize that the automatic
aggregation of interactive commercials is certainly
one step forward to harness the potential of the IDTV
technologies to support t-commerce and m-commerce
platforms —as argued in the survey of (Myers Group,
2006), that potential is enormous, but remains heav-
ily underexploited. Our current work is precisely de-
voted to this part of our system, in an attempt to port
our solutions to the standard technologies used for au-
tomatic service composition in the field of Web Ser-
vices (Milanovic and Malek, 2004).
SIGMAP 2007 - International Conference on Signal Processing and Multimedia Applications
220
ACKNOWLEDGEMENTS
This work has been supported by the Spanish Min-
istry of Education and Science (Project TSI2004-
03677).
REFERENCES
Blanco-Fernández, Y., Pazos-Arias, J., López-Nores,
M., Gil-Solla, A., and Ramos-Cabrer, M. (2006).
AVATAR: An improved solution for personalized TV
based on semantic inference. IEEE Transactions on
Consumer Electronics, 52(1):223–231.
DVB consortium (2004). Transmission system for handheld
terminals. ETSI standard EN 302 304.
DVB consortium (2006). Globally Executable MHP. ETSI
standard TS 102 819.
Kim, P. (2006). Advertisers face TV reality. Business View
Trends, Forrester Research.
Lekakos, G. and Giaglis, G. (2004). A lifestyle-based ap-
proach for delivering personalised advertisements in
digital interactive television. Journal of Computer-
Mediated Communications, 9(2).
Manjunath, B. S., Salembier, P., and Sikora, T. (2002). In-
troduction to MPEG-7: Multimedia content descrip-
tion language. John Wiley & Sons.
Milanovic, N. and Malek, M. (2004). Current solutions for
Web Service composition. IEEE Internet Computing,
8:51–59.
Montaner, M., López, B., and de la Rosa, J. (2002). Improv-
ing case representation and case base maintenance in
recommender agents. Lecture Notes in Computer Sci-
ence, 2416:234–248.
Myers Group (2006). T-commerce case studies.
http://www.researchandmarkets.com.
TV-Anytime forum (2003). TV-Anytime specification se-
ries. ETSI standard TS 102 822.
SPONTANEOUS AND PERSONALIZED ADVERTISING THROUGH MPEG-7 MARKUP AND SEMANTIC
REASONING - Exploring New Ways for Publicity and Marketing over Interactive Digital TV
221