APPROACH TO MANAGE SEMANTIC INFORMATIONS
FROM UGC
Maria Ilaria Lunesu, Filippo Eros Pani and Giulio Concas
Department of Electrics and Electronics Engineering, University of Cagliari, Piazza d'Armi, Cagliari, Italy
Keywords: User Generated Content, multimedia object, ontology, mapping, Knowledge-base.
Abstract: The purpose of this work is to face the issue of classification variety and non-homogeneity, especially in
Web 2.0, for User Generated Content coming from popular digital platforms. The solution offered to this
problem is an approach based on an ontology that can represent information, typically associated with
UGCs, integrated with a unique mapping technique amongst ontology contents and UGCs contents coming
from other platforms. Regarding standard information and information shared by many of these objects,
existing relations are exploited through mapping, when possible; otherwise new ones are created when it is
deemed necessary. Such an ontology can represent, as embedded information, folksonomies and all non-
standard information. That kind of information, despite being unclassifiable by means of standard schemas
like the UGC ones, can be mapped. Rather than representing all properties of digital content, we were
concerned with having an ontology that could associate semantic value to every tag, standard and not.
1 INTRODUCTION
This work deals with the issue of usability of
different content types: bibliographical, digital text,
customised, as well as content types that, stemming
from different digital platforms, offer different
means of representing information. Software
platforms that handle great quantities of multimedia
content, especially User Generated Content, are
steadily increasing nowadays. The prominent
features of such platforms are their ease of use, the
possibility for users to create and manage their own
spaces (personal channels or pages), the
implementation of efficient content research and
localization methods, and the definition of access
and usage types. Objects with these features appeal
to more and more users, requiring an ever-growing
number of applications. The main drawback of such
platforms lies in their poor interoperability, which
does not allow for a complete usability of contents.
Thus contents cannot be shared with other platforms,
in particular aggregation or multi-source ones. These
problems drove us to using a semantic representation
of data and resources. We started with the idea of a
Knowledge-base, based on an ontology able to
provide contents originated in UGC sources like
Flickr, Youtube and Wikipedia, along with what was
entered by users of the system itself. The
knowledge-base had to be able to receive and
manage contents, assigning them an unambiguous
meaning. We then followed an approach based on
choosing an ontology already in existence, able to
represent the semantics of known multimedia
content. As a starting point, we chose the most used
standards to represent metadata in this domain:
Adobe XMP, DUBLIN CORE, EXIF, IPTC and
other types of standardised metadata. Typical tags
for UGCs were analysed next, and a rule was
defined to reconcile the tags with metadata of the
chosen ontology as well as to import the other tags
as embedded in specialised metadata (e.g.,
folksonomies in Flickr).
This document is organised as follows: section 2
deals with the state of the art for multimedia content
ontologies; section 3 offers a review of the state of
the art for User Generated Contents tags, while
section 4 hosts observations on issues concerning
mapping and the description of the proposed
approach. Lastly, conclusions are listed in section 5.
2 MULTIMEDIA ONTOLOGIES
When multimedia content ontologies (Schreiber,
Dubbeldam, Wielemaker and Wielinga, 2001) are
described, the ones with a working range including
470
Lunesu M., Eros Pani F. and Concas G..
APPROACH TO MANAGE SEMANTIC INFORMATIONS FROM UGC.
DOI: 10.5220/0003690704700473
In Proceedings of the International Conference on Knowledge Engineering and Ontology Development (KEOD-2011), pages 470-473
ISBN: 978-989-8425-80-5
Copyright
c
 2011 SCITEPRESS (Science and Technology Publications, Lda.)

many application fields, (such as Content
Visualization, Content Indexing, Learning,
Reasoning and Sharing) are considered. The solution
is in the usage of semi-automated building
techniques with the purpose of simplification of
previous steps. Many attempts at building
multimedia ontologies can be referenced.
In (Jaimes and Smith, 2003), ontologies were
built manually. Textual information provided in
videos was manually extracted and assigned to
concepts, properties, or relations within the
ontology.
New methods for semantics knowledge
extraction from annotated images are presented by
Benitez and Chang (2003). Perceptive knowledge is
built by organising the images in clusters based on
their visual and textual features. Semantic
knowledge is extracted removing all semantic
ambiguity, using WordNet and image clusters.
In Strintzis, Bloehdom, Handschuh, Staab,
Simou, Tzouvatras, Petridis, Kompatsiaris and
Avrithis (2004), a Visual Descriptors Ontology and
a Multimedia Structure Ontology, respectively based
on MPEG-7 Visual Descriptors and MPEG-7 MDS,
are used together with a domain ontology so as to
support content annotation.
In Bertini, Cucchiara, Del Bimbo and Torniai
(2005), ontologies enhanced with images were
introduced to automatically annotate videos. Clip
highlights were considered as examples of ontology
concepts and were directly related to corresponding
concepts, grouped into subclasses based on their
perceptive similarity. BOEMIE (Bootsrapping
Ontology Evolution con Multimedia Information
Extraction) uses a synergistic approach that ties
multimedia extraction to ontology evolution, by
operating automatically a continuous extraction of
semantics information from multimedia contents.
This action aims to create and enhance ontologies.
On the other hand it aims to spread the ontology to
improve on the resilience of the extraction system.
MOM (Multimedia Ontology Manager), as seen in
Bertini, Del Bimbo, Torniai, Cucchiara and Grana
(2006), is a complex system, developed according to
the principles and concepts of ontologies, enhanced
through images.It supports dynamic creation and
update of multimedia ontologies; and offers
functionalities to automatically perform annotations
and create extended textual comments. It also allows
for complex queries on video databases. Based on
the same ontology, there is also OntoMedia: a
multimedia ontology based on an information
system. Its main purpose is managing big
multimedia collections using semantic metadata
integration techniques. The annotations on
multimedia documents were generally developed
according to two different routes. Both approaches
were focused on low-level descriptors. Our ontology
(for which a brief description will be provided in the
following sections) is conceived as a tool able to
exploit pre-made schemas in order to represent
content belonging to various types and coming from
different sources. Such schemas are typical of
standards and were used as means to model the
domain.
3 UGC
The most powerful applications and the most
common platforms usually have these features: easy
and fast content search by keywords, link usage for
easy navigation in contents, content editing by users
themselves either iteratively (Wikipedia) or
cumulatively (blogs and forums), content
classification through “tags”, possibility to direct
users to offers (any kind) through “collaborative
filtering”-type algorithms, real time notices through
RSS for content change or editing. The usage of all
those new technologies encouraged the success of
such systems for socialising, where a remarkable
exchange of information of many types (text, video,
audio) and from different sources takes place. Users
create communities, sharing comments, opinions and
above all their own knowledge and experience.
The term 'UGC' nothing but points out how the
Web is evolving more and more towards being a
product by its very users, labelled with the new
name of 'prosumer' (producers and consumers).
Every publicly accessible content type, with an
added share customised by the user, is part of the
UGC universe. We considered two kinds of such
content, for which we analysed and compared
metadata, that is contents from Youtube and Flickr.
The differences can be immediately noticed. In the
first case, there is mapping possibility, directly or
not, through schemas and standard properties; in the
second, the usage of a new cataloguing method,
typical of the platform and coming from a new
school of thought, with no compliance with any
standard.
An example of this can be the atom id tag, that
“specifies a URN that uniquely and permanently
identifies a feed or video entry.”
The Dublin Core Standard element dc.identifier
is “An unambiguous reference to the resource within
a given context.”
APPROACH TO MANAGE SEMANTIC INFORMATIONS FROM UGC
471

Given the semantics in common between the
objects we can thus perform a direct mapping
between this Atom tag and the dc.identifier element
from Dublin Core.
Table 1: Disambiguation: direct mapping
Mapped Tag atom:id
Semantic description
Specifies a URN that uniquely and
permanently identifies a feed or
video entry.
Standard Metadata dc.identifier
Identifier
An unambiguous reference to the
resource within a given context.
The problem with User Generated Content,
coming from the various platforms considered, is
that they display marked differences in classification
of associated information. We considered also Flickr
and Youtube, which, despite handling customizable
multimedia content customized by users, consider a
different way to represent information. In Youtube's
case, it is often possible to create relation via a direct
mapping in general, and indirect mapping in some
special cases. Regarding Flickr, instead, we faced a
“customisation” of the information, exploiting
Folksonomies to catalogue all information without a
standard. Some information is natively represented,
other is included in metadata. The latter can be
mapped in an ontology that, starting from the usage
of standards, provides for receiving and cataloguing
information in existing tags or tags for which a new
relation was created.
4 PROPOSED APPROACH
The purpose of this work was to offer a different
approach for the usability issue of User Generated
Content coming from the most popular digital
platforms. In order to solve this problem we
propose, as a solution, an approach articulated in
few easy steps: 1) Selection of an ontology able to
represent information typically associated with such
contents, among what is already available. 2)
Improving this ontology with a number of mapping
rules that allow for representing information coming
from sources like UGCs not complying to standards.
This technique can exploit different relations when
possible, or creating new ones whenever necessary.
This is especially true for information proper and
common to many contents. 3) Integrating other
pieces of information of this ontology in fields that
can store non-mappable information with the above
mentioned technique. In such fields typical tags for
the platforms, as well as tags defined by users
(folksonomies) can be stored. With this approach,
we can store all non-standard and unclassifiable
information inside the ontology, by pre-made
schemas. Rather than representing all properties of
digital content, we were concerned with having an
ontology that could associate semantic value to all
non standard, mappable tags as well as storing also
information found in non-mappable tags.
The ontology does not need to be able to
represent everything, but to use what is already
available for representing known and classified
information such as author, URL, etc. It also must
use that mapping amongst infrastructures and
information provided by the platform.
For every information for which no schemas or
tags are present, i.e. for everything non-standard,
like user comments and other new default
information, folksonomies are used.
Below is a peculiar example of Web content
acquisition for content related to an image stored in
the Flickr platform, and of related metadata
management. For example, Flickr provides a tool,
flickr.photos.getExif, that allows for reading the
metadata set associated to a given content. Entering
the last number of the address into the tool, a list of
tags that include that information, is returned. In
order to enter such data in the ontology, it is
necessary to create various instances to represent
content, format, the schema-Exif describing it,
instances for each data type associated to each tag
and related values. As you can see, not all retrieved
data were created inside our ontology, so an Atom
feed needed to be associated to the content, so that it
can collect unknown metadata in bulk. These
metadata are partly complying with the Exif
standard (and mapped with the typical rules of the
standard as such) and partly belong to
Folksonomies. The above mentioned mapping rules
were applied, and part of the data were inserted,
while the rest was inserted in the
FlickrFolksonomies class. As for the mapping, it
was necessary to manually enter what was not
provided for by the scheme of the ontology. We
inserted the information related to all properties and
created the link amongst them and between them
and the various metadata so that they could be
represented univocally and no information could be
lost. In our example the first thing to be created was,
with the aid of the tool, the MultimediaContent
class; the name 'CastellodiArco' was then associated
to it, exploiting the 'instance browser. It could be
noticed that, for the properties previously created,
the hasMetadataLocation and doesExpress fields
KEOD 2011 - International Conference on Knowledge Engineering and Ontology Development
472

appear already compiled. On the other hand, we had
to define the elements to insert in the
hasMetadataDescription field and the ExifSchema,
ExifSchemaCastellodiArco, UnknownMetadata and
UnknownmetadataCastellodiArco instances.
The latter belongs to the class devoted to the
representat ion of unknown metadata belonging to a
standard.
At this stage, the ExifSchemaCastellodiArco
instance could be filled out with all the fields
returned by the Flickr tool. In this way an univocal
correspondence between information and metadata
related to it was created. The entire Exif schema
must be checked in order to know which tags of the
picture are present or not. We entered the missing
data manually.
Once the values were ready to be entered into the
tags, we created a different data-type instance for
each data. Afterwards a Date_1-type instance was
created for the tiff:dateTime tag. Since the data type
belongs to the Exif schema, it requires some
additional attributes for temporal information
(exif:subSecTimeDigitized,exif:subSecTimeOriginal
exif:subSecTime); thanks to the existing relations,
the fields related to such attributes were displayed as
well.
5 CONCLUSIONS
In this work, we suggested a new approach to solve
the problem of actual availability of UGC. This
approach is especially suited for all those instances
when a multimedia content is considered for which
associated information do not comply with standard
in categorizing metadata. Special attention has to be
paid to widespread standards like Adobe XMP,
DUBLIN CORE, EXIF, IPTC.
In fact, in those cases, a synergistic integration of
an ontology based on the standard with the usage of
a clearly set mapping technique allows for
representing a great number of contents and
metadata. This mapping technique was especially
useful to sort out a vast and complex knowledge
field such as multimedia content. Dealing with
mapping arose the necessity of using shared
standards rather than proprietary ones, now very
widespread. The proposed approach may be used as
support for a software platform that allows different
actors to develop added-value services. Such
services could be based on multimedia content
insertion into a semantic organisation context. It is
clear that such an approach should rely on a
powerful tool which could map all the information
concerning entered contents in relation to the form
decided as representation standard within itself.
REFERENCES
Benitez, A., Chang S. (2003). Automatic multimedia
knowledge discovery, summarization and evaluation.
IEEE Transactions on Multimedia.
Bertini, M., Cucchiara, R., Del Bimbo, A., Torniai, C.
(2005). Video annotation with pictiorally enriched
ontologies. In Proceedings of IEEE International
Conference on Multimedia and Expo (ICME 2005).
Castello di Arco picture. Retrieved from:
http://www.flickr.com/photos/cristina63/3830632607/
Groza, T., Handschuh, S. (2009). A Hybrid Approach
Towards Information Expansion based on Shallow and
Deep Metadata. In Proceedings of KEOD 2009 (pp.
109-116).
Jaimes, A., Smith, J. (2003). Semi-automatic, data-driven
construction of multimedia ontologies. Proceedings of
IEEE International Conference on Multimedia and
Expo (ICME 2003, Vol. 2).
Petridis, K., Anastasopoulos, D., Saatho, C.,
Timmermann, N., Kompatsiaris, I., Staab, S. (2006).
M-OntoMat-Annotizer: Image Annotation. Linking
Ontologies and Multimedia Low-Level Features.
Engineering Applications of Semantic Web Session
(SWEA) at the 10th International Conference on
Knowledge-Based and Intelligent Information and
Engineering Systems.
Picca, D. (2010). Building Multilingual Lexical Resources
On Semiotic Principles. In Proceedings of KEOD
2010 (pp. 412-415).
Schreiber, A. Th., Dubbeldam, B., Wielemaker, J.,
Wielinga, B. (2001). Ontology-Based Photo
Annotation. IEEE Intelligent Systems (Vol. 16, pp. 66-
74).
Strintzis, J., Bloehdom, S., Handschuh, S., Staab, S.,
Simou, N., Tzouvatras, V., Petridis, K., Kompatsiaris,
I., Avrithis, Y. (2004). Knowledge representation for
semantic multimedia content analysis and reasoning.
In Proceedings of the Europe an Workshop on the
Integration of Knowledge, Semantics and Digital
Media technology (EWIMT 2004).
APPROACH TO MANAGE SEMANTIC INFORMATIONS FROM UGC
473