Towards Personalised Multimedia Applications
A Literature Review
Sebastian Sastoque H.
1
, Oscar Avila
1
and Marcela Iregui
2
1
Department of Systems and Computing Engineering, School of Engineering, Universidad de los Andes, Bogota, Colombia
2
ACCEDER Research Group, Faculty of Engineering, Universidad Militar Nueva Granada, Bogota, Colombia
Keywords:
Multimedia, Personalisation, Retrieval, Annotation, Application Development, Systems, Models, Literature
Review.
Abstract:
Multimedia applications are now commonly used in daily life for several domains as marketing, health, learn-
ing and entertainment, among others. As the number of available applications increases, a competitive factor
is the level of alignment to personal preferences. Indeed, the role of multimedia content has been crucial to
generate user centred applications. However, multimedia content personalisation requires complex systems
that execute diverse tasks such as representation, modelling, annotation and retrieval. Research on this field
has been focused on content annotation and retrieval perspectives. Despite this, these domains do not address
two of key personalisation factors, i.e., considering personal preferences and contextual knowledge. This work
presents a literature review aimed to identify theoretical elements related to personalisation purposes, which
could be integrated to the most common approaches. As a result, a road map for future research is established.
1 INTRODUCTION
Nowadays the development of new technologies and
the expansion of the Internet have broadened the
range of applications and systems that use multimedia
content. Diverse fields such as education, medicine,
entertainment, marketing and business, among oth-
ers, have benefited from this phenomenon. While
the number and variety of applications are growing,
users change their needs, expectations and demands,
looking for ever further personalised experiences (Lu
et al., 2011). For instance, users prefer watching
movies of their favourite genres, reading news about
their interest topics or listening music according to
their mood. In this context, multimedia content per-
sonalisation has become a crucial factor to enrich the
user experience. For example, Facebook became the
main leader of social networks apps, thanks to person-
alised social interactions through multimedia content,
that was its main differential within the existing com-
petitors at the moment.
The concept of personalisation can be defined as
the process of customising data output in order to re-
flect users interests, preferences and situations and
meet their requirements (Boll, 2003). Multimedia
content itself is not enough for personalizing appli-
cations. Thus, complex systems are needed for repre-
senting, modelling, indexing, retrieving and present-
ing media content according to user needs and the ap-
plication domain (Lu et al., 2011). This is known as
the semantic gap, which constitutes one of the main
challenges in multimedia research (Lew et al., 2006).
Aiming at filling this gap, proposals are presented
in two different areas: content features extraction and
annotation, and content retrieval (Lu et al., 2011).
Although, research in these fields contributes to per-
sonalisation, they do not address the user preferences
knowledge domain, which is a key factor in user
centred applications. Therefore, some complemen-
tary areas could contribute to this field by extending
the main functionalities obtained by using approaches
only based on annotation and retrieval.
Some approaches have adopted an holistic per-
spective of the problem (Sastoque et al., 2014; Gior-
dano et al., 2011; Aldu
´
an et al., 2011; Scherp et al.,
2007). However, these works do not present an ex-
haustive review of the literature to present their pro-
posals. In fact, after a preliminary searching process,
it was not found a work aimed to review personalised
multimedia applications research field. The found re-
views are focused on the area of Multimedia Informa-
tion Retrieval (Lu et al., 2011; Sebe and Tian, 2007;
Lew et al., 2006).
Accordingly, this work presents a literature review
H., S., Avila, O. and Iregui, M.
Towards Personalised Multimedia Applications - A Literature Review.
DOI: 10.5220/0006326701270134
In Proceedings of the 19th International Conference on Enterprise Information Systems (ICEIS 2017) - Volume 3, pages 127-134
ISBN: 978-989-758-249-3
Copyright © 2017 by SCITEPRESS Science and Technology Publications, Lda. All rights reserved
127
aimed (i) to identify the research process behaviour in
the domain of personalised multimedia applications
and (ii) to establish which approaches from comple-
mentary areas could be adapted for personalisation
purposes. The results of this review are used to deter-
mine what are the main challenges and open research
questions in the area.
The remaining of the paper is organised as fol-
lows: Section 2 presents related works and introduce
the mains concepts for the review. Section 3 intro-
duces the methodology of literature review and the
results of the two first steps, namely, planning and
Conducting and Material Collection and Evaluation.
Section 4 presents a qualitative and quantitative anal-
ysis of the review, which corresponds to the third step
of the methodology. Section 5 presents the challenges
related to the area. Finally, section 6 concludes the
paper.
2 RELATED WORKS
Given the variety of contributions in the field of per-
sonalisation in Multimedia Information Retrieval, this
section summarises some reviews aimed to synthesise
the literature in this area to bring an overview of the
related works.
Lu et al., (Lu et al., 2011) present a survey aimed
to overview the current research in multimedia anno-
tation, relation and visualisation. The survey analyse
a sequence of applications useful for content-based
multimedia retrieval. Then, the authors select the best
models and techniques used to represent context and
user preferences, conceptualise ontologies, display
content and retrieve multimedia information. Finally,
the selected models and techniques are mixed to pro-
pose the concept of Intelligent Multimedia Content.
Sebe et al., (Sebe and Tian, 2007) present an analysis
of the topics related to personalisation in Multimedia
Information Retrieval aimed to identify the common
tendencies to achieve personalisation. Particularly,
this work addresses two issues related to: (i) the users
preferences for personalised access to the content and
(ii) the devices capabilities and applications to display
content. The authors conclude that there exist chal-
lenges in personalisation related to Multimedia Col-
laboration, Interactive Search with Agent Interfaces
and Folksonomies, i.e., the use of crowd wisdom to
annotate and retrieve the content. Lew et al., (Lew
et al., 2006) review a set of 100+ published articles
before 2006 in the area of content-based multime-
dia information retrieval. They studied the effect of
search paradigms, affective computing, learning, se-
mantic queries and evaluation techniques in content-
based retrieval. The authors conclude that the im-
portant challenges are: (i) semantic search in media
with complex backgrounds, (ii) multimodal analysis,
(iii) exploration of media collections, (iv) interactive
search and (v) evaluation methodologies.
Although these reviews are a great contribution to
the state of the art in the multimedia field, they only
are focused on the retrieval area and how to represent
the user preferences. It leaves a gap related to the
current state of the development of personalised mul-
timedia applications and what are the main challenges
to achieve next level of personalisation, which are the
most relevant topics of this work.
3 LITERATURE REVIEW
A literature review is a systematic and reproducible
design to identify, evaluate and interpret a set of ex-
isting documents (Seuring and M
¨
uller, 2008). The re-
views usually aim two objectives: (i) to summarise
the existing research of a field by identifying issues,
patterns and contributions and (ii) to identify the con-
ceptual framework of a field and contribute to its
development (Webster and Watson, 2002). The de-
sign of a literature review should follow a methodol-
ogy that ensures the comprehension and analysis of
the content, where quantitative and qualitative analy-
sis should be addressed to describe the research field
(Seuring and M
¨
uller, 2008). In this work the se-
lected methodology was based on the model proposed
by (Mayring, 2014) and adopted the best practises
proposed by (Webster and Watson, 2002). Thus the
literature review followed the next steps: Planning
(section 3.1), Conducting and material collection and
evaluation (section 3.2) and Descriptive review anal-
ysis (section 4).
3.1 Planning
This step defines the review questions, the criteria to
conduct the search, the delimitation of the material
and the method to validate contributions. Accord-
ing to the review objectives introduced in section 1,
the proposed review questions are: (i) What are the
main contributions of the different approaches?; (ii)
What are the approaches field of application?; (iii)
How multimedia content is related to the user domain
context?; (iv) Which techniques are used to represent
and store content annotations? and (v) How personal-
isation is addressed?
Regarding the material delimitation to reduce the
number of proposals to analyse, the review process is
based on the following considerations: (i) the analysis
ICEIS 2017 - 19th International Conference on Enterprise Information Systems
128
scope includes only peer-reviewed publications writ-
ten in English; (ii) the main area of the review is mod-
els for personalised multimedia applications; (iii) the
following complementary areas are included to iden-
tify the common approaches: personalised content re-
trieval and personalised content annotation, (iv) only
those works focused in models, architectures, plat-
forms, software, systems or frameworks are included,
(v) the proposals are in the design, development or
implementation stage (vi) to ensure the objectivity of
the review the work selection is performed by the au-
thors separately and (vii) the works are published until
April of 2016.
3.2 Conducting and Material Collection
and Evaluation
In this step it is performed a comprehensive and ex-
haustive search for primary approaches using the cri-
teria defined in the previous step. Then, a validation
and assessment process is performed to select the set
of approaches that will be included in the review.
The search of related publications was conducted
as a structured keyword search. The search process
used the Scopus database (Burnham, 2006). This
database was selected because it indexes quality per-
reviewed works presented in journals and confer-
ences of main scientific publishers, such as, IEEE
Explore, ACM, Elsevier, among others (Burnham,
2006). The query used in the search was: (TITLE-
ABS-KEY(“Personalised multimedia application”))
OR (TITLE-ABS-KEY(“Application” OR “Model”
OR “Architecture” OR “Framework” OR “System”
OR “Platform” OR “Software”) AND TITLE-ABS-
KEY(“Personalization” OR “Personalisation” OR
“Personalized”) AND TITLE-ABS-KEY(“Multimedia
Content” OR “Multimedia” OR ( (“Media” OR
“Content” OR “Multimedia”) AND (“Retrieval” OR
“Annotation”)) ) AND TITLE-ABS-KEY(“Design”
OR “Implementation” OR “Development”))
The Scopus engine resulted in 851 candidate arti-
cles. The evaluation of the material process to limit
the articles used the following filters: (i) Reading of
the title and keywords reduced the numbers of works
to 388. (ii) Reading of the abstracts to filter those
ones that do not have hints to answer the review ques-
tions, limited the number of articles to 123. (iii) A
full reading of the articles concludes the evaluation to
the selection of a final set of 76 articles. In this work
we synthesise the most common approaches and only
cite the most relevant works for each approach.
4 DESCRIPTIVE REVIEW
ANALYSIS
It assesses formal aspects of the material. Two kinds
of analysis are performed for providing theoretical
background from contributions or identifying open re-
search challenges. The first analysis is quantitative,
which provides information about the area trends,
e.g., the number of publications per year. The sec-
ond analysis is qualitative, corresponding to the main
concepts descriptions regarding to predefined review
questions, e.g., common methods and techniques used
in one task.
4.1 Quantitative Analysis
Descriptive dimensions are defined for performing
quantitative analysis and classification of each article.
The dimensions are time dependant and answer the
following questions: (i) how frequent are the publica-
tions?, (ii) which research methodologies are applied,
(iii) what are common terms used in the works? (iv)
what are the publications type?, and (v) which are the
most common journals and conferences?. The results
of the quantitative analysis using these dimensions are
presented below.
After the first work, from 1991 to 2002, the publi-
cation frequency remained below one paper per year.
The number of publications increased in 2005 and
since then it varies between 2 to 9. However, it could
be expected that with emerging technologies and de-
vices, in the next future this frequency would show a
significance increase.
The methodologies reported by the reviewed pub-
lications are among these four types: (i) theoretical
and conceptual; (ii) based on case studies and domain
dependant applications; (iii) modelling founded and
(iv) literature reviews.
Figure 1: Common terms.
An analysis of the common terms used in the
publications is presented in Figure 1. This analysis
was performed with the tool (Sinclair and Rockwell,
Towards Personalised Multimedia Applications - A Literature Review
129
2016). As expected the most common terms are Mul-
timedia and Content. However, the broad use of terms
such as knowledge, semantic and ontology shows that
exists a trend in the different approaches to include
semantic and knowledge representation issues. The
most common media types addressed in the papers
are videos and images. Finally, the works commonly
present frameworks and models and the most frequent
application is content retrieval.
Figure 2: Publications per type.
The publications are equally distributed between
conferences and journals (Figure 2). Although there
is not a conference or journal that has a representative
frequency of publications, the journal with the largest
number of contributions is Multimedia Tools and Ap-
plications. The highest number of conferences related
to the field publish their proceedings in Lecture Notes
in Computer Science, IEEE Conference on Multime-
dia and CEUR Workshop Proceedings.
4.2 Qualitative Analysis
The qualitative analysis of the review is presented ac-
cording to the questions introduced in section 3.1.
4.2.1 What are the Main Contributions of the
Different Approaches?
The principal contributions of the approaches are di-
vided in five categories: (i) design of architectures,
(ii) methods for content representation, (iii) models
for user profiling and development of applications,
(iv) study cases and applications for personalisation
in specific domains and (v) surveys and literature re-
views.
Related to the category of design of architectures,
the main contributions are associated with: the use
of specific technologies and standards, such as cloud
computing, big data and MPEG-7 (Zeng, 2016; Guo
et al., 2015); handle, deliver, process, annotate and
retrieve content (Aldu
´
an et al., 2011; de Fez et al.,
2015); general purpose related with personalisation
based on services or the definition of components
(Scherp et al., 2007).
The approaches related to content representation
commonly address two tasks: first, the representa-
tion of content using low level features, such as, vi-
sual structures, signal descriptors or descriptive lan-
guages (Rinaldi, 2014); and second, the semantic rep-
resentation of the content by using meta-data and al-
gorithms to relate low level features with semantics
(Mallik et al., 2013).
The contributions corresponding to modelling cat-
egory aim to model: user profiles for specific context
applications (Park et al., 2012); applications for spe-
cific domains as museums, news, searches or sports
(Appalla et al., 2016); techniques for content retrieval,
annotation and semantic relation (Sun et al., 2011;
Scherp et al., 2007; Sebe and Tian, 2007); frame-
works to the development and implementation of ap-
plications (Weiß et al., 2008).
The articles that present study cases and appli-
cations have as main contributions: the definition
of concepts and requirements for content distribu-
tion and personalisation (Evans et al., 2006); the im-
plementation of applications for content digitization,
management and processing (Chen et al., 2012); the
use of domain knowledge in applications related to
the context, such as, vigilance, georeferencing and
digital museums (Xue et al., 2012; Mylonas et al.,
2008).
Finally, a set of articles present literature reviews
and surveys related to the state for multimedia re-
trieval, recommendation, personalisation and annota-
tion (Lu et al., 2011; Lew et al., 2006).
4.2.2 What are the Approaches Field of
Application?
The adjacent fields, that are related to multimedia
content personalisation, presented in the review are:
(i) retrieval, (ii) annotation and indexing, (iii) recom-
mendation and (iv) adaptation and personalisation.
The multimedia content retrieval aims to the ex-
traction of semantic information from multimedia
data sources to retrieve content according to a query
(Furht, 2008). The queries could be composed as
textual keywords, content examples or mixing both
techniques. The approaches that address retrieval
commonly aims the inclusion of semantic relations
to the retrieval process. The common methods have
three principal steps: (i) feature extraction and con-
tent summarising , (ii) filtering and (iii) description
(Aldu
´
an et al., 2011; Nie et al., 2016; Scherp et al.,
2007; Sebe and Tian, 2007).
ICEIS 2017 - 19th International Conference on Enterprise Information Systems
130
Annotation and Indexing techniques supports the
content retrieval process. The multimedia content an-
notation refers to methods that allows semantic and
low level descriptions of content (Furht, 2008). These
annotations could follow specific frameworks or mod-
els related to domain context (semantic descriptions)
and content features (low level descriptions) (Park
et al., 2014; Mallik et al., 2013). Multimedia in-
dexing refers to content pre-processing aimed to cre-
ate indexes by using features (Mylonas et al., 2008).
The indexation objective is to increase searching effi-
ciency on large databases. This must support similar-
ity queries, in which the similarity measure is defined
previously by a domain expert.
Recommendation processes aim to represent users
preferences with the objective of suggest unseen items
(Furht, 2008). The final outcome of recommendation
is a set of new elements for a specific person by em-
ploying a profile and information about other users
and items features (de Fez et al., 2015). Another ap-
proaches seek to guide the user in a personalized way
to elements of his interest in a large space of possible
options (Karaman et al., 2014).
Personalisation and Adaptation present the con-
tent to the user in an individualised way according
to their tastes and constraints. Personalisation pro-
cess allows the adaptation of specific multimedia con-
tent in accordance to individual standards, tastes and
preferences (Furht, 2008). Approaches that address
personalisation seeks to extract user profiles and re-
lates them to semantic descriptions of the multime-
dia content for presentation purposes (Dolbear et al.,
2008). Adaptation is the process to transform a con-
tent to another representation that accommodates to
restricted device input and output capabilities (Furht,
2008), e.g., converting images to text descriptions due
to bandwidth constrains of mobile devices (De Vrieze
et al., 2005).
4.2.3 How Multimedia Content is Related to the
User Domain Context?
The reviewed works that address relation between
multimedia content and user domain context use the
following approaches: (i) ontologies, (ii) models and
(iii) annotations.
An ontology is ”a formal, explicit specification
of a shared conceptualisation“ (Studer et al., 1998).
According to (Guarino et al., 2009), this conceptual-
isation refers to an abstract and simplified view of a
domain that formally represent its knowledge by in-
cluding entities and their relationships. In this con-
text, ontologies are used to relate the content to spe-
cific domains by associating: low level features and
descriptors with semantic categories; content descrip-
tion and annotations with semantic domain rules and;
user profiles with semantic categories and content de-
scriptors (Aldu
´
an et al., 2011; de Fez et al., 2015).
The approaches that propose models report a set
of specific classes, attributes and methods for pro-
viding semantic representation of multimedia con-
tent. Specifically, some works propose domain mod-
els (Mylonas et al., 2008), others use graphs and
semantic networks (Weiß et al., 2008), and those
that propose existing models such as OWL triplets,
SCORM elements or descriptive languages, etc (Nie
et al., 2016).
Finally, the last set of approaches relate seman-
tics to the content by meta-data annotations regarding
domain concepts. These annotations are commonly
stored in XML schemes attached to the content file
or linked databases (Yang et al., 2014; Scherp et al.,
2007).
4.2.4 Which Techniques are Used to Represent
and Store Content Annotations?
Content annotations are commonly obtained by us-
ing automatic, semi-automatic and manual methods.
The automatic ones refer at the extraction of features
and descriptors by computational systems. In semi-
automatic methods the systems extract some features,
which are complemented by the user. In the man-
ual method, the user is the responsible of annotate
the content according to his knowledge. Regardless
of the method used to extract features and descrip-
tors, the annotations proposed by the reviewed works
are represented by using (i) MPEG7, (ii) raw meta-
data and (iii) models; and are stored by using (i)
XML Schemes, (ii) raw data, (iii) ontologies and (iv)
databases.
MPEG-7 is a Multimedia Content Description In-
terface that describes the features and meta-data of
multimedia content through a standardized scheme
(Furht, 2008). These scheme use different types of
descriptors according to the content type, i.e., visual
descriptors for images and videos or signal descrip-
tors for audios and videos, among others. The stor-
age of MPEG-7 descriptors in the reviewed works use
databases, XML schemes attached to the binary files
of the content and ontologies (Park et al., 2014; Sebe
and Tian, 2007).
Other works used to create annotations is the stor-
age of key and value pairs, in which each pair repre-
sent a feature or description of the multimedia con-
tent. These works store the annotations in databases,
raw files, ontologies and XML schemes (Guo et al.,
2015; Scherp et al., 2007; Aldu
´
an et al., 2011).
Finally, some approaches create models of the an-
notations according to the application needs. These
Towards Personalised Multimedia Applications - A Literature Review
131
models represent low level and semantic descriptors.
The models are stored in databases, XML schemes
and ontologies (de Fez et al., 2015).
4.2.5 How Personalisation is Addressed?
Personalisation is addressed by the creation and de-
velopment of user profiles. The latter are a set of at-
tributes that represent users tastes, behaviour, prefer-
ences and patterns (Furht, 2008). The works reviewed
address personalisation in two ways: first, by integrat-
ing user context information, and second by modify-
ing content presentation. The first type of approaches
integrates user context to recommend and provide a
set of items that match the user profile (de Fez et al.,
2015; Evans et al., 2006) or that match other user con-
text data such as location, mental abilities or device
capabilities (Karaman et al., 2014; Weiß et al., 2008).
The second type of approaches present the content on
an individual basis according to user profile. In these
approaches the presentation of the content is the more
important factor to address personalisation (Aldu
´
an
et al., 2011; Mylonas et al., 2008; Sebe and Tian,
2007; Scherp et al., 2007).
5 OPEN RESEARCH
CHALLENGES
Notwithstanding the significant progress of academic
research on personalised multimedia applications,
there has been little impact in the development of
commercial applications that use these research re-
sults. Although advances in various complementary
areas, such as annotation, representation and retrieval
of multimedia content have contributed to the person-
alisation field, there are still open challenges ahead
for increase the use of research results in industries.
5.1 Architectures and Models to Exploit
the Advantages of New Technologies
With the rapid and steady evolution and expansion of
new technologies, such as, Cloud Computing, Inter-
net of Things and Big Data, in order to generate in-
novations in the media industry, these technologies
must be considered to complement personalised mul-
timedia applications. For example, with architecture
models based on Cloud Computing technologies, the
design and test of new applications could be easily
addressed, allowing to developers undertaking, their
new projects with few resources in relatively short
time. In this context, personalisation could be eas-
ily addressed through Cloud Computing by permit-
ting storage and content processing in a distributed
manner, avoiding full installation, saving computa-
tion processing in the client side and reducing soft-
ware upgrading and maintenance. Regarding Inter-
net of Things, real-time multimedia communication
could be applied in different contexts, increasing per-
sonalised experiences. For example, in emergency
calls, user could share detailed information about an
incident by transmitting images and/or videos from
near cameras, permitting a real time analysis of the
incident nature and severity.
Therefore, a current challenge in personalised
multimedia applications development is the design of
architectures models easily portable to the cloud, to
take advantage of all its benefits and enable new pos-
sibilities for data analysis and new businesses devel-
opment.
5.2 Evolution of User Profiles
Other current challenge is the proposal of advanced
models for user profiles. Models may include con-
text information of the content as well as attributes
that represent users context. User models need to in-
clude elements to describe demographic and personal
information, local settings, interests, skills, goals,
tastes, behaviours, patterns and individual character-
istics. These types of information should be interre-
lated with knowledge representation to enable contex-
tual reasoning between the multimedia content, the
application domain and the user profile. Therefore,
as the user context is highly changing, user profile
should evolve over time and should be dynamically
updated and refined to reflect external events, includ-
ing the user interaction history.
5.3 Discovery of Data Semantic
Concepts and Knowledge Base
Evolution
Multimedia applications should have the capability to
add new content for personalising user experience. In
this sense, the applications should have the capabil-
ity to discover the semantics of new incoming con-
tent, update the knowledge base and determine rela-
tionships. This can be achieved by using low-level
features in conjunction with machine learning tech-
niques to identify high level content semantics. In ad-
dition, the knowledge base should be adaptable to the
application domain and the user profile. User mod-
elling deals with individual abilities, such as, cogni-
tive, social and linguistic skills and personal and en-
vironmental circumstances. In this sense, an open
challenge is the personalisation of knowledge bases
ICEIS 2017 - 19th International Conference on Enterprise Information Systems
132
in conjunction with easy maintainability and interop-
erability.
5.4 User-centred Design of Applications
Although today literature refers to ubiquitous, mobile,
pervasive, social and personalised applications, ad-
vances in research and development are still far from
achieving them. Most common technologies and ap-
plications are unfriendly, cultural biased, unnatural
and difficult to use. Thus, a gap between user needs
and personalised applications development is pointed
out. For example, cognitive science has shown the
benefits of multi-modal interfaces to improve, user
understanding and interactions, due to the apparent
parallelism on the brain in response to multiple in-
put stimulus (Dumas et al., 2009). This means, from
an application development standpoint that it is use-
ful design meaning interfaces that exploit this kind of
advantage. However, most of the reviewed works do
not include these kind of considerations in the content
personalisation. Accordingly, it is important to create
new frameworks that include user-centred design ap-
proaches to personalised multimedia applications de-
velopment.
5.5 New Types of Content and Data
Most analysed works only focus the personalisation
on audio, images, text or video. However, new types
of multimedia content are now present in the litera-
ture, e.g., multimedia presentations, interactive docu-
ments, geo-referenced maps or 3d models. Therefore,
personalisation research should focus on the analy-
sis of new media types and techniques for multime-
dia integration. Important questions remain regarding
methods for effective content selection, media alloca-
tion, and modality selection.
6 CONCLUSION
Personalisation of Multimedia Applications in an
emerging research area that has received growing
attention in the research community over the past
decade. This work presents a literature review that
identify the contributions of complementary areas
to personalisation fields. The results of the review
make evident that indeed exists two classical work ar-
eas aimed to address personalisation (annotation and
retrieval). Although some works try to mix them
to achieve personalisation (Giordano et al., 2011;
Aldu
´
an et al., 2011), they only address the extrac-
tion and annotation of semantic features for content
retrieval purposes. The following complementary ar-
eas that supports personalisation were identified at the
review: (i) knowledge representation, (ii) user profil-
ing, (iii) recommendation, (iv) content adaptation, (v)
content retrieval, (vi) content annotation and index-
ing, and (vii) user-centred software methodologies.
Even thought some proposals for multimedia person-
alisation are in the intersection of several of these ar-
eas, there are no works implementing an holistic ap-
proach that include all of them filling the existing gap.
From the reviewed works, those that have reached
a higher level of maturity are in the areas of content
retrieval and annotation, knowledge representation
and user profiling. However, their influence on the
development of multimedia personalised application
are still low and frameworks and architectures that in-
cludes all their benefits deserves more research effort.
From there, a set of open research challenges were
identified: (i) architectures and models to exploit the
advantages of new technologies such as cloud com-
puting and Internet of things; (ii) evolution of user
profiles; (iii) discovery of data semantic concepts and
knowledge base evolution; (iv) user-centred applica-
tions design; and (v) new types of content and data.
Future work are related to fulfil the open chal-
lenges and to the proposal of methodologies, frame-
works and architectures that allows the implemen-
tation of personalised multimedia applications. In
this way, the development of multimedia applications
should be based on contributions that support person-
alisation by: (i) relating the content with its semantic
significance, (ii) knowing the user patterns and pref-
erences, (iii) presenting the content according to the
capabilities of the devices and user cognition abili-
ties, and (iv) managing the annotation, storage and
retrieval of multimedia files.
REFERENCES
Aldu
´
an, M., S
´
anchez, F., Alvarez, F., Jim
´
enez, D.,
Men
´
endez, J. M., and Cebrecos, C. (2011). System
architecture for enriched semantic personalized media
search and retrieval in the future media internet. Com-
munications Magazine, IEEE, 49(3):144–151.
Appalla, P., Kuthadi, V. M., and Marwala, T. (2016). An
efficient educational data mining approach to support
e-learning. In Information Systems Design and Intel-
ligent Applications, pages 63–75. Springer.
Boll, S. (2003). Mm4u-a framework for creating personal-
ized multimedia content. In Proceeding of the Interna-
tional Conference on Distributed Multimedia Systems.
Burnham, J. F. (2006). Scopus database: a review. Biomed-
ical digital libraries, 3(1):1.
Chen, M., Ma, L., and Cai, W. (2012). A comprehen-
sive multimedia material library system with seman-
Towards Personalised Multimedia Applications - A Literature Review
133
tic expansion retrieval. Advanced Science Letters,
10(1):450–458.
de Fez, I., Gil, M., Fons, J., Guerri, J. C., and Pelechano,
V. (2015). A personalized system for scalable dis-
tribution of multimedia content in multicast wire-
less networks. Multimedia Tools and Applications,
74(21):9595–9621.
De Vrieze, P., Van Bommel, P., Van Der Weide, T., and
Klok, J. (2005). Adaptation in multimedia systems.
Multimedia Tools and Applications, 25(3):333–343.
Dolbear, C., Hobson, P., Vallet, D., Fernndez, M., Cantador,
I., and Castells, P. (2008). Personalised multimedia
summaries. Semantic Multimedia and Ontologies.
Dumas, B., Lalanne, D., and Oviatt, S. (2009). Multimodal
interfaces: A survey of principles, models and frame-
works. In Human machine interaction, pages 3–26.
Evans, A., Fern
´
andez, M., Vallet, D., and Castells, P.
(2006). Adaptive multimedia access: from user needs
to semantic personalisation. In Circuits and Systems,
2006. ISCAS 2006. Proceedings. 2006 IEEE Interna-
tional Symposium on, pages 4–pp. IEEE.
Furht, B. (2008). Encyclopedia of multimedia. Springer
Science & Business Media.
Giordano, D., Kavasidis, I., Pino, C., and Spampinato,
C. (2011). A semantic-based and adaptive architec-
ture for automatic multimedia retrieval composition.
In Content-Based Multimedia Indexing (CBMI), 2011
9th International Workshop on, pages 181–186. IEEE.
Guarino, N., Oberle, D., and Staab, S. (2009). What is an
ontology? In Handbook on ontologies, pages 1–17.
Guo, K., Pan, W., Lu, M., Zhou, X., and Ma, J. (2015). An
effective and economical architecture for semantic-
based heterogeneous multimedia big data retrieval.
Journal of Systems and Software, 102:207–216.
Karaman, S., Bagdanov, A. D., Landucci, L., DAmico,
G., Ferracani, A., Pezzatini, D., and Del Bimbo, A.
(2014). Personalized multimedia content delivery on
an interactive table by passive observation of museum
visitors. Multimedia Tools and Applications.
Lew, M. S., Sebe, N., Djeraba, C., and Jain, R. (2006).
Content-based multimedia information retrieval: State
of the art and challenges. ACM Transactions on Mul-
timedia Computing, Communications, and Applica-
tions (TOMM), 2(1):1–19.
Lu, Y., Sebe, N., Hytnen, R., and Tian, Q. (2011). Personal-
ization in multimedia retrieval: A survey. Multimedia
Tools and Applications, 51(1):247–277.
Mallik, A., Ghosh, H., Chaudhury, S., and Harit, G. (2013).
Mowl: an ontology representation language for web-
based multimedia applications. ACM Transactions on
Multimedia Computing, Communications, and Appli-
cations (TOMM), 10(1):8.
Mayring, P. (2014). Qualitative content analysis: theoretical
foundation, basic procedures and software solution.
Mylonas, P., Athanasiadis, T., Wallace, M., Avrithis, Y., and
Kollias, S. (2008). Semantic representation of mul-
timedia content: Knowledge representation and se-
mantic indexing. Multimedia Tools and Applications,
39(3):293–327.
Nie, W., Liu, A., and Su, Y. (2016). Cross-domain seman-
tic transfer from large-scale social media. Multimedia
Systems, 22(1):75–85.
Park, K.-W., Hong, H.-K., Lee, M., and Lee, D.-H.
(2014). Unified multimedia annotation system us-
ing mpeg-7 visual descriptors and semantic event-
templates. In Electronics, Information and Commu-
nications (ICEIC), 2014 International Conference on,
pages 1–2. IEEE.
Park, W.-I., Kang, S., and Kim, Y.-K. (2012). A person-
alized multimedia contents recommendation using a
psychological model. Computer Science and Infor-
mation Systems, 9(1):1–21.
Rinaldi, A. M. (2014). A multimedia ontology model based
on linguistic properties and audio-visual features. In-
formation Sciences, 277:234–246.
Sastoque, S., Narv
´
aez, C., and Iregui, M. (2014). Ontolog-
ical knowledge based model for personalizing multi-
media applications. Ventana Inform
´
atica, 30.
Scherp, A., Boll, S., and Cremer, H. (2007). Emergent se-
mantics in personalized multimedia content. Journal
of Digital Information Management, 5(2):55.
Sebe, N. and Tian, Q. (2007). Personalized multimedia re-
trieval: the new trend? In Proceedings of the interna-
tional workshop on Workshop on multimedia informa-
tion retrieval, pages 299–306. ACM.
Seuring, S. and M
¨
uller, M. (2008). From a literature re-
view to a conceptual framework for sustainable sup-
ply chain management. Journal of cleaner production,
16(15):1699–1710.
Sinclair, S. and Rockwell, G. (2016). Voyant tools analysis.
Studer, R., Benjamins, V. R., and Fensel, D. (1998). Knowl-
edge engineering: principles and methods. Data &
knowledge engineering, 25(1):161–197.
Sun, Y., Chen, H.-x., Chen, M.-s., Sang, A.-j., Liu, W.-j.,
and Yang, S.-y. (2011). Multimedia high-level se-
mantic framework and retrieval algorithm. Journal
of Jilin University (Engineering and Technology Edi-
tion), 1:048.
Webster, J. and Watson, R. T. (2002). Analyzing the past
to prepare for the future: Writing a. MIS quarterly,
26(2):13–23.
Weiß, D., Duchon, M., Fuchs, F., and Linnhoff-Popien,
C. (2008). Context-aware personalization for mobile
multimedia services. In Proceedings of the 6th Inter-
national Conference on Advances in Mobile Comput-
ing and Multimedia, pages 267–271. ACM.
Xue, M., Zheng, S., and Zhang, C. (2012). Ontology-based
surveillance video archive and retrieval system. In
Advanced Computational Intelligence (ICACI), 2012
IEEE Fifth International Conference on, pages 84–89.
IEEE.
Yang, Y., Ha, H.-Y., Fleites, F. C., and Chen, S.-C. (2014).
A multimedia semantic retrieval mobile system based
on hcfgs. MultiMedia, IEEE, 21(1):36–46.
Zeng, Z. (2016). Design of a cloud services platform for
a multimedia teaching environment. World Trans-
actions on Engineering and Technology Education,
14(1):173–178.
ICEIS 2017 - 19th International Conference on Enterprise Information Systems
134