Towards Personalised Multimedia Applications

A Literature Review

Sebastian Sastoque H.

, Oscar Avila

and Marcela Iregui

Department of Systems and Computing Engineering, School of Engineering, Universidad de los Andes, Bogota, Colombia

ACCEDER Research Group, Faculty of Engineering, Universidad Militar Nueva Granada, Bogota, Colombia

Keywords:

Multimedia, Personalisation, Retrieval, Annotation, Application Development, Systems, Models, Literature

Review.

Abstract:

Multimedia applications are now commonly used in daily life for several domains as marketing, health, learn-

ing and entertainment, among others. As the number of available applications increases, a competitive factor

is the level of alignment to personal preferences. Indeed, the role of multimedia content has been crucial to

generate user centred applications. However, multimedia content personalisation requires complex systems

that execute diverse tasks such as representation, modelling, annotation and retrieval. Research on this ﬁeld

has been focused on content annotation and retrieval perspectives. Despite this, these domains do not address

two of key personalisation factors, i.e., considering personal preferences and contextual knowledge. This work

presents a literature review aimed to identify theoretical elements related to personalisation purposes, which

could be integrated to the most common approaches. As a result, a road map for future research is established.

1 INTRODUCTION

Nowadays the development of new technologies and

the expansion of the Internet have broadened the

range of applications and systems that use multimedia

content. Diverse ﬁelds such as education, medicine,

entertainment, marketing and business, among oth-

ers, have beneﬁted from this phenomenon. While

the number and variety of applications are growing,

users change their needs, expectations and demands,

looking for ever further personalised experiences (Lu

et al., 2011). For instance, users prefer watching

movies of their favourite genres, reading news about

their interest topics or listening music according to

their mood. In this context, multimedia content per-

sonalisation has become a crucial factor to enrich the

user experience. For example, Facebook became the

main leader of social networks apps, thanks to person-

alised social interactions through multimedia content,

that was its main differential within the existing com-

petitors at the moment.

The concept of personalisation can be deﬁned as

the process of customising data output in order to re-

ﬂect users interests, preferences and situations and

meet their requirements (Boll, 2003). Multimedia

content itself is not enough for personalizing appli-

cations. Thus, complex systems are needed for repre-

senting, modelling, indexing, retrieving and present-

ing media content according to user needs and the ap-

plication domain (Lu et al., 2011). This is known as

the semantic gap, which constitutes one of the main

challenges in multimedia research (Lew et al., 2006).

Aiming at ﬁlling this gap, proposals are presented

in two different areas: content features extraction and

annotation, and content retrieval (Lu et al., 2011).

Although, research in these ﬁelds contributes to per-

sonalisation, they do not address the user preferences

knowledge domain, which is a key factor in user

centred applications. Therefore, some complemen-

tary areas could contribute to this ﬁeld by extending

the main functionalities obtained by using approaches

only based on annotation and retrieval.

Some approaches have adopted an holistic per-

spective of the problem (Sastoque et al., 2014; Gior-

dano et al., 2011; Aldu

an et al., 2011; Scherp et al.,

2007). However, these works do not present an ex-

haustive review of the literature to present their pro-

posals. In fact, after a preliminary searching process,

it was not found a work aimed to review personalised

multimedia applications research ﬁeld. The found re-

views are focused on the area of Multimedia Informa-

tion Retrieval (Lu et al., 2011; Sebe and Tian, 2007;

Lew et al., 2006).

Accordingly, this work presents a literature review

H., S., Avila, O. and Iregui, M.

Towards Personalised Multimedia Applications - A Literature Review.

DOI: 10.5220/0006326701270134

In Proceedings of the 19th International Conference on Enterprise Information Systems (ICEIS 2017) - Volume 3, pages 127-134

ISBN: 978-989-758-249-3

127

aimed (i) to identify the research process behaviour in

the domain of personalised multimedia applications

and (ii) to establish which approaches from comple-

mentary areas could be adapted for personalisation

purposes. The results of this review are used to deter-

mine what are the main challenges and open research

questions in the area.

The remaining of the paper is organised as fol-

lows: Section 2 presents related works and introduce

the mains concepts for the review. Section 3 intro-

duces the methodology of literature review and the

results of the two ﬁrst steps, namely, planning and

Conducting and Material Collection and Evaluation.

Section 4 presents a qualitative and quantitative anal-

ysis of the review, which corresponds to the third step

of the methodology. Section 5 presents the challenges

related to the area. Finally, section 6 concludes the

paper.

2 RELATED WORKS

Given the variety of contributions in the ﬁeld of per-

sonalisation in Multimedia Information Retrieval, this

section summarises some reviews aimed to synthesise

the literature in this area to bring an overview of the

related works.

Lu et al., (Lu et al., 2011) present a survey aimed

to overview the current research in multimedia anno-

tation, relation and visualisation. The survey analyse

a sequence of applications useful for content-based

multimedia retrieval. Then, the authors select the best

models and techniques used to represent context and

user preferences, conceptualise ontologies, display

content and retrieve multimedia information. Finally,

the selected models and techniques are mixed to pro-

pose the concept of Intelligent Multimedia Content.

Sebe et al., (Sebe and Tian, 2007) present an analysis

of the topics related to personalisation in Multimedia

Information Retrieval aimed to identify the common

tendencies to achieve personalisation. Particularly,

this work addresses two issues related to: (i) the users

preferences for personalised access to the content and

(ii) the devices capabilities and applications to display

content. The authors conclude that there exist chal-

lenges in personalisation related to Multimedia Col-

laboration, Interactive Search with Agent Interfaces

and Folksonomies, i.e., the use of crowd wisdom to

annotate and retrieve the content. Lew et al., (Lew

et al., 2006) review a set of 100+ published articles

before 2006 in the area of content-based multime-

dia information retrieval. They studied the effect of

search paradigms, affective computing, learning, se-

mantic queries and evaluation techniques in content-

based retrieval. The authors conclude that the im-

portant challenges are: (i) semantic search in media

with complex backgrounds, (ii) multimodal analysis,

(iii) exploration of media collections, (iv) interactive

search and (v) evaluation methodologies.

Although these reviews are a great contribution to

the state of the art in the multimedia ﬁeld, they only

are focused on the retrieval area and how to represent

the user preferences. It leaves a gap related to the

current state of the development of personalised mul-

timedia applications and what are the main challenges

to achieve next level of personalisation, which are the

most relevant topics of this work.

3 LITERATURE REVIEW

A literature review is a systematic and reproducible

design to identify, evaluate and interpret a set of ex-

isting documents (Seuring and M

uller, 2008). The re-

views usually aim two objectives: (i) to summarise

the existing research of a ﬁeld by identifying issues,

patterns and contributions and (ii) to identify the con-

ceptual framework of a ﬁeld and contribute to its

development (Webster and Watson, 2002). The de-

sign of a literature review should follow a methodol-

ogy that ensures the comprehension and analysis of

the content, where quantitative and qualitative analy-

sis should be addressed to describe the research ﬁeld

(Seuring and M

uller, 2008). In this work the se-

lected methodology was based on the model proposed

by (Mayring, 2014) and adopted the best practises

proposed by (Webster and Watson, 2002). Thus the

literature review followed the next steps: Planning

(section 3.1), Conducting and material collection and

evaluation (section 3.2) and Descriptive review anal-

ysis (section 4).

3.1 Planning

This step deﬁnes the review questions, the criteria to

conduct the search, the delimitation of the material

and the method to validate contributions. Accord-

ing to the review objectives introduced in section 1,

the proposed review questions are: (i) What are the

main contributions of the different approaches?; (ii)

What are the approaches ﬁeld of application?; (iii)

How multimedia content is related to the user domain

context?; (iv) Which techniques are used to represent

and store content annotations? and (v) How personal-

isation is addressed?

Regarding the material delimitation to reduce the

number of proposals to analyse, the review process is

based on the following considerations: (i) the analysis

ICEIS 2017 - 19th International Conference on Enterprise Information Systems

128

scope includes only peer-reviewed publications writ-

ten in English; (ii) the main area of the review is mod-

els for personalised multimedia applications; (iii) the

following complementary areas are included to iden-

tify the common approaches: personalised content re-

trieval and personalised content annotation, (iv) only

those works focused in models, architectures, plat-

forms, software, systems or frameworks are included,

(v) the proposals are in the design, development or

implementation stage (vi) to ensure the objectivity of

the review the work selection is performed by the au-

thors separately and (vii) the works are published until

April of 2016.

3.2 Conducting and Material Collection

and Evaluation

In this step it is performed a comprehensive and ex-

haustive search for primary approaches using the cri-

teria deﬁned in the previous step. Then, a validation

and assessment process is performed to select the set

of approaches that will be included in the review.

The search of related publications was conducted

as a structured keyword search. The search process

used the Scopus database (Burnham, 2006). This

database was selected because it indexes quality per-

reviewed works presented in journals and confer-

ences of main scientiﬁc publishers, such as, IEEE

Explore, ACM, Elsevier, among others (Burnham,

2006). The query used in the search was: (TITLE-

ABS-KEY(“Personalised multimedia application”))

OR (TITLE-ABS-KEY(“Application” OR “Model”

OR “Architecture” OR “Framework” OR “System”

OR “Platform” OR “Software”) AND TITLE-ABS-

KEY(“Personalization” OR “Personalisation” OR

“Personalized”) AND TITLE-ABS-KEY(“Multimedia

Content” OR “Multimedia” OR ( (“Media” OR

“Content” OR “Multimedia”) AND (“Retrieval” OR

“Annotation”)) ) AND TITLE-ABS-KEY(“Design”

OR “Implementation” OR “Development”))

The Scopus engine resulted in 851 candidate arti-

cles. The evaluation of the material process to limit

the articles used the following ﬁlters: (i) Reading of

the title and keywords reduced the numbers of works

to 388. (ii) Reading of the abstracts to ﬁlter those

ones that do not have hints to answer the review ques-

tions, limited the number of articles to 123. (iii) A

full reading of the articles concludes the evaluation to

the selection of a ﬁnal set of 76 articles. In this work

we synthesise the most common approaches and only

cite the most relevant works for each approach.

4 DESCRIPTIVE REVIEW

ANALYSIS

It assesses formal aspects of the material. Two kinds

of analysis are performed for providing theoretical

background from contributions or identifying open re-

search challenges. The ﬁrst analysis is quantitative,

which provides information about the area trends,

e.g., the number of publications per year. The sec-

ond analysis is qualitative, corresponding to the main

concepts descriptions regarding to predeﬁned review

questions, e.g., common methods and techniques used

in one task.

4.1 Quantitative Analysis

Descriptive dimensions are deﬁned for performing

quantitative analysis and classiﬁcation of each article.

The dimensions are time dependant and answer the

following questions: (i) how frequent are the publica-

tions?, (ii) which research methodologies are applied,

(iii) what are common terms used in the works? (iv)

what are the publications type?, and (v) which are the

most common journals and conferences?. The results

of the quantitative analysis using these dimensions are

presented below.

After the ﬁrst work, from 1991 to 2002, the publi-

cation frequency remained below one paper per year.

The number of publications increased in 2005 and

since then it varies between 2 to 9. However, it could

be expected that with emerging technologies and de-

vices, in the next future this frequency would show a

signiﬁcance increase.

The methodologies reported by the reviewed pub-

lications are among these four types: (i) theoretical

and conceptual; (ii) based on case studies and domain

dependant applications; (iii) modelling founded and

(iv) literature reviews.

Figure 1: Common terms.

An analysis of the common terms used in the

publications is presented in Figure 1. This analysis

was performed with the tool (Sinclair and Rockwell,

Towards Personalised Multimedia Applications - A Literature Review

129

2016). As expected the most common terms are Mul-

timedia and Content. However, the broad use of terms

such as knowledge, semantic and ontology shows that

exists a trend in the different approaches to include

semantic and knowledge representation issues. The

most common media types addressed in the papers

are videos and images. Finally, the works commonly

present frameworks and models and the most frequent

application is content retrieval.

Figure 2: Publications per type.

The publications are equally distributed between

conferences and journals (Figure 2). Although there

is not a conference or journal that has a representative

frequency of publications, the journal with the largest

number of contributions is Multimedia Tools and Ap-

plications. The highest number of conferences related

to the ﬁeld publish their proceedings in Lecture Notes

in Computer Science, IEEE Conference on Multime-

dia and CEUR Workshop Proceedings.

4.2 Qualitative Analysis

The qualitative analysis of the review is presented ac-

cording to the questions introduced in section 3.1.

4.2.1 What are the Main Contributions of the

Different Approaches?

The principal contributions of the approaches are di-

vided in ﬁve categories: (i) design of architectures,

(ii) methods for content representation, (iii) models

for user proﬁling and development of applications,

(iv) study cases and applications for personalisation

in speciﬁc domains and (v) surveys and literature re-

views.

Related to the category of design of architectures,

the main contributions are associated with: the use

of speciﬁc technologies and standards, such as cloud

computing, big data and MPEG-7 (Zeng, 2016; Guo

et al., 2015); handle, deliver, process, annotate and

retrieve content (Aldu

an et al., 2011; de Fez et al.,

2015); general purpose related with personalisation

based on services or the deﬁnition of components

(Scherp et al., 2007).

The approaches related to content representation

commonly address two tasks: ﬁrst, the representa-

tion of content using low level features, such as, vi-

sual structures, signal descriptors or descriptive lan-

guages (Rinaldi, 2014); and second, the semantic rep-

resentation of the content by using meta-data and al-

gorithms to relate low level features with semantics

(Mallik et al., 2013).

The contributions corresponding to modelling cat-

egory aim to model: user proﬁles for speciﬁc context

applications (Park et al., 2012); applications for spe-

ciﬁc domains as museums, news, searches or sports

(Appalla et al., 2016); techniques for content retrieval,

annotation and semantic relation (Sun et al., 2011;

Scherp et al., 2007; Sebe and Tian, 2007); frame-

works to the development and implementation of ap-

plications (Weiß et al., 2008).

The articles that present study cases and appli-

cations have as main contributions: the deﬁnition

of concepts and requirements for content distribu-

tion and personalisation (Evans et al., 2006); the im-

plementation of applications for content digitization,

management and processing (Chen et al., 2012); the

use of domain knowledge in applications related to

the context, such as, vigilance, georeferencing and

digital museums (Xue et al., 2012; Mylonas et al.,

2008).

Finally, a set of articles present literature reviews

and surveys related to the state for multimedia re-

trieval, recommendation, personalisation and annota-

tion (Lu et al., 2011; Lew et al., 2006).

4.2.2 What are the Approaches Field of

Application?

The adjacent ﬁelds, that are related to multimedia

content personalisation, presented in the review are:

(i) retrieval, (ii) annotation and indexing, (iii) recom-

mendation and (iv) adaptation and personalisation.

The multimedia content retrieval aims to the ex-

traction of semantic information from multimedia

data sources to retrieve content according to a query

(Furht, 2008). The queries could be composed as

textual keywords, content examples or mixing both

techniques. The approaches that address retrieval

commonly aims the inclusion of semantic relations

to the retrieval process. The common methods have

three principal steps: (i) feature extraction and con-

tent summarising , (ii) ﬁltering and (iii) description

(Aldu

an et al., 2011; Nie et al., 2016; Scherp et al.,

2007; Sebe and Tian, 2007).

ICEIS 2017 - 19th International Conference on Enterprise Information Systems

130

Annotation and Indexing techniques supports the

content retrieval process. The multimedia content an-

notation refers to methods that allows semantic and

low level descriptions of content (Furht, 2008). These

annotations could follow speciﬁc frameworks or mod-

els related to domain context (semantic descriptions)

and content features (low level descriptions) (Park

et al., 2014; Mallik et al., 2013). Multimedia in-

dexing refers to content pre-processing aimed to cre-

ate indexes by using features (Mylonas et al., 2008).

The indexation objective is to increase searching efﬁ-

ciency on large databases. This must support similar-

ity queries, in which the similarity measure is deﬁned

previously by a domain expert.

Recommendation processes aim to represent users

preferences with the objective of suggest unseen items

(Furht, 2008). The ﬁnal outcome of recommendation

is a set of new elements for a speciﬁc person by em-

ploying a proﬁle and information about other users

and items features (de Fez et al., 2015). Another ap-

proaches seek to guide the user in a personalized way

to elements of his interest in a large space of possible

options (Karaman et al., 2014).

Personalisation and Adaptation present the con-

tent to the user in an individualised way according

to their tastes and constraints. Personalisation pro-

cess allows the adaptation of speciﬁc multimedia con-

tent in accordance to individual standards, tastes and

preferences (Furht, 2008). Approaches that address

personalisation seeks to extract user proﬁles and re-

lates them to semantic descriptions of the multime-

dia content for presentation purposes (Dolbear et al.,

2008). Adaptation is the process to transform a con-

tent to another representation that accommodates to

restricted device input and output capabilities (Furht,

2008), e.g., converting images to text descriptions due

to bandwidth constrains of mobile devices (De Vrieze

et al., 2005).

4.2.3 How Multimedia Content is Related to the

User Domain Context?

The reviewed works that address relation between

multimedia content and user domain context use the

following approaches: (i) ontologies, (ii) models and

(iii) annotations.

An ontology is ”a formal, explicit speciﬁcation

of a shared conceptualisation“ (Studer et al., 1998).

According to (Guarino et al., 2009), this conceptual-

isation refers to an abstract and simpliﬁed view of a

domain that formally represent its knowledge by in-

cluding entities and their relationships. In this con-

text, ontologies are used to relate the content to spe-

ciﬁc domains by associating: low level features and

descriptors with semantic categories; content descrip-

tion and annotations with semantic domain rules and;

user proﬁles with semantic categories and content de-

scriptors (Aldu

an et al., 2011; de Fez et al., 2015).

The approaches that propose models report a set

of speciﬁc classes, attributes and methods for pro-

viding semantic representation of multimedia con-

tent. Speciﬁcally, some works propose domain mod-

els (Mylonas et al., 2008), others use graphs and

semantic networks (Weiß et al., 2008), and those

that propose existing models such as OWL triplets,

SCORM elements or descriptive languages, etc (Nie

et al., 2016).

Finally, the last set of approaches relate seman-

tics to the content by meta-data annotations regarding

domain concepts. These annotations are commonly

stored in XML schemes attached to the content ﬁle

or linked databases (Yang et al., 2014; Scherp et al.,

2007).

4.2.4 Which Techniques are Used to Represent

and Store Content Annotations?

Content annotations are commonly obtained by us-

ing automatic, semi-automatic and manual methods.

The automatic ones refer at the extraction of features

and descriptors by computational systems. In semi-

automatic methods the systems extract some features,

which are complemented by the user. In the man-

ual method, the user is the responsible of annotate

the content according to his knowledge. Regardless

of the method used to extract features and descrip-

tors, the annotations proposed by the reviewed works

are represented by using (i) MPEG7, (ii) raw meta-

data and (iii) models; and are stored by using (i)

XML Schemes, (ii) raw data, (iii) ontologies and (iv)

databases.

MPEG-7 is a Multimedia Content Description In-

terface that describes the features and meta-data of

multimedia content through a standardized scheme

(Furht, 2008). These scheme use different types of

descriptors according to the content type, i.e., visual

descriptors for images and videos or signal descrip-

tors for audios and videos, among others. The stor-

age of MPEG-7 descriptors in the reviewed works use

databases, XML schemes attached to the binary ﬁles

of the content and ontologies (Park et al., 2014; Sebe

and Tian, 2007).

Other works used to create annotations is the stor-

age of key and value pairs, in which each pair repre-

sent a feature or description of the multimedia con-

tent. These works store the annotations in databases,

raw ﬁles, ontologies and XML schemes (Guo et al.,

2015; Scherp et al., 2007; Aldu

an et al., 2011).

Finally, some approaches create models of the an-

notations according to the application needs. These

Towards Personalised Multimedia Applications - A Literature Review

131

models represent low level and semantic descriptors.

The models are stored in databases, XML schemes

and ontologies (de Fez et al., 2015).

4.2.5 How Personalisation is Addressed?

Personalisation is addressed by the creation and de-

velopment of user proﬁles. The latter are a set of at-

tributes that represent users tastes, behaviour, prefer-

ences and patterns (Furht, 2008). The works reviewed

address personalisation in two ways: ﬁrst, by integrat-

ing user context information, and second by modify-

ing content presentation. The ﬁrst type of approaches

integrates user context to recommend and provide a

set of items that match the user proﬁle (de Fez et al.,

2015; Evans et al., 2006) or that match other user con-

text data such as location, mental abilities or device

capabilities (Karaman et al., 2014; Weiß et al., 2008).

The second type of approaches present the content on

an individual basis according to user proﬁle. In these

approaches the presentation of the content is the more

important factor to address personalisation (Aldu

et al., 2011; Mylonas et al., 2008; Sebe and Tian,

2007; Scherp et al., 2007).

5 OPEN RESEARCH

CHALLENGES

Notwithstanding the signiﬁcant progress of academic

research on personalised multimedia applications,

there has been little impact in the development of

commercial applications that use these research re-

sults. Although advances in various complementary

areas, such as annotation, representation and retrieval

of multimedia content have contributed to the person-

alisation ﬁeld, there are still open challenges ahead

for increase the use of research results in industries.

5.1 Architectures and Models to Exploit

the Advantages of New Technologies

With the rapid and steady evolution and expansion of

new technologies, such as, Cloud Computing, Inter-

net of Things and Big Data, in order to generate in-

novations in the media industry, these technologies

must be considered to complement personalised mul-

timedia applications. For example, with architecture

models based on Cloud Computing technologies, the

design and test of new applications could be easily

addressed, allowing to developers undertaking, their

new projects with few resources in relatively short

time. In this context, personalisation could be eas-

ily addressed through Cloud Computing by permit-

ting storage and content processing in a distributed

manner, avoiding full installation, saving computa-

tion processing in the client side and reducing soft-

ware upgrading and maintenance. Regarding Inter-

net of Things, real-time multimedia communication

could be applied in different contexts, increasing per-

sonalised experiences. For example, in emergency

calls, user could share detailed information about an

incident by transmitting images and/or videos from

near cameras, permitting a real time analysis of the

incident nature and severity.

Therefore, a current challenge in personalised

multimedia applications development is the design of

architectures models easily portable to the cloud, to

take advantage of all its beneﬁts and enable new pos-

sibilities for data analysis and new businesses devel-

opment.

5.2 Evolution of User Proﬁles

Other current challenge is the proposal of advanced

models for user proﬁles. Models may include con-

text information of the content as well as attributes

that represent users context. User models need to in-

clude elements to describe demographic and personal

information, local settings, interests, skills, goals,

tastes, behaviours, patterns and individual character-

istics. These types of information should be interre-

lated with knowledge representation to enable contex-

tual reasoning between the multimedia content, the

application domain and the user proﬁle. Therefore,

as the user context is highly changing, user proﬁle

should evolve over time and should be dynamically

updated and reﬁned to reﬂect external events, includ-

ing the user interaction history.

5.3 Discovery of Data Semantic

Concepts and Knowledge Base

Evolution

Multimedia applications should have the capability to

add new content for personalising user experience. In

this sense, the applications should have the capabil-

ity to discover the semantics of new incoming con-

tent, update the knowledge base and determine rela-

tionships. This can be achieved by using low-level

features in conjunction with machine learning tech-

niques to identify high level content semantics. In ad-

dition, the knowledge base should be adaptable to the

application domain and the user proﬁle. User mod-

elling deals with individual abilities, such as, cogni-

tive, social and linguistic skills and personal and en-

vironmental circumstances. In this sense, an open

challenge is the personalisation of knowledge bases

ICEIS 2017 - 19th International Conference on Enterprise Information Systems

132

in conjunction with easy maintainability and interop-

erability.

5.4 User-centred Design of Applications

Although today literature refers to ubiquitous, mobile,

pervasive, social and personalised applications, ad-

vances in research and development are still far from

achieving them. Most common technologies and ap-

plications are unfriendly, cultural biased, unnatural

and difﬁcult to use. Thus, a gap between user needs

and personalised applications development is pointed

out. For example, cognitive science has shown the

beneﬁts of multi-modal interfaces to improve, user

understanding and interactions, due to the apparent

parallelism on the brain in response to multiple in-

put stimulus (Dumas et al., 2009). This means, from

an application development standpoint that it is use-

ful design meaning interfaces that exploit this kind of

advantage. However, most of the reviewed works do

not include these kind of considerations in the content

personalisation. Accordingly, it is important to create

new frameworks that include user-centred design ap-

proaches to personalised multimedia applications de-

velopment.

5.5 New Types of Content and Data

Most analysed works only focus the personalisation

on audio, images, text or video. However, new types

of multimedia content are now present in the litera-

ture, e.g., multimedia presentations, interactive docu-

ments, geo-referenced maps or 3d models. Therefore,

personalisation research should focus on the analy-

sis of new media types and techniques for multime-

dia integration. Important questions remain regarding

methods for effective content selection, media alloca-

tion, and modality selection.

6 CONCLUSION

Personalisation of Multimedia Applications in an

emerging research area that has received growing

attention in the research community over the past

decade. This work presents a literature review that

identify the contributions of complementary areas

to personalisation ﬁelds. The results of the review

make evident that indeed exists two classical work ar-

eas aimed to address personalisation (annotation and

retrieval). Although some works try to mix them

to achieve personalisation (Giordano et al., 2011;

Aldu

an et al., 2011), they only address the extrac-

tion and annotation of semantic features for content

retrieval purposes. The following complementary ar-

eas that supports personalisation were identiﬁed at the

review: (i) knowledge representation, (ii) user proﬁl-

ing, (iii) recommendation, (iv) content adaptation, (v)

content retrieval, (vi) content annotation and index-

ing, and (vii) user-centred software methodologies.

Even thought some proposals for multimedia person-

alisation are in the intersection of several of these ar-

eas, there are no works implementing an holistic ap-

proach that include all of them ﬁlling the existing gap.

From the reviewed works, those that have reached

a higher level of maturity are in the areas of content

retrieval and annotation, knowledge representation

and user proﬁling. However, their inﬂuence on the

development of multimedia personalised application

are still low and frameworks and architectures that in-

cludes all their beneﬁts deserves more research effort.

From there, a set of open research challenges were

identiﬁed: (i) architectures and models to exploit the

advantages of new technologies such as cloud com-

puting and Internet of things; (ii) evolution of user

proﬁles; (iii) discovery of data semantic concepts and

knowledge base evolution; (iv) user-centred applica-

tions design; and (v) new types of content and data.

Future work are related to fulﬁl the open chal-

lenges and to the proposal of methodologies, frame-

works and architectures that allows the implemen-

tation of personalised multimedia applications. In

this way, the development of multimedia applications

should be based on contributions that support person-

alisation by: (i) relating the content with its semantic

signiﬁcance, (ii) knowing the user patterns and pref-

erences, (iii) presenting the content according to the

capabilities of the devices and user cognition abili-

ties, and (iv) managing the annotation, storage and

retrieval of multimedia ﬁles.

REFERENCES

Aldu

an, M., S

anchez, F., Alvarez, F., Jim

enez, D.,

Men

endez, J. M., and Cebrecos, C. (2011). System

architecture for enriched semantic personalized media

search and retrieval in the future media internet. Com-

munications Magazine, IEEE, 49(3):144–151.

Appalla, P., Kuthadi, V. M., and Marwala, T. (2016). An

efﬁcient educational data mining approach to support

e-learning. In Information Systems Design and Intel-

ligent Applications, pages 63–75. Springer.

Boll, S. (2003). Mm4u-a framework for creating personal-

ized multimedia content. In Proceeding of the Interna-

tional Conference on Distributed Multimedia Systems.

Burnham, J. F. (2006). Scopus database: a review. Biomed-

ical digital libraries, 3(1):1.

Chen, M., Ma, L., and Cai, W. (2012). A comprehen-

sive multimedia material library system with seman-

Towards Personalised Multimedia Applications - A Literature Review

133

tic expansion retrieval. Advanced Science Letters,

10(1):450–458.

de Fez, I., Gil, M., Fons, J., Guerri, J. C., and Pelechano,

V. (2015). A personalized system for scalable dis-

tribution of multimedia content in multicast wire-

less networks. Multimedia Tools and Applications,

74(21):9595–9621.

De Vrieze, P., Van Bommel, P., Van Der Weide, T., and

Klok, J. (2005). Adaptation in multimedia systems.

Multimedia Tools and Applications, 25(3):333–343.

Dolbear, C., Hobson, P., Vallet, D., Fernndez, M., Cantador,

I., and Castells, P. (2008). Personalised multimedia

summaries. Semantic Multimedia and Ontologies.

Dumas, B., Lalanne, D., and Oviatt, S. (2009). Multimodal

interfaces: A survey of principles, models and frame-

works. In Human machine interaction, pages 3–26.

Evans, A., Fern

andez, M., Vallet, D., and Castells, P.

(2006). Adaptive multimedia access: from user needs

to semantic personalisation. In Circuits and Systems,

2006. ISCAS 2006. Proceedings. 2006 IEEE Interna-

tional Symposium on, pages 4–pp. IEEE.

Furht, B. (2008). Encyclopedia of multimedia. Springer

Science & Business Media.

Giordano, D., Kavasidis, I., Pino, C., and Spampinato,

C. (2011). A semantic-based and adaptive architec-

ture for automatic multimedia retrieval composition.

In Content-Based Multimedia Indexing (CBMI), 2011

9th International Workshop on, pages 181–186. IEEE.

Guarino, N., Oberle, D., and Staab, S. (2009). What is an

ontology? In Handbook on ontologies, pages 1–17.

Guo, K., Pan, W., Lu, M., Zhou, X., and Ma, J. (2015). An

effective and economical architecture for semantic-

based heterogeneous multimedia big data retrieval.

Journal of Systems and Software, 102:207–216.

Karaman, S., Bagdanov, A. D., Landucci, L., DAmico,

G., Ferracani, A., Pezzatini, D., and Del Bimbo, A.

(2014). Personalized multimedia content delivery on

an interactive table by passive observation of museum

visitors. Multimedia Tools and Applications.

Lew, M. S., Sebe, N., Djeraba, C., and Jain, R. (2006).

Content-based multimedia information retrieval: State

of the art and challenges. ACM Transactions on Mul-

timedia Computing, Communications, and Applica-

tions (TOMM), 2(1):1–19.

Lu, Y., Sebe, N., Hytnen, R., and Tian, Q. (2011). Personal-

ization in multimedia retrieval: A survey. Multimedia

Tools and Applications, 51(1):247–277.

Mallik, A., Ghosh, H., Chaudhury, S., and Harit, G. (2013).

Mowl: an ontology representation language for web-

based multimedia applications. ACM Transactions on

Multimedia Computing, Communications, and Appli-

cations (TOMM), 10(1):8.

Mayring, P. (2014). Qualitative content analysis: theoretical

foundation, basic procedures and software solution.

Mylonas, P., Athanasiadis, T., Wallace, M., Avrithis, Y., and

Kollias, S. (2008). Semantic representation of mul-

timedia content: Knowledge representation and se-

mantic indexing. Multimedia Tools and Applications,

39(3):293–327.

Nie, W., Liu, A., and Su, Y. (2016). Cross-domain seman-

tic transfer from large-scale social media. Multimedia

Systems, 22(1):75–85.

Park, K.-W., Hong, H.-K., Lee, M., and Lee, D.-H.

(2014). Uniﬁed multimedia annotation system us-

ing mpeg-7 visual descriptors and semantic event-

templates. In Electronics, Information and Commu-

nications (ICEIC), 2014 International Conference on,

pages 1–2. IEEE.

Park, W.-I., Kang, S., and Kim, Y.-K. (2012). A person-

alized multimedia contents recommendation using a

psychological model. Computer Science and Infor-

mation Systems, 9(1):1–21.

Rinaldi, A. M. (2014). A multimedia ontology model based

on linguistic properties and audio-visual features. In-

formation Sciences, 277:234–246.

Sastoque, S., Narv

aez, C., and Iregui, M. (2014). Ontolog-

ical knowledge based model for personalizing multi-

media applications. Ventana Inform

atica, 30.

Scherp, A., Boll, S., and Cremer, H. (2007). Emergent se-

mantics in personalized multimedia content. Journal

of Digital Information Management, 5(2):55.

Sebe, N. and Tian, Q. (2007). Personalized multimedia re-

trieval: the new trend? In Proceedings of the interna-

tional workshop on Workshop on multimedia informa-

tion retrieval, pages 299–306. ACM.

Seuring, S. and M

uller, M. (2008). From a literature re-

view to a conceptual framework for sustainable sup-

ply chain management. Journal of cleaner production,

16(15):1699–1710.

Sinclair, S. and Rockwell, G. (2016). Voyant tools analysis.

Studer, R., Benjamins, V. R., and Fensel, D. (1998). Knowl-

edge engineering: principles and methods. Data &

knowledge engineering, 25(1):161–197.

Sun, Y., Chen, H.-x., Chen, M.-s., Sang, A.-j., Liu, W.-j.,

and Yang, S.-y. (2011). Multimedia high-level se-

mantic framework and retrieval algorithm. Journal

of Jilin University (Engineering and Technology Edi-

tion), 1:048.

Webster, J. and Watson, R. T. (2002). Analyzing the past

to prepare for the future: Writing a. MIS quarterly,

26(2):13–23.

Weiß, D., Duchon, M., Fuchs, F., and Linnhoff-Popien,

C. (2008). Context-aware personalization for mobile

multimedia services. In Proceedings of the 6th Inter-

national Conference on Advances in Mobile Comput-

ing and Multimedia, pages 267–271. ACM.

Xue, M., Zheng, S., and Zhang, C. (2012). Ontology-based

surveillance video archive and retrieval system. In

Advanced Computational Intelligence (ICACI), 2012

IEEE Fifth International Conference on, pages 84–89.

IEEE.

Yang, Y., Ha, H.-Y., Fleites, F. C., and Chen, S.-C. (2014).

A multimedia semantic retrieval mobile system based

on hcfgs. MultiMedia, IEEE, 21(1):36–46.

Zeng, Z. (2016). Design of a cloud services platform for

a multimedia teaching environment. World Trans-

actions on Engineering and Technology Education,

14(1):173–178.

ICEIS 2017 - 19th International Conference on Enterprise Information Systems

134