The Information Value of Context for a Mobile News Service

Toon De Pessemier, Kris Vanhecke and Luc Martens

iMinds - WiCa, Ghent University, Gaston Crommenlaan 8 box 201, B-9050 Ghent, Belgium

Keywords:

Recommender System, News, User Study, Context, User Interaction.

Abstract:

Traditional recommender systems provide personal suggestions based on the user’s preferences, without tak-

ing into account any additional contextual information such as time or device type. However, in many ap-

plications, this contextual information may be relevant for the human decision process, and as a result, be

important to incorporate into the recommendation process, which gave rise to context-aware recommender

systems. However, the information value of contextual data for the recommendation process is highly depen-

dent on the application domain and the users’ consumption behavior in different contextual situations. This

research aims to assess the information value of context for a recommender system of a mobile news service by

analyzing user interactions and feedback. A large-scale user study shows that context-aware recommendations

outperform traditional recommendations, but also indicates that the accuracy improvement might be limited

in a real-life situation. Service usage takes place in a limited number of different contexts due to user habits

and repetitive behavior, leaving little room for optimization based on the context. Data fragmentation over

different contextual situations strengthens the sparsity problem, thereby limiting the user-perceived accuracy

gain obtained by incorporating context in the recommender. These ﬁndings are important for news providers

when considering to offer context-aware recommendations.

1 INTRODUCTION

Recommender systems are software tools and tech-

niques providing suggestions for items to be of in-

terest to a user such as videos, songs, or news arti-

cles. The consumption of these audiovisual media and

the accessing of information always happen in a cer-

tain context (Ricci, 2012), i.e. conditions or circum-

stances that signiﬁcantly affect the decision behavior.

This gave rise to the development of context-aware

recommender systems (CARS), which take this con-

textual information into account when providing rec-

ommendations.

For various application domains, the user con-

text has gained an increased interest from re-

searchers (Adomavicius and Tuzhilin, 2011). For

context-aware music recommendations for example,

the user’s emotions can be used as input by using

support vector machines as emotional state transi-

tion classiﬁer (Han et al., 2010). In the applica-

tion domain of tourism for example, various applica-

tions use the current location or activity of the user to

personalize and adapt their content offer to the cur-

rent user needs (Ricci, 2010; De Pessemier et al.,

2014). Personal recommendations for points of in-

terest can be provided based on the user’s proximity

of the venue (Kenteris et al., 2010).

In the domain of audiovisual media, more specif-

ically news content, the inﬂuence of context on the

consumption behavior and personal preferences is

less obvious. However, research (Yu et al., 2006) has

shown that the situation of the user (location, activ-

ity, time), as well as the device and network capabil-

ities are important contextual parameters for context-

aware media recommendations on smartphones.

The growth of the digital news industry and espe-

cially the development of mobile products is boom-

ing. Mobile has become, especially amongst younger

media consumers, the ﬁrst gateway to most online

news brands. In a recent survey (Reuters Institute

for the Study of Journalism, 2014), conducted in 10

countries with high Internet penetration, one-ﬁfth of

the users now claim that their mobile phone is the pri-

mary access point for news.

Despite this shift of news consumption to the mo-

bile platform, the study of Weiss (Weiss, 2013) high-

lights that a gap exists between what news consumers,

particularly young adults, are doing and using on their

smartphones and what news organizations are able to

provide. In most cases, news organizations disregard

contextual data or they are only using geo-location

features in their mobile apps for trafﬁc and weather;

363

De Pessemier T., Vanhecke K. and Martens L..

The Information Value of Context for a Mobile News Service.

DOI: 10.5220/0005408603630369

In Proceedings of the 11th International Conference on Web Information Systems and Technologies (WEBIST-2015), pages 363-369

ISBN: 978-989-758-106-9

 2015 SCITEPRESS (Science and Technology Publications, Lda.)

they do not anticipate the high use of location-based

services by smartphone consumers.

2 RELATED WORK

In the domain of digital news services, various ini-

tiatives to personalize the offered news content have

been proposed.

SCENE (Li et al., 2011) is such a news ser-

vice. It stands for a SCalable two-stage pErsonalized

News rEcommendation system. The system consid-

ers characteristics such as news content, access pat-

terns, named entities, popularity, and recency of news

items when performing recommendation. The pro-

posed news selection mechanism demonstrates the

importance of a good balance between user interests,

the novelty, and diversity of the recommendations.

The News@hand system (Cantador et al., 2008)

is a news recommender which applies semantic-based

technologies to describe and relate news contents and

user preferences in order to produce enhanced rec-

ommendations. This news system ensures multi-

media source applicability and multi-domain porta-

bility. The resultant recommendations can be adapted

to the current context of interest, thereby emphasizing

the importance of contextualization in the domain of

news. However, context is not the main focus of this

study and the inﬂuence of context on the consumption

behavior is not investigated.

The News Recommender Systems Chal-

lenge (Said et al., 2013) focused on providing

live recommendations for readers of German news

media articles. This challenge highlighted why news

recommendations have not been as analyzed as some

of the other domains such as movies, books, or music.

Reasons for this include the lack of data sets as well

as the lack of open systems to deploy algorithms

in. In the challenge, the deployed recommenders

for generating news recommendations are: Recent

Recommender (based only on the recency of the

articles), Lucene Recommender (a text retrieval sys-

tem built on top of Apache Lucene), Category-based

Recommender (using the article’s category), User

Filter (ﬁlters out the articles previously observed

by the current user), and Combined Recommender

(a stack or cascade of two or more of the above

recommenders).

Although the various initiatives emphasize the im-

portance of a personalized news offer, most of them

focus on the recommendation algorithms and ignore

the contextual information that is coupled with the in-

formation request, the user, and the device. In this

study, the focus is not on improving state of the art

recommendation algorithms, but rather on investigat-

ing the inﬂuence of context on the consumption of

news content by means of a large-scale user study.

In many cases, the research on CARS remains

conceptual (Adomavicius and Tuzhilin, 2011), where

a certain method has been developed, but testing is

limited to an ofﬂine evaluation or a short-term user

test with only a handful of people, often students or

colleagues who are not representative for the popula-

tion. In contrast, this research investigates the role of

context for news recommendations, based on a large-

scale empirical study. Users could utilize a real news

service

that offers content of four major Flemish

news companies on their own mobile devices, in their

everyday environment, where and when they wanted,

i.e., in a living lab environment.

Living lab experiments are an extension towards

more natural and realistic research test environ-

ments (Følstad, 2008). Living labs allow to evalu-

ate research hypotheses by users representative for

the target population who satisfy their information

need in a real context. Since users are following

their own agenda, laboratory biases on their behav-

ior can be neglected (Hopfgartner et al., 2014). Al-

though less transparent and predeﬁned, living lab ex-

periments aim to provide more natural settings for

studying users’ behavior and their experience.

Especially for context-aware applications, in

which the user’s environment has an inﬂuence on the

way the application works and/or on the offered con-

tent, a realistic setting is essential for a reliable eval-

uation. Therefore, this paper investigates the inﬂu-

ence of context and the beneﬁt of context-aware rec-

ommendations for a real news service by means of a

large-scale user panel, in a realistic environment, over

a longer period of time. Since a user study can pro-

vide reliable explanations as to which recommenda-

tion method is the best, and why one method is better

than the other (Shani and Gunawardana, 2013), three

alternative recommendation methods for the news ser-

vice are compared through such a user study.

3 EXPERIMENTAL SETUP

The news service that was used in this experiment

aggregates content of different premium content

providers: newspapers, magazines, but also content

of television as short video clips. Figure 1 shows a

screenshot of the user interface offering the content

with a reference of the provider on top of each con-

tent item. The aggregated content provides users a

http://www.iminds.be/en/projects/2014/04/17/

stream-store

WEBIST2015-11thInternationalConferenceonWebInformationSystemsandTechnologies

364

more complete and varied overview of the news than

traditional services do. To anticipate the abundance

of news content and the associated choices that peo-

ple have to make, the news service offers personalized

recommendations.

During the experiment, the device type (smart-

phone or tablet) and the time of the day (morning,

noon, daytime, or evening) are studied as contextual

inﬂuences of the news consumption. The news ser-

vice is accessible through a mobile application, which

is available on Android and iOS for tablets as well as

smartphones. As a result, the type of device that is

used for consuming the news content is an interesting

contextual factor.

Compared to the well-established application do-

mains of recommender systems, such as movies or

books, news items have a shorter lifespan and are fre-

quently updated. Consequently, consulting the news

on a daily basis, or even multiple times a day, can be

interesting, which makes the time aspect another im-

portant contextual factor. The time is closely related

to the location of the user, as was also witnessed dur-

ing the analysis of users’ interactions and consump-

tion behavior. A frequently recurring pattern was as

follows: in the morning, users are at home; during

daytime, they are at work; and during the evening,

they are again at home. Therefore, and to prevent over

speciﬁcation (Section 4.3), the location is not adopted

as a separate contextual factor.

For the evaluation of this service and its recom-

mendations, 120 test users were recruited by an ex-

perienced panel manager from iMinds-iLab.o

(i.e. a

research division with a strong expertise in living lab

research and panel management). These test users, all

interested in news and owning a smartphone and/or

tablet, belong to the target group of an online news

service. The test users could install the mobile appli-

cation of the news service on their smartphone and/or

tablet and freely use the service during the evalua-

tion period of around 5 weeks. These test users were

divided into three groups, each receiving a different

type of recommendations, as explained in Section 4.

The test users’ interactions with the service were

logged to analyze their consumption behavior and to

get insight in the actual use of the news service and

their overall experience: 10 test users did not install

the app, or did not use the news service during the

evaluation period. They are excluded from the analy-

sis, so that the number of actual participants was re-

duced to 110.

http://www.openlivinglabs.eu/livinglab/iminds-ilabo

4 NEWS RECOMMENDATIONS

The experiment takes three different approaches to

recommend interesting news. Each user received only

one recommendation type during the whole evalua-

tion period. To avoid any bias, test users were not

informed about the existence of multiple types of rec-

ommendations.

4.1 Recommendations based on Explicit

Static Preferences

Before the actual experiment, test users were asked

about their preferences for different categories of

news content (National, International, Culture, Econ-

omy, Lifestyle, Politics, Sports, and Interesting facts)

through an online questionnaire. Users could specify

their interests on a 5-point rating scale for each cate-

gory and reﬁne this score for different times of the day

(morning, noon, daytime, and evening). The answers

on this questionnaire constitute the user proﬁle that

is used for generating news recommendations. Dur-

ing the experiment, these preferences are considered

static; user proﬁles are not updated based on explicit

or implicit feedback on the content, and the recom-

mender is not learning from the user’s behavior.

4.2 Content-based Recommendations

The content-based recommendations are not based on

a prior questionnaire but use the implicit and explicit

feedback users provide during the evaluation period.

A request to read one of the recommended news items

is considered as positive implicit feedback. Eval-

uating the news recommendation by means of the

‘Thumbs Up’ and ‘Thumbs Down’ icons in the user

interface provides explicit feedback.

This feedback is gradually collected during the

usage of the service. As a result, the user proﬁles

of the content-based recommender are dynamic and

constantly change as users interact with the news ser-

vice. As the user is utilizing the news service and

provides feedback, the recommender is learning the

user’s preferences.

A content-based recommendation algorithm was

chosen because of the availability of informative

metadata about the content items, the sparsity of the

data set, and the cold start problem associated with the

start-up phase. As content-based solution, the ‘Inter-

estLMS algorithm’ of the Duine framework (Telem-

atica Instituut / Novay, 2009) was adopted. The In-

terestLMS algorithm builds a user proﬁle by inferring

personal preferences from the metadata describing the

news items that are requested (implicit feedback) or

TheInformationValueofContextforaMobileNewsService

365

Figure 1: Screenshot of the user interface of the news service.

evaluated (explicit feedback). Subsequently, the algo-

rithm determines the news items that best match the

user’s proﬁle.

News items are characterized by eight different

categories, the same as used in the questionnaire to

elicit the explicit static preferences of the users. In

addition, keywords provide more detailed info about

the news items. These keywords are not predeﬁned

but are extracted from the text of the article using

OpenCalais (Thomson Reuters, 2013). OpenCalais is

a Web service that automatically creates rich semantic

metadata for the content. It analyzes the news article

and ﬁnds the entities within it, but it returns the facts

and events hidden within the text as well. This way,

the news article is tagged and analyzed with the aim

of checking whether it contains information what the

user cares about.

For the content-based recommendations, contex-

tual aspects are not taken into account. So, contextual

data, such as the device type and the time of the day,

are ignored during the creation of the proﬁle and the

calculation of the recommendations.

4.3 Context-aware Content-based

Recommendations

Just like the content-based recommendations, the

context-aware content-based recommendations are

not using a prior questionnaire but are self-learning

based on the explicit and implicit feedback users pro-

vide during the experiment. For this type of recom-

mendations, the InterestLMS algorithm of the Duine

framework is extended to take into account the con-

text of the user. Before generating the recommenda-

tions, the user feedback is processed by a contextual

pre-ﬁlter (Adomavicius and Tuzhilin, 2008). Contex-

tual information is used to determine the relevance of

the feedback and ﬁlter these data based on the cur-

rent situation. For instance, if a user wants to read

news during the evening, an exact pre-ﬁlter (Ado-

mavicius and Tuzhilin, 2011) selects only feedback

gathered during the evening to calculate the recom-

mendations. Therefore, the day is partitioned into

four non-overlapping intervals: morning from 6:00 to

11:00, daytime from 11:00 to 12:00 and from 13:00

to 18:00, noon from 12:00 to 13:00 and evening/night

from 18:00 to 6:00.

One major advantage of the contextual pre-

ﬁltering approach is that it allows deployment of any

of the traditional recommendation techniques (Ado-

mavicius and Tuzhilin, 2005). This makes it pos-

sible to use the same underlying algorithm for the

context-aware content-based recommendations as for

the content-based recommendations, which enables

the comparison of both types of recommendations and

to investigate the inﬂuence of contextual information.

Different pre-ﬁltering techniques have proven

their efﬁcacy in literature (Baltrunas and Ricci, 2009).

They all have to cope with the problem of context

over-speciﬁcation: focusing on the exact context is of-

ten a too narrow limitation. An overly speciﬁed con-

text may not have enough training examples for accu-

rately estimating the user’s interests. For example, if

a user rarely utilizes a tablet during noon to read news

articles, the exact context (noon + tablet) may not pro-

vide enough data (feedback from the user) for accu-

rately calculating the recommendations, which gives

rise to the ‘sparsity’ problem. As a result, insufﬁcient

WEBIST2015-11thInternationalConferenceonWebInformationSystemsandTechnologies

366

feedback is available for generating reliable recom-

mendations (Papagelis et al., 2005).

An appropriate solution for context over-

speciﬁcation is to use a more general context

speciﬁcation by applying context generaliza-

tion (Adomavicius and Tuzhilin, 2011). Since certain

aspects of the overly speciﬁc context may be less

signiﬁcant, the data ﬁltering can be made more

general in order to retain more data after the ﬁltering

for calculating recommendations.

In this experiment, context generalization is ap-

plied in two phases in case of insufﬁcient feedback

data. In a ﬁrst phase, the time frame is broadened. For

instance, if recommendations are needed for a user

who is reading news on a tablet during noon, the time

restriction “noon” is dropped ﬁrst. The data gathered

in that speciﬁc context is supplemented with the user’s

feedback gathered on a tablet during other time peri-

ods. If the amount of feedback is still insufﬁcient after

this ﬁrst generalization, the context is further gener-

alized. In a second phase, the device type is broad-

ened. More speciﬁcally, the user’s feedback provided

on a speciﬁc type of device (e.g., a tablet) is supple-

mented with the user’s feedback provided on other de-

vice types (e.g., a smartphone). We opted to apply the

generalization ﬁrst on the time aspect of the context,

and in a second phase on the device type, since many

users are utilizing the service during different time pe-

riods but on only one type of device.

5 FEEDBACK ANALYSIS

To quantify the added value of a dynamic proﬁle

and contextual information, the users’ interactions

in terms of feedback with each of the three recom-

menders are analyzed. Figure 2 gives an overview of

the user feedback on the news service during the eval-

uation period. The chart distinguishes implicit feed-

back, i.e. requesting to ‘view’ a news item, and ex-

plicit feedback, i.e., evaluating a news item by pro-

viding a ‘Thumbs Up’ or ‘Thumbs Down’ rating.

In Figure 2, this user feedback is aggregated over

all users and partitioned by the type of recommenda-

tions that the users received. Since some test users

dropped out just before the evaluation period, the dif-

ferent recommender types are not evaluated by the

same number of test users. Table 1 shows the number

of test users assigned to each type of recommenda-

tions, which has a direct inﬂuence on the total amount

of feedback gathered for that type.

During the evaluation period, 2728 positive eval-

uations (‘Thumbs Up’) of a news recommendation

were registered for all types of recommendations

Figure 2: The amount of user interaction with the news ser-

vice for each type of recommendations, partition according

to the type of interaction.

together. In contrast, only 949 times a negative

evaluation (‘Thumbs Down’) was provided by the

users. These aggregated values (‘Thumbs Up’ 74.2%

- ‘Thumbs Down’ 25.8%) are an indication for the

general satisfaction of the users with the news that

they get recommended.

However, signiﬁcant differences for the different

types of recommendations can be witnessed. Com-

pared to the recommendations that are based on the

explicit static preferences of the users, less views (to-

tal number, but also number per user) are obtained for

the content-based and context-aware content-based

recommendations. This might indicate that users get

the interesting news more quickly using the more ad-

vanced algorithms, since they also spent more time

per news item.

Comparing the different types of recommenda-

tions in terms of negative feedback (‘Thumbs Down’)

demonstrates the added value of personal feedback

and the context of the user for the recommender sys-

tem. Recommendations based on explicit static pref-

erences received 411 times ‘Thumbs Down’ from

users who are not satisﬁed with the news content. The

content-based recommender uses the implicit feed-

back (requests to view a news item) and explicit eval-

uations (‘Thumbs Up & Down’) as personal feedback

during the evaluation period. Compared to the recom-

mendations based on explicit static preferences, less

negative (309 times ‘Thumbs Down’) and more pos-

itive evaluations are provided for the content-based

recommendations. The lowest number of negative

evaluations (229 times ‘Thumbs Down’) was received

for the context-aware content-based recommender,

which suggest news items based on the personal feed-

back of the user and by taking into account the user’s

current context. A Wilcoxon rank-sum test showed

these differences are signiﬁcant (p = 0.043 < 0.05).

Table 1 shows the ratio of the number of positive

TheInformationValueofContextforaMobileNewsService

367

Table 1: Comparison of the recommendation types.

Recommender Type Explicit Static Preferences Content-based Context-aware Content-based

Input Data Preliminary Questionnaire Personal Feedback Personal Feedback + Context

Number of Test Users 38 37 35

#Thumbs up / #(Thumbs up + down) 0.67 0.75 0.79

evaluations (# Thumbs Up) and the number of explicit

evaluations (# Thumbs Up + Down) for the different

types of recommendations. The results conﬁrm the

increase in accuracy by making the system dynamic

(content-based recommendations) and taking into ac-

count the context (context-aware content-based rec-

ommendations).

6 DISCUSSION

Although the context-aware content-based recom-

mendations received the most positive evaluation

(highest ratio in Table 1), the difference with the tradi-

tional content-based recommendations is limited be-

cause of two reasons.

Firstly, many test users are not active in differ-

ent contextual situations. They tend to read the news

each day at the same time, using the same device. In

this study, 10 users (9.1%) utilized the service dur-

ing only one time period (morning, noon, daytime or

evening), 17 users (15.5%) have viewed news items

during two different time periods, 36 users (32.7%)

were active during three different time periods, and

47 users (42.7%) have requested news content during

morning, noon, daytime, and evening (all four time

periods). In terms of time period, users were far most

active during the evening (45.0% of the news request),

followed by the morning (26.7%). Over the evalua-

tion period of 5 weeks, only 47 users (42.7%) utilized

the service at least once during all four time periods

(morning, noon, daytime, and evening).

In terms of device types, 91 users (82.7%) uti-

lized only one device to access the news content (ei-

ther smartphone or tablet) and only 19 users (17.3%)

used both devices at least once for reading news. In

addition, users that used both device types may have

the habit or preference to read news on only one of

them. On the smartphone, news is more often con-

sumed during different times of the day (76.8% used a

smartphone during 3 or 4 time periods) than on tablets

(56.7% used a tablet during 3 or 4 time periods). Peo-

ple generally keep their phone more closely through-

out the whole day than their tablet, which is often used

only once during the day, typically at home. Because

some users consume news in ﬁxed usage patterns,

with limited variations in their context, the context-

aware recommender cannot fully exploit the contex-

tual information for this news service.

Secondly, the results are based on the evaluation

period of approximately 5 weeks, and CARS require

sufﬁcient time to learn user preferences in different

contextual situations. Because of the cold start prob-

lem (i.e. the issue that the system cannot draw any

inferences for users or items about which it has not

yet gathered sufﬁcient information) and data fragmen-

tation over the different contextual situations, we be-

lieve that there is still room for accuracy improvement

of the context-aware content-based recommendations

by gathering additional user feedback over a longer

time period. The required number of ratings to over-

come the cold start problem depends on various fac-

tors such as the algorithm parameters, the content do-

main, and the speciﬁc items that are rated. Studies

have shown that, in general, more than 20 - 30 ratings

are necessary for the system to recommend relevant

items to the user (Lee et al., 2007). In our user study,

some of the users did not achieve enough ratings for

each contextual situations.

Additional data can help to learn patterns in the

users’ behavior and preference differences for vari-

ous contextual situations, thereby further improving

the accuracy of the context-aware content-based rec-

ommendations.

7 CONCLUSIONS

In this paper, a start-up news service offering personal

recommendations is evaluated by an empirical user

study. Three types of recommendations are tested:

recommendations based on an explicit static pro-

ﬁle, content-based recommendations using the actual

user behavior but ignoring the context, and context-

aware content-based recommendations incorporating

user behavior as well as context. The study aimed to

assess the importance of context in the recommender

of a real-life mobile news service by focusing on two

contextual aspects: device type and time. The results

conﬁrm the added value of contextual information for

the recommender, but also indicate that the accuracy

improvement might be limited in a real-life situation.

Two conditions must be met in order to improve tra-

ditional recommenders with contextual information.

Firstly, users have to utilize the service in different

contextual situations. Logged user behavior showed

WEBIST2015-11thInternationalConferenceonWebInformationSystemsandTechnologies

368

that users typically access the news service with only

one device (either smartphone or tablet) at different

times of the day, but especially in the evening. Sec-

ondly, sufﬁcient training data have to be available to

learn user preferences and variations in these prefer-

ences over different contextual situations. In context-

aware recommender systems, the cold start problem

is strengthened by the fragmentation of the consump-

tion data over different contextual situations. Context

generalization can be a partial solution, but choosing

the right contextual aspect to optimally broaden the

context is often difﬁcult.

ACKNOWLEDGEMENTS

This research was performed in the context of the

iMinds-MIX Stream Store project, which is cofunded

by iMinds, a research institute founded by the Flem-

ish Government.

REFERENCES

Adomavicius, G. and Tuzhilin, A. (2005). Toward the

next generation of recommender systems: a survey

of the state-of-the-art and possible extensions. IEEE

Transactions on Knowledge and Data Engineering,

17(6):734–749.

Adomavicius, G. and Tuzhilin, A. (2008). Tutorial on

context-aware recommender systems. In Proceedings

of the second ACM conference on Recommender Sys-

tems (RecSys ’08).

Adomavicius, G. and Tuzhilin, A. (2011). Context-aware

recommender systems. In Ricci, F., Rokach, L.,

Shapira, B., and Kantor, P. B., editors, Recommender

Systems Handbook, pages 217–253. Springer US.

Baltrunas, L. and Ricci, F. (2009). Context-based splitting

of item ratings in collaborative ﬁltering. In Proceed-

ings of the Third ACM Conference on Recommender

Systems, RecSys ’09, pages 245–248, New York, NY,

USA. ACM.

Cantador, I., Bellog

ın, A., and Castells, P. (2008).

News@hand: A semantic web approach to recom-

mending news. In Nejdl, W., Kay, J., Pu, P., and

Herder, E., editors, Adaptive Hypermedia and Adap-

tive Web-Based Systems, volume 5149 of Lecture

Notes in Computer Science, pages 279–283. Springer

Berlin Heidelberg.

De Pessemier, T., Dooms, S., and Martens, L. (2014).

Context-aware recommendations through context and

activity recognition in a mobile environment. Multi-

media Tools and Applications, 72(3):2925–2948.

Følstad, A. (2008). Living labs for innovation and develop-

ment of information and communication technology:

A literature review. Electronic Journal of Organiza-

tional Virtualness, 10:99–131.

Han, B.-J., Rho, S., Jun, S., and Hwang, E. (2010). Music

emotion classiﬁcation and context-based music rec-

ommendation. Multimedia Tools Appl., 47(3):433–

460.

Hopfgartner, F., Kille, B., Lommatzsch, A., Plumbaum, T.,

Brodt, T., and Heintz, T. (2014). Benchmarking news

recommendations in a living lab. In Kanoulas, E.,

Lupu, M., Clough, P., Sanderson, M., Hall, M., Han-

bury, A., and Toms, E., editors, Information Access

Evaluation. Multilinguality, Multimodality, and Inter-

action, volume 8685 of Lecture Notes in Computer

Science, pages 250–267. Springer International Pub-

lishing.

Kenteris, M., Gavalas, D., and Mpitziopoulos, A. (2010). A

mobile tourism recommender system. In Proceedings

of the The IEEE symposium on Computers and Com-

munications, ISCC ’10, pages 840–845, Washington,

DC, USA. IEEE Computer Society.

Lee, H., Kim, J., and Park, S. (2007). Understanding col-

laborative ﬁltering parameters for personalized rec-

ommendations in e-commerce. Electronic Commerce

Research, 7(3-4):293–314.

Li, L., Wang, D., Li, T., Knox, D., and Padmanabhan,

B. (2011). Scene: A scalable two-stage personal-

ized news recommendation system. In Proceedings of

the 34th International ACM SIGIR Conference on Re-

search and Development in Information Retrieval, SI-

GIR ’11, pages 125–134, New York, NY, USA. ACM.

Papagelis, M., Plexousakis, D., and Kutsuras, T. (2005). Al-

leviating the sparsity problem of collaborative ﬁlter-

ing using trust inferences. In Herrmann, P., Issarny,

V., and Shiu, S., editors, Trust Management, volume

3477 of Lecture Notes in Computer Science, pages

224–239. Springer Berlin Heidelberg.

Reuters Institute for the Study of Journalism

(2014). Digital News Report. Available at

http://www.digitalnewsreport.org/.

Ricci, F. (2010). Mobile recommender systems. Informa-

tion Technology & Tourism, 12(3):205–231.

Ricci, F. (2012). Contextualizing recommendations. In

ACM RecSys Workshop on Context-Aware Recom-

mender Systems (CARS ’12), In conjunction with the

6th ACM Conference on Recommender Systems (Rec-

Sys ’12). ACM.

Said, A., Bellog

ın, A., and de Vries, A. (2013). News

recommendation in the wild: Cwis recommendation

algorithms in the nrs challenge. In Proceedings of

the 2013 International News Recommender Systems

Workshop and Challenge. NRS, volume 13.

Shani, G. and Gunawardana, A. (2013). Tutorial on

application-oriented evaluation of recommendation

systems. AI Communications, 26(2):225–236.

Telematica Instituut / Novay (2009). Duine Framework.

Available at http://duineframework.org/.

Thomson Reuters (2008-2013). Open Calais. Available at

http://www.opencalais.com/.

Weiss, A. S. (2013). Exploring news apps and location-

based services on the smartphone. Journalism & Mass

Communication Quarterly, 90(3):435–456.

Yu, Z., Zhou, X., Zhang, D., Chin, C.-Y., Wang, X., and

men, J. (2006). Supporting context-aware media rec-

ommendations for smart phones. Pervasive Comput-

ing, IEEE, 5(3):68–75.

TheInformationValueofContextforaMobileNewsService

369