Rating Prediction with Contextual Conditional Preferences

Aleksandra Karpus

, Tommaso di Noia

, Paolo Tomeo

and Krzysztof Goczyła

Faculty of Electronics, Telecommunications and Informatics, Gda

nsk University of Technology, Gda

nsk, Poland

Department of Electrical & Electronics Engineering, Polytechnic University of Bari, Bari, Italy

Keywords:

Recommender Systems, Context Awareness, Conditional Preferences, Rating Prediction, Cold-start Problem.

Abstract:

Exploiting contextual information is considered a good solution to improve the quality of recommendations,

aiming at suggesting more relevant items for a speciﬁc context. On the other hand, recommender systems

research still strive for solving the cold-start problem, namely where not enough information about users and

their ratings is available. In this paper we propose a new rating prediction algorithm to face the cold-start

system scenario, based on user interests model called contextual conditional preferences. We present results

obtained with three publicly available data sets in comparison with several state-of-the-art baselines. We show

that usage of contextual conditional preferences improves the prediction accuracy, even when all users have

provided a few feedbacks, and hence small amount of data is available.

1 INTRODUCTION

Exploiting contextual information is considered a

good solution to improve the quality of recommen-

dations, aiming at suggesting more relevant items for

a speciﬁc context (Lombardi et al., 2009). During

last decade many context-aware approaches were pro-

posed. However, they usually consider the situation

where a lot of data is available. On the other hand,

recommender systems research still strive for solving

the cold-start problem, namely where we do not have

enough information about users and their ratings. For

example, matrix factorization methods do not work

well in cold start scenarios (Kula, 2015).

Different situations described in the literature are

called cold-start problem. Two of them are well-

known and have also another name, respectively: new

item and new user cold-start problem. Both occur

when recommender system is well-established and a

lot of ratings are available. When we introduce a new

item into such system, in many recommendation algo-

rithms it will not be recommended to users, because

of the lack of its history, i.e. user ratings. The same

happens when a new user registers into the recom-

mender system. He will not receive interesting rec-

ommendations just because the system does not know

his preferences yet (Jannach et al., 2010).

However, to the best of our knowledge, a little

work was done on the third kind of the cold-start prob-

lem, i.e. a new system cold-start problem. The lack of

interest in this particular problem could be justiﬁed by

the facts that it is rather rare and that a company, when

runs a new system, does not have enough resources

to support research. Nevertheless, this scenario when

we do not have may users, items and ratings is very

interesting and deserves further consideration.

Besides the well-know cold start problem, we

could distinguish the continuous cold-start problem

which is characteristic for some speciﬁc domains such

as tourism or job recommendations (Bernardi et al.,

2015). It was noticed that in some domains users

never become warm, i.e. we never have many ade-

quate ratings, because a user searches for items very

rarely and changes his preferences over time. For ex-

ample, for young people it is better to sleep in a cheap

hostel than in an expensive hotel during a trip, while

older people could think the opposite.

In this paper we introduce an algorithm for the

rating prediction task in a new system cold-start

situations. It is based on a user model called

contextual conditional preferences (Karpus et al.,

2016) which represents user interests in items in a

compact way. We run our experiments on three

context-aware data sets publicly available in the Web,

i.e. LDOS-CoMoDa, Unibz-STS and Restaurant &

consumer data, which are quite small and ﬁt well

for our purposes. We conﬁrmed that our algorithm

works well in the cold-start scenarios. Because the

similarity in the continuous cold-start problem and a

new system cold-start problem characteristics, we be-

Karpus, A., Noia, T., Tomeo, P. and Goczyla, K.

Rating Prediction with Contextual Conditional Preferences.

DOI: 10.5220/0006083904190424

In Proceedings of the 8th International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management (IC3K 2016) - Volume 1: KDIR, pages 419-424

ISBN: 978-989-758-203-5

419

lieve that this method could help to resolve the con-

tinuous cold-start problem. However, this is the next

step in our research.

The remainder of the paper is constructed as fol-

lows. Section 2 brieﬂy introduces contextual condi-

tional preferences. In Section 3 a rating prediction al-

gorithm is presented. The data sets used are described

in Section 4. Section 5 provides our evaluation ap-

proach and obtained results. Related works are pre-

sented in Section 6. Conclusions close the paper.

2 CONTEXTUAL CONDITIONAL

PREFERENCES

Contextual conditional preferences (CCPs) intro-

duced in (Karpus et al., 2016) are a compact represen-

tation of user interests in items in different situations.

This model describes the relations between the con-

text related to the user’s ratings and the item content,

and consists of a set of conditional preferences.

We deﬁne the Contextual Conditional Preference

(CCP) as an expression of the form:

(γ

= c

) ∧ . . . ∧ (γ

= c

) | (α

= a

)  (α

= a

) ∧

. . . ∧ (α

= a

)  (α

= a

)

with γ

being contextual variables and α

item at-

tributes, and c

, ..., c

, a

, ..., a

, a

being concrete

values of these parameters.

The above preference is read as given the context

(γ

= c

) ∧ . . . ∧ (γ

= c

) I prefer a

over a

for α

and a

over a

for α

. An example of the CCP is

shown below.

dominant emo = 7 ∧ decision = 1

∧ end emo = 7 ∧ physical = 1

| genre ∈

{

1, 3, 7, 8, 10, 19, 21

}

 genre ∈

{

6, 13, 14, 17

}

∧ actor ∈

{

1, 6, 8, 15

}

 actor ∈

{

2, 7, 9, 10, 13

}

It means that for a given context (e.g. decision is 1 - it

was a user’s decision to watch a certain movie) a user

prefers genres with id 1, 3, 7, 8, 10, 19 or 21 to those

with 6, 13, 14 or 17 and actors from clusters 1, 6, 8

and 15 to those from clusters 2, 7, 9, 10 or 13 etc.

We distinguish two types of CCPs: individual and

general. Individual CCP (ICCP) represents prefer-

ences of a single user, while general CCP (GCCP)

catches a general trend of interests for all users in a

certain contextual situation, i.e. we treat ratings from

all users like they were made by one person.

During our experiments we automatically gener-

ated CCPs. For more details about the algorithm

of the preferences extraction please refer to (Karpus

et al., 2016).

3 RATING PREDICTION WITH

CONTEXTUAL CONDITIONAL

PREFERENCES

Having a concrete user and his context, and wanting

to predict his rating for some item, ﬁrst we need to

ﬁnd the best CCPs (his or general ones) that will be

used during a prediction process. In this case, the best

preferences are those which are the most similar to

the considered context. In order to count a contextual

similarity between a CCP p and a current user context

ctx(u) we used the following metric:

sim(p, ctx(u)) =

∑

(γ

)∈p

overlap(ctx(u), (γ

, c

)).

We also used the overlap function deﬁned as:

overlap(ctx(u), (γ

, c

)) =







1 (γ

, c

) ∈ ctx(u);

0.5 c

= −1;

0 otherwise.

The overlap function returns 1 when we are sure that

the pair (γ

, c

) is contained both in the contextual part

of p and in the current user context ctx(u). When it is

uncertain, i.e. when the value c

for the dimension γ

is equal −1 (the unknown value), it returns 0.5. Oth-

erwise 0 is returned. Please notice, that the current

user context ctx(u) is also a set of pairs (γ

, c

), i.e.

the name of the contextual variable and its value.

For an item that we want to predict a rating, we

construct a list containing this item and items seen by

the user in the context similar to current context in at

least some percentage (this value is conﬁgurable and

could depend on the data set - the conﬁguration for

each data set that we used is presented in Table 1).

Identiﬁed in the previous step best preferences are

used to order constructed list. For each pair of items,

we choose the one that has the most similar values for

the attributes attr(i) (a set of attribute name and value

pairs (α

, a

)) with the CCP p. For this purpose we

used another similarity measure and overlap function

deﬁned as:

sim

cont

(p, attr(i)) =

∑

(α

)∈p

overlap(attr(i), (α

, a

))

overlap(attr(i), (α

, a

)) =







1 (α

, a

) ∈ attr(i);

0 a

= −1;

−1 otherwise.

The overlap function used here is quite different from

the one used above. In the case of item features it

is more crucial to have strict matching. This is the

reason why we do not reward the unknown value and

why we give penalty for unmatched parameter values.

It should be noticed that we need to compare the

similarity of the item attributes with both sides of

KDIR 2016 - 8th International Conference on Knowledge Discovery and Information Retrieval

420

Table 1: The conﬁguration for experiments with three data

sets.

LDOS STS RC

% of sim. 75 95 75

Other algo- Glob. Item Avg, User Avg,

rithms used Avg Glob. Avg Glob. Avg

Table 2: Basic statistics of three data sets.

LDOS STS RC

# of users 121 325 138

# of items 1232 249 130

# of ratings 2296 2534 1161

Max # of ratings / user 275 175 18

Min # of ratings / user 1 1 3

Avg # of ratings / user 18.98 7.80 8.41

Max # of ratings / item 26 282 36

Min # of ratings / item 1 1 3

Avg # of ratings / item 1.86 10.18 8.93

the preference relation in the current preference state-

ment.

The process of reordering is repeated as long as

nothing could be changed. Depending on the ﬁnal

place of the considered item in the list, we compute

its rating. If the context is new, i.e. if there is no

other item in the list, we rate the current item with

some baseline algorithm (it is a conﬁgurable option,

see other algorithms used in Table 1). If the item is

ﬁrst or last on the list, we assign to it a rating of the

nearest neighbor. Otherwise, we compute the rating

as an average of two or four, depending on the size

of a list, nearest neighbors’ ratings, i.e. the one/two

above considered item and one/two below it. We as-

sume that we do not have much data, so we cannot

take more than four neighbors.

4 DATA SETS

We performed our experiments with three data sets,

i.e. the LDOS-CoMoDa

data set (LDOS), the Unibz-

STS data set (STS) and the Restaurant & consumer

data set

(RC). Basic statistics of data sets are pre-

sented in Table 2.

The LDOS-CoMoDa (Kosir et al., 2011) contains

user interaction with the system, i.e. the rating on

a 5-star scale, basic users’ information, content in-

formation about multiple item dimensions and twelve

additional contextual information about the situation

The data is available at http://212.235.187.145/

spletnastran/raziskave/um/comoda/comoda.php.

The data sets are available at https://github.com/

irecsys/CARSKit/tree/master/context-aware data sets.

when the user consumed the item. According to (Odic

et al., 2013) the choice of contextual variables to be

used is crucial because of a different amount of in-

formation they gain. To eliminate irrelevant variables

we computed correlation coefﬁcients between context

related attributes. We found only two of them to be

strongly correlated, i.e. city and country. However,

values of these attributes remain the same for a sin-

gle user, so they were not taken into account for fur-

ther consideration. In (Odic et al., 2013) six variables

in the LDOS-CoMoDa were identify as informative.

Since we focus on the cold start situation problem in

this paper, we want to limit sparsity of the data as

much as possible. Therefore, we chose four of six

best contextual variables, i.e. dominant emotion, end

emotion, physical and decision, to use in our further

work presented in this paper.

The Unibz-STS (Braunhofer et al., 2013) data set

was collected by a mobile application that recom-

mends places of interests (POIs) in South Tyrol in

Italy. The recommender is called South Tyrol Sug-

gests (STS). The data set contains ratings on a 5-star

scale, information about a users’ personality (e.g. ex-

traversion, emotional stability), a context of visiting

a POI (e.g. weather, season, companion) and a POI’s

category. Like for the LDOS-CoMoDa data set, we

did not consider ﬁxed user information as contextual

variables. For further consideration we chose the two

most informative (according to (Odic et al., 2013))

context parameters, i.e. weather and companion.

The Restaurant & consumer data (Vargas-Govea

et al., 2011) consists of three types of information: a

restaurant data (e.g. cuisine, smoking, dress), a user

information (e.g. smoker, dress preference, trans-

port) and a rating that a user gave to a restaurant.

In this data set ratings are expressed on a 0-2 scale.

Contextual parameters such as information about a

user’s mood or companion are not available. Thus,

for further work we chose most informative fea-

tures: smoker, drink level, dress preference, ambi-

ence, transport, personality and color.

5 EVALUATION

To simulate a new recommender system we split the

data set twice. Firstly, we generate 5 subsets by

putting all ratings of a single user into one subset.

Users were picked according to the number of ratings

to achieve sets with comparable size. Secondly, we

split every of 5 subsets into training and test sets. Be-

cause there is no time stamp of the ratings, the assign-

ment of a single rating was made randomly using just

one condition: if there are already 8 ratings of con-

Rating Prediction with Contextual Conditional Preferences

421

sidered user in the training set, put all other ratings

of that user into test set. Consequently, every test set

consists of ratings of users that have rated more than 8

items (number 8 was chosen to have reasonable num-

ber of test cases). Every test instance contains a user

context part, an item content part and a rating that the

user gave to the item.

ti = (r, (γ

, c

), ...(γ

, c

), (α

, a

), ..., (α

, a

), i),

where r is a rating that user gave to an item i, (γ

, c

)

is part of contextual information (context(ti) which

means that contextual parameter γ

has value c

, and

(α

, a

) is part of item content information (con-

tent(ti)) which means that content feature α

has value

We evaluated our approach by doing hold-out val-

idations on ﬁve different training and test subsets

described above. We tested our algorithm in three

conﬁguration: using both ICCPs and GCCPs, us-

ing only ICCPs and using only GCCPs. As mea-

sures we used the mean average error (MAE) and

the root mean square error (RMSE) which are com-

monly used for evaluating the accuracy of a rating

in the rating prediction task. We compared our ap-

proach with nine baseline algorithms, i.e. Random

Guess, Item Average, User Average, Item K-Nearest

Neighbors (Item KNN), User K-Nearest Neighbors

(User KNN), SVD++, Biased Matrix Factorization

(Biased MF), Bayesian Probabilistic Matrix Factor-

ization (Bayesian Probabilistic MF) and Probabilis-

tic Matrix Factorization (Probabilistic MF). For this

purpose we used the LibRec library

. The last

three algorithms are extensions of Matrix Factoriza-

tion method, which are available for the rating pre-

diction task in this library. The values of MAE and

RMSE measures could be seen in Figure 1.

As seen in Figure 1, our method outperforms

known baseline algorithms in most of the cases when

considering a median value. The best median re-

sults were obtained when we used only ICCPs for

LDOS-CoMoDa and Unibz-STS data sets. An excep-

tion is Restaurant & consumer data set for which

we did not obtain any result in many cases. The rea-

son of this behavior is that we do not have truly con-

textual data for training the model. Parameters that

were used as context variables are ﬁxed for a cer-

tain user, thus could not be considered as a user con-

text. Of course they could be used as context infor-

mation for GCCP, which is conﬁrmed by obtained

results. It could lead us to the conclusion that the

method is more general and could be used also for

quasi-contextual data.

http://www.librec.net/

We performed the Wilcoxon test to prove a statis-

tical signiﬁcance of presented results. The p-values

vary for different data sets, kinds of preferences and

pairs of algorithms, and conﬁrm observations from

Figure 1 that our method is at least as good as base-

line algorithms. The best results were obtained on

the LDOS-CoMoDa data set. The Wilcoxon test con-

ﬁrmed a statistical signiﬁcance of a prediction ac-

curacy improvement on all algorithms for CCP and

GCCP with the p-value smaller than 0.05 (and even

smaller than 0.01 for some cases). It should be no-

ticed that two of three Matrix Factorization algo-

rithms perform pretty weak in comparison with other

baseline methods, e.g. Probabilistic MF returned

worse results than Random Guess. It conﬁrms the

fact that Matrix Factorization does not work well in

the cold-start scenarios.

6 RELATED WORK

A Context-Awareness in Recommender Systems is

a well-established research area. Many recommen-

dation techniques were already proposed, so they

were classiﬁed in three types of them according to

the phase in the recommendation process in which

the context is incorporated, i.e. pre-ﬁltering tech-

nique, post-ﬁltering technique and contextual model-

ing (Adomavicius and Tuzhilin, 2011).

A multi-agent system for making context and

intention-aware recommendations of Points of Inter-

est (POI) was presented in (Costa et al., 2012). The

tasks of collecting information about POIs and stor-

ing users’ proﬁles data were divided into two kinds

of agents. The user’s Personal Assistant Agent is

responsible for receiving queries, storing user data,

computing recommendations and updating user pref-

erences according to his feedback. Authors incorpo-

rated not only the context but also a user’s goal in visit

the POI.

An interesting approach for a context-awareness

was proposed in (Baltrunas and Amatriain, 2009).

Authors introduced micro proﬁles which split a user

proﬁle into partitions depending on the values of con-

text parameters. They showed that usage of such mi-

cro proﬁles gives a signiﬁcant improvement in the

prediction accuracy in the movie domain while con-

sidering time as a context variable. Contextual Con-

ditional Preferences could be seen as a kind of micro

proﬁling, because each preference statement consists

of user interests and a context in which it is true.

In (Lee and Lee, 2007) a new context-aware music

recommender system was presented. As a main rec-

ommendation technique authors used case-base rea-

KDIR 2016 - 8th International Conference on Knowledge Discovery and Information Retrieval

422