SEREBIF

Searc

h Engine Result Enhancement by Implicit Feedback

Ralph Weires, Christoph Schommer and Sascha Kaufmann

University of Luxembourg, Faculty of Sciences, Technology and Communication

6, rue Richard Coudenhove-Kalergi, L-1359 Luxembourg

Keywords:

Information Retrieval, Relevance Feedback, Implicit Feedback, Collaborative Retrieval.

Abstract:

Web search engines often include results for a query which are not really relevant for a user. SEREBIF is an

approach for incorporating feeback from the users into the search engine results to increase the result quality.

We especially focus on implicit feedback, which does not require the users to put any additional effort than

usual into the search process. The captured information (e.g. entered queries, clicked results) is afterwards

analyzed, and the results are then taken into account for further search sessions. SEREBIF can generally be

used on top of an existing search engine to improve its results. In this paper, we explain the basic idea of

SEREBIF, the current state of the prototype we realized and ﬁrst results.

1 INTRODUCTION

Today web search engines use more and more com-

plex and elaborated ranking functions to deliver the

proper results for a given query. On the other hand,

website owners are trying to exploit the ranking meth-

ods to get their pages as high as possible in the result

lists. In the end, it often happens that items not rele-

vant for a user show up in the result list, too.

SEREBIF is an approach which makes use of im-

plicit feedback for a given search engine, and incorpo-

rates this information into the results to increase their

quality. The overall goal is to analyze the preferences

and in general the behavior of users utilizing a search

engine to get information about the real relevance of

the results. For example, if the users always tend to

click on the second given result for a certain query

and usually leave out the ﬁrst one, the ranking does

not seem to be appropriate and could be changed ac-

cordingly (Joachims, 2002). So it should be possible

to increase the quality of the results over time, just by

taking the implicitely collected information about the

clicked results into account.

Section 2 describes some related work, whereafter

the SEREBIF system is explained in more detail in

section 3. Section 4 shows the current status of our

prototype, followed by the concluding section 5.

2 RELATED WORK

Relevance feedback (Salton and Buckley, 1990) can

be divided into explicit and implicit feedback. The

former type requires the users to actively give feed-

back, e.g. by answering questions. In the context of

search results, this could mean to ask users which of

the given results have actually been relevant to them.

This type of feedback requires user cooperation, so

the users need to be willing to spend additional time

into giving the feedback. However, the information

gained by this way is rather reliable, since it was ex-

plicitely given by the users. For the results of web

search engines, few users would be willing to actually

rate the given results, so explicit feedback would most

likely be of limited practical use. Implicit feedback on

the other hand is automatically collected information

about the user behaviour (Kelly and Teevan, 2003).

By this way, it is possible to get some information

about what was really interesting to the users, with-

out the need of explicitly asking them about it. So

the users do not need to invest any extra time to give

(explicit) feedback; they just do what they need to do

anyway in order to ﬁnd the relevant information that

they are looking for. This makes it rather easy to ob-

tain large amounts of data, for we do not need to moti-

vate the users to actively give feedback to the system.

Important users’ interactions which can be taken

into account in the context of search engines are en-

tered queries, clicked results, and the time that users

263

Weires R., Schommer C. and Kaufmann S. (2008).

SEREBIF - Search Engine Result Enhancement by Implicit Feedback.

In Proceedings of the Four th International Conference on Web Information Systems and Technologies, pages 263-266

DOI: 10.5220/0001513202630266

 SciTePress

spend on the result sites. The mere click on an item

in the result list for a query can already be seen as an

indication of relevance for that item. Moreover, the

actual time which a user spends watching the result

site can give a closer indication of relevance for that

result (Kelly and Belkin, 2004). The gained infor-

mation by this way is not as reliable as with explicit

feedback, though. Users can of course also click on

results which they later see are not important to them.

However, we take the general assumption that clicks

on results and the visiting time are generally a rele-

vance indicator. This means that if many users click

on a certain result for a given query, there is a high

probability for this result being indeed relevant.

Furthermore, we want to keep track of the con-

nections between multiple queries in a search session.

It often happens that users cannot ﬁnd the informa-

tion they are looking for with a certain query and

therefore use another, reformulated query afterwards.

Such so-called query chains (Radlinski and Joachims,

2005) can be used to propose alternative queries to

later users, helping them to ﬁnd the desired informa-

tion quicker.

3 ARCHITECTURE

This section outlines the basic ideas and the architec-

ture used for SEREBIF. We generally require a search

engine to be given whose results we want to improve.

3.1 Feedback Capturing

We decided to build SEREBIF as a proxy-like sys-

tem, looking like a search engine to the users. Fig-

ures 1 and 2 show how the communication works.

The ﬁrst picture shows a search query coming from a

user, which is forwarded to the underlying search en-

gine. The results are returned to the user in a slightly

modiﬁed way. This modiﬁcation mainly affects the

HTML links in the result page so that they point back

to SEREBIF instead of the actual site of the result. By

this way, SEREBIF is able to get to know on which

results the users click afterwards. To the users, this

does not make much of a difference, because a result

request is immediately redirected to the actual result

site as shown in ﬁgure 2.

The information we capture is recorded on a ses-

sion basis only. This means that we do not keep pro-

ﬁles of individual people (or say, computers) over a

longer period of time, since we do not track IP ad-

dresses or anything alike. So if the same user visits

SEREBIF on two different days, the system will treat

Searchengine

SEREBIFUser

query

queryresults

modifiedresults

logfile

Searchengine

SEREBIFUser

query

queryresults

modifiedresults

logfile

Figure 1: Communication for a search query.

Resultsite

SEREBIFUser

requestforresult

redirect

resultresource

logfile

requestforresult

Resultsite

SEREBIFUser

requestforresult

redirect

resultresource

logfile

requestforresult

Figure 2: Communication for a result request.

this as two completely different sessions (of presum-

ably different users). The reason for choosing this be-

haviour lies in our intention to keep the privacy of the

users at the highest possible level while still making

use of the feedback information. Information we keep

track of for each session are:

• Entered queries, including timestamps

• Returned results delivered by the underlying

search engine (limited to the ﬁrst page of results)

• Clicked result links, including timestamps

3.2 Feedback Processing

After capturing, we apply a ﬁrst preprocessing to di-

vide the data into search sessions. These sessions

contain one or more queries (possibly including query

chains), the results from the underlying search engine

for the queries, the clicked results for each query and

an estimation of the duration the user stayed on each

clicked result. We also plan to include external infor-

mation into these sessions, such as synonyms for the

contained query terms. To do this, we can make use

of an external thesaurus like WordNet (Miller, 1995).

Using such additional information, we can later apply

techniques like query expansion to reﬁne the results

(Gong et al., 2005).

In this preprocessing step, we have to deﬁne the

visiting duration of the result sites a user clicked on.

The captured data only contains the time a user re-

quested a result, and maybe timestamps of subsequent

actions in this session. In the simplest case, we can

WEBIST 2008 - International Conference on Web Information Systems and Technologies

264

assume that the visiting time of a result is the dura-

tion from the result request to the next known action

for that session (if any). However, this is certainly

no precise measure, since a user can open multiple

results for a query one after another in different win-

dows (or browser tabs), without having looked at any

of them already. Therefore, we experiment with dif-

ferent approaches to build an estimation model for the

result visit duration.

The gathered information is afterwards merged

in a network storage based on the ANIMA system

(Schommer and Schroeder, 2005), (Schroeder et al.,

2007). The reason for choosing ANIMA is mainly

due to its ability to cope with transactional stream-

ing data. The input data contains information about

search sessions of users, which can easily be broken

down into transactions for ANIMA. In addition, AN-

IMA is able to store only the most important aspects

of the information without the need of storing all feed-

back data. This helps to keep the network at a rea-

sonable size, especially if we think of having large

amounts of captured feedback data. So if a certain re-

sult has only been clicked on by a single user for a

given query, this information will be forgotten after

a certain amount of time. This makes sense because

we want to store the general trend of all users, with-

out giving too much weight to single and isolated user

actions.

The search sessions are considered as transactions

for the ANIMA network. Building up such a network

provides us with information about typical queries,

results, and measures about the relevance of the re-

sults by the underlying search engine. Figure 3 shows

how the storage looks like for SEREBIF. For our stor-

age network, we use three different types of nodes (in

the picture arranged in different layers):

Query terms (T): The single terms contained in user

queries

Queries (Q): The full queries that have been entered

Documents (D): The resulting documents that have

Documents

Queries

Queryterms

DocumentsDocuments

QueriesQueries

QuerytermsQueryterms

Figure 3: Storage architecture.

been provided by the underlying search engine

Different links between these nodes can exist, and

they might be weighted to indicate the strength of the

relationship. We propose the following connections:

TxT: Relationships between query terms. This can

be based on co-occurence (terms tend to occur in

queries together) but can also indicate a semanti-

cal relationship like synonyms.

TxQ: The connection between queries and query

terms is a rather trivial one - it just indicates that a

term was contained in that query.

QxQ: Connections between different queries can be

established if it is found that users tried to ﬁnd

some information with multiple consecutive (re-

formulated) queries.

QxD: These links indicate that documents are of

some relevance for a query. They are initially

given by the information of the underlying search

engine. The connection weights are subject to

changes, so the importance of documents for

queries can shift as time goes by. The weights are

for example increased for documents which users

actually looked (clicked) upon. The visiting time

of documents is also taken into account here.

3.3 Feedback Integration

The information stored in the network can later be

used in different ways to enhance further search en-

gine results. First and most importantly, the clicks and

visit times of the results can inﬂuence the ranking of

the results. This means that if, for a given query and

result list, the users steadily tend to visit the ﬁfth result

more often or longer than the ﬁrst four results, it will

be promoted higher in the list. However, we can also

use the storage for other purposes. The connections

known by query chains can help to guide users to the

right information. If a user enters a certain query for

which there are reformulated queries known from pre-

vious searches, it is possible to suggest these reformu-

lated queries as alternatives. Similar things can also

be done using the connections between query terms.

Either additional terms can be suggested to the user,

or they can automatically be included into the query

which SEREBIF forwards to the underlying search

engine. By that way, we support the users in reﬁn-

ing the search into the desired direction and possibly

eliminating ambiguities that might exist for a query.

SEREBIF - Search Engine Result Enhancement by Implicit Feedback

265

4 CURRENT STATUS

The current prototype can be seen at the URL

http://mine.uni.lu/

weires/serebif/search.php. To the

user, SEREBIF looks just like a usual simple search

engine (see ﬁgure 4), even though it in fact only redi-

rects the queries to the underlying search engine. The

Figure 4: SEREBIF user interface.

implementation of SEREBIF is currently able to cap-

ture the user interaction as described. As underlying

search engine, we are using the Google SOAP Search

API

Since the prototype is running just for a few weeks

now, the recorded data set is not very big yet. We pro-

moted our system mainly to fellow researchers and

captured user information for about 150 sessions, in-

cluding more than 650 queries and a total of over 1900

query terms up to now. We expect this amount of data

to be steadily growing while we continue realizing our

storage architecture.

5 CONCLUSIONS

In this paper, we described the SEREBIF system,

which captures implicit feedback collected from the

users of an existing search engine. This information

is used to enhance the results of later search queries.

To a certain extent, the search system adapts to the

behaviour of the majority of the users, which is sup-

posed to increase the quality of the results.

Our main future work currently lies in ﬁnishing

and evaluating our prototype as described. There are

also some other problems that we want to deal with in

the future. We have to deﬁne how to merge the feed-

back information with the (unknown) ranking func-

tion of the underlying search engine in an appropriate

http://code.google.com/apis/soapsearch/

way. Since we do not know the detailed ranking val-

ues of the results returned by the search engine, it is

difﬁcult to determine how much feedback from the

users is needed to push a result up the list a certain

amount. However, we plan to develop an own search

engine to base SEREBIF upon, to be more indepen-

dent of external systems. In this case, we would be

able to access the exact ranking values for the result

list, which makes this problem much easier to deal

with.

Another problem of our approach is the vulner-

ability against fake feedback, which could possibly

be exploited to artiﬁcially inﬂuence the ranking. If

such a user-based approach would be included into a

large commercial search engine, people could try to

automatically generate false feedback for the system

to push certain pages up the result list. This could e.g.

be done by writing programs which submit a certain

query to SEREBIF and always request the result in the

result list which is desired to be promoted. Solutions

have to be found to cope with this, too.

REFERENCES

Gong, Z., Cheang, C. W., and U, L. H. (2005). Web query

expansion by wordnet. In Andersen, K. V., Deben-

ham, J. K., and Wagner, R., editors, DEXA, volume

3588 of Lecture Notes in Computer Science, pages

166–175. Springer.

Joachims, T. (2002). Optimizing search engines using click-

through data. In KDD, pages 133–142. ACM.

Kelly, D. and Belkin, N. J. (2004). Display time as implicit

feedback: understanding task effects. In Sanderson,

M., J

arvelin, K., Allan, J., and Bruza, P., editors, SI-

GIR, pages 377–384. ACM.

Kelly, D. and Teevan, J. (2003). Implicit feedback for in-

ferring user preference: a bibliography. SIGIR Forum,

37(2):18–28.

Miller, G. A. (1995). Wordnet: A lexical database for en-

glish. Commun. ACM, 38(11):39–41.

Radlinski, F. and Joachims, T. (2005). Query chains: learn-

ing to rank from implicit feedback. In Grossman, R.,

Bayardo, R., and Bennett, K. P., editors, KDD, pages

239–248. ACM.

Salton, G. and Buckley, C. (1990). Improving retrieval per-

formance by relevance feedback. JASIS, 41(4):288–

297.

Schommer, C. and Schroeder, B. (2005). Anima: Associate

memories for categorical data streams. In Proceed-

ings of the 3rd International Conference on Computer

Science and its Applications (ICCSA).

Schroeder, B., Hilker, M., and Weires, R. (2007). Dynamic

association networks in information management. In

Proceedings of the 4th International Conference on

Machine Learning and Data Analysis (MLDA).

WEBIST 2008 - International Conference on Web Information Systems and Technologies

266