Trust Building in Recommender Agents

Li Chen

, Pearl Pu

Human Computer Interaction Group, School of Computer and Communication Sciences,

Swiss Federal Institute of Technology at Lausanne (EPFL), Switzerland

EPFL IC IIF GR-PU, BC-144, Station 14, CH-1015 Lausanne, Switzerland

EPFL IC IIF GR-PU, BC-107, Station 14, CH-1015 Lausanne, Switzerland

Abstract. Trust has long been regarded as an important factor influencing us-

ers’ decision to buy a product in an online shop or to return to the shop for more

product information. However, most notions of trust focus on the aspects of be-

nevolence and integrity, and less on competence. Although benefits clearly ex-

ist for websites to employ competent recommender agents, the exact nature of

these benefits to users’ trusting intentions remains unclear. This paper presents

some preliminary results of these issues based on a trust model that we have

developed for recommender agents. We describe a carefully constructed survey

in an attempt to reveal the relationship between users’ perception of the agent’s

trustworthiness based on its competence and consumer trusting intentions, and

more importantly, the role of explanation-based recommendation interfaces and

their media format on trust promotion.

Keywords: recommender agents, trustworthiness, explanation-based interfaces,

competence, trusting intentions, e-commerce

1 Introduction

In online commerce (or e-commerce), the traditional salesperson is often replaced by

a product recommender agent (or a virtual salesperson). Given the lack of face-to-

face interaction consumer, trust is difficult to build and easy to lose in a virtual store,

which has impeded customers from participating in e-commerce environments. Thus,

trust has been established to be a key factor to the success of e-commerce [4, 8]. It is

widely accepted that trust in a technological item (like the recommender agent) is

based on competence, benevolence, and integrity, just like trust in a person [10].

Although trust-related issues have been explored broadly in the fields of e-

commerce and human computer interaction, many limitations still exist. As a matter

of fact, most notions of trust have concentrated on how to improve the online shop’s

security, privacy policy and reputation, i.e. the benevolence and integrity of trust

constructs, and less on its competence. As recommender agents have been increas-

Chen L. and Pu P. (2005).

Trust Building in Recommender Agents.

In Proceedings of the 1st International Workshop on Web Personalisation, Recommender Systems and Intelligent User Interfaces, pages 135-145

DOI: 10.5220/0001422901350145

 SciTePress

ingly employed in websites to assist users in choosing products and making decisions,

it is necessary to pay attention to how an agent’s competence influences and builds

consumer’s trust. That is, it is meaningful to investigate how the system design fea-

tures, such as its interface display techniques, recommender algorithms, and user-

system interaction models interact within the trust building process.

Another main limitation of the field is the lack of empirical studies detailing the

exact nature of trust-induced benefits. In the electronic environment, trust is widely

defined as a kind of behavioral intention [5], referred as “trusting intentions” by

McKnight et al. [14]. It has been established that customer trust is positively associ-

ated with customer’s intention to transact, purchase a product, or return to the website

[4, 8, 11]. However, there is no further exploration of which construct of trust most

contributes to one specific intention. Moreover, it is unclear whether users, rather

than e-stores, can actually benefit from the trust. For example, can users actually

improve their task performance due to their increased trust in the recommender

agent?

The contribution of this paper is the development of a trust model, which identifies

a set of system features that contribute towards building competence-inspired trust in

recommender agents. It considers different aspects of the system design, in particular

the role of explanation-based recommendation interfaces and their media format on

building trust. To understand the effect of these design issues, we have conducted a

survey among 53 users to understand the benefits of trusting a recommender agent

based on its competence and the effective means to develop trust using explanation-

based interfaces. The results showed that a positive perception of a recommender

agent’s competence increases users’ intention to return to the agent, but does not

necessarily affect their intention to purchase. The explanation facility integrated in the

recommendation interface was positively correlated with users’ trust-building in the

agent. The survey further demonstrated that an alternative organization-based expla-

nation technique was more effective than the simple “why” construct used in most e-

commerce websites.

This paper is organized as follows: section 2 presents our trust model for recom-

mender agents and its different constituents; section 3 introduces different explana-

tion techniques applicable in recommendation interfaces; section 4 reports our hy-

potheses and survey results; and section 5 concludes the paper’s work followed by

several directions for future research.

2 Trust Model

We have conceptualized a general trust model for recommender agents. It consists of

three components: system features, trustworthiness of the agents, and trusting inten-

tions (see Fig. 1). The system features mainly deal with those design aspects of a

recommender agent that can contribute to the promotion of its trustworthiness. We

classified them into three groups: the interface display techniques, the recommender

algorithms (e.g. collaborative filtering, case-based reasoning, and preference-based

search techniques), and user-system interaction models such as how an agent elicits

users’ preferences. In this paper, we focus our treatment of systems features on inter-

136

face display techniques, especially explanation-based interfaces, and we detail how to

select content, media format and richness for such interfaces.

The agent trustworthiness is a trust formation process based on users’ perception

of the agent’s competence, reputation, integrity, and benevolence, which has been

regarded as the main positive influence on the trusting intentions [6, 13]. In this paper,

we primarily consider the competence perception and its essential contribution to

trust-induced benefits.

The trust intentions are the benefits expected from users once trust has been estab-

lished by the recommender agents. Trusting intentions include the intention to pur-

chase a recommended item, to return to the store for more information on products or

purchase more recommended products, and to save effort. The intention to save effort

is of particular interest to us because it examines whether upon establishing a certain

trust level with the agent, users will exert less effort to process all information them-

selves by selecting the recommended items much earlier in the recommendation cy-

cles.

In addition to the agent trustworthiness, another influence on trusting intentions

would be the individual propensity to trust. Studies of trust as a purely psychological

attribute revealed that each person possesses a stable personality characteristic, which

influences one’s willingness to extend trust in specific situations [3]. We are inter-

ested to know whether this factor will make an impact on users' behavior intentions in

recommender agents.

Fig. 1. Trust model for recommender agents

3 Building Trust Using Explanation-based Interfaces

Explanation has long been employed as one main approach to improve system’s

transparency in the domains of expert system [9], recommender systems [7], and

interactive data exploration systems [1]. Practical application of explanation-based

interfaces can be found in decision support systems (Logical Decisions:

www.logicaldecisions.com), and commercial web sites (Active Decisions:

www.activedecisions.com).

System Features

Trustin

Intentions A

Trustworthiness

Recommender

rithm

Intention to bu

Intention to

return

Intention to save

effort

Benevolence

utation

Com

etence

Inte

rit

Pro

ensit

to trust

Interface display

lanation

User-system

act

137

However, the benefits of explanation for trust formation have not been well estab-

lished. Herlocker et al. [7] have shown that in automated collaborative filtering (ACF)

based recommender systems, providing explanation facility of recommendations can

improve the acceptance of the system and filtering performance of users, but no fur-

ther work on the relationship between explanation and trust building. More specifi-

cally, it is still unclear whether explaining how recommendations are computed can

increase user’s trust in the recommender agent. In this section, we therefore primarily

consider trust building by the different design dimensions of explanation-based inter-

faces. In particular, we investigate the modality of explanation, e.g., the use of graph-

ics vs. text, the amount of information used to explain, e.g., whether long or short text

is more trust inspiring, and most importantly whether alternative explanation tech-

niques exists that are more effective in trust building than the simple “why” construct

currently used in most e-commerce websites.

Fig. 2. The “why” explanation facility used in most e-commerce websites

The explanation generation comprises the steps of content selection and organiza-

tion, media allocation, and media realization and coordination [1]. Content selection

determines what information should be included in the explanations. For instance, the

neighbors’ ratings can be included to explain the recommended items in the collabo-

rative filtering based recommender systems [7]. Once the content is selected, we must

know how to organize and display it. The simplest strategy is to display the content in

a rank ordered list with a “why” tool tip for each recommendation (see Fig. 2).

Fig. 3. Organization-based explanation interface, where the category title replaces the “why”

component

138

As an alternative and potentially more effective technique, we have designed an

organization-based explanation interface where recommendations that provide trade-

off alternatives are grouped in one category (see Fig. 3). This idea was inspired by

McCarthy et al.’s work [12] which showed that suggesting products in groups of

compound critiques enabled users to reach their decisions much faster.

The ranch house seems better than Japa-

nese house according to your preferences,

since it has advantages on garage size,

condition, needed repairs, purchase price,

systems, kitchen and other features. How-

ever, the Japanese house still has some

benefits on surroundings quality, operating

costs, exterior appearance and upstairs

size.

Fig. 4. Explanation realized in text vs. graphics. The right figure (adapted from logical deci-

sions software) is using graphics to explain the difference between two houses in terms of their

attribute values. The left text gives the same content in the style of conversational sentences

Media allocation and realization considers the concrete mapping between the dif-

ferent portions of the selected content and the corresponding media. Currently, there

are mainly two media used to implement explanation. One medium is text (see Fig. 4),

which is used in many commercial web sites (see Fig. 2) and expert systems [9]. The

research direction has been to make the explanation more conversational and argu-

mentative to make people feel at ease.

Another medium uses graphics for realizing explanation content (see Fig. 4). The

advantage is that visual information can allow people to develop a deep understand-

ing of the data. Herlocker et al. have proven that the histogram with grouping of

neighbor ratings was the best performing explanation component for collaborative

filtering recommendations [7]. However, their experiment didn’t compare the histo-

gram with text for the same explanation content. In general, few existing works indi-

cate which media is more preferred by users in general or in a specific circumstance.

House 18 is an interesting house. In fact, it

has a convenient location in the Ecublens

neighborhood. House 18 is close to work (1.7

miles).

House 18 is an interesting house. In fact, it

has a convenient location in the Ecublens

neighborhood. Even though house 18 is

somewhat far from the park (3 miles), it is

close to work (1.7 miles) and a rapid trans-

portation stop (1 mile). House 18 offers a

beautiful view, and it has a wonderful

exterior.

Fig. 5. Short and concise explanation sentences vs. Long and detailed ones

The issue of media richness of explanation is also not well understood. For exam-

ple, is a short and concise explanation text preferred to a long and concise one (see

139

Fig. 5)? Carenini and Moore [2] have developed one method to generate argumenta-

tive text tailored to the user's multi-criteria preference model. They also indicated that

the effective arguments should be concise, presenting only pertinent and cogent in-

formation. However, their evaluation was specific in the domain of searching for a

house, and did not measure the effectiveness from the aspect of trust building.

4 Survey

We have conducted a survey among 53 users in order to understand the interaction

among the three components of our trust model: the effect of an agent’s competence

in building users’ trust, the influence of trust on users’ problem solving efficiency and

other trust intentions, and the effective means to build trust using explanation-based

interfaces. The result of this survey will hopefully help us focus our future direction

in conducting empirical studies to understand the quantitative relationships among

these components of trust.

4.1 Hypotheses and Survey Questions

We have developed 9 hypotheses and divided them into three categories: agent trust-

worthiness in terms of its competence, trusting intentions assessment, and explanation

techniques on trust formation. For each hypothesis, we have designed a question for

participants to respond on a 5-point Likert scale (see the hypotheses and correspond-

ing questions in the following tables). To illustrate the hypothesized scenarios, a set

of pre-designed interfaces was used as references. For instance, the interface inte-

grated with the “why” construct (adapted from a website powered by Active Deci-

sions, see Fig. 2) was shown along with another similar display without such explana-

tion facility (see Fig. 6) to our participants when they were asked whether they would

trust more in the recommender agent which could explain how it works than the agent

without any explanation.

Fig. 6. The recommendation interface without “why” explanation facility

140

The first hypothesis tested in our survey was about the direct relationship between

the recommender agent’s competence and users’ trust in the agent.

Hypothesis Question

Hypothesis 1: A positive perception of the

recommender agent’s competence will induce

the user’s tendency to trust that agent.

The recommender agent gave me some

really good suggestions. Therefore, the

agent can be trusted.

We further predicated that a positive perception of the agent's competence could

increase a user's intention to return and save effort, but not his/her intention to pur-

chase because purchase intention would depend on other variables as well. Therefore,

we have developed the following three hypotheses related to the effect of agent's

competence on trusting intentions.

Hypothesis Question

Hypothesis 2: A positive perception of the

recommender agent’s competence may not be

the only element contributing to users’ dispo-

sition to buy a product from the website.

Even though I got some really good sug-

gestions from the agent, I am not yet in-

clined to buy the product from the website

where I found the recommender agent.

Hypothesis 3: A positive perception of the

recommender agent’s competence may neces-

sarily lead to users’ intention to return to the

agent for other product recommendation.

The recommender agent gave me some

really good suggestions. Therefore, I will

return to this website for other product

recommendations.

Hypothesis 4: A high level of trust in the

recommender agent may necessarily lead to

users’ intention to completely rely on the

agent to make a decision.

If I trust the recommender agent, I will rely

on it more to help me make a decision,

rather than processing all of the informa-

tion myself.

Then we measured the effectiveness of explanation-based display techniques on

trust building in recommender agents. The hypotheses 5 and 6 were about the benefits

of explanation on trust promotions, and the remaining hypotheses aimed at determin-

ing the effect of media format and the richness of explanation, and more importantly

whether an alternative organization-based explanation technique (see Fig. 3) would

perform better than the simple “why” construct (see Fig. 7).

Hypothesis Question

Hypothesis 5: Explanation is positively

correlated with user’s trust in the recom-

mender agent.

If there are two recommender agents, one

with an explanation of how it works (see

Fig. 2), and another one without (see Fig.

6), I will definitely trust the first one more.

Hypothesis 6: Explaining how suggestions

are computed increases users’ trust in the

agent.

If I know how the suggestions are com-

puted and ranked, I will be less likely to

want to see the alternatives the agent does

not suggest.

Hypothesis 7: The explanation of suggestions

in text form is more effective than in graph-

ics.

I prefer to see an explanation in familiar

language rather than in diagrams such as a

histogram or a table (see Fig. 4).

Hypothesis 8: Explanation in short and con-

cise sentences is preferred to long and de-

tailed ones.

I prefer short and concise explanation

sentences to long and detailed ones (see

Fig. 5).

Hypothesis 9: Well-organized recommenda-

tions are more effective than a simple list of

If the suggestions are well organized into

different groups according to their differ-

141

suggestions with explanations. ences (see Fig. 3), it will be easier for me

to compare them and make a quicker

choice, compared to a rank-ordered sug-

gestions with detailed explanations (see

Fig. 7).

Fig. 7. The recommendations with simple “why” explanation component

4.2 Survey Participants and Procedure

A total of 53 (7 females) undergraduate students taking the Human Computer Interac-

tion course participated in the survey for partial course credit. To make sure that all of

them have had some experience in online shopping environments before the survey,

we have instructed these student participants to search for a Tablet PC at PriceGrab-

ber (www.pricegrabber.com).

This survey was conducted in the form of a carefully constructed questionnaire,

based on a series of hypothesis and corresponding applicable questions. In the begin-

ning, participants were required to attend a pre-test of their familiarity with e-

commerce environment and propensity to trust, with the aim to check whether these

factors would influence their survey answers. Afterwards, the survey started by ask-

ing users to respond to each of the nine questions on a 5-point Likert scale from

“Strongly disagree” to “Strongly agree” respectively. Since most of the students’

native language is French, each question was accompanied with a corresponding

translation so as to avoid any language misunderstanding.

4.3 Results and Discussion

The survey results give us some useful implications on trust building in recommender

agents (see Table 1). It indicates that the competence of recommender agents would

not be the only contribution to users’ trust formation process (hypothesis 1: median

=3 “not sure” and mode=3), but it is positively correlated with the trusting intention

to return. In fact, most of participants agreed with the items measured for hypothesis

2 and 3 (mean>3, median=4 “agree” and mode=4). This indicates that if users pos-

sessed a high perception of the recommender agent’s competence, they would be

more inclined to return to the agent for other products’ information and recommenda-

142

tion, but they would not necessarily intend to buy the product from the website where

the agent was found. Post-survey discussion indicated that they would visit more

websites to compare the product’s prices before making a purchase. The website’s

security, reputation, delivery service and privacy policy were also their important

considerations in buying a product. As for the trust benefit to users’ problem solving

efficiency (hypothesis 4), the most frequently recurring answer was “disagree”

(mode=2), implying that many participants would still want to take time to process

information themselves, rather than entirely relying on the agent to choose an item.

Table 1. The analysis results of survey on hypotheses (possible range from 1: strongly disagree

to 5: strongly agree)

Mean (St.d.)

N=53

Median

N=53

Mode

N=53

Hypothesis 1 3.15 (0.73) 3 3

Hypothesis 2 3.55 (0.94) 4 4

Hypothesis 3 4.23 (0.63) 4 4

Hypothesis 4 2.89 (1.09) 3 2

Hypothesis 5 3.64 (0.99) 4 4

Hypothesis 6 3.06 (0.98) 3 4

Hypothesis 7 2.38 (0.94) 2 2

Hypothesis 8 2.85 (1.12) 3 2

Hypothesis 9 3.91 (1.03) 4 4

The positive responses to hypothesis 5, 6 and 9 (mean>3 and mode=4) indicate

that explanation can be an effective means to achieve user’s trust, and the organiza-

tion would be a more effective explanation technique than the simple “why” con-

struct. However, the other two aspects of explanation, i.e. the modality (hypothesis 7)

and richness (hypothesis 8), were not conclusive in this study (mean<3 and mode=2).

That is, it is unclear whether graphics or text is more effective to realize explanation,

and whether long or short explanation text would be more trust inspiring. From the

participants’ viewpoints, these two aspects were mostly dependent on the concrete

product domain. Users would prefer a short and concise conversational sentence for

the so-called low-risk products such as movies and books, but if they were looking

for products carrying high financial and emotional risks such as cars and houses, a

more detailed and reasonable explanation would be favored. In addition, people from

different educational background seemed to have different preferences on the media

richness. For example, students majoring in mathematics were more likely to prefer

the explanation using graphics than explanation in text form.

The correlations between pre-tested variables with measured hypotheses indicated

that participants’ propensity to trust and familiarity levels with the e-commerce didn’t

have significant correlations with their resulting ratings on the hypothesized items,

except the frequency of using online shopping tools was significantly positively cor-

related with the hypothesis 9 (p<0.05), which suggests that if participants had more

online shopping experience, they would more likely prefer a well-organized recom-

mendations to a simple list with the “why” explanation construct.

143

5 Conclusion and Future Work

This article describes the early stage of our investigation of trust issues in recom-

mender agents and the qualitative relationship between consumers’ perception of

trustworthiness based on an agent’s competence and their trusting intentions.

Through a carefully designed survey, we have shown that a recommender agent’s

competence is positively correlated with users’ intention to return, but not necessarily

with their intention to purchase. Further, explanation-based interfaces provide a

promising approach to build a competence-inspired trust relationship. More impor-

tantly, an organization-based explanation technique is likely to be more effective than

the simple “why” construct.

These initial results provide useful insights to several directions for future work.

For example, we have recently started a large-scale empirical study to quantitatively

measure the difference of user’s speed in identifying their most preferred items be-

tween the interface using the “why” construct and the interface using the information

organization strategies. We are also planning a controlled experiment to measure if

users can improve their decision accuracy using the explanation-based interface. In

addition, we also believe that users will likely save their decision-making effort with

agents which are trustworthy.

References

1. Carenini, G. and Moore, J. Multimedia Explanations in IDEA Decision Support System.

Working Notes of the AAAI Spring Symposium on Interactive and Mixed-Initiative Decision

Theoretic Systems (1998)

2. Carenini, G. and Moore, J. An Empirical Study of the Influence of Argument Conciseness

on Argument Effectiveness. Proceedings of the 38th Annual Meeting of the Association for

Computational Linguistics (ACL-00) (2000)

3. Chopra, K. and Wallace, W. A. Trust in Electronic Environments. In Proceedings of the 36

Hawaii International Conference on System Sciences (HICSS'03) (2003)

4. Gefen, D. E-Commerce: The Role of Familiarity and Trust. Omega: The International Jour-

nal of Management Science, 28, 725-737 (2000)

5. Gefen, D., Rao, V.S. and Tractinsky, N. The Conceptualization of Trust, Risk, and Their

Relationship in Electronic Commerce: The Need for Clarifications, Working paper (2003)

6. Gefen, D. and Straub, D.W. Building Consumer Trust in Electronic Commerce, Working

paper (1999)

7. Herlocker, J. L., Konstan, J. A. and Riedl, J. Explaining Collaborative Filtering Recommen-

dations. In ACM Conference on Computer Supported Cooperative Work (2000)

8. Jarvenpaa, S. L., Tractinsky, N. and Vitale, M. Consumer trust in An Internet store. Informa-

tion Technology and Management, 1(1-2), 45-71 (2000)

9. Klein, D. A. and Shortliffe, E. H. A Framework for Explaining Decision-theoretic Advice.

Artificial Intelligence 67: 201-243 (1994)

10. Komiak, S., Wang, W. and Benbasat, I. Comparing Customer Trust in Virtual Salespersons

with Customer Trust in Human Salespersons. Proceedings of 38

Hawaii international confer-

ence on system sciences (2005)

11. Koufaris, M., Hampton-Sosa, W. Customer Trust Online: Examining the Role of the Ex-

perience with the Web Site, CIS Working paper series (2002)

144

12. McCarthy, K., Reilly, J., McGinty, L. and Smyth, B. Experiments in Dynamic Critiquing,

Proceedings of the International Conference on Intelligent User Interfaces (2004) 175-182

13. McKnight, D.H., and Chervany, N.L. What Trust Means in e-Commerce Customer Rela-

tionships: Conceptual Typology. International Journal of Electronic Commerce, 6(2), 35-59

(2002)

14. McKnight, D. H., Cummings, L. and Chervany, N. L. Trust Formation in New Organiza-

tional Relationships. Working paper (1995)

145