RESEARCH ON THE BAYESIAN LEARNING MODEL FOR

SELECTING ARGUMENTS ON ARGUMENTATION-BASED

NEGOTIATION OF AGENT

Guorui Jiang, Xiaoyu Hu and Xiuzhen Feng

School of Economics and Management, Beijing University of Technology

No. 100 Pingleyuan, Chaoyang District, Beijing, P.R. China

Keywords: Argumentation-based negotiation, Agent, Selecting arguments, Bayesian learning.

Abstract: In the Argumentation-based negotiation of agent, it is important to enhance the agent’s ability according to

the environment, which would improve the argumentation efficiency significantly. Introducing Bayesian

learning model to select arguments in Argumentation-based negotiation, the agent is able to learn and adjust

itself according to a dynamic environment. This helps in making more rational and scientific choice for

advancing efficiency of argumentation, when it is facing a variety of options for sending arguments. Finally,

an example was presented for showing the rationality and validity of the model.

1 INTRODUCTION

As a widespread and important phenomenon in the

society, negotiation not only brings opposition to all

parties as they hold differences, but also makes them

dependent on each other as they all commit

themselves to consistency. With the rapid

development of economic globalization and market

networking, the traditional business negotiation has

been replaced by e-business negotiation because of

the shortcomings on the efficiency and effectiveness.

In order to improve the decision-making of

participants more scientifically and reasonably

during the negotiation, the theory and technology of

agent in Artificial Intelligence have been introduced;

such as Game-theoretic approaches, Heuristic-based

approaches and Argumentation-based approaches

(Rahwan et al., 2003). However, in most game-

theoretic and heuristic models, agents are not

allowed to exchange any additional information

other than what is expressed in the proposal (i.e.

potential agreements or potential deals) itself;

another limitation of the two approaches is they both

assume that agents’ utilities or preferences have

been fixed, which means that one agent cannot

directly influence another agent’s preference model,

or any of its internal mental attitudes (e.g., beliefs,

desires, goals, etc.) when it is generating its own

preference model. Therefore, more and more

researchers are contributing their study on

Argumentation-based negotiation of agent in the

field of e-business negotiation.

The selection of argument is one of the hot topics

on studying Argumentation-based negotiation of

agent. Kraus studied this topic much earlier with all

argument types from the weakest one to the most

aggressive one. The mechanism of selection is that

the agent will first try to use the weakest argument.

If it does not succeed, it will go further with the

following stronger arguments (Kraus et al., 1998); in

this case, we could see that the agent has to face

varied negotiation environments when it was ready

to send an argument. Obviously, the rules proposed

by Kraus can not be universally applied. Recently,

many researches have been focusing on evaluating

the strength of various arguments and making the

final choice with the comparison among all

evaluation results. The representative one is the

evaluation model contributed by Amgoud (Amgoud

et al., 2004; Amgoud et al., 2005). An extended

model in order to select sending argument is

present, which makes the evaluation of strength as a

core of the selection, chooses the certainty level (or

priority level) of the knowledge and goal related to

the argument as the main influencing factors of the

strength; and for the vague and qualitative

characteristics of evaluation of above factors,

introduces some methods of representation and

measurement in fuzzy mathematics; finally, makes

the comprehensive evaluation as the scientific basis

317

Jiang G., Hu X. and Feng X. (2010).

RESEARCH ON THE BAYESIAN LEARNING MODEL FOR SELECTING ARGUMENTS ON ARGUMENTATION-BASED NEGOTIATION OF AGENT.

In Proceedings of the 2nd International Conference on Agents and Artiﬁcial Intelligence - Artiﬁcial Intelligence, pages 317-322

DOI: 10.5220/0002727603170322

 SciTePress

for the selection of the argument’s type and content

(Guorui Jiang et al., 2009). However, the values of

the main influencing factors of the arguments’

strength in the whole argumentation interaction are

fixed, and they do not fully take into account the

dynamic of the actual negotiating environment, as

well as how the agent can adapt to its dynamic

environment.

To solve the problem mentioned above, a

representative study is the models of opponent

agents proposed by Carabelea (Carabelea, 2002);

agents can adjust their strategy for argumentation

through building and modifying models of

opponents during and after the negotiation process.

But how to build and modify the opponent’s model

was not specifically explained in the paper. In

addition, we can find in Agent-based negotiation,

through giving agents certain ability to learn, to have

access to more information of opponents’

preferences and the negotiating environment during

the negotiation interaction, you can effectively

improve their self-regulating capacity to the

dynamic environment, so as to achieve the purpose

of improving the efficiency of the negotiations.

Bayesian learning method is common in traditional

e-business negotiation (Zeng et al., 1998), it mainly

focus on learning to the feedback of negotiating

opponent, but the research on its use in

Argumentation-based e-business negotiation is

unusual now. Saha first proposed a Bayesian

network approach to build opponents belief model to

help agent to select a more effective argument.

Unfortunately, the paper only demonstrated the

possibility of the method, but it did not give an

opponent model which can be updated in the true

sense (Saha et al., 2004; Saha et al.,

2005).Vreeswijk and Nielsen also introduced

Bayesian network to generation or comparison of

arguments(Vreeswijk, 2005; Nielsen et al.,

2007).But we can find, these research mostly

focused on traditional social Argumentation-based

negotiation, not the Multi-issue e-business

Argumentation-based negotiation; and the agent’s

belief model (including information about

opponent’s goals and the negotiation environment)

can not be specifically formalized, thus can not

effectively influence the strategy for argumentation

like selection of arguments in negotiation.

Based on the background upwards, in this paper,

we will introduce Bayesian learning to

Argumentation-based negotiation. The agent is

empowered the ability of learning and adjusting

itself according to a dynamic environment, which

helps in making more rational and scientific choice

when it is facing a variety of options for an

argument. In this way, the efficiency of the

argumentation will be improved. At the same time,

in order to be easy understood, we verify the

rationality and validity of the model with calculation

and analysis of an example at the end of the paper.

2 CLASSIFICATION OF THE

ARGUMENTS

2.1 Threat

During a negotiation an agent A can force another

agent B to do α by threatening to do an action β, to

achieve the goal himself. A threat is then made up of

three parts: the knowledge relative to this threat (the

threat itself), the goal that A wants to achieve, and

finally the goal of the threatened agent B (Amgoud

et al., 2004; Amgoud et al., 2005).

Example 1 During an e-business negotiation, the

buyer agent A wants the seller agent B to lower the

price (LowPri) in a proposal, but it was refused. In

this status, A may put forward that it will choose

another seller (ChosAnoSel) as the threat to make B

modify his beliefs and accept the proposal as soon

as possible. And this threat can formally be

expressed as follows:

{},,

LowPri ChosAnoSel LowPri ChosAnoSel

<¬ → ¬ >

2.2 Reward

During a negotiation an agent A can entice another

agent B to do α by offering to do an action β as a

reward, to achieve the goal himself. A reward is then

made up of three parts: the knowledge relative to

this reward (the reward itself), the goal that A wants

to achieve, and finally the goal of the rewarded

agent B (Amgoud et al., 2004; Amgoud et al.,

2005).

Example 2 During an e-business negotiation, the

buyer agent A wants the seller agent B to lower the

price (LowPri) in a proposal, but it was refused. In

this status, A may put forward that it will buy some

other related products (BuyOthPro) from B as the

reward to make B modify his beliefs and accept the

proposal as soon as possible. And this reward can

formally be expressed as follows:

{},,

A LowPri BuyOthPro LowPri BuyOthPro

<→ >

2.3 Appeal

During a negotiation, an agent A may refer to some

positive or negative facts as examples to persuade

another agent B to do the business with it as an

ICAART 2010 - 2nd International Conference on Agents and Artificial Intelligence

318

appeal, to achieve the goal himself and the biggest

profit of both of them (Amgoud et al., 2005; Jinghua

Wu et al., 2006). Besides, we found that the appeals

also concern the goals of the receiving agent like

threats and rewards. So, in this paper we extend the

definition of Appeal proposed by Amgoud. An

appeal is made up of three parts: the knowledge

relative to this appeal (the appeal itself), the goal

that agent A wants to achieve, and finally the goal of

the appealed agent B.

Example 3 In the same negotiation stalemate

mentioned above in Examples 1 and 2, the buyer

agent A may refer to some interests (SelfValue) that

the seller agent B doesn’t know but these behaviors

may bring to such as many other agents will come to

buy this product after seeing it through the use of A

to persuade B to accomplish the business, and that

will also achieve the goal of high buying quantity

(HigBuy) of B. This appeal can formally be

expressed as follows:

{},,

A SelfValue LowPri LowPri Higbuy=< → >

3 FORMAL MODEL OF

EVALUATION ON THE

ARGUMENTS’ STRENGTH

AND SELECTING

ARGUMENTS

During the course of evaluation, we can conclude

three main factors influencing the argumentation

strength: the certainty level of the knowledge related

to the argument, the priority of the goal that the

agent sending argument wants to achieve, and the

priority of the goal that the agent receiving argument

wants to achieve. However, evaluation on these

factors can be hardly found with accurate value in

practice, it could be “very high” or “high”,

“Medium” and other vague concepts. Problems

related to those vague concepts can not be solved by

the traditional mathematics and statistics.

Accordingly, we would introduce the methods in

fuzzy mathematics to quantify the qualitative

factors, and make the comprehensive evaluation

finally. The details of the model mentioned above

can be found in the previous study (Guorui Jiang et

al., 2009).

4 BAYESIAN LEARNING MODEL

FOR SELECTING

ARGUMENTS ON

ARGUMENTATION-BASED

NEGOTIATION

4.1 Bayesian Learning Model

The essence of the Bayesian learning model consists

of using the Bayesian formula to processing the

received information to amend the prior knowledge

of learning objects. The Bayesian formula can be

expressed as follows:

There is a set of events A

, A

… A

concern with

event H set:

1）P (A

)>0；2）A

∩A

，i≠j；3）

∪

) =Ω,

Where P (A

) is priori probability; Ω is the union of

events A

, A

…A

The Bayesian formula is defined as follows:

(/) (/)()/ (/)()

iiiii

PA H PH A PA PH APA

∑

(1)

Where P (H/A

) is conditional probability, which

means that the probability of occurrence of event H

on the condition of the occurrence of event A

; P

/H) is posterior probability, which means the

understanding of learning objects after revising.

4.2 The Basic Contents of Bayesian

Learning Model on

Argumentation-based Negotiation

In Argumentation-based negotiation, the basic

framework of Bayesian learning model can be

summarized as follows:

 Learners: participating agents; in this paper, we

stand on the buyer agent’s point of view as the

learner.

 Learning objects: the information of opponent’s

preferences and negotiating environment. In

this paper, they are referred to the main factors

of the assessment of argument’s strength,

namely, "the certainty level of knowledge in

argumentation" and "the priority level of

related goals of negotiating partner in

argumentation".

 Priori knowledge: the sample space and

distribution of probability of the two main

factors mentioned above.

 Information: the interactive information of

learning object received during the course of

negotiations. In this paper, the information is

RESEARCH ON THE BAYESIAN LEARNING MODEL FOR SELECTING ARGUMENTS ON

ARGUMENTATION-BASED NEGOTIATION OF AGENT

319

the responded message as accepting or

rejecting from seller agent after receiving an

argument.

 Bayesian beliefs: in this paper, it refers to the

estimate of the seller’s responded policy of

after receiving an argument; it is the basis to

obtain the conditional probability. For

example, in the buyer’s opinion, the higher the

priority level of seller’s related goals in the

argument is, the higher the probability of the

argument’s acceptance by the seller will be;

On the contrary, the lower priority level is, the

lower probability of acceptance will be.

 Posterior knowledge: after obtaining the

conditional probability, combined with a

priori probability, the posterior probability is

calculated by Bayesian formula, which is the

updated knowledge of learning objects after

the buyer agent’s Bayesian learning.

4.3 The Process of Bayesian Learning

for Selecting Arguments

In e-commerce negotiations, the buyer sends an

argument to the seller firstly, the seller would make

a decision to accept or reject after the assessment of

the intensity of the argument. The buyer receives a

response message and analyzes it, updating

information of negotiating partner’s preferences or

the information of negotiating environment by

Bayesian learning Model. This paper updates the

certainty of relevant knowledge that concerns with

the argument and the priority of relevant objectives

of seller. The buyer will make assessment of

arguments with different types or contents according

to the updated value of intensity factors, and then

sends a new argument.

For example, let

{ 1, 2,..., }RRii n== be the

assuming set relating to the priority of the seller-

related goals of the buyer agent. Based on their prior

knowledge, each hypothesis has a probability

estimate, constitutes a probability

set

() {( ) 1,2,...,}PR PRi i n==

, which satisfy

with

()1

PRi

∑

. Subsequently, the buyer will

receive the feedback signal e from the seller (this

shall be accepted or rejected), according to the

current observed domain knowledge, the buyer

assumes a priori for each conditional

probability

()PeRi . At this point, the buyer

generates hypothetical posterior probability based on

the Bayesian learning model as:

()( )

()

()( )

ti i

tk k

PRPeR

PRe

PRPeR

−

∑

(2)

where

()tiPRe

indicates the probability of iR in the

interaction of t-th round of argumentation, after the

buyer agent receives the seller's feedback e.

Therefore, in the t-th round, the buyer will re-predict

priority of target related of the seller as:

Pr ( )j

gti

PReRi−

∑

(3)

The updated value of the corresponding intensity

factors will also be applied to evaluate the strength

of the optional arguments, and thus choose to send a

new round of argumentation. The details on the

evaluation and selection can be found in (Guorui

Jiang et al., 2009).

5 ANALYSIS OF AN EXAMPLE

5.1 Negotiation Parameters and Basic

Assumptions

This paper discusses mainly the buyer's selection on

sending in Argumentation-based e-commerce

bilateral negotiations. The main negotiation terms

are price, quality and delivery time.

The buyer agent will first estimate the certainty

level of knowledge and the priority level of the seller

agent’s goals related to the optional

arguments(initial beliefs).Here, we will explain this

by an example, suppose the buyer sends the first

argument as “a reward , if the price is cut by seller,

the buyer will buy more such products from him";

Table 1 shows the buyer’s priori probability

estimation of the priority level of the seller agent’s

goals related to the argument; Table 2 shows the

priori conditional probability when the seller accepts

or rejects the argument based on table 1.These priori

knowledge are all gained by the past online

transactions between the buyer and the seller. For

example, in the buyer’s common cognizance, the

higher the priority of seller’s related goals in the

argument is, the higher the probability of the

argument’s acceptance by the seller will be; On the

contrary, the lower priority level is, the lower

probability of acceptance will be.

ICAART 2010 - 2nd International Conference on Agents and Artificial Intelligence

320

Table 1: The buyer’s priori probability estimation of the priority level of the seller agent’s goals related to the argument.

The priority level of the

seller agent’s goals

R1:

very low

R2:

low

R3:

lower

R4:

medium

R5:

higher

R6:

high

R7:

very high

Probability P

(Ri) 0 0 0 0.1 0.3 0.4 0.2

Table 2: The priori conditional probability when the seller accepts or rejects the argument.

The priority level of the seller

agent’s goals related

The seller’s response to the argument

Acceptance Rejection

Very low 0.125 0.875

Low 0.25 0.75

Lower 0.375 0.625

Medium 0.5 0.5

Higher 0.625 0.375

High 0.75 0.25

Very high 0.875 0.125

5.2 The Process of Bayesian Learning

for Selecting Arguments

During e-business negotiation, the proposal given by

the buyer agent A is quite different from the

expectation of the seller agent B, for its profit, B

may reject, which may bring the negotiation into a

stalemate. In this situation, to guarantee the profit of

both of them and the negotiation to be continued

successfully, A will send argument to B to persuade

it make some concession, suppose there are three

optional arguments including:

(1) price cut, A will

purchase more such products from B if the price

cuts, (2) improve the quality, A will purchase more

such products from B if the quality improves, (3)

shorten the delivery time, B has promised A to

shorten delivery time in the past; the above optional

arguments can formally be expressed as follows:

{},,

LowPri BuyMorePro LowPri BuyMorePro=< → >

{},,

HigQua BuyMorePro HigQua BuyMorePro=< → >

{},,A PastPromise ShortDelivery ShortDelivery Hi

Rep=< → >

A send the first argument

, then receive the

acceptance by B to

.Based on the feedback by B

and the priori knowledge of B(for example as Table

1 and Table 2), A use Bayesian learning model to

update the belief on the certainty level of the

knowledge and the priority level of the seller agent’s

goals related to

.Here, we explain this by showing

updating of the priority level of the seller agent’s

goals related to

as follows: according to formula

(2) and data from Table 1 and Table 2, we can

achieve the results after calculation, for example,

04 4

()( )

()

()( )

0.1 0.5

0.1 0.5 0.3 0.625 0.4 0.75 0.2 0.875

0.07

PRPeR

PRe

PRPeR

×+× +× +×

≈

∑

in it,

14()PRe indicates the buyer A’s posterior

probability to

4R after it received the acceptance of

from B. Similarly, there are

15 16 17( ) 0.263, ( ) 0.421, ( ) 0.246PRe PRe PRe≈≈≈

and

01 0 2 03() () () 0PR PR PR

== because

01 0 2 03() () ()0PR PR PR

==.

Before A receives the feedback from B to

, its

belief about the priority level of the seller agent’s

goal (as buy more such products) can be calculated

by formula (3) as follows:

Pr ( )

0.1 (0.4,0.45,0.55,0.6)+0.3 (0.55,0.6,0.7,0.75)

+0.4 (0.7,0.75,0.85,0.9)+0.2 (0.85,1,1,1)

=(0.655,0.725,0.805,0.845)

PRi Ri

=×

=× ×

××

∑

During this, we have used knowledge in fuzzy

mathematics, the detail can be found in (Guorui

Jiang et al., 2009). Similarly, after A receives the

feedback, its belief about the priority level of the

seller agent’s goal will be updated as follows:

Pr' ( )

0.07 (0.4,0.45,0.55,0.6)+0.263 (0.55,0.6,0.7,0.75)

+0.421 (0.7,0.75,0.85,0.9)+0.246 (0.85,1,1,1)

=(0.67645,0.75105,0.82645,0.86415)

PRi Ri

=×

=× ×

××

∑

And, the probability distribution of the priority level

of the B-related goal "buy more products

(BuyMorePro)" will be updated as Table 3:

RESEARCH ON THE BAYESIAN LEARNING MODEL FOR SELECTING ARGUMENTS ON

ARGUMENTATION-BASED NEGOTIATION OF AGENT

321

Table 3: The updated probability distribution of the priority level of B’s related goal.

The priority level of the

seller agent’s goals related

R1:

very low

R2:

low

R3:

lower

R4:

medium

R5:

higher

R6:

high

R7:

very high

Probability P

(Ri) 0 0 0 0.07 0.263 0.421 0.246

After the buyer agent respectively updated the

certainty level of the knowledge and the priority

level of the seller agent’s goal related to

, it will

evaluate the strength of the rest optional arguments

according to the updated values of the main

influencing factors, and thus choose to send a new

argument. The details can be found in (Guorui Jiang

et al., 2009). In the example, as an agent can not

allowed to send two same arguments continuously,

and 3

are the new optional arguments, we

finally select to send

in a new round after the

evaluation. We can find that it is precisely because

the priority level of the B-related goal "buy more

products" has been updated after the first round,

while this goal is also B’s related to

, allowing

be the better choice; also the choice is more

effective and scientific because of update of agent’s

belief on the dynamic negotiation environment.

6 CONCLUSIONS

In this paper, a Bayesian learning model has been

introduced along with Argumentation-based

negotiation, and the process of Bayesian learning for

argument selection has been analyzed with an

example. This could make more scientific and

rational choice of different types and contents of

arguments and increase the possibility of acceptance

of arguments. It gives an agent a certain ability to

learn by Bayesian learning model. So that it can be

continuously adjusted according to dynamic

environment, self-awareness, and effectively

improve the efficiency of argumentation. It

overcomes shortcomings of argument selection

followed a static environment in the previous study,

and appears to be more practical and effective.

But, in the model, more complete prior

knowledge is needed and the reasonableness and

accuracy of prior distribution also need to be further

improved. At the same time, more information from

the opponent and the negotiation environment

should be considered into the learning object of the

agent, to help it to greatly improve its self-adaptive

ability to the dynamic negotiation environment.

ACKNOWLEDGEMENTS

This work is supported by the National Natural

Foundation of China (No: 70940005).

REFERENCES

Amgoud, L., Cayrol, C., 2004. Generation and Evaluation

of Different Types of Arguments in Negotiation. In

Proceedings of International Workshop on Non-

Monotonic Reasoning (NMR'04), pp. 10-15.

Amgoud, L., Prade, H., 2005. Formal Handling of Threats

and Rewards in a Negotiation Dialogue. In

Proceedings of the 4th International joint Conference

on Autonomous Agents and Multi-Agent Systems,

AAMAS'2005, pp. 529-536.

Carabelea, C., 2002. Adaptive Agents in Argumentation-

based Negotiation. Multi-Agent Systems and

Applications. 2322, 87-122.

Guorui Jiang, Xiaoyu Hu, 2009. Research on the

Evaluation Model for Selection of the Argument’s

Type and Content in Argumentation-based

Negotiation of Agent. In Proceedings of the 11th

International Conference on Informatics and

Semiotics in Organisations, pp. 173-180.

Jinghua Wu, Guorui Jiang and Tiyun Huang, 2006. Using

Two Main Arguments in Agent Negotiation. Agent

Computing and Multi-Agent Systems. 4088, 578-583.

Kraus, S., Sycara, K., McBurney, P., and Evenchik, A.,

1998. Reaching Agreements through Argumentation.

Artificial Intelligence. 104(1-2), 1-69.

Nielsen, S. H., Parsons, S., 2007. An Application of

Formal Argumentation: Fusing Bayesian networks in

multi-agent systems. Artificial Intelligence. 171(10-

15), 754-775.

Rahwan, I., Ramchurn, S. D., McBurney, P., Parsons, S.,

and Sonenberg, L., 2003. Argumentation-based

Negotiation. The Knowledge Engineering Review.

18(4), 343-375.

Saha, S., Sen, S., 2004. A Bayes Net Approach to

Argumentation. In Proceedings of the 3rd

International Joint Conference on Autonomous Agents

and Multiagent Systems, pp. 1436-1437.

Saha, S. and Sen, S., 2005. A Bayes Net Approach to

Argumentation Based Negotiation. Artificial

Intelligence. 3366, 208-222.

Vreeswijk, G. A. W., 2005. Argumentation in Bayesian

belief networks. Artificial Intelligence. 3366, 111-129.

Zeng, D., Sycara, K., 1998. Bayesian learning in

negotiation. Human-Computer Studies. 48(1), 125-141.

ICAART 2010 - 2nd International Conference on Agents and Artificial Intelligence

322