EVOLVING STRUCTURES FOR PREDICTIVE DECISION

MAKING IN NEGOTIATIONS

Marisa Masvoula, Panagiotis Kanellis and Drakoulis Martakos

Department of Informatics and Telecommunications, National and Kapodistrian University of Athens

University Campus, Athens 15771, Greece

Keywords: Evolving Connectionist Systems, Negotiation forecasts, Predictive Decision Making.

Abstract: Predictive decision making increases the individual or joint gain of negotiators, and has been extensively

studied. One particular skill of predicting agents is the forecast of their opponents’ future offers. Current

systems focus on enhancing learning techniques in the decision making module of negotiating agents, with

the purpose to develop more robust systems. Empirical studies are conducted in bounded problem spaces,

where data distribution is known or assumed. Our proposal concentrates on the incorporation of learning

structures in agents’ decision making, capable of forecasting opponents’ future offers even in open problem

spaces, which is the case in most negotiation situations.

1 INTRODUCTION

Electronic Marketplaces (E-markets), is an

important component of e-business that brings

demand and supply of commodities and services into

balance. The term e-market is used in a broad sense

and incorporates the various types and

configurations of markets, stores, agoras and other

meeting places where transactions about tangible or

intangible objects take place (Kersten, Chen,

Neumann, Vahidov, and Weinhardt, 2008). Our

focus lies on the negotiation mechanism, which is

defined as an iterative communication and

distributed decision-making process, where

participants, humans or agents acting on their behalf,

are searching for an agreement. Several scientific

fields have made contributions to the development

of negotiation theory. In particular models that

follow normative, prescriptive or descriptive

approaches derived from the application of

economic theories, management and social sciences

respectively. The current trend concentrates on the

development of learning techniques, incorporated

either in support systems that assist human

negotiators, or in software agents that are capable to

fully automate the process. It is proved that humans

or agents that act in open, dynamic environments

where minimal knowledge is available are

particularly benefited by learning techniques that

seem to “extend” their cognitive abilities. In section

2 we give a brief review of the learning techniques

employed by negotiators, and particularly focus on

forecasting opponents’ offers. In section 3 we

discuss limitations and weaknesses and in section 4

we propose a structure that is expected to advance

the state-of-the art in predictive decision-making.

Finally, in section 5 we describe the expected results

of this proposal.

2 LEARNING IN

NEGOTIATIONS

The majority of research efforts regarding the

learning techniques in order to support the various

negotiation activities are concentrated in the

adoption of optimal or satisfying strategies, in

understanding negotiating partners and in identifying

individual preferences and objectives. This is due to

the fact that negotiators deal with vague and

incomplete information. The common case is to be

ignorant about their opponents’ preferences and

strategy. Nevertheless negotiation result, measured

in terms of individual or joint satisfaction, highly

depends on the negotiating behaviors of the engaged

parties, reflected through the different strategies. We

devise current state-of-the-art agents into those that

follow explorative, repetitive or predictive strategies.

391

Masvoula M., Kanellis P. and Martakos D. (2010).

EVOLVING STRUCTURES FOR PREDICTIVE DECISION MAKING IN NEGOTIATIONS.

In Proceedings of the 12th International Conference on Enterprise Information Systems - Artiﬁcial Intelligence and Decision Support Systems, pages

391-394

DOI: 10.5220/0002895003910394

 SciTePress

The former category consists of agents that search

the strategy space usually through trial-and-error

learning processes, the second category consists of

agents who repeat strategies that have proved

efficient in past similar situations, while the third

category consists of agents who adopt a strategy,

based on estimations of environmental parameters

and/or opponent. We focus on the latter category

and particularly to the issue of estimating opponents’

future offers, which has proved to add value to

negotiators in various domains. The learning

methods used to provide opponent’s forecasts

summarize to statistical models, mathematical

models and neural networks. In section 2.1 we

present current systems of negotiation forecasts.

2.1 Forecasting Opponent’s offers

Forecasting opponents’ offers has proved valuable

for various reasons. We discriminate between single

and multi-lag predictions. Single-lag predictions,

which involve the estimation of the opponents’ next

offer, encourage more sophisticated decision making

mechanisms. Oprea (2003) discusses the

development of SmartAgent enhanced with a feed

forward artificial neural network, to facilitate trading

scenarios via an internet platform. The agent uses

the predicted value of his opponents’ next offer in

order to refine his proposal and increase individual

gain. Carbonneau, Kersten, and Vahidov (2008)

depict the development of a neural network

predictive model in order to facilitate “What-if”

analysis and generate optimal offers. It is proved that

even small variations in the current offer can have

important impact on the expected counter-offer from

the opponent. A similar negotiation support tool is

applied by Lee and Ou-Yang (2009) in a supplier

selection auction market, where the demander

benefits from the suppliers’ forecasts, by selecting

the most appropriate alternative in each round.

Papaioannou, Roussaki, and Anagnostou (2006)

discuss a predictive model, based on neural

networks (MLPs and RBFs), with the purpose to

refine the agents’ pre-final offering decision and

produce more beneficial outcomes. The difference

with this approach is that the prediction mechanism

is run only once, when agent is approaching his

deadline. Brzostowski and Kowalczyk (2006)

implemented a non-linear regression model to

forecast opponents’ next offer; they describe an

iterative procedure in order to foresee the whole

negotiation thread, based on standard concessions.

The objective is to identify the optimal strategy in

order to attain the most beneficial discourse.

Moving to the realm of multi-lag predictions, an

interesting approach based on non-linear regression

can be found in Hou (2004), where prediction of

opponents’ future offers, combined with the

estimation of his strategic parameters, has been used

to effectively detect and withdraw from pointless

negotiations, where agreement could not have been

established. This line of inquiry has also been

followed by Roussaki, Papaioannou and Anagnostou

(2007), where the decision of the agents to withdraw

or not from the current negotiation was taken at an

early round through the forecast of the providers’

offer before the clients’ deadline, with the use of

MLPs and RBFs. Finally, predictions have been

used to avoid negotiation breakdown whilst making

a best deal at the opponents’ deadline (Hou, 2004).

Current systems have been assessed by a series of

experiments with opponents who use pure and in

some cases mixed static strategies in various

domains, and it has been proved that predicting

agents gain in utility compared to the non-predicting

ones.

3 PROBLEM STATEMENT

When it comes to forecasting the partners’ future

offers, techniques can be summarized into those

based on statistical approaches (non-linear

regression), mathematical models, based on

arithmetic analysis and connectionist approaches,

particularly some special types of neural networks

(MLPs and RBFs). We are not concerned with

mathematical models, since experiments have

proved that they give poorer results when compared

to non-linear regression or neural networks. The

agents enhanced with non-linear regression methods

are more restrictive than those who use artificial

neural networks, in that they are particularly tied to

specific offer generation functions which have been

described by Faratin, Sierra, and Jennings (1998).

On the contrary neural networks do not assume a

known function form and this makes them more

robust in the general case. We trust that the current

trend on providing offer forecasts lies on neural

networks, also due to the fact that they have been

applied in different negotiation problems and

domains.

Nevertheless, in all aforementioned systems, the

networks are trained once in an off-line mode and

are set to operate in a real environment. This implies

high dependency of the predictors’ accuracy to the

available training data which are initially presented.

In reality, an electronic market place is a highly

ICEIS 2010 - 12th International Conference on Enterprise Information Systems

392

turbulent environment; data distributions may

change as stakeholders enter and leave the e-market,

or as individual preferences and strategies change

over time. If an agent changes the negotiable

attributes’ reservation values, his concession

strategy or the available time to negotiate, a different

negotiation thread, series of offers, will be produced.

As the predicting agent uses the neural network with

different data, the accuracy of the system is expected

to decrease. Neural networks that are used for

predictions comprise of a hidden layer with sigmoid

or tangent hyperbolic transfer functions and of an

output layer with linear transfer functions. The

transfer function of the nodes in the hidden layer

acts as a squashing function which returns values in

[-1,1]. Therefore if the new input deviates from min

and max values of input data in the training vector,

the network will not be able to produce accurate

results. Existing systems have not been tested in

dynamic environments with changing data

distributions. Since they are trained only once, how

can we expect to provide the network with data that

exhaust all possible interactions?

To tackle with the problem of changing

distributions, it is evident that models must engage

in on-line learning, where learning takes place

during operation, as new input patterns are

presented. A stated risk of this approach is

catastrophic forgetting; previously learned patterns

are forgotten with the presentation of new data.

Albesano Gemello, Laface, Mana, and Scanzio

(2008) state that catastrophic forgetting is

particularly high when a connectionist network is

adapted with new data that do not adequately

represent the knowledge included in the original

training data. The question we pose is the

following: how can the accuracy of a model engaged

in on-line, life-long learning be preserved even in an

environment with unknown data distributions?

4 PROPOSAL

In order to advance the current state of the art we

propose the use of a model capable of adapting to

new data of unknown distributions without

forgetting previously learned patterns. The above

characteristics are met by Evolving Intelligent

Systems (EIS), which trace and understand the

dynamics of the modeled processes, automatically

evolve rules, solve problems of complex domains

and continuously improve performance. Methods of

(EIS) are consolidated in Evolving Connectionist

Systems (ECoS), and have been studied in various

domains. “An ECOS is an adaptive, incremental

learning and knowledge representation system that

evolves its structure and functionality, where in the

core of the system is a connectionist architecture that

consists of neurons and connections between them”

(Kasabov, 2007). ECoS have the following attractive

features: they may evolve in open space, engage in

incremental lifelong learning in an online mode,

learn both as individual systems and as evolutionary

populations of such systems, partition the problem

space locally, allowing for fast adaptation, have

evolving structures and trace the evolving processes

over time. We propose the integration of Evolving

Fuzzy Neural Networks (EFuNNs) (Kasabov 2007),

which are evolving connectionist structures, in the

decision-making mechanism of negotiating agents.

EFuNNs translate the input and output space to

fuzzy input and fuzzy output space. The objective is

to provide appropriate mappings of input to output

subspaces. This is realized with the use of

intermediate rule nodes which move as new patterns

are presented and the data associations change.

Additionally new rule nodes may be created to

represent new associations. With this technique the

system is always consistent with current data,

without any assumptions of data distributions. In

more detail EFuNNs have a five layer structure as

shown in figure 1, where new nodes and connections

are created and connected as data examples are

presented. The first layer represents the input

variables and the second represents fuzzy

quantization of each input variable. The third layer

contains rule nodes that evolve through supervised

learning and represent prototypes of input-output

data associations. The fourth layer represents fuzzy

quantization of the output variables and the fifth

layer represents the values of the output variables.

Figure 1: Evolving fuzzy neural network (Kasabov, 2007).

Rule nodes move to accommodate new input-output

examples. The networks’ structure is not predefined

but changes according to incoming data (rules are

updated or new rules are inserted). This special

characteristic of EFuNNs allows for adaptation to

dynamic environments. Additionally each rule node

is separately trained (implements local learning), and

EVOLVING STRUCTURES FOR PREDICTIVE DECISION MAKING IN NEGOTIATIONS

393

this allows for learning new patterns without

forgetting the previously learned ones. Our belief

that EFuNNs are appropriate to guide predictive

decision making in negotiations is strengthened by

the fact that they can learn any dataset in various

problems (function approximation, time-series

prediction, and classification) and have been tested

in various domains. For example, (Kasabov, 2007)

demonstrates that EFuNNs are capable to learn

complex chaotic functions through incrementally

adaptive learning from one-pass data propagation.

5 EXPECTED RESULTS

Our research attempts to advance the state of the art

in predictive decision making with the proposal of

agents that are capable of providing predictions even

in dynamic environments with changing data

distributions. We distinguish two cases, bounded

and open problem spaces: (a) “in bounded problem

spaces, if sufficient examples are presented after a

time moment, the input and output space will be

covered by hyperspheres of the evolved rules, and

the system will reach the desired accuracy”

(Kasabov, 2007). It has been proved that EFuNNs

are universal function approximators in bounded

problem spaces; the proof is based on the well-

known Kolmogorov theorem and is analogous to the

proof that MLPs with two layers are universal

function approximators. In such cases we expect

EFuNNs to be as accurate as MLPs in the task of

forecasting opponents’ offers. (b) “In open problem

spaces, where data dynamics and distribution may

change over time in a continuous way, the error of

EFuNNs will depend on the closeness of the new

input to the existing rule nodes” (Kasabov, 2007).

Such spaces have not been considered in existing

literature and we argue that current systems are not

adequate to model evolving lifelong learning

processes. The use of EFuNNs in the decision-

making of existing negotiating agents adds value to

the field, as more accurate results are expected even

in open problem spaces. Empirical evaluation of our

proposal will be provided through a number of

experiments simulating different situations.

REFERENCES

Albesano, D., Gemello, R. Laface, P., Mana, F., Scanzio.

S. (2006) Adaptation of Artificial Neural Networks

Avoiding Catastrophic Forgetting. In Proceedings of

IJCNN 2006 (pp. 1554-1561).

Brzostowski, J. and Kowalczyk, R. (2006). Adaptive

Negotiation with On-Line Prediction of Opponent

Behaviour in Agent-Based Negotiations. In

Proceedings of the IEEE/WIC/ACM international

Conference on intelligent Agent Technology (pp. 263-

269). Washington, DC: IEEE Computer Society.

Carbonneau, R., Kersten, G. E., and Vahidov, R. (2008).

Predicting opponent's moves in electronic negotiations

using neural networks. Expert Systems with

Applications: An International Journal, 34 (2), 1266-

1273.

Faratin, P., Sierra, C. and Jennings, N. R. (1998).

Negotiation Decision Functions for Autonomous

Agents. Int. Journal of Robotics and Autonomous

Systems, 24 (3 - 4), 159-182.

Hou, C. (2004). Predicting Agents Tactics in Automated

Negotiation. In Proceedings of the intelligent Agent

Technology, IEEE/WIC/ACM international

Conference (pp. 127-133). Washington, DC : IEEE

Computer Society.

Kasabov, N. (2007). Evolving Connectionist Systems: The

Knowledge Engineering Approach. London: Springer-

Verlag,

Kersten, G. E, Chen, E., Neumann, D., Vahidov, R.,

Weinhardt, D. (2008). On Comparison of Mechanisms

of Economic and Social Exchanges: The Times

Model, Negotiation, Auctions and Market

Engineering. Heidelberg: Springer Berlin.

Lee, C. C. and Ou-Yang, C. (2009). A neural networks

approach for forecasting the supplier's bid prices in

supplier selection negotiation process. Expert Systems

with Applications, 36(2), 2961-2970

Oprea, M. (2003) The Use of Adaptive Negotiation by a

Shopping Agent in Agent-Mediated Electronic

Commerce. In: Proceedings of the 3rd International

Central and Eastern European Conference on Multi-

Agent Systems, CEEMAS03. (pp. 594-605).

Heidelberg: Springer Berlin.

Papaioannou, I. V., Roussaki, I. G., and Anagnostou, M.

E. (2006). Comparing the Performance of MLP and

RBF Neural Networks Employed by Negotiating

Intelligent Agents. In Proceedings of the

IEEE/WIC/ACM international Conference on

intelligent Agent Technology (602-612). Washington,

DC: IEEE Computer Society.

Papaioannou, I., Roussaki, I., and Anagnostou, M. (2008).

Detecting Unsuccessful Automated Negotiation

Threads When Opponents Employ Hybrid Strategies.

In D. Huang, D. C. Wunsch, D. S. Levine, and K. Jo,

(Eds.), Proceedings of the 4th international

Conference on intelligent Computing (pp. 27-39).

Heidelberg: Springer Berlin.

Roussaki, I., Papaioannou, I., Anagnostou, M. (2007).

Building Automated Negotiation Strategies Enhanced

by MLP and GR Neural Networks for Opponent

Agent Behaviour Prognosis. In Sandoval, F., Gonzalez

Prieto, A., Cabestany, J., Grana, M. (eds.) IWANN

2007 (pp. 152-161) Heidelberg: Springer Berlin.

ICEIS 2010 - 12th International Conference on Enterprise Information Systems

394