SERIOUS GAMING, MANAGEMENT AND LEARNING

An Agent Based Perspective

Marco Remondino, Anna Maria Bruno and Marco Pironti

University of Turin, Torino, Italy

Keywords: Business game, Knowledge transmission, Enterprise management.

Abstract: We present the construction and experimental application of a web based system for teaching topics of

Business Administration. The same concepts can be easily extended to other formative areas, and used to

transfer knowledge (learning by doing). The system realizes a cooperative behaviour of human agents

(learners) who interactively take decisions for a simulated profit oriented enterprise. The technical design is

based on System Dynamics and Artificial Agent modelling. An agent based framework is applied to the

model in the form of virtual tutoring system for learners; the cognitive agents learn through a trial and error

technique. After the trial period, they can be used as a decision support system for the human learners.

1 INTRODUCTION

Business Games (BG) can be considered role

playing games, characterized by a managerial

context. The players usually face some situations

typical for enterprise management and must take

various core decisions, mainly about marketing,

logistics, production, research and development

politics and so on. A very interesting feature of

business games is that they can be employed as a

teaching instrument and for training; the

students/trainee can learn some important concepts

about enterprise management, by trying them on the

field, instead of just studying them on books. This is

regarded as "learning by doing" concept. The main

didactic goals for BGs are to refine the decision

capacities of the learners when facing situations of

uncertainty, and above all their ability to take

managerial decisions when there is a trade-off

between risk and profit. Besides, through a BG,

some advanced managerial techniques can be

reached, and so can be the interaction among the

different enterprise functions.

The BG presented in this work is built on the

System Dynamics methodology (Forrester, 1961)

and following the specifications given in Bussolin

(1979); this means that the mechanisms of the game

are based on finite differences equations and curves

defining the main parameters of the game itself.

The innovative part is constituted by an agent

based framework applied in the form of virtual

tutoring system for learners; the intelligent agents

learn by trial and error, based on Reinforcement

Learning paradigms, by practicing the system. After

this trial period, they form a model of the

cause/effect relations among the decisions and the

observed results and can then be used as a decision

support system for human learners, during the game.

2 MODEL STRUCTURE

The model is built using a structure based on the

theory of System Dynamics. The model itself is

considered as an artifact (Simon, 1996), i.e.: an

interface between the internal structure

(implemented in Java) and the external environment,

i.e.: the physical one, in which the system itself is

used by the learners. There are six main subsystems,

mutually connected, in the simulated enterprise:

production, finance, implants, research and

development, marketing and sales. Some of these

subsystems are divided into other subsystems, if

needed (e.g.: national sales and sales to the rest of

the World). The model is a dynamic system and the

temporal walkthrough in the system has been

converted into a set of differential equations and

laws that can generate the walkthrough itself. This

description consists into a constant relation between

402

Remondino M., Bruno A. and Pironti M. (2009).

SERIOUS GAMING, MANAGEMENT AND LEARNING - An Agent Based Perspective.

In Proceedings of the First International Conference on Computer Supported Education, pages 402-405

DOI: 10.5220/0001967604020405

 SciTePress

the system status in a generic time T and the status

after a brief time interval "delta T" (DT). Two are

the main variable types in the model: the stock type

and the flow type (or rate). The latter are used to

recalculate the former after each DT. Many of these

flows are generated by the "actions" of the learners,

i.e.: their decisions, in order to modify the states of

the system. Not all the states are modified by

external actions, though.

There exist some inner actions and regulations

that act as "internal implicit decisions" performed by

the system, used to normalize the levels. The choice

of the configuration and balance among the external

decisions and implicit decisions identifies the nature

and type of knowledge that has to be transferred to

the learner in a direct or indirect way.

The external decisions are those that make it

possible for the individual learners to know the

object of their studies, since it is directly "acted

upon" by them. This kind of actions are simply

referred to as "decisions", since they can be carried

on by the learners. The other kind of decisions are

those that make it possible to keep the system

"alive" even when the learners (for a lack of

knowledge) has not been able to lead the system.

The enterprise, here seen as a complex system, is

part of a bigger external environment with which it

continuously interacts. This is configured by some

other sub-systems, like the banking system (able to

supply the financial means for the developing of

new technologies, new products and the enterprise

itself), the market system (where the demand is

generated in the form of orders for the enterprise),

the technology system (that determines what kinds

of technologies are available at a certain time step),

the suppliers system and customers system

(respectively simulating those sides) and the

workforce system (determining the average wages,

the work supply on the market and so on). The

equations in the model are in the form of:

SFi = SSi + (RIi –ROi) * DT (1)

Where SFi at the first member is the i-th Stock

Variable at the end of a DT, while the SSi on the

right is the same variable at the beginning of the DT.

RIi and ROi are respectively the Input Rate and

Output Rate relative to the i-th stock variable.

The variation is then depicted as a difference

among the Input and Output rates during the

considered DT; this is summed to the previous stock

value, to calculate the new one. The algebraic

difference among the two rates is then to be

weighted by the time in which that rates applied.

The units of measurement in the system derive

from the above equation. The time is measured in

months and the stocks are measured in units. The

rates are then units/month and DT is again measured

in months. DT is a very brief time period; for

simplicity, in the model it’s set to 1/100 of a month.

3 COGNITIVE AGENTS

Reactive agents don’t own an internal representation

of the environment and react to the stimuli coming

from it, by retrieving wired behaviours similar to

reflexes without maintaining any internal state.

Cognitive agents’ behaviour, on the contrary, is

goal-directed and reason-based; i.e. is “intentional”.

Basing on the final goal, the agent chooses its

action (or set of actions) according to its beliefs

(knowledge) of the world. At a higher level,

cognitive agents can choose a specific goal from a

set of achievable ones. Cognitive agents’ behaviour

can be seen as a two steps process: 1) goal selection

and 2) action selection to reach the selected goal.

The action selection problem at time t+1, along

with the goal selection, at a macro level, are central

topics, when the agents must learn how the models

works, by experimenting on it and being able to act

as a decision support system for human users.

So, in the following, by action selection we do

not strictly mean the problem of choosing which

action to take at a micro level (agent level), but also

which one, among the possible goals, to select.

In order to decide which actions to perform, the

utility for each of them must be evaluated; specific

Reinforcement Learning (RL) algorithms are used,

which transform quantitative data (the payoff) in

behavioural patterns for the agents. An agent

endowed with some RL algorithm, when in a

particular state of the world (x), performs an action

(a) and gets a payoff (r), calculated by a reward

function based on the consequences of the action

itself. Through a trial & error mechanism the agent

learns what are the actions that maximize this

numerical value and computes an internal table,

linking actions to states. It’s straightforward that in

the presented model the macro-goals are multiple

and typical of enterprise management (e.g.:

maximizing the profit, improving the implants,

expanding the research & development and so on).

Besides, the actions to be performed to achieve

any of these goals are in the form on a vector,

containing the decisions affecting each part of the

enterprise and the relative strategies.

SERIOUS GAMING, MANAGEMENT AND LEARNING - An Agent Based Perspective

403

4 ENTERPRISE MANAGEMENT

The model supplies the user (and the agents) with a

set of generated reports, typical of Management and

enterprise analysis. The users, by reading and

analyzing them, can track down the influence of the

single decision – or even better the aggregate effects

coming from two or more decisions – on the

synthetic results, representing the monthly

performance of the whole enterprise.

According to the traditional use of the above

quoted modelling, the design of a whatever

economic system being simulated provides a

discussion of its results “at the end” of simulation,

and the traditional transfer of knowledge suffer from

the paradigm of “coeteris paribus” i.e. teaching is

concerned with the behaviour of just given variables

at a time, while keeping the remaining ones “still”.

On the contrary, in the presented approach, the

learner (or better, the team of learners) “drives” the

whole system by his own decisions during the

decision making process, learns through the

interaction and the process of “role playing” within

the group, and at the end of each month gets a set of

reports, that will constitute the basis for the

decisions to be taken during the following one.

The users, by reading and analyzing them, have

to track down the influence of the single decision –

or even better the aggregate effects coming from two

or more decisions – on the synthetic results,

representing the monthly performance of the whole

enterprise.

While the model has been originally conceived

as a teaching platform, in Universities and schools

for transmitting such concepts as “double-entry

accounting”, and the way in which the decisions

taken in a real enterprise affect the synthetic results,

at the end of each period (month), it can be

employed also as a simulator for managing

purposes. After tuning it basing on a certain

enterprise, it can be used as a what-if analysis tool,

i.e.: a simulator in which certain changes can be

done, in order to see how the system reacts to them,

before doing them in the real world.

The cognitive agents can help both when the

system is used as a teaching platform in schools, and

when it’s used as a real simulator. In fact, in the first

case, the agents can correct the common mistakes of

the students, by learning a correct policy (or, better,

a correct set of action for a given managing strategy)

and aid the students when they need to keep their

decision. For an example, it’s possible to think about

a simulated enterprise, in which the R&D

department is particularly weak, while the income is

high, but the products are getting old and – possibly,

will turn obsolete in few months. Unskilled students,

that of course are not used to manage real enterprise,

will probably get excited by the good and steady

incomes, and will likely not invest in the R&D

department. In few months, the competitors will

have better products on the market and suddenly the

enterprise will drastically diminish the sold

quantities. Unfortunately, it will be too late to start

an R&D campaign, since that’ll likely require

several months (or even year) to develop a new

competitive product from scratch.

A cognitive agent, when turned on, by means of

a dynamic hint could inform the students that their

product is in the maturity phase of its life cycle, and

that soon it’ll face a probable decline. If, that

notwithstanding, the students still do not invest in

R&D, then the agent will suggest this as a possible

strategy to prevent a forthcoming decline for the

product, that will lead to a likely drastic reduction in

the sales and will directly point to R&D as a way to

overcome this in advance, by developing new

products or improving the old ones.

When used a dynamic real-time simulator for

what-if analysis, the agent has a different task; since

we can aspect that users are, in this case,

experienced managers, the agent could help them in

finding the cause-effect relations basing on historical

data. For example, when the system is tuned on a

real enterprise, and empirically validated on its raw

data, it could be used to conduct a scenario analysis.

If we consider, for instance, the introduction of a

new machinery in a manufacturing enterprise, then

the system will keep track of its costs (variable and

fixed) and all the interactions it could have with the

rest of the environment (e.g.: required labour force,

energy, training and so on).

Though, without an intelligent system acting as a

supervisor, many of these relations will result in

black boxes, exactly as it happens in the real world.

An intelligent agent could supply step-by-step

explanations for that, by monitoring all the data

flow, through statistical and data-mining techniques.

Besides acting as a decision support system for

human learners experiencing with the model,

artificial agents can be used to supervise the decision

taken by learners, in order to interpret them in a

cognitive way.

For example, in the previously mentioned

enterprise accounting model, some users could

immediately pursue an high profit, while others

could be concerned first with the expansion of their

enterprise on new markets. Others could choose to

improve industrial plants, while others could want to

CSEDU 2009 - International Conference on Computer Supported Education

404

differentiate production and invest on research &

development and marketing. All these decisions are

complex, since they are determined by the

combination of many different variables.

Sometimes the learners won’t even realize that

they are pursuing a strategy instead of another one,

and they often won’t foresee what the selected

strategy could bring.

5 REACTIVE AGENTS

The agents can also constitute some parts of the

model itself (Remondino, 2003); in the considered

enterprise accounting model, some reactive agents

can form the supply chain, or the warehouses, or

even the competitors operating on the same market.

When dealing with reactive agents, the action

selection problem is to be found at a macro

(aggregate) level, i.e.: population level. If reactive

agents are the competitors of human learners in the

simulated world, they could have a fixed rule of

behaviour over time. Some evolutionary algorithms

could be embedded in the agents, so that the best

players on the market could merge, to form some

other artificial players with an even better behaviour.

In this way it’s possible to start with a population of

agents with a random behaviour, facing the standard

decisions in the model, and select – through the

various “generations” – the best ones.

So it’s not the single agent that selects his

behaviour by updating its own policy (that remains

the same, being the agent a reactive one), but the

population that evolves over time, through the

mechanism of reproduction and mutation. This is an

approach often used when the rules of the

environment are given and the main task is to

observe some emerging aggregate behaviour arising

from simple entities, i.e.: reactive agents.

Since these agents does not feature a goal based

– pro-active – behaviour, the way they act tends to

be deeply dependent on the choices made by the

designer. In order to design flexible systems, the

aggregate behaviour (at population level, i.e.: macro

level) can be made self-adaptive through the

implementation of an evolutionary algorithm (EA).

In this case the agents will have a wired random

behaviour at the beginning, and evolve according to

the environment in which they act, through a

selection mechanism.

6 CONCLUSIONS

A cognitive business game has been presented in

this paper, used to form learners in the Universities

and schools. The structure of the model is built on

the theory of System Dynamics. The inner structure

of the model has been briefly described in the paper,

along with the main sub-systems tied to form the

whole. The users of the system must take decisions

at each time step, after which the system calculates

the corresponding results, showing them according

to the principles of double-entry accounting.

Cognitive agent based paradigms are then described

as a development for the system itself. The agent

based framework constitutes a form of virtual

tutorship for the learners. The agents act as a

decision support system for the decisions to be

taken, and can explain some cause/effect relations.

The agents themselves learn how the model work

by practicing it, through some reinforcement

learning techniques, and are then able to assist the

learners in the decision process. A brief description

about how reactive agent can be used as a part of the

model itself is also described.

ACKNOWLEDGEMENTS

We would like to gratefully acknowledge the key

support of prof. Gianpiero Bussolin, Full Professor

(1989-2004) for “Economia e Gestione delle

Imprese”, University of Torino, who originally

designed the Enterprise Simulator and participated

to its implementation. His work has been the starting

point for most ideas described in this paper. We

would also like to thank and acknowledge the

managerial board of Fondazione CRT.

REFERENCES

Bussolin, G., 1979. Simulazione interattiva aziendale.

Costruzione di un modello e risultati della sua

applicazione, Rivista di informatica.

Forrester, Jay W., 1961. Industrial Dynamics. Waltham,

MA: Pegasus Communications. 464 pp.

Remondino, M., 2003. Agent Based Process Simulation

and Metaphors Based Approach for Enterprise and

Social Modeling, ABS 4 Proceedings, SCS Europ.

Publish. House – ISBN 3-936-150-25-7, pp.93-97.

Simon, H. A., 1996. The Sciences of the Artificial, (third

ed.). Cambridge, MA, MIT Press.

SERIOUS GAMING, MANAGEMENT AND LEARNING - An Agent Based Perspective

405