Using Game AI to Control a Simulated Economy
Paul McCarlie and Aaron Hunter
British Columbia Institute of Technology, 3700 Willingdon Avenue, Burnaby, Canada
Keywords: AI for Games, Rule-based Systems, Machine Learning.
Abstract: We explore the use of Artificial Intelligence (AI) to manipulate a simulated economy. Towards this end, we
present work in progress on a macroeconomic simulation that can be controlled by a game player. We view
this simulation as a sort of serious game; it can be played as a competition, but it can also be an educational
tool through which players learn both about economic principles and the behaviour of AI controllers. The
main contribution of this paper is the comparative study of the effectiveness of different AI agents for the
manipulation of a simulated environment. Focusing on AI approaches that are common in the gaming industry,
we implement four players that use intelligent methods to control the simulation by trying to maximize the
economic output. The aim of our work is to illustrate that simple methods from the game AI community can
be used to control a complex economic simulation effectively. This work, therefore, supports the common
position in the gaming community that simple character-based AI methods can produce competitive game
play even for complex tasks. Moreover, we demonstrate that, in the case of this particular simulation, a rule-
based reasoner outperforms more sophisticated AI agents.
1 INTRODUCTION
Simulating an economy is an important and
challenging problem in many video games. While the
economic simulations for games are simpler than
those used for real-life forecasting, they are often
complex and based on the same economic models. In
order to produce Artificial Intelligence (AI)
opponents in games of economic simulation, we need
to develop tools and frameworks for decision making
and manipulation of economic artifacts. In this paper,
we present a comparative study of several different
AI techniques for controlling a simulated economy.
This is work in progress, aimed at determining which
AI methods can perform the task effectively. The goal
is to create agents that manipulate the economy at a
level that is competitive with a human player, using
only the basic AI tools that are typically employed in
game development. The restriction to common
methods from game AI is significant, as we want to
explore how effective our AI controllers can be under
standard constraints on the development of AI for
games. We are also interested in determining which
specific AI models produce the strongest controllers.
This paper makes several contributions to existing
literature. First, in order to even explore the main
problem, we need to develop a realistic economic
simulation based on real economic models. We then
develop prototype software that implements four
different AI controllers for manipulating the economy
through simple directions. We illustrate that each
approach functions better than a baseline controller,
and we determine which approach is the most
effective. We argue that these results are immediately
applicable in game AI. Our prototype software
illustrates which AI technique is most effective at
improving the economy, restricting attention to the
simple techniques often employed in games. We
suggest that, when fully developed, this software will
provide a useful testbed for learning about economic
principles, as well as learning about the utility of AI
for decision making about an economy.
2 BACKGROUND
2.1 Economic Modelling
In this section, we provide a basic introduction to
economic modelling. Of course, a detailed discussion
of macroeconomic theory is beyond the scope of this
paper. We refer the reader to (Ragan, 2020) for a
complete introduction.
McCarlie, P. and Hunter, A.
Using Game AI to Control a Simulated Economy.
DOI: 10.5220/0010212306290634
In Proceedings of the 13th International Conference on Agents and Artificial Intelligence (ICAART 2021) - Volume 2, pages 629-634
ISBN: 978-989-758-484-8
Copyright
c
2021 by SCITEPRESS Science and Technology Publications, Lda. All rights reserved
629
Our simulation uses the Aggregate Supply-
Aggregate Demand (AS-AD) model to determine
price level based on supply and demand. This model
is used to track taxation, spending, investing and
debit. Basically, the price for goods is calculated as
an aggregate of consumption, investment and
government spending.
To track long term change in an economy’s
output, we use the Solow-Swan Growth Model
(Donghan et al., 2014). The economic output in this
model is determined as an aggregate of technology,
labour, productivity and capital stock. Capital stock
change is in turn calculated by a set of different
equations, relating population and capital growth.
2.2 Game AI
In this section, we briefly summarize the AI
approaches used in our simulation. While game AI
borrows techniques from the academic AI literature,
there is a distinct approach that requires distinct
methods. In particular, the goal in game AI is
generally to simulate intelligent characters, rather
than to simulate human reasoning; as such there are
particular tools and approaches that tend to be
implemented and used widely. The AI techniques
described in this section are all standard approaches
in game development, described in more detail in
(Millington, 2019).
The first three approaches are essentially based on
rules. First, we use a classic rule-based system, in
which economic rules based on expert knowledge are
defined. These are simple if-then rules, where we
keep track of knowledge base that changes as
antecedents are triggered. This framework then
dictates the actions that the AI should take to grow the
economy. The second approach is similar, but it uses
fuzzy logic in reasoning about the rules. In other
words, the rules are based on fuzzy concepts with
degrees of truth rather than classical logic (Köse,
2012). The third approach to AI employs goal-based
behaviour, where the AI has explicit goals and actions
rather than simple rules. This approach permits
reactive planning in the classical sense of (Georgeff
and Lansky, 1987). The advantages of this approach
have been discussed in (She and Grogono, 2009).
The last approach to AI used in our simulation is
Machine Learning (ML). Specifically, we use
regression to learn the relationships between
dependent variables based on past information. In this
simulation, we actually use a combination of linear
regression, Gaussian regression, and Sequential
Minimal Optimization regression (SMOreg). In the
present work, we use the Weka framework to
implement the ML agent (Frank et al., 2016).
3 ECONOMIC SIMULATION
3.1 Overview
Our software is a game that includes an interactive
macroeconomic simulation. The simulation can be
controlled by a human player, or by an AI agent. The
objective is not only to produce a playable game, but
also to provide a useful testing environment for
experimenting with economic policies. At the same
time, we wanted to determine if standard game AI
approaches can operate the simulation more
effectively than a human player.
A number of variables are simulated. Broadly
speaking, the primary ones are:
Debt
Economic Output
Taxation
Government spending
Public investment
Public consumption
Technology
The idea of the game is to maximize the economic
growth of your economy while keeping debt under
control.
There are two separate “debts” simulate in our
platform: government debt and public debt.
Government debt is determined by taxation and
spending; if players spend more than they are taking
in from taxes, government debt must increase. Public
debt is a bit more complicated. Public spending
depends on the interest rate, which depends on the
money supply; a higher money supply means a lower
interest rate and therefore more spending. The money
supply itself is determined by owned bonds divided
by the reserve requirement. In order to increase public
spending, the player must either increase the money
supply or decrease the reserve requirement. Public
income cannot be directly changed; it is determined
as a percentage of output. As is to be expected, if
spending exceeds income, public debt is incurred.
Debt is automatically paid back periodically,
being subtracted from spending, and it must be paid
back with interest. The interest is determined by how
large the debt is; higher debt means a higher interest
rate.
The other important variable is technology. This
is what will increase our long term economic output.
ICAART 2021 - 13th International Conference on Agents and Artificial Intelligence
630
It is advanced by public and government spending.
This is where the critical tradeoff is: we want to spend
enough to maximize investment and long term
growth, but not so much that we take on too much
debt. If debt gets large enough that it’s interest
payments cannot be completely paid from spending,
and it will directly technological advancement
negatively.
3.2 Gameplay
The actual gameplay consists of adjusting policies
over game cycles. A game cycle consists of the
following:
1. Run the Solow-Swan Growth model.
2. Run the AS-AD model and show the user its
indicators.
3. Make policy changes.
4. Run the AS-AD model again and show the user its
indicators.
At the beginning of a cycle, the player is presented
with the choice of running the simulation manually or
with the AI. If they choose manual, then they are
shown current economic indicators, and must decide
what they wish to change (if anything).
The player has control over 4 different policy
levers:
Taxation
Spending
Bonds owned
Reserve requirements
When prompted how they would like to change each
policy, players simply type their responses into the
console. When they have made their decisions, the
new policies are applied and the player is shown new
economic indicators which reflect their changes.
In order to use the AI component, we choose how
many cycles we want to simulate. The game then runs
as many game cycles as is specified with the AI
making the policy change decisions instead of the
player.
The program is written in Java and should work
on most major Operating systems, dependencies are
managed with Gradle.
3.3 Illustrative Example
The game is currently played at the command line.
The intention is that this simulation can be a
component of a larger game that involves an economy
at an underlying component. This is the case, for
example, in the Democracy series of games.
When the game is launched, the player is
prompted as follows:
Pressmformanualplay,pressaforaiplay
If manual play is selected, then a manual cycle starts
with a description of both the Solow Model and the
AS-AD model:
*SolowModelInformation*‐
PopulationGrowthrate:0.0
TotalOutput:564.6216173286173
*ASADModelInformationpreadjustment*‐
*OutputGapData*‐
LongRunAggregateSupply:564.62161732861
Additional information is displayed, related to
aggregate demand, taxation, government spending,
inflation and debt. The player is then given options
for policy adjustment:
Selectoptionforpolicyadjustment:
tfortaxes
gforgovernmentspending
mformoneysupply
rforreserverequirement
The player can select any option. If they select
government spending, for example, they will be
prompted as follows:
Howmuchdoyouwishtochangespendingby?
Sizeofspendingchangeneededtoclosethegap:
14.003562437128632
A number can be entered to increase or decrease
government spending, and the simulation will then
print out all of the AS-AD model information again.
The second line indicates how much change is
required to make a difference in the economy; this
information is included to help new users. The new
model produced after any action will show changes
that have occurred as a result of the action taken.
If the player starts off by selecting the AI option,
they will be prompted as follows:
EnternumberofcyclesforAItorun
The given number of cycles will be simulated, using
one of the AI models. Results are displayed in the
same format as they are for the manual run.
3.4 AI Control
We now briefly describe the implementation of each
AI controller, starting with the rule-based reasoner.
Using Game AI to Control a Simulated Economy
631
3.4.1 Rule-based Reasoner
The rule set used for the rule-based reasoner is
simple. We describe the basic rule set here
informally. One module of the rule set addresses the
situation where the public is rich.
(PubBalance > GovBalance)
PublicIsRich
(PublicIsRich & OutputGap > 0)
IncreaseSpending
(PublicIsRich & OutputGap > 0)
DecreaseSpending
(PublicIsRich & OutputGap < 0)
BuyBonds
(PublicIsRich & OutputGap < 0)
DecreaseReserve
These rules are applied to determine possible
changes. The overall goal here is to reduce the output
gap. As such, when there are two different changes
are triggered, selection between the two is based on
which action will yield the biggest change.
A similar set of rules is included for handling the
situation where the public balance is less than the
government balance.
3.4.2 Fuzzy Logic
The fuzzy logic module operates similarly to the rule-
based reasoner. However, rather than simply
comparing the public balance and the government
balance, we use fuzzy valued variables to describe the
properties of the model. The fuzzification of the
variable debt, for example, is defined in an external
file as follows:
TERM debt :=
(balanceHighNegative,1.0)
(balanceNeutralNegative,0.0)
In other words, we use defined terms such as
HighNegative and NeutralNegative as fuzzy-valued
properties. We have corresponding defuzzification
definitions for our output variables:
TERM surplus :=
(spendingHighNegative,1)
(spendingNeutralNegative, 0);
These blocks let us assign numeric values to concepts
like “high spending” or “neutral spending.” Overall,
the fuzzy logic AI differs from the rule-based
implementation in that we do not have to hard code
increases or decreases in values; the actual numeric
values are determined by the fuzzy membership
values.
3.4.3 Goal-oriented Behaviour
When Goal-Oriented Behaviour is used, the AI loops
through all nine possible policy changes and tests the
result for each. The software implements this
capability through two functions:
tryOption(i): Returns the complete ASAD model
that will be generated by option i.
getEconomicHealth(): Uses a formula to estimate
the economic health, after selecting a particular
option.
Hence, the Goal-Oriented behaviour option
essentially uses a Markovian approach to make each
choice without looking ahead beyond immediate
effects.
3.4.4 Regression
The Machine Learning Regression AI uses a classifier
learned from past data, using the Weka
implementation for regression. The classifier looks at
the current state of the AS-AD model, and chooses
the policy change that best fits the current state by
classifying it with respect to the training data.
4 TESTING
4.1 Simulation Testing
Before discussing the performance of the game and
the AI player, it is important to note that the
simulation itself was extensively tested to ensure that
it worked correctly. In other words, each action that a
player can take was tested to determine if the
simulated economy changed in the expected manner.
Consider, for example, increasing taxation. The
expected outcomes of increasing taxation include the
following:
Equilibrium output should decrease
Government Balance and Total Government
Balance should increase
Inflation Rate and Average Inflation Rate should
decrease
Price Level should decrease
All of these changes were validated in our testing.
Similarly, it was verified the expected changes occur
for all of the other changes a player can make in the
system.
ICAART 2021 - 13th International Conference on Agents and Artificial Intelligence
632
4.2 Preliminary User Testing for
Manual Gameplay
A small collection of four players tested manual
gameplay with an early version of the software.
Several superficial changes to the software were
made to address their concerns.
The current block-based layout of the AS-AD
display was developed in response to readability
concerns.
The output gap was included in the AS-AD model
display for different parameters, as it helps users
determine how much they should change property
values.
Cycle numbers were added to the output.
The AS-AD model and the Solow model are
actually both displayed in sequence, as users
found it confusing when they had to choose
between the two models.
Overall, our users found the final interface presented
here to be understandable and usable for gameplay.
Of course, a larger user study is required for further
improvement and validation.
4.3 AI Testing
Each AI algorithm was tested to determine how well
it manages the simulated economy. The focus of this
testing is on the Long Run Aggregate Supply
(LRAS). This is an indicator that essentially measures
the long-term economic output; when running the
simulation, a high LRAS score indicates strong
economic performance.
For comparison, we first ran a baseline test, in which
10 cycles run without any manipulation of the
economy. We then ran the same simulation for 10
cycles with a human player, as well as each AI
algorithm. The results are given in Table 1.
Table 1: Comparing Player Performance.
Pla
y
er Final LRAS
Baseline 784
Human 899
Rule-
b
ased Reasone
r
892
Fuzzy Logic 879
Goal-Oriented Behaviou
r
860
Machine Learnin
g
834
Before analysing the results, we emphasize this is
a simple test of a prototype system. More detailed
testing and refinement would be beneficial.
Nevertheless, several interesting observations can be
made about the test results:
All of the AI players outperform the baseline; it
appears that manipulations made are beneficial.
All of the AI players perform worse than a human
with game experience.
The best AI player is the rule-based version, while
the worst is the machine learning version.
The results here are somewhat disappointing in terms
of the machine learning approach, as we expected it
to be more effective. Instead, the most effective AI
follows very simple rules that essentially encode
human knowledge. However, on the whole, the resuls
are encouraging. It is easy to bankrupt the economy
through poor play, but this did not happen. All of our
AI players were better than a null agent, suggesting
that they each have some value as opponents.
5 DISCUSSION
5.1 Economic Simulation
The simulation used in this prototype is unique from
the perspective of gaming. We are not aware of any
games that allow the user to “play” as the central
bank. We are also not aware of any other game where
the Solow Model and the AS-AD model are
combined into a single macroeconomic simulation.
The result is a complex model, with many variables
that interact in a manner that is difficult to predict. As
such, we suggest that this simulation actually
provides an interesting framework that can be used in
games involving commerce.
Hence, from the perspective of pure gameplay, we
argue that our prototype has been successful. We also
remark that, while our focus has been on treating the
simulation as a game, it can also be used as a tool for
studying real economies. As such, it is possible to
actually experiment and learn about effective
economic policies through this simple game.
5.2 Game AI
In terms of AI, our focus here has really been on
experimenting with different kinds of AI players.
Informally, we had two main goals:
The AI players should be based on simple
techniques that are common in the game AI
community.
The AI players must perform better than a static
or random player.
We were able to achieve both of these goals and
implement four different approaches for
experimentation.
Using Game AI to Control a Simulated Economy
633
In terms of the comparative results, our work is
consistent with the standard perspective for AI in
games. While sophisticated AI based on machine
learning is incredibly useful for many practical
problems, it is often the case that these methods are
not as beneficial in creating interesting AI opponents.
In this paper, we have seen that the best AI is also the
simplest: a player that uses a simple rule-based
system.
Hence, even when a game is based on a complex
simulation, our work suggests that one need not
employ complex AI to create competitive and
believable opponents.
5.3 Future Work
There are several directions for future work. Clearly,
one direction is pure software development. While
our simulation is interesting to study in the abstract,
it is not currently a very interesting game. In order to
improve this situation, it needs to be packaged as a
framework that can be implemented in a more
stimulating game environment.
Ideally, our simulation will be delivered as a
component for games that involve gameplay beyond
economic simulation. Hence, the economy
underlying a complex simulation game will be
simulated in a realistic manner and controlled by an
appropriate artificial agent. We are specifically
interested in packaging our simulation as a
component of educational software, in which people
can use the simulation to learn about the nature of
economic decision making and manipulation.
In terms of the AI, we intend to implement and
test additional approaches. For example, we have not
included any AI methods based on rigorous
Knowledge Representation formalisms, nor have we
included any neural-network based Machine
Learning. It would be valuable to test such methods
as part of a complete implementation. Moreover, we
intend provide more detailed testing with a range of
economies. At present, we believe that the poor
performance of the ML agent is partially due to the
paucity of starting data. We would therefore like to
provide more a detailed comparison with more initial
data, and more varied test cases. We leave this
detailed comparative analysis for future work.
6 CONCLUSION
This paper is a report on work in progress, focused on
using AI agents to control a simulated ecnomy in a
game setting. We have described a realistic economic
simulation that can be controlled as a playable game.
The player of the game acts as the central bank (or
government) and takes actions to try and improve the
economic output. We have defined four different AI
algorithms for playing the game, based on standard
AI methods used in the gaming community. While the
AI players do not outperform a skilled human at
present, the AI players do make decisions that
improve economic output when compared to a simple
baseline controller.
Overall, this work demonstrates that simple game
AI methods can effectively control a simulated
economy. The comparison between different AI
methods is still at an early stage, but preliminary
results suggest that a simple rule-based system
provides better performance than more complex
methods, including an approach based on machine
learning.
REFERENCES
Donghan, C., Hui, Y., and Longfei, G., "A Generalized
Solow-Swan Model", Abstract and Applied Analysis,
2014.
Frank, E., Hall, M. and Witten, I. The WEKA Workbench.
Online Appendix for "Data Mining: Practical Machine
Learning Tools and Techniques", Morgan Kaufmann,
Fourth Edition, 2016.
Georgeff, M. and Lansky, A. Reactive reasoning and
planning. In Proceedings of AAAI, 677–682, 1987.
Köse, U. Developing a fuzzy logic based game system.
Computer Technology and Application. 3, 510-517,
2012.
Millington, I. AI for Games. CRC Press, 2019.
Ragan, C. Macroeconomics. Pearson Canada, 2020.
She, Y. and Grogono, P. A procedural planning system for
goal oriented agents in games. In Proceedings of the
22nd Canadian Conference on Artificial Intelligence,
245–248, 2009.
Winterstein, D. “A Simple Intro to Gaussian Processes (a
Great Data Modelling Tool).” Platypus Innovation: A
Simple Intro to Gaussian, Platypus Innovation, 2016.
ICAART 2021 - 13th International Conference on Agents and Artificial Intelligence
634