impact than the previous existing trends. Going from
277 wrong predictions at no noise to 297 at maximum
noise it nearly tripled the previous prediction error
change with a better fit.
For the large scale simulation repeated predictions
and real trials appeared to be a must.
2.4.5 Large-scale Experiment: Repeated
Predictions & Randomized Schedules
One additional change was made in addition to the
previously discussed ones. Due to its low perfor-
mance and unlikeliness of being the best on its own
merit, SNPR traders were taken out of the pool of pos-
sibilities. This resulted in a number of changes to the
starting parameters of the simulation:
• The total number of participating traders de-
creased with the number of available strategies to
4 ∗ 3 = 12 for the prediction trials and 12 + 2 for
the real trials
• The total number of possible trader combinations
dropped from 455 to 55
• The maximum possible noise probability lowered
to 66.6%
Due to how passive and nonadaptive SNPR traders
have proven to be, this change had an additional bene-
ficial effect. The three advanced algorithms were now
in closer contest with fewer bystanders, meaning that
the profit ratio being measured is closer to 1. Previ-
ously all of them had a fair amount of extra profit just
by taking advantage of bad SNPR trades.
This large trial was done on 10 different and ran-
domized order schedules. In one of the schedules
the randomizer produced an unfavourable schedule,
resulting in few to none trades in most of the trials
and as such the data from that was discarded. The
other schedules were tested at 14 distinct noise prob-
abilities in the range of 0% − 65% with increments
of 5%. Each probability point was tested for all 55
trader combinations and each trader combination had
a set of 50 prediction subtrials and 50 real subtrials.
Before the summaries relating to the main hypoth-
esis of this research are presented, a slight correction
in evaluation methods must be made. Previous, less
exhaustive experiments did not show the phenomena
presented below in an impactful way when plotting
profit so the credibility of using least squares line fits
for them has not changed. For this subsection how-
ever, the least squares fit on its own was no longer
an accurate enough tool. The market average profit
should, under every condition, stay constant. In cases
where a line fit would indicate this not being the case,
the line fit is wrong. The market average does not
change with noise and it was taken from all trader
combinations equally for each probability. The target
of analysis is the difference in slopes between the two
lines of market average and enhanced trader average.
Analysis of cases where the market average was not
flat was problematic - though less so than it appears
due to the comparison being point-wise rather than
line-wise. Still, the least-squares fit in some complex
schedules should be taken with a grain of salt.
The observed phenomena still hinted at being the
closest to fitting on a line. Fits of higher degree poly-
nomials were attempted but the higher-rank coeffi-
cients were close to 0. Even after increasing pre-
diction accuracy some outlier removal steps still had
to be taken, especially with the re-inclusion of more
complex order schedules. The data range has a com-
plex structure with traders sometimes earning extraor-
dinarily high profits - outliers on the upper end were
far more common than those on the lower end. Out-
liers were removed with a method common in statis-
tics but with slightly different parameters. Generally,
points outside 1.5 times the inter-quartile range are
considered outliers. For the purposes of not losing
too much important data - it should be noted that even
these outliers awerere a result of 50 + 50 individual
data points - this interquartile range requirement was
loosened. The central range was the interdecile range
(between 10% and 90%). The multiplier for accept-
able points was 1 times this range from the edges.
Figure 3 shows the line fits on all schedules. The
colour of the line indicates how good the line fit is; a
darker colour has closer to 0 residuals. The lightest
colour was set as the maximum residuals out of the 9
line fits.
The majority of trends displayed a clear down-
wards slope, among which were the three closest fits.
One third of the studied order schedules displayed an
upwards slope of moderate uncertainty. These trends
are closely paired with the prediction accuracy graphs
in Figure 4, which shows four lines with a downwards
slope where upwards was the expected direction -
more noise meaning more prediction errors. Three of
those coincided with the upwards profit schedules and
one poor line fit of prediction accuracy had a down-
ward slope on the profit graph.
The overall result of the large-scale experiment
supported the research hypothesis. The majority of
order schedules tested show an above-average profit
earned from good predictions that steadily decreased
over increasing prediction noise in strategic informa-
tion. Figure 5 is a visualization of distribution dif-
ferences in the trial profits. It presents comparisons
through nonparametric Wilcoxon tests of the distribu-
tions involved in the nine order schedules.
ICAART 2021 - 13th International Conference on Agents and Artificial Intelligence
202