ware development life-cycle has grown shorter over
time, a steady system state is no longer a guarantee.
Since the publication of Salfner et al.’s survey, it
has been pointed out that while many effective tech-
niques for predicting failure exist, these techniques
are too difficult to maintain and consequently are not
being used. In 2015, Irrera et al. published a frame-
work called the Adaptive Failure Prediction (AFP)
framework for dealing with this problem that auto-
mated the process of retraining a failure prediction
algorithm after an underlying system change (Irrera
et al., 2015). After a system change, a virtual clone of
the production system is made and then load against
this clone is generated. After the cloned system is suf-
ficiently loaded, faults are injected which quickly lead
to failure. This failure is captured, labeled, and used
to train a new predictor. The new predictor is com-
pared against the old one and replaces it if the new
outperforms the old.
The target system for this research is a Microsoft
Windows active directory domain services server and,
as a result, full-stack authenticated session traffic is
required in order to sufficiently load the service. In
this work, we seek to enable the generalization of the
AFP framework and were unable to find a sufficient
load generation tool to carry out the automated re-
training of a predictor defined in AFP.
2.2 Network Load & Traffic Generation
Many tools exist for the purpose of generating net-
work traffic. Generally, these tools are classified
into three categories: application-level, flow-level,
and packet-level generators (Botta et al., 2012; Zach
et al., 2013). Application-level generators emulate
traffic produced by applications on a network, flow-
level generators replicate actual traffic using statisti-
cal modeling, and packet-level generators create and
inject packets into the network. Network traffic gen-
erators are further classified as open- or closed-loop.
Open-loop generators use a packet arrival model for
packet timing, whereas closed-loop generators wait
for a response to a sent request prior to sending the
next request (Weigle et al., 2006). Unfortunately, as
far as we can tell, none of the tools available generate
the necessary interaction with a deployed Microsoft
Windows active directory environment necessary to
facilitate the implementation of the AFP framework.
Active directory implements the Kerberos authentica-
tion protocol in Windows domains and due to its cryp-
tographic nature cannot be tested against replayed or
random traffic; rather, a sequence of valid and invalid
requests and responses are necessary to stress test this
framework. Indeed, multi-step “handshakes” are nec-
essary for rich service delivery and this capability is
not realized by the current tools with any degree of
modularity or extensibility.
A brief review of the traffic generators consid-
ered when researching this problem follows. The
Distributed Internet Traffic Generator (D-ITG) (Botta
et al., 2012) is, as its name implies, a distributed
traffic generator capable of performing application,
flow, and packet-level generation using both open-
and closed-loop operations – sessions are initiated at
specific time intervals and, within each session, new
requests are not sent prior to receiving a response to
the previous request. Sadly, D-ITG currently only
supports TCP, UDP, ICMP, DNS, Telnet and VoIP
which does not suit our needs.
NTG (Zach et al., 2013) is an application-level,
distributed network traffic generator which is both
open- and closed-loop. A key feature of NTG, as it
relates to our problem, is that it interacts with existing
network services. Unfortunately, it is only limited to
web, mail, and multimedia servers/services, which is
insufficient for our purposes.
Swing (Vishwanath and Vahdat, 2009) is a flow-
level, closed-loop traffic generator that observes live
network traffic, extracts distributions from the traf-
fic, and generates new traffic in a manner consistent
with the observed traffic distributions. While this tool
provides the ability to generate statistically-realistic
traffic from generators to listeners across a link, the
lack of both two-way traffic and interaction with ex-
isting services (specifically authentication services)
does not satisfy the requirements for our problem.
A final tool worth mentioning, while not a net-
work traffic generator, is Microsoft’s Active Direc-
tory Performance Testing Tool (ADTest)
†
. Offi-
cial Microsoft documentation is limited, however
in (Bijaoui, 2011; Morowczynski, 2014; Suyanto
and Tiwari, 2010a; Suyanto and Tiwari, 2010b) we
find that ADTest assesses the ability of Microsoft
2003/2008/2012 Active Directory Lightweight Direc-
tory Services (AD LDS) servers to add organization
units and users, and make various changes to Active
Directory to aid in developing requirements for an AD
LDS deployment. It is important to note that Mi-
crosoft no longer supports this tool (Morowczynski,
2014). Also of importance, ADTest is not capable
of testing other services that rely on active directory
domain services for authentication (e.g. RDP, SMB,
etc), nor can it be extended to do so, and is therefore
insufficient for our goals.
While all of these tools are, in general, sufficient
for generating traffic in a network, they do not gener-
†
https://www.microsoft.com/en-us/download/
details.aspx?id=15275
SIMULTECH 2016 - 6th International Conference on Simulation and Modeling Methodologies, Technologies and Applications
196