Advancing Protocol Fuzzing

for Industrial Automation and Control Systems

Steffen Pfrang, David Meier, Michael Friedrich and J

urgen Beyerer

Fraunhofer IOSB, Fraunhoferstr. 1, 76131 Karlsruhe, Germany

Keywords:

Security Testing, Fuzzing, Network Protocols, IACS, Industrial Automation and Control Systems, Vulnerabi-

lities, Device Under Test.

Abstract:

Testing for security vulnerabilities is playing an important role in the changing domain of industrial automa-

tion and control systems. These systems are increasingly connected to each other via networking technology

and are faced with new cyber threats. To improve the security properties of such systems, their robustness

must be ensured. Security testing frameworks aim at enabling the assurance of robustness even at the time of

development and can play a key role in bringing security into the industrial domain.

Fuzzing describes a technique to discover vulnerabilities in technical systems and is best known from its usage

in IT security testing. It uses randomly altered data to provoke unexpected behaviour and can be used in com-

bination with regular unit testing. Combined with the power of fuzzing, the effectiveness of security testing

frameworks can be increased.

In this work, different fuzzing tools were evaluated for their properties and then compared with the requi-

rements for an application in the industrial domain. As no fuzzer was fully satisfying these requirements,

a new fuzzer, combining the strength of different others, was designed and implemented, and then evalua-

ted. The evaluation includes a real-world application where multiple vulnerabilities in industrial automation

components could be identiﬁed.

1 INTRODUCTION

Security testing is becoming an important part of the

development of industrial automation and control sy-

stems (IACS). Through new trends, like Industrie 4.0

or the (Industrial) Internet of Things, these systems

get increasingly connected with each other, exposing

possible network vulnerabilities. Because many of

these systems were not intended for this connected

world, security testing was formerly neglected.

Fuzzing is successfully used in the blackhat com-

munity for ﬁnding zero-day-vulnerabilities in code

(BlackHat, 2017). In contrast to penetration testing,

using randomly altered data to test an interface of a

test subject does not need a deep technical understan-

ding of vulnerabilities and how to exploit them.

To provide robust and secure IACS for the future,

security testing needs to be performed throughout the

development life cycle, as standards like IEC 62443

are already demanding. To improve and simplify such

testing, highly optimized testing platforms and frame-

works are developed, that are able to carry out prede-

ﬁned tests. But they are still lacking the possibility of

incorporating fuzzing into the test cycles, partly be-

cause the availability of fuzzers for IACS is limited.

Therefore, this work aims at providing an easily ap-

plicable and effective fuzzer to enable developers and

implementers of IACS to test their systems using fuz-

zing techniques.

This paper is organized as follows: Starting with

an introduction to IACS and security testing in section

2, the ISuTest framework is introduced as host for the

newly integrated fuzzing technology. In section 3,

a broad overview of fuzzing characteristics is given.

A comparative study of available network fuzzers re-

sults in a compilation of requirements for our develo-

ped solution. The design of the newly integrated fuz-

zing functionality is being described in section 4. The

evaluation in section 5 is split into two parts: Starting

with a functional requirement evaluation, the section

continues with a practical test of the developed solu-

tion in a test bed revealing several vulnerabilities in

IACS.

Pfrang S., Meier D., Friedrich M. and Beyerer J.

Advancing Protocol Fuzzing for Industrial Automation and Control Systems.

DOI: 10.5220/0006755305700580

In Proceedings of the 4th International Conference on Information Systems Security and Privacy (ICISSP 2018), pages 570-580

ISBN: 978-989-758-282-0

2 BACKGROUND & RELATED

WORK

This section gives a short introduction on the topics

addressed in this paper. It brieﬂy describes the current

state of industrial automation and security testing of

industrial systems. It also describes requirements for

industrial security testing, as well as an example for a

testing framework fulﬁlling these requirements. The

background is concluded by presenting the basics of

fuzzing in security research.

2.1 Industrial Automation and Control

Systems

The modern industry landscape shows a high degree

of automation enabled by industrial automation and

control systems. Nowadays, they are connected via

network technology to enable the exchange of cont-

rol and production data to increase the efﬁciency of

the automation processes. This increasing intercon-

nectivity is made possible by the introduction of stan-

dard IT network technology in the industrial domain.

The old, proprietary bus systems are replaced by com-

munication protocols based on Ethernet. But, as the

industrial domain still has special requirements, spe-

cialized protocols need to be used, that are able to,

for example, provide real-time communication. Ex-

amples for such protocols are Proﬁnet (PNIO) or Et-

herCAT. All of these established industrial Ethernet

protocols have in common that they were designed

with no security considerations in mind. This makes

networks built on these protocols vulnerable and puts

importance on the security properties of the individual

devices used in these networks.

2.2 Security Testing for IACS

Security testing is used to reveal vulnerabilities of a

system. For this, the Device under Test (DuT) is ex-

amined in a test bed. The DuT is provided with in-

put by a testing system and its reaction and behavior

(output) is evaluated. When an unexpected reaction

is registered, further examination is used to determine

which kind of problem was encountered and if this

problem is caused by a vulnerability.

When security testing is performed without kno-

wledge about the inner state or function of the DuT,

this is called black box testing. The testing can be furt-

her classiﬁed into positive, providing the system with

input that should lead to an expected reaction, and

negative testing, where the system is provided with

unexpected input and is expected to maintain normal

functionality.

Vulnerabilities can be classiﬁed by their origin as

elaborated in (Pfrang et al., 2017): Vulnerabilities

that originate from the design, the implementation and

from the conﬁguration of a system.

Based on the IEC 62443 standard on industrial se-

curity, the ISASecure EDSA (Embedded Device Se-

curity Assurance) certiﬁcation program (ISA Security

Compliance Institute (ISCI), 2016) is a well-known

standard in the industry and intended for the testing of

IACS. It includes multiple areas of requirements, in-

cluding secure development (SDA-E), functional se-

curity (FSA-E) and device robustness testing (ERT).

For the device robustness testing, the Communication

Robustness Testing (CRT) deﬁnes test cases for mul-

tiple network protocols.

The special requirements and conditions of the in-

dustrial domain, as described in the previous section,

need to be thoroughly considered in the security tes-

ting of IACS, as well as requirements originating

from security standards like the ISASecure EDSA. In

(Pfrang et al., 2017), we gave an overview of multi-

ple security testing frameworks intended for the ap-

plication in the IACS domain, formulated key requi-

rements and, ﬁnally, introduced ISuTest, a new and

modular security testing framework for IACS that is

intended to fulﬁll these requirements.

2.2.1 Industrial Security Testing Requirements

Following a review of existing security testing sys-

tems and test speciﬁcation from standards like the IEC

62443, we formulated key requirements for an indus-

trial security testing system. Such a system needs to

be able to test arbitrary IACS for vulnerabilities (I),

independent from type, manufacturer or technologies

used in the IACS. The system also needs to be modu-

lar and extensible (II) to account for future application

within these evolving technologies. The modularity

includes the possibility to test different network pro-

tocols in different layers (IIa) and to test the different

possibilities of input and output of data an IACS has

(IIb). The results of such tests need to be evaluated

with suitable methods (IIc). Another important re-

quirement is a precise way to describe security tests

through declarative test scripts (IIIa) that allow for

an arbitrary combination of input and output hand-

ling (IIIb), as well the evaluation of the result against

an expected behavior (IIIc). As there is the necessity

to execute a high number of individual tests, the sy-

stem needs to support the fully automated execution

of tests (IIId) that also need to be reproducible (IIIe).

2.2.2 ISuTest Framework

The ISuTest framework is a novel security testing fra-

mework for IACS that is intended to fullﬁll the re-

quirements presented in the previous section. ISuTest

is a highly modular security testing framework that

is adaptable to different test environments and scena-

rios. It is able to incorporate different technologies

and therefore able to test systems of very different

forms.

Figure 1: Architecture of the ISuTest framework.

As shown in ﬁgure 1, the core of the framework

is formed by the scanner, handling the whole testing

logic, and the result component that is responsible for

the creation of ﬁnal test results. The extensible part

of the framework is consisting of test scripts, modu-

les and evaluations. Test scripts specify individual

test cases, including preparation, execution and result

evaluation. The test scripts rely on modules to test sy-

stems and are executed by the scanner. These modules

provide the functionality to interact with a DuT, inde-

pendent of the used communication technology. For

the creation of results, different evaluators can then

interpret the reaction of the DuT to detect malfuncti-

ons and vulnerabilities. The framework is compatible

with the OpenVAS Scanner Protocol and can there-

fore be used with this vulnerability scanner.

2.3 Fuzzing

Fuzzing characterizes a negative testing methodology

driven by random alteration of inputs. The test subject

is tested by providing predominantly invalid, unex-

pected input to it. This approach is highly automata-

ble and offers a high chance of ﬁnding unknown vul-

nerabilities since it explores unexpected input. But

fuzzing can only cover a small subset of all possible

input in a feasible amount of time. Consequently, one

of the main goals is to select the fuzzed data as ef-

ﬁciently as possible, so that the fuzzed data covers a

subset with as high probability for detecting vulnera-

bilities.

Figure 2: The phases of a fuzzing process.

The fuzzing process, as shown in ﬁgure 2, is divi-

ded into three phases: Generation of data, transmis-

sion of data and monitoring the DuT. Data is created

and altered by the fuzzer. The alteration of the data

should ensure that the data is still processed by the

test subject, i.e. the alterations should not destroy the

format completely. Altered data is transmitted to the

test subject via arbitrary interfaces. Whilst being tes-

ted, the subject is monitored in order to detect anoma-

lies in its behaviour. Detection of crashes, unavailable

services or unexpected behaviour is an evidence for a

successful fuzzing test.

3 ANALYSIS

In this section, we provide a terminology for the clas-

siﬁcation of properties of fuzzers. We then conduct

a comparative study of available fuzzers and motivate

why there is the need of a new fuzzer for IACS.

3.1 Classiﬁcation of Fuzzers

In order to compare different approaches and types of

fuzzers, it is useful to describe each of them using ort-

hogonal categories. In the following, we present four

categories to characterize fuzzers which are derived

from (Sutton et al., 2007) and (McNally et al., 2012):

Input crafting, operation mode, extensibility and or-

ganizational features.

3.1.1 Input Crafting

Crafting of fuzzed input data is realizable in many dif-

ferent ways. Depending on the scenario, data is either

generated based on knowledge about the data format

or mutated during transmission. Additionally, a fuz-

zer either generates the input data whilst testing the

DuT (online) or precomputes the fuzzed data (ofﬂine).

The main reasons for ofﬂine generation of data are ti-

ming requirements of real time components used in

IACS.

3.1.2 Operation Mode

In order to fuzz stateful protocols efﬁciently, the fuz-

zer must track the state of the protocol being tested.

Rearranging fuzzers are sometimes referred to as be-

havioural fuzzers. They do not fuzz data but data se-

quences which can trigger a different kind of faults

in the DuT compared to the data fuzzing. Rearran-

ging fuzzing is only relevant for stateful protocols be-

cause stateful protocols can have a wrong order of va-

lid data. Adaptive fuzzers use the reaction of the DuT

for adjusting their input in order to increase the proba-

bility of provoking an anomaly. If fuzzers are equip-

ped with special means to monitor IACS outputs other

then Ethernet-based ones, they provide IACS monito-

ring.

3.1.3 Extensibility

This category deals with the extensibility of the fuz-

zer. If a fuzzer is learning, it is able to extend its

knowledge of protocols and data structures automa-

tically by observing a given communication record.

In contrast, other fuzzers rely on Expert knowledge.

Some fuzzers are tailored to a set of speciﬁc proto-

cols. Others offer the opportunity to extend the pro-

tocol knowledge. From a technical point of view, it is

important on which network layer of the ISO/OSI mo-

del (Zimmermann, 1980) the fuzzers are based. There

are three common options for network fuzzers: Ether-

net (layer 2), IP (layer 3) or TCP/UDP (layer 4). Par-

ticularly, if protocols need to be extended or learned,

an Ethernet-based protocol cannot be implemented in

an IP-based fuzzer.

3.1.4 Organizational Features

Some organizational features are listed in this cate-

gory. They are particularly important if an existing

solution should be extended. So here is listed whet-

her the source code of the fuzzer is available online

and under which license it can be used. Additionally,

if known, the programming language and the time of

the last update of the fuzzer are registered.

3.2 Comparative Study of Fuzzers

Based on the classiﬁcation provided in section 3.1, we

performed a comparative study on available fuzzers.

The results are presented in table 1 and described in

this section. The selection criteria for the studied fuz-

zers were the following: All fuzzers are either open

source or were described comprehensively in scienti-

ﬁc papers. Additionally, all fuzzers provide the possi-

bility to create fuzzed network packets.

Two learning fuzzers, Pulsar (Gascon et al., 2015)

and AutoFuzz (Gorbunov and Rosenbloom, 2010),

use machine learning algorithms to infer the ﬁnite au-

tomaton of a stateful packet and heavily depend on

the completeness of provided data. While AutoFuzz

is not maintained since 2010, Pulsar got recent upda-

tes. Nevertheless, Pulsar crashed in our tests when

using it with other data then with the included exam-

ples. The maintainers conﬁrmed this behaviour.

The General Purpose Fuzzer (GPF) learns proto-

col formats from recorded trafﬁc and displays the pro-

tocol format in a human-readable way. Based on the

graphical representations, the fuzzer is conﬁgurable

to alter the displayed data.

Lzfuzz (Bratus et al., 2008) uses the General Pur-

pose Fuzzer (GPF) as its fuzzing engine. It uses a

compression algorithm to learn data structures and

fuzzes data mutatively as a man in the middle when

the data is transmitted. Lzfuzz can monitor the DuT,

but does not offer generative and rearranging fuzzing.

An example for a commercial fuzzing framework

is Peach (Peach Fuzzer Website, 2016). It is providing

the ability to deﬁne data formats and fuzzing scena-

rios in a xml format. It is available in an open-source

community edition with a reduced subset of functi-

ons. The code of Peach is written in C#.

Another unmaintained fuzzing framework using

data and protocol deﬁnitions based on xml is Snooze

(Banks et al., 2006), which is a research prototype de-

veloped in 2006.

A popular fuzzing framework, which has been ex-

tended by several research projects, is Sulley (Amini

and Portnoy, 2007). It has not been maintained since

the year 2007 but extended in various projects to

adapt it to certain test scenarios. Sulley offers sta-

teful fuzzing of network protocols, using block-based

deﬁnitions to craft data packets and applying an in-

expandable set of heuristics to alter said data. The

Sulley framework only offers TCP and UDP as trans-

port protocols, whereas IACS often use Ethernet-

based protocols. Monitoring of the DuT is supported

by Sulley with automated logging of network trafﬁc

and storing it. After every fuzzing step, Sulley re-

sets the connection and iterates through its monitors

before continuing fuzzing the DuT in order to ensure

a well-deﬁned state of the DuT before sending fuzzed

data. This feature is not conﬁgurable and causes a low

testing rate of one fuzzed data packet per second. Sul-

ley uses an outdated data format to illustrate the proto-

Table 1: Network fuzzer comparison.

Pulsar

Autofuzz

GPF

lzfuzz

Peach

Snooze

Sulley

ProFuzz

MTF

PropFuzz

Ideal

Input crafting

Generative X X X X X X X X

Mutative X X X X

Online X X X X X X X X X X

Ofﬂine X X X X

Operation mode

Stateful X X X X X X

Rearranging X X X X

Adaptive X X X X

IACS monitoring X X

Extensibility

Learning X X X X X X (X)

Expert Knowl. X X X X X

Protocol extens. X

X X X X X X

ISO/OSI layer 3 4 3 3 2 4 4 2 4 4 2

Organizational

Source code X X X

– X – X X – – X

License BSD n/a GPL – MIT – GPL GPL – – Open

Language Pyth./R Java Pyth. n/a C# Pyth. Pyth. Pyth. Pyth. Pyth. –

Last updates 2017

2010 2007 2008 2016

2006 2015

2012 2016 2017 –

via archive.org,

known bugs, see text,

Community Edition,

ofﬁcially unmaintained

col state machine which cannot be opened easily, im-

peding development of individual fuzzers within the

framework.

ProFuzz (HS Augsburg, 2012) and Modbus/TCP

Fuzzer (MTF) (Voyiatzis et al., 2015) are fuzzers dis-

tinctly developed for fuzzing automation protocols.

They are targeted to one speciﬁc protocol, Proﬁnet

and respectively Modbus, and are not extendible for

usage with other data formats.

PropFuzz (Niedermaier et al., 2017) is a fra-

mework using statistical computation to analyze the

structure of proprietary protocols during the DuT’s

communication with the vendor’s integrated develop-

ment environment (IDE). It is able to fuzz such com-

munication, which has to be based on TCP. This ge-

nerates a limited range of fuzzed data since the input

is limited to only the valid communication with the

IDE.

Concluding, none of the considered fuzzers and

fuzzing frameworks shown in table 1 ﬁt our scenario

perfectly and include the properties required for pro-

viding an efﬁcient and extendable fuzzing framework.

3.3 IACS Fuzzer Requirements

Following the comparative study of different fuzzers

presented in the previous section and based on the re-

quirements for security testing of IACS described in

section 2.2.1, the following requirements for an ideal

fuzzer of IACS can be formulated.

(A) The fuzzer must be able to use existing protocol

deﬁnitions and therefore be generative. This ena-

bles the integration of expert knowledge into the

fuzzing process for enabling the test of arbitrary

systems based on varying technologies.

(B) The fuzzer must be able to craft the desired input

online and ofﬂine. Online input crafting is needed

to be able to exercise adaptive fuzzing, whereas

ofﬂine crafting might be needed to prepare input

to achieve high transmission rates.

of the network protocols. There are many state-

ful industrial protocols and the fuzzer needs these

states to be able to achieve a high coverage of pos-

sible device conditions.

(D) The fuzzer must be able to use knowledge of the

protocol procedures to create invalid protocol se-

quences, enabling fuzzing by rearranging (beha-

vioral). This increases the effectiveness in the tes-

ting of network protocol implementations.

(E) The fuzzer must be adaptive in its interaction

with the examined DuT. This enables the fuzzer

to react on the responses of the device and is nee-

ded to reach certain states in stateful protocols.

(F) The fuzzer must be able to monitor the DuT as

thoroughly as possible, i.e. on all available inter-

faces. Especially IACS can exhibit multiple inter-

faces of different nature, like network or electrical

I/O.

(G) The fuzzer must be extensible to enable the adap-

tion to different test scenarios and different net-

work technologies. The extensibility includes

simple ways for adding new protocols by expert

knowledge, and optionally by automated learning.

Source code availability and suitable, open licen-

sing is also part of the extensibility properties of a

fuzzer.

(H) The fuzzer must be able to craft network pac-

kets on the Ethernet-Layer (ISO/OSI layer 2), to

enable the fuzzing of industrial Ethernet protocols

that are often used in the IACS domain.

(I) The fuzzer shall create reproducible test cases and

data. This is important for the availability of repe-

ated testing in order to localize found vulnerabili-

ties and test mitigations.

An ideal fuzzer for IACS needs to meet all these

requirements. Its properties, resulting from the requi-

rements, are also depicted in table 1. The results in

the table also show, that the examined existing fuzzers

cover only a subset of the presented requirements. In

order to offer an improved and extendable solution,

this work aims at providing a fuzzer more suitable to

the IACS domain.

4 DESIGN

Based on the identiﬁed requirements, we present in

this section a generic fuzzer which meets the require-

ments as well as allows the exchanging of its compo-

nents in order to extend or replace functionality of the

framework. Integrated in the ISuTest framework, the

fuzzing framework is composed of different ISuTest

modules: data generation, protocols, data transmis-

sion and monitoring modules. The modules and their

interaction are shown in ﬁgure 3.

Figure 3: Architecture of the fuzzing framework.

The ISuTest scanner interprets a test script which

deﬁnes a fuzzing test. It initializes the fuzzer accor-

dingly to the speciﬁed testing functionality. On the

one hand, the fuzzer makes use of the heuristics to ge-

nerate the requested input data and puts resulting fuz-

zing instructions into the queue. On the other hand,

the fuzzer instructs the monitor. Once the fuzzing test

is being started, the protocol instance pulls fuzzing

instructions from the queue and crafts network pac-

kets by using scapy/automata (Biondi, 2010) deﬁniti-

ons. Concurrently, the protocol sends the packets to

the DuT while the monitor observes the DuT’s reacti-

ons.

All tests conducted with the fuzzing framework

must be reproduceable. Therefore, a pseudo random

number generator is used, which provides determinis-

tic values depending on a provided seed.

In the next subsections, the design is being explai-

ned following the phases of a fuzzing process.

4.1 Data Generation

The packet formats for the protocols, which are to be

fuzzed, are encoded in packet deﬁnitions. Modelling

of stateful protocols is done with a ﬁnite automaton.

The automaton must enable state transitions without

sending and receiving the packets which are normally

necessary for state transitions in order to realize re-

arranging fuzzing. For generative fuzzing, the state

transitions must enforce sending and receiving data

according to the protocol. This enables a targeted

fuzzing of protocol states which have to be reached

before fuzzed data is sent.

The data generation itself is twofold: Fuzzing in-

structions are generated in the fuzzer module with the

help of heuristics, and alteration of valid data in the

protocol module within the ﬁnite automaton. The fuz-

zing instructions contain information which ﬁeld is to

be fuzzed and in which manner, i.e. it contains the

value the given ﬁeld is to be applied by the protocol

module. The protocol module uses a packet deﬁni-

tion to craft a data packet containing the altered va-

lue. Lose coupling between fuzzer and protocol with

well deﬁned interfaces in-between makes the frame-

work highly ﬂexible. The fuzzer is also logging seed

values for every generation of pseudo-random values,

enabling to recreation of the same values again.

4.1.1 Heuristics

The fuzzer component uses heuristics to determine

how a ﬁeld of a given data type is to be fuzzed, whe-

reas the protocol module crafts the altered data. The

need for using heuristics results from the impossibi-

lity to test every possible value in a reasonable time.

Therefore, promising values have to be chosen.

The used heuristics rely on experience from previous

projects, e.g. Sulley, and are extendable. Expert kno-

wledge should additionally be used to tailor the heu-

ristics to a given test scenario in order to maximize

the chance for ﬁnding a vulnerability.

The fuzzing framework currently makes use of

four different heuristics. The Integer heuristic pro-

duces numbers like 0, the minimum and the maxi-

mum value as well as boundary cases at power of 2

minus 1. The String heuristic provides, for example,

escape sequences, terminal commands and SQL sta-

tements. The MAC heuristic creates randomized Mac

addresses while the UUID heuristic chooses randomi-

zed Universally Unique Identiﬁers.

4.2 Transmission of Data

The ﬁnite state machine, modeling the protocol to

be tested, alters ﬁelds of given data packets crafted

from the packet deﬁnition according to the instructi-

ons received from the fuzzer. Instructions are trans-

mitted between the fuzzer and the protocol instance

via a queue. This enables the fuzzer and the proto-

col modules to be separate entities within the ISu-

Test framework. The execution of the two entities is

loosely decoupled to enable running them asynchro-

nously and increase overall performance of the tes-

ting framework. Furthermore, the fuzzer and protocol

modules are reusable for other kinds of security tests

within the ISuTest framework.

The communication between the fuzzing frame-

work and the DuT is realized by a communication

module which provides transparent connectivity on

a certain layer. For example, a communication mo-

dule providing a TCP connection handles the esta-

blishment and maintaining of said connection. Based

on that connection layer, the protocol modules only

need to manage data based on the connection and for-

ward it to the communication module.

4.3 Monitoring of the DuT

White box fuzzers have access to the internal state of

the subject under test, e.g. by using a debugger to

observe the reactions of a program to fuzzed input.

Industrial devices however do not offer the possibi-

lity to observe internal reactions to a certain input.

Instead, the state of the DuT has to be inferred by

monitoring interfaces which it provides. This inclu-

des the Ethernet interface over which it communica-

tes with the fuzzing framework, as well as other data

buses and I/O.

By running a known program on the DuT, the ex-

pected output of the DuT can be compared with the

actual output in order to detect vulnerabilities of the

DuT. Monitoring of analogue and digital I/O requires

appropriate measurement and processing of the sig-

nals in order to be logged and used in the fuzzer. In

particular, the rate of false positives arising from me-

asurement inaccuracies must be kept to a minimum.

4.4 Fuzzing Procedure

The fuzzer operates in two modes: synchronous or

asynchronous. When operating synchronously, the

fuzzer checks the state of the DuT after each in-

struction, respectively after each sent data packet. A

more time efﬁcient way of monitoring the DuT’s he-

alth is monitoring it asynchronously, after a bulk of

data packets has been sent. Since the fuzzer and the

protocol is decoupled, the fuzzer requests the health

information about the DuT instead of being notiﬁed

and can only map anomalies to messages which have

been sent since the last check. A tradeoff between

efﬁciency and accuracy is found here in order to in-

crease the number of performed tests to a reasonable

number.

After every monitoring check with a positive re-

sult, a well-deﬁned state must be reached by the DuT.

Resetting the DuT enables the fuzzing to be targeted

to a certain state. The efﬁciency of fuzz testing is in-

creased by selectively fuzzing states of a stateful pro-

tocol instead of sending fuzzed data without conside-

ration of the DuTs state. Without considering the pro-

tocol state machine, a deep exploration of said state

machine is not feasible since most of the fuzzed data

resets the DuT into an initial state.

5 EVALUATION

In this evaluation, the properties of the presented fuz-

zer will be compared to the requirements, as presented

in section 3.3. For a practical evaluation, the fuzzer

was also used to test multiple IACS, which will also

be described in the following sections, including the

test results.

5.1 Requirement Evaluation

The presented fuzzer design uses protocol descripti-

ons, including structure of individual packets and lo-

gic of the protocol sequences (section 4), and is able

to generativly create the test data, satisfying require-

ment (A). The test data can be generate online, during

the the test execution, or ofﬂine, as demanded by re-

quirement (B).

The design enables the fuzzing of stateful pro-

tocols as well as behavioral fuzzing by rearran-

ging packet sequences (section 4.1) and thus satis-

ﬁes requirements (C) and (D). Through the usage of

Scapy/Automata, as well as modules from the ISu-

Test framework, the responses of the DuT are availa-

ble to the fuzzer (section 4.2) and enable the adaption

to these reactions, fulﬁlling requirement (E).

As described in section 4.3, the fuzzer is able to

monitor a wide range of the output possibilities of the

DuT, ranging from network responses to digital and

analog I/O, satisfying requirement (F). This form of

protocol description format also enables the fuzzer to

adapt to new test scenarios, including additional pro-

tocols, as demanded in requirement (G).

The fuzzer is based on open source technology,

enabling open adaption to new technologies (require-

ment (G)). Based on Scapy/Automata, the fuzzer is

able to craft packets from the Ethernet layer upwards

and enabling the implementation of typical automa-

tion protocols (requirement (H)). Requirement (I) sta-

tes the reproducibility of test cases, which the presen-

ted fuzzer can satisfy by logging conﬁguration and

seed values, as described in section 4.1.

5.2 Test Bed Evaluation

The fuzzer has been implemented in Python 3 as ISu-

Test modules. It makes use of Scapy deﬁnitions for

network and automation protocols. States of automa-

tion protocols, in particular PNIO, have been imple-

mented using Automata in Scapy.

Four PNIO bus couplers were selected as DuTs

to perform the evaluation. Bus couplers are used to

operate and control remote periphery in terms of digi-

tal and analog signals. In PNIO terminology, the bus

couplers act as PNIO devices which are managed by

a PNIO controller.

5.2.1 Testing Methodology

The test bed in which the evaluation took place, follo-

wing the ISuTest speciﬁcation, consists of the devices

depicted in ﬁgure 4. Depicted in the center, the indus-

trial network Switch is providing communication.

Above the switch, the infrastructure components

are shown: The Test device, a Linux-based PC, runs

the ISuTest framework. A programmable logic con-

troller (PLC) acts as IO device which monitors the di-

gital outputs of the DuTs. Finally, the Remote control-

lable power supply (RCPS) is used to reset the DuTs

automatically. Below the switch, there are the four

bus couplers labelled as DuT 1 to 4.

Figure 4: Test bed evaluation setup.

Beside the switch, the DuT master is depicted with

a dashed connection. Initially, the DuT master is used

as PNIO controller which runs the automation process

and controls the four DuTs as PNIO devices. The au-

tomation process emits a reference signal on one digi-

tal output of every DuT. It consists of three repeating

cycles in which the ﬁrst one is a 1 and the next two

ones are a 0 each. This cycle is based on the EDSA

CRT deﬁnition, as mentioned in section 2.

In the setup phase, the Test device observes the

communication setup between the DuT master and

the DuTs. During the testing phase, the Test device

spoofs both the MAC and the IP address of the DuT

master while the DuT master gets disconnected phy-

sically from the switch. The spooﬁng enables the Test

device to act as the legitimate PNIO controller for the

DuTs. This is important because it allows for setting

up a valid communication between the Test device

and the DuTs, which ensures that the DuTs process

the fuzzed packets as intended.

5.2.2 Essential Services Monitoring

ISuTest monitors the functionality of the DuTs while

those are being tested. In table 2, the essential servi-

ces of the DuTs 1 to 4 that are taken into account are

listed. A check mark represents the fact whether the

DuT supports the respective service.

Table 2: Essential services provided by the DuTs and chec-

ked by ISuTest.

Service

DuT 1

DuT 2

DuT 3

DuT 4

Ping X X X X

Web server X – X –

Digital Output X X X X

CM Connect X X X X

Ping tests are performed by the ICMP module

from ISuTest. It sends ICMP echo requests and waits

for echo replies. Checking the availability of the web-

based management interface is performed by ISu-

Test’s HTTP module. It requests the home page of

the DuT and checks the HTTP status code.

In order to check for the digital output, the refe-

rence signal is being used. The digital output pin of

the DuT is connected to a digital input pin of the IO

device. The Digital IO module checks for both the

existence of the test signal and its jitter.

Checking for the PNIO-CM Connect functionality

is the most complex part, because it consists of more

than one single request and response packet but in-

cludes different protocols. It makes use of the PNIO

module being described in the following and checks

whether the real-time communication lasts at least

three seconds.

5.2.3 Test Execution

The fuzzing tests have been performed on each DuT

for the following four protocols: PNIO-DCP, PNIO-

CM and PNIO-RT, as an example for industrial au-

tomation protocols, and IPv4, as an example for an

internet protocol.

Table 3 represents an example for a PNIO-DCP

network packet requesting the identiﬁcation of a

Table 3: Example of the fuzzing conﬁguration of a DCP

message requesting the identiﬁcation of a PNIO device

(IdentReq). Length in bit, coverage in %.

Name Len. Heur. Iter. Cover.

FrameID 16 Integer 2000 3,05

ServiceID 8 Integer 256 100

Service-Type 8 Integer 256 100

xid 32 Integer 2000  0,1

ResponseDel. 16 Integer 2000 3,05

DCPDataLen. 16 Integer 2000 3,05

blockType 16 Integer 2000 3,05

dcpBlockLen. 16 Integer 2000 3,05

blockInfo 16 Integer 2000 3,05

nameOfStat. 1152 String 2500  0,1

Sum 17012

PNIO device with the respective nameOfStation as

the identiﬁer. Starting with the name and the length

(in bit) of a ﬁeld, the following columns represent the

applied heuristics, the number of iterations performed

while testing and the resulting coverage of the search

area in percent.

As one can see, the bigger the ﬁelds, the smaller

the coverage of the overall value space. Since there

are deﬁned multiple block types for getting and set-

ting conﬁguration options, the number of fuzzing ite-

rations for the PNIO-DCP protocol multiplies.

Table 4: Excerpt of the fuzzing conﬁguration of a PNIO-

CM Connect request message. Length in bit.

Name Len. Heur. Iter.

args max 32 Integer 2000

args length 32 Integer 2000

max count 32 Integer 2000

offset 32 Integer 2000

actual count 32 Integer 2000

block type 16 Integer 2000

block length 16 Integer 2000

block version high 8 Integer 256

block version low 8 Integer 256

ARType 16 Integer 2000

ARUUID 128 UUID 2000

SessionKey 16 Integer 2000

CMInitiatorMacAdd 48 MAC 2000

CMInitiatorObjectUUID 128 UUID 2000

. . .

Table 4 represents the fuzzing of PNIO-CM as an

example for a stateful network protocol. The depicted

network packet shows an excerpt of a Connect re-

quest from the DuT master to the DuT. The PNIO-

CM-related part of the packet has 80 distinctive ﬁelds

with a total length of 1,232 bit. In order to fuzz all

possible conﬁgurations, one had to fuzz 2

1,232

pac-

kets (≈ 7.4 · 10

370

). Using the heuristics, this amount

of test cases can be reduced to 106,000, which can be

conducted in a reasonable amount of time.

5.2.4 Vulnerability Discoveries

During the execution of the fuzzing tests, we disco-

vered multiple vulnerabilities in each of the DuTs.

On the one hand, there were expected vulnerabilities

which are present in the DuTs because they originate

from weaknesses in the design of PNIO. A prominent

example for this, which is also mentioned in (Pfrang

and Meier, 2017), is a DCP Set request which chan-

ges the name of the PNIO device (nameOfStation).

Since there is limited use in testing if the communica-

tion breaks if the device is renamed to a or b or c, we

limited those test cases.

On the other hand, we discovered vulnerabilities

which are located in the implementation. As an exam-

ple, we will describe a vulnerability in the PNIO-CM

protocol which two DuTs have in common. When

setting up an application relationship, both communi-

cation partners conﬁgure the frequency in which they

exchange real time data messages. The respective pa-

rameter is called SendClockFactor and deﬁned in a 2

byte ﬁeld.

When setting this parameter to the values of 8, 9,

10, 11 or 65524, two DuTs crash and reboot with all

status LEDs ﬂashing shortly. Usually, devices loosing

a connection have to send alarm frames. This does not

work either. The monitoring also conﬁrms that neither

are the devices pingable nor can their web server be

reached. Their digital outputs stop working and the a

CM connect is not possible for half a minute.

In real factories, it is common to ﬁx failures of au-

tomation equipment by power cycling the respective

device. But this vulnerability cannot be ﬁxed by this

means, because the device has lost its device name

which is indispensable in a PNIO communication.

Probably those values cause a memory corruption. In

order to cure this behaviour, one has to reset the de-

vice name using the PNIO-DCP protocol.

6 CONCLUSION

Security testing for IACS is crucial. By detecting vul-

nerabilities and ﬁxing them before they can be abused

by attackers, manufacturers can improve the overall

security of automation systems. The ISuTest frame-

work enables developers to perform security tests on

their IACS automatically. Fuzzing is another, inte-

resting technique to discover vulnerabilities. By ad-

ding this functionality to the ISuTest framework, its

effectiveness in ﬁnding vulnerabilities could be im-

proved extraordinarily.

Within this work, ﬁrst, available fuzzers and fuz-

zing frameworks have been characterized and com-

pared. Then, requirements for effective fuzzing of

IACS, based on existing security testing require-

ments, were developed. By comparing the studied

fuzzers to these requirements, it was shown that none

of the fuzzers fulﬁlled them completely. As a result, a

new fuzzer was developed and described in this paper,

which can be integrated into the ISuTest framework as

well as meet the requirements for an ideal IACS fuz-

zer.

During the evaluation, it was shown that the new

fuzzer is able to fulﬁll these requirements. The fuz-

zer was evaluated further by setting up a test bed and

conducting fuzzing test against multiple Proﬁnet de-

vices. During this evaluation, several vulnerabilities

could be discovered. The fact that two devices suf-

fered from the same vulnerability stresses the impor-

tance of security tests not only for developers but also

for integrators of IACS. As shown, the fuzzing frame-

work proved its usefulness successfully.

6.1 Future Work

During this work, it became clear that future work

should focus on the precise description and classiﬁca-

tion of the vulnerabilities. This is important to be able

to clearly classify detected vulnerabilities and report

them to the manufacturer of the device. This includes

formal methods taking into account the special requi-

rements for IACS.

Further work needs also to be put into the deﬁni-

tion of new protocols following the presented fuzzer

deﬁnition. As the IACS domain consists of highly

heterogeneous technology, it is important to cover as

many technologies and protocols as possible.

REFERENCES

Amini, P. and Portnoy, A. (2007). Sulley: Fuz-

zing framework. http://www.fuzzing.org/wp-content/

SulleyManual.pdf. [Online; accessed 2016-10-23].

Banks, G., Cova, M., Felmetsger, V., Almeroth, K., Kem-

merer, R., and Vigna, G. (2006). SNOOZE: toward

a stateful NetwOrk prOtocol fuzZEr. In International

Conference on Information Security, pages 343–358.

Springer.

Biondi, P. (2010). Scapy documentation. http://

www.secdev.org/projects/scapy/doc/. [Online; acces-

sed 2017-11-07].

BlackHat (2017). Fuzzing for vulnerabilities.

https://www.blackhat.com/us-17/training/fuzzing-for-

vulnerabilities.html. [Online; accessed 2017-11-07].

Bratus, S., Hansen, A., and Shubina, A. (2008). LZfuzz: a

fast compression-based fuzzer for poorly documented

protocols. Technical Report TR2008-634, Dartmouth

College.

Gascon, H., Wressnegger, C., Yamaguchi, F., Arp, D., and

Rieck, K. (2015). Pulsar: Stateful black-box fuzzing

of proprietary network protocols. In International

Conference on Security and Privacy in Communica-

tion Systems, pages 330–347. Springer.

Gorbunov, S. and Rosenbloom, A. (2010). Autofuzz: Au-

tomated network protocol fuzzing framework. Inter-

national Journal of Computer Science and Network

Security, 10(8):239.

HS Augsburg (2012). ProFuzz. https://github.com/

HSASec/ProFuzz. [Online; accessed 2017-11-07].

ISA Security Compliance Institute (ISCI) (2016). Em-

bedded Device Security Assurance (EDSA) - version

2.0.0. http://www.isasecure.org/en-US/Certiﬁcation/

IEC-62443-EDSA-Certiﬁcation. [Online; accessed

2017-05-18].

McNally, R., Yiu, K., Grove, D., and Gerhardy, D. (2012).

Fuzzing: the state of the art. http://oai.dtic.mil/oai/

oai?verb=getRecord&metadataPreﬁx=html&identiﬁer

=ADA558209. [Online; accessed 2016-09-30].

Niedermaier, M., Fischer, F., and von Bodisco, A. (2017).

Propfuzz-an it-security fuzzing framework for pro-

prietary ics protocols. International Conference on

Applied Electronics (AE).

Peach Fuzzer Website (2016). Peach fuzzer:

Discover unknown vulnerabilities. http://

www.peachfuzzer.com/. [Online; accessed 2016-11-

01].

Pfrang, S. and Meier, D. (2017). On the Detection of Replay

Attacks in Industrial Automation Networks Operated

with Proﬁnet IO. In Proceedings of the 3rd Internati-

onal Conference on Information Systems Security and

Privacy, pages 683–693.

Pfrang, S., Meier, D., and Kautz, V. (2017). Towards a mo-

dular security testing framework for industrial auto-

mation and control systems: Isutest. In Proceedings of

the 22nd IEEE International Conference on Emerging

Technologies and Factory Automation, ETFA 2017.

Sutton, M., Greene, A., and Amini, P. (2007). Fuzzing:

Brute Force Vulnerability Discovery. Pearson Educa-

tion.

Voyiatzis, A. G., Katsigiannis, K., and Koubias, S. (2015).

A modbus/TCP fuzzer for testing internetworked in-

dustrial systems. In 20th Conference on Emerging

Technologies & Factory Automation (ETFA), pages 1–

6. IEEE.

Zimmermann, H. (1980). Osi reference model–the iso mo-

del of architecture for open systems interconnection.

IEEE Transactions on communications, 28(4):425–

432.