Figure 2: Vulnerability Testing Process
The reason we choose TTCN-3 for vulnerability
test specification is that, unlike TTCN-2 which is
specifically designed for conformance testing,
TTCN-3 has been developed with the objective of
being a flexible and powerful test specification
language applicable for all types of testing,
including robustness testing. Although TTCN-3 is
conceptually extended from TTCN-2, it is
syntactically very different from TTCN-2, and is
therefore named differently. Being a standard, the
use of TTCN-3 will ensure wide portability and
understandability, and the generated test suite will
be shared with the community for verification
without any modification required.
After the vulnerable test suite (VTS) is specified,
we need to use TTCN-3 tools to convert the VTS to
Executable Test Suite (ETS). Then we can feed the
ETS into the test tool (test execution module) to
execute against the implementation under test (IUT),
and finally we obtain the test verdicts. The process
of ETS translation and execution using a toolset (e.g.
TTthree and µTTman) is illustrated in Figure 3. This
process was applied to a real-life Megaco product
IUT. First, the VTS is generated via some creative
parameter variation heuristics and specified in
TTCN-3, and the VTS specification is stored in the
file megacoTest.ttcn3. The abstract VTS
specification in TTCN-3 is then transformed into an
ETS (via the TTthree tool) for actual application in
the vulnerability testing of the IUT (via µTTman
tool).
First, the VTS specification in TTCN-3 is
compiled into a programming language, e.g. Java or
C/C++. We are using the TTthree tool developed by
Testing Technologies™ to compile the VTS into
Java, specifically from megacoTest.ttcn3 to
megacoTest.jar. Next, we develop a test adapter,
written in Java, to map the abstract test system
interface referred to in TTCN-3 into a real test
system connected to the IUT. An example is the
mapping of an abstract port in TTCN-3 into a real
TCP or UDP port opening. In addition, we develop
an encoder, also written in Java, to map TTCN-3
data structures into real messages for injecting the
test messages, and a decoder to map real messages
into TTCN-3 data structures for receiving the
responses. For convenience, the above three java
programs are compiled into a single file:
megacoTestAdapter.jar.
To facilitate the testing, it is desirable to have a
test manager to inject the test cases in sequences,
receive the responses, and display the results. We
are using the µTTman tool, also developed by
Testing Technologies. The µTTman tool uses a
Module Loader File written in XML to reference the
test suite, the test adapter and the codec (voice
encoder/decoder) to be used, which in our case, are
the megacoTest.jar and megacoTestAdapter.jar.
3.2 VTS Generation – Mutation
Method
We adopt syntax testing method to identify software
bugs. Syntax testing is a way to test system
robustness. In a sense, if a system is very robust,
then it is really hard to be broken into, hence more
secure. In syntax testing, the test cases, i.e. the input
to the software, are created based on the
specifications in languages understood by the
interface software (Beizer, 1990). The motivation
for syntax testing springs from the fact that each
interface has a language, whether it is hidden or
open, from which effective tests can be created with
a relatively small effort (Kaksonen, 2000). The
communication protocol between two entities is a
perfect interface language.
The syntax for the interface language is usually
represented by formal grammars such as "Backus
Naur Form" (BNF). Following the rules of the
grammar, the defined formal language will produce
a right sentence which is basically sequences of
bytes. The selection of test cases in syntax testing
could start with single-error sentences that follow
the defined syntax. By single-error, we mean only
one grammar element in the right sentence is
replaced with some error or exceptional value. An
exceptional element value is an input that has been
designed to provoke undesired behavior in the
implementation, and we regard these as parameter
mutation (or parameter variation) from the normal
valid value. An example would be to replace a valid
integer value by a float number in the sentence. The
exceptional input is usually not considered seriously
Protocol Spec
VTS
Generation
VTS Æ ETS
Test
Execution
Verdict
IUT
ICETE 2004 - SECURITY AND RELIABILITY IN INFORMATION SYSTEMS AND NETWORKS
218