STACKFENCES: A RUN-TIME APPROACH FOR DETECTING

STACK OVERFLOWS

Andr

e Z

uquete

IEETA / UA

Campus Univ. de Santiago, 3810-193 Aveiro, Portugal

Keywords:

Buffer overﬂows, run-time detection, run-time correctness assessment, damage containment, dependability

Abstract:

This paper describes StackFences, a run-time technique for detecting overﬂows in local variables in C pro-

grams. This technique is different from all others developed so far because it tries to detect explicit overﬂow

occurrences, instead of detecting if a particular stack value, namely a return address, was corrupted because of

a stack overﬂow. Thus, StackFences is useful not only for detecting intrusion attempts but also for checking

the run-time robustness of applications. We also conceived different policies for deploying StackFences, al-

lowing a proper balancing between detection accuracy and performance. Effectiveness tests conﬁrmed that all

overﬂows in local variables are detected before causing any severe damage. Performance tests ran with several

tools and parameters showed an acceptable performance degradation.

1 INTRODUCTION

The exploitation of overﬂow vulnerabilities has been

one of the most popular forms of computer attacks

during the last 15 years. According to the ICAT

Metabase vulnerability statistics

, about 20% of the

vulnerabilities published in the last 4 years are related

with buffer overﬂows. Such vulnerabilities existed in

compiled C or C++ code used for different purposes:

operating system (OS) kernel; server and client appli-

cations. For some people the problem will continue to

exist as long C is used, and can only be minimized or

eliminated by using other languages instead of C (Mc-

Graw, 2002). However, C is still widely used, and

probably will continue to be, because it generates fast

compiled code and there is a large repository of legacy

code written in C. Therefore, there is a real need for

techniques and tools that help to minimize the number

and the risk of overﬂows in old or new C programs.

StackFences is a solution for detecting buffer over-

ﬂows affecting local variables, i.e., variables allocated

in stack frames. The key idea that we explored is

an extension to the canary mechanism, introduced

in StackGuard (Cowan et al., 1998), complemented

with the XOR canary mechanism introduced also in

StackGuard. The original canary mechanisms were

deployed for protecting only return values stored in

http://icat.nist.gov

the stack. We extended the protection scope and used

canaries for controlling all stack areas susceptible to

overﬂow. This way, we hope to detect each and ev-

ery stack overﬂow, either affecting highly sensible

values, like return addresses, or not. We also con-

ceived different policies for balancing detection accu-

racy and performance. Namely, run-time validations

can be performed in two different ways: (i) one more

detailed, allowing a more accurate and timely detec-

tion of overﬂows, more suitable for development sce-

narios, and (ii) one lazier, checking only when abso-

lutely necessary, more suitable for production envi-

ronments.

For testing StackFences we developed a prototype

for Linux systems using TCC (Tiny C Compiler). C

modules compiled with StackFences are fully com-

patible with the standard C libraries and with mod-

ules compiled with other compilers or compilation

options.

2 OVERVIEW

There are mainly two reasons for the buffer overﬂow

problems in C programs. First, the language does not

check for any boundaries around variables and allows

programmers to manage memory areas at will, with-

out any run-time control, using pointers, type casts

Zúquete A. (2004).

STACKFENCES: A RUN-TIME APPROACH FOR DETECTING STACK OVERFLOWS.

In Proceedings of the First International Conference on E-Business and Telecommunication Networks, pages 76-84

DOI: 10.5220/0001398000760084

 SciTePress

and pointer arithmetic. Second, many standard C

library functions are intrinsically unsafe concerning

buffer overﬂows (e.g. the infamous gets() and sev-

eral functions for manipulating strings).

Buffer overﬂows are a problem because they can

be used to modify data that controls the execution of

a victim process or OS kernel. The exploitation of

a buffer overﬂow vulnerability can expose a victim

application to two different risks:

Denial of Service (DoS): an attacker may interfere

randomly with the application’s execution ﬂow,

eventually making it fail after some illegal opera-

tion.

Penetration: An attacker may take control of the

application’s execution ﬂow by performing a crafty

buffer overﬂow.

Ideally, one would like to avoid both risks, but

that may be difﬁcult or even impossible to achieve

with current hardware architectures and existing soft-

ware. Comparing both risks, the risk of penetration is

greater than the risk of DoS. Therefore, it seems use-

ful to avoid penetration risks by transforming them

into DoS risks. We followed this reasoning in the de-

sign of StackFences.

2.1 Detection vs. prevention

Buffer overﬂows may be detected using static analy-

sis, i.e., before actually compiling and deploying the

code (e.g. (Wagner et al., 2000; Larochelle and Evans,

2001)). Another possibility is to tackle buffer over-

ﬂows dynamically, or at run-time, during the execu-

tion of the vulnerable applications. This was the ap-

proach that we followed in StackFences.

Dealing with buffer overﬂows at run-time implies

either prevention or detection. Prevention attempts to

conceal overﬂow vulnerabilities or to make their oc-

currence partially or totally harmless. Though preven-

tion does not help ﬁnding problems out, it is highly

desirable for avoiding both DoS and penetration risks.

However, total prevention is difﬁcult to achieve.

Detection attempts to detect the occurrence of ab-

normal facts, such as buffer overﬂows. Detection only

mitigates the problem, since the usual reaction is to

raise some sort of alert and immediately terminate the

affected application or OS kernel, eventually leading

to a DoS situation. Therefore, detection helps to as-

sess correctness on the execution of applications with

overﬂow vulnerabilities, which is important for im-

proving damage containment and reducing penetra-

tion risks. StackFences is a detection solution.

2.2 Anatomy of a stack overﬂow

The overﬂow of the stack memory reserved for a C

variable can corrupt the execution ﬂow of an applica-

tion in many ways. Assuming only overﬂows across

memory with growing addresses and an x86 micro-

processor, we have at least the following scenarios:

1. By setting the value of a neighbouring variable be-

low in the stack.

2. By setting the value of a saved frame pointer of

a previous stack frame to another address. When

the modiﬁed frame pointer is recovered, the appli-

cation will use, in that stack frame, different local

variables and function arguments.

3. By setting the return address for the calling func-

tion. When the function returns it will continue to

execute at an address chosen by the attacker.

These attacks can be complemented by inserting

auxiliary data, like microprocessor instructions or

function parameters, into input buffers, overﬂowed or

not. The auxiliary data can be used later for a pene-

tration bootstrap if the attacker succeeds in using it.

Most overﬂow attacks are stack-smashing at-

tacks (Aleph One, 1996), i.e., attacks of the third type

referred above, that overwrite return addresses and

jump into bootstrap penetration code. The famous In-

ternet Worm of 1988 did it (Spafford, 1989), as well

as many other attacks thereafter.

But the two other types are also risky. J. Wilander

and M. Kambar pointed out, in (Wilander and Kam-

bar, 2002), that a general weakness of the solutions

presented so far for run-time detection or prevention

is that they protect known attack targets, mainly re-

turn addresses in the stack, instead of protecting all

targets. From a pragmatic point of view, we can un-

derstand why that happens: stack-smashing attacks

are the simplest and most popular ones. But for deal-

ing with future and more sophisticated attacks ex-

ploiting buffer overﬂows we need to change the pro-

tection paradigm, like we did with StackFences.

3 RELATED WORK

In this section we describe the approach followed

by several run-time overﬂow detection or prevention

techniques. We do not address any static detection

solutions.

Many run-time protection techniques were devel-

oped to protect the most common target of overﬂow

attacks: the return pointer. StackGuard (Cowan et al.,

1998) uses canaries, which are speciﬁc values that are

placed between the local variables and the return ad-

dress in the same stack frame. The canary is installed

in the function prologue and checked in the epilogue.

If its value changed in the meanwhile then there was

an overﬂow and the process is halted; otherwise the

function returns normally.

STACKFENCES: A RUN-TIME APPROACH FOR DETECTING STACK OVERFLOWS

Propolice (Etoh and Yoda, 2000) enhances the

basic protection of StackGuard by rearranging local

variables. The assumptions of propolice are that

(i) only character arrays are vulnerable to overﬂows

and (ii) function arguments do not contain character

arrays. Thus, vulnerable local variables — charac-

ter arrays or structures with character arrays — are

packed together in a vulnerable location next to the

canary (guard). All other variables are placed above

in the stack. This way, overﬂows can occur inside

vulnerable locations but cannot affect non-vulnerable

variables neither the return address, which is pro-

tected by the canary. However, propolice assump-

tions are not complete, because there are other kinds

of vulnerable variables besides character arrays, and

ignores overﬂows affecting only variables in a vulner-

able area.

Another way of protecting return addresses is by

hiding them with a XOR canary, or cookie. The XOR

canary is XORed with the return address in a func-

tion’s prologue and again in the epilogue. Attackers

not knowing the value of the XOR canary are un-

able to modify return addresses in a useful manner.

StackGuard was the ﬁrst to use XOR canaries to frus-

trate attacks (to return addresses) circumventing the

basic canary mechanism

. StackGhost (Frantzen and

Shuey, 2001) is kernel-level solution for Sparc archi-

tectures that protects return addresses with XOR ca-

naries. It uses either per-kernel or per-process XOR

canaries, but the ﬁrst is too weak for competent hack-

ers. Overﬂow attacks affecting return values produce

wrong return addresses. These can be automatically

detected in 75% of the cases because Sparc instruc-

tions must be aligned on a 4-byte boundary; on the

other 25% the program will run uncontrolled.

StackGuard’s MemGuard (Cowan et al., 1998) pro-

tection makes return addresses in the stack read-only

during the normal execution of functions. This way,

any attempts to overwrite them raise a memory ex-

ception. However, the performance penalty of this

approach is huge.

Another way of protecting return addresses is to

keep a separate copy of them in a return-address

stack. Return addresses are stored in and fetched from

the return-address stack in the function’s prologue

and epilogue, respectively. Vendicator’s StackShield

Global Ret Stack protection and Secure Return

Address Stack (Xu et al., 2002) use only the copied

values, thus preventing attacks affecting return ad-

dresses stored in the normal stack. But these solutions

fail completely in protecting any other stack values

from overﬂows.

The Return Address Defender (Chiueh and Hsu,

2001) also uses a return-address stack but provides

This mechanism was introduced in version 1.12.

http://www.angelfire.com/sk/stackshield

only detection because return addresses on the Re-

turn Address Repository are compared with the ones

in the ordinary stack before being used, and the pro-

cess is halted if they are different. StackShield’s Ret

Range Check is similar but stores a copy of the

current return address in a global variable. They are

both useless against overﬂow attacks overwriting the

return address with its exact value, which is not difﬁ-

cult to guess for a competent hacker.

Libverify (Baratloo et al., 2000) also uses a return-

address stack but it can be transparently applied to

existing binary code by means of a dynamic library.

The drawback of Libverify is that all protected code

must be copied into the heap to overwrite instructions

in the prologue and epilogue of all functions. This

means that processes are unable to share the code they

effectively run in main memory and absolute jumps

within the text area must be handled with traps.

Libsafe (Baratloo et al., 2000) is a dynamic library

that replaces unsafe functions of the standard C li-

brary that are typically used for performing buffer

overﬂows. All Libsafe functions compute the upper

bounds of destination’s buffers before actually trans-

ferring data into memory. The upper bounds are de-

ﬁned by the location of return addresses. But the pro-

tection is limited and misses unsafe functions com-

piled inline within existing applications or libraries.

PointGuard (Cowan et al., 2003) is an extension

of the original XOR canary mechanism for protect-

ing pointer variables. Pointers are stored encrypted

(XORed with the XOR canary) and are decrypted

when loaded into CPU registers. However, this ap-

proach raises problems when integrating mix-mode

code (some PointGuard, some not).

Practically all protection mechanisms look only at

the effects of overﬂows within the current stack frame

— exceptions are PointGuard and Libsafe. Stack-

Guard 3 (Wagle and Cowan, 2003) and Libsafe also

protect saved frame pointers, which are collocated

with return addresses. And propolice, under their

assumptions, further protects all variables that are not

character arrays or structures with character arrays.

Most of the protection mechanisms described are

added at compile time; exceptions are Libsafe, Lib-

verify and StackGhost. StackFences is also added at

compile time.

With StackFences we tried to further improve the

detection of stack overﬂows. Namely, we tried to (i)

detect overﬂows within all existing stack frames, not

only the current one; and to (ii) detect overﬂows from

all variables that could be overﬂowed. In this par-

ticular case we extended propolice’s and Point-

Guard’s notions of vulnerable areas and we do not

ignore overﬂows within variables belonging to a vul-

nerable area, as propolice does.

ICETE 2004 - SECURITY AND RELIABILITY IN INFORMATION SYSTEMS AND NETWORKS

4 OUR CONTRIBUTION:

StackFences

To detect overﬂows in stack variables we use

StackGuard-like boundary canaries between them.

The management of canaries — allocation, set-up and

checking — depends on the kind of stack variables we

are dealing with: local variables or function parame-

ters. In this paper we handle only the management of

boundary canaries for local variables.

The value of canaries is protected with a XOR ca-

nary, that is a per-process random 32-bit value. The

canary is stored in a publicly known variable (cXor

hereafter) and should not be modiﬁed after being

setup. Adaptive attacks trying to guess the correct

value of a process’ canary are infeasible, in theory,

because attacked processes should terminate after an

attack with an unsuitable, tentative canary value. We

assume, like for StackGuard, StackGhost and Point-

Guard, that attackers cannot get dumps of memory

areas containing any well-known values XORed with

the XOR canary.

4.1 Overview

Boundary canaries are located near local variables, on

higher addresses. The questions now are (i) which

variables we want to, or should, bind with canaries,

(ii) how do we ﬁnd the canaries and check their value;

and (iii) when do we want to, or should, check the

value of canaries.

4.1.1 Variables to bind with canaries

Following a high security policy, we should bind all

stack variables. But this is a sort of brute force ap-

proach that has considerable impact in the perfor-

mance of modiﬁed applications.

A more relaxed policy, which we chose for Stack-

Fences, is to bind only potentially vulnerable vari-

ables. These are the variables for which a pointer is

taken and used in the current function or other func-

tion called upon it. This policy is more efﬁciency than

the previous one and should tackle most stack over-

ﬂow problems, since they usually derive from deﬁ-

cient uses of legitimate stack addresses.

Because the size of C structures cannot be altered,

canaries cannot be added between members of local

struct variables. Therefore, overﬂows may still oc-

cur inside structures, but not affecting external mem-

ory areas.

4.1.2 Looking for and checking canaries

We used a function-independent way of locating and

checking canaries, because it appeared to be more

can

n−3

can

cTail

function parameters

prev frame pointer

return address

can

n−2

local variable

can

n−1

local variable

can

local variable

cHead

current stack frame

previous stack frames

stack bottom

prev cHead

Figure 1: Example of the list of canaries, starting in the cur-

rent stack frame (can

) and until the ﬁrst one inserted at

the beginning of the process execution (can

). Variables

cHead and

cTail point the head and tail of the list. Shaded

boxes represent canaries XORed with the process’ XOR ca-

nary.

ﬂexible and simple to implement and test. The set

of all boundary canaries forms a linked list, as shown

in Fig. 1. The location of the head and tail canaries of

the list are stored in publicly known variables (cHead

and cTail in this text). The virtual address (of an-

other canary) stored in each canary is protected using

the XOR canary previously referred. This way an at-

tacker causing the overﬂow of a stack variable cannot

easily guess valid values for boundary canaries be-

tween the overﬂowed variable and the neighbouring

variables to be tampered.

The full list of canaries, or particular sub-lists, can

easily be checked by dereferencing canaries, XOR-

ing the obtained value with the XOR canary and test-

ing whether the result is a valid address. Testing the

validity of an address is straightforward: (i) it must

be higher than the previous one, because the list goes

strictly from the top to the bottom of the stack, and

(ii) it cannot be higher than a target canary address

that we want to reach. Any violation of these asser-

tions is an overﬂow evidence.

4.1.3 Triggering canary checking

There are at least two distinct situations that should

trigger canary checking:

Before doing some operation related with the ex-

ternal perception of the application’s behaviour.

Broadly, this means that checking should be done

before any I/O attempt;

Just before the return of a function. In this case,

we should check for overﬂows within the current

stack frame, i.e., caused in local variables or pa-

rameters, which will disappear because the stack

STACKFENCES: A RUN-TIME APPROACH FOR DETECTING STACK OVERFLOWS

frame will be released.

In the ﬁrst case, we should check the full list of

canaries, from cHead to cTail, because we are

looking for any stack overﬂow. The more times it is

checked, the more timely we can ﬁnd stack overﬂows,

but with a signiﬁcant impact on the performance of

the application.

In the second case, we should check only the list of

canaries belonging to the local stack frame (can

can

n−2

in Fig. 1) since we are looking for evidences

of local stack overﬂows that are about to disappear.

If no canaries exist in the current stack frame then no

local checking is required. Checking a local list of

canaries is also an iterative walk starting in cHead

but ending in the head canary of the previous stack

frames (can

n−3

in Fig. 1).

4.2 Management of the canary list

StackFences’ canaries are similar to local variables,

but are invisible to application code and have a

compiler-deﬁned (initial) value. The stack space for

canaries is allocated when the compiler deﬁnes the

location of local variables. The setup of the canaries

should occur after the normal C prologue.

The list of canaries is increased after the prologue

of a function and decreased when the function returns.

Increasing the list consists of adding all canaries in the

local stack frame to the head of the list and storing in

cHead the address of the new top-most stack canary

(can

in Fig. 1). Decreasing the list consists simply

of setting the value of cHead using the address of the

head canary of the previous stack frames (can

n−3

Fig. 1).

The list of canaries must also be increased when the

program calls alloca. The space requested should

be increased to accommodate a canary at the end of it,

which becomes the new list head. The list must also

be decreased when the program calls longjmp; in

this context it is similar to a long return. The value

of cHead must be set with the address of the head

canary in the stack frame we are jumping into, or in

some other stack frame below.

4.3 Policies for canary checking

Checking boundary canaries is a potentially expen-

sive operation, thus it needs to be carefully managed

in order to balance two requirements: (i) effective de-

tection of stack overﬂows and (ii) efﬁcient execution

of the program. Furthermore, in terms of effective-

ness, two natural approaches should be contemplated

considering the software life cycle: (i) in develop-

ment stages, the sooner overﬂows are spotted the bet-

ter, while (ii) in production stages, preventing applica-

tions from “making damage” may be enough for most

cases.

Thus, considering the two execution environments

mentioned above – development and production – we

conceived two different policies for checking the cor-

rectness of boundary canaries. The two policies de-

ﬁne when two lists of canaries are checked: (i) the list

of canaries belonging to the current stack frame and

(ii) the full list of canaries.

4.3.1 Checking local canaries

As previously explained, boundary canaries on the

current stack frame should be checked when the stack

frame is about to be released; otherwise, we could

miss local overﬂows. Consequently, the reduction of

the canary list, both within a normal function return

and within a call to longjmp, always checks the con-

sistency of all released canaries.

The extra code for checking local canaries and re-

ducing the list of canaries was placed in a function

(canReduce) that is called before the function’s epi-

logue. It gets, as a parameter, the address of the

head canary in the previous stack frames (can

n−3

Fig. 1). This address is stored (in clear) at the top

of the current stack frame in the function’s prologue

(prev cHead in Fig. 1). The function canReduce

checks the canaries from cHead until prev cHead

and, at the end, stores prev cHead in cHead.

For handling the reduction of the canary list af-

ter a longjmp call we use a function similar to

canReduce. The function starts from cHead and

walks along the canary list until ﬁnding a canary with

an address higher than the stack pointer saved in the

jump context. The address of that canary will be

stored in cHead. Jump contexts stored by setjmp

and used by longjmp are not modiﬁed; only the

functionality of longjmp needs to be extended.

4.3.2 Checking all canaries

We conceived two policies for checking the full list

of canaries: one more suitable for development sce-

narios, another more suitable for production environ-

ments. In either case, we tried to prevent an attacked

process from doing any I/O after a stack buffer over-

ﬂow. The checking function is named canWalk.

Development policy: In a development scenario

we want to catch an overﬂow as accurately as pos-

sible, in order to simplify the process of ﬁnding and

ﬁxing the vulnerability. It is, thus, natural to sacriﬁce

execution efﬁciency in favour of debugging effective-

ness. Our development policy consists of a canWalk

call before each function call. This approach is com-

putationally costly but has the advantage of detecting

overﬂows not far from where they occurred.

Production policy: In a production scenario we

want the applications to run efﬁciently and, yet, we

ICETE 2004 - SECURITY AND RELIABILITY IN INFORMATION SYSTEMS AND NETWORKS

want to detect overﬂows before they can interfere with

their I/O. Our production policy consists of a watch-

dog process to catch all the system calls of the target

process and to call canWalk before each I/O sys-

tem call requested by the target process. We deﬁne

an I/O system call as a system call interacting with

I/O objects (ﬁles, pipes, sockets, etc.) or with other

operating system resources (e.g. send signals to other

processes, manage virtual memory attributes, change

the ownership of the process, etc.).

5 IMPLEMENTATION

For implementing all the mechanisms and policies

previously described for detecting stack buffer over-

ﬂows we modiﬁed a C compiler and developed 3

auxiliary C modules for Linux systems that must

be linked with the applications we want to protect.

The applications only need to be recompiled with the

modiﬁed compiler and linked with some of the mod-

ules to use StackFences.

For the C compiler we used the version 0.9.20

of TCC (Tiny C Compiler), a simple but complete

C compiler developed by Fabrice Bellard (Bellard,

2003). We chose this compiler mainly because it al-

lows a fast prototyping for getting a proof of concept.

The ﬁrst auxiliary module deﬁnes variables and

functions that are common to all overﬂow checking

facilities — the public variables cXor, cHead and

cTail; the XOR canary setup function (that uses the

ﬁle /dev/urandom), the function canReduce;

and a new longjmp function.

The second auxiliary module deﬁnes a new main

and all the checking/aborting functions for the de-

velopment policy. The new main deﬁnes the initial

boundary canary, initiates variables in the previous

module and calls the application’s original main.

The third auxiliary module deﬁnes a new main and

all the checking/aborting functions for the production

policy. The new main creates a child process for run-

ning the application and the initial process stays as the

watchdog of the new one (using ptrace). The code

of new main executing in the child process is basi-

cally identical to the one of the second module.

The watchdog triggers a call to canWalk before

allowing the execution of any requested I/O system

call. The canWalk function is similar to the one

in the second module but fetches canary values with

ptrace because it runs in a different process.

6 EFFECTIVENESS

EVALUATION

We tested the effectiveness of StackFences with the

test suite described in (Wilander and Kambar, 2002)

and kindly provided by the authors. The results were

the best possible: StackFences detected and halted all

12 attacks overﬂowing stack variables. We were also

able to do so using either of the canary checking poli-

cies described in §4.3.2.

The empirical results obtained with the same test

suite and using 4 protection tools — StackGuard,

StackShield, propolice and Libsafe/Libverify –

showed that the tools where able to handle, in the best

case, 10 out of the 12 attacks (with propolice).

Note, however, that:

• the test suite is not complete, it only overﬂows

character arrays, which are exactly the vulnerable

variables considered by propolice. But Stack-

Fences is more powerful, being able to detect over-

ﬂows in local variables other than character arrays.

Therefore, StackFences is much better suited for

assessing the correctness of applications in run-

time than propolice, but that cannot be fully

demonstrated with this test suite.

• propolice does not detect any overﬂows within

consecutive character arrays, as StackFences does,

and such vulnerability is also not explored by the

test suite.

7 PERFORMANCE EVALUATION

The performance penalties introduced by Stack-

Fences depend on several factors, namely: (i) the

number of vulnerable local variables; (ii) the number

of function calls; (iii) the length of the list of canaries

when canWalk is called; and (iv) the number of I/O

operations that trigger the call to canWalk (if using

the production policy).

We evaluated StackFences with both micro and

macro benchmarks. The micro benchmarks provide

upper bounds to the overheads caused by setting and

checking canaries in each function’s prologue and

epilogue, respectively. The macro benchmarks help

us to have an idea about the relative cost of each

checking policy, because they control the calls to

canWalk, and also to get an idea about the space

occupied by canaries. But, for lack of space, micro

benchmarks are not further referred in this article.

For the evaluation of StackFences with macro

benchmarks we chose 3 tools with moderate ﬁle I/O

and high CPU activity: ctags, tcc and bzip2.

These 3 tools where compiled in 4 different ways —

with gcc, tcc, and tcc with the two StackFences

STACKFENCES: A RUN-TIME APPROACH FOR DETECTING STACK OVERFLOWS

Table 1: Results of the execution of macro benchmarks with existing applications for evaluating the total overheads introduced

by StackFences and statistic data regarding the checking of StackFences’ canaries.

elapsed time (ms) StackFences statistics

tool local variables arguments StackFences gcc tcc canReduce canWalk

total vulnerable policy -O3 orig. with calls length calls length

StackFences avg. max. avg. max.

bzip2 development 70 122 149 (+22%) 1,667 5.9 11 437,946 21.8 24

sources production 126 (+3%) 48 7.4 13

bzip2 462 302 tcc development 207 324 396 (+22%) 1,970 6.0 11 1,127,521 22.1 24

(65%) sources production 334 (+3%) 62 8.4 13

ctags development 230 360 439 (+22%) 1,683 5.9 11 1,207,621 22.3 24

sources production 371 (+3%) 60 8.5 13

bzip2 development 89 167 419 (+151%) 411,256 2.4 10 2,927,026 30.8 165

sources production 217 (+30%) 1,250 14.6 65

tcc 978 603 tcc development 97 179 517 (+189%) 464,208 2.5 10 3,737,023 33.5 252

(62%) sources production 222 (+24%) 928 15.9 60

ctags development 304 541 1,149 (+112%) 1,231,803 2.3 10 8,297,317 24.1 163

sources production 724 (+34%) 4,682 13.9 102

bzip2 development 36 36 49 (+36%) 3,338 1.4 3 1,334,837 2.1 5

sources production 44 (+22%) 137 2.3 4

ctags 911 178 tcc development 91 92 132 (+43%) 13,899 1.4 3 3,599,858 2.2 5

(20%) sources production 116 (+26%) 207 2.3 4

ctags development 91 98 138 (+41%) 16,530 1.4 3 3,480,356 2.2 5

sources production 138 (+41%) 391 2.2 4

checking policies — and executed with 3 different pa-

rameters — the sources of each of the 3 tools. All

benchmarks ran in a Red Hat 8.0 Linux box (kernel

2.4.18-27.8.0) with a Pentium IV at 2.4 GHz, 256 MB

RAM and 512 KB cache.

The results of the evaluation are presented in Ta-

ble 1: the second and third columns show the num-

ber of local variables used by the tools and the num-

ber and percentage of them that are vulnerable and

checked by StackFences; the fourth column shows the

arguments used with the tools; the ﬁfth column shows

the protection policy used with StackFences. In the

rest of the columns we have execution results: the

sixth and seventh columns show the elapsed time ob-

served with the tools compiled with gcc (with max-

imum optimisation) and the normal tcc (that has no

optimisations); the eighth column shows the elapsed

time observed with the tools compiled with tcc and

StackFences, using both security policies, and the

overhead in percentage comparing with the results

of the previous column; the last six columns show

the number of calls to canReduce and canWalk

and the average and maximum number of canaries

checked per call. StackFences’ auxiliary modules

were written in C and compiled with gcc and max-

imum optimisation. The elapsed times are the mini-

mum observed in 100 consecutive runs of each test.

The values provided for gcc are only indicative,

because many dynamic solutions for dealing with

buffer overﬂows were implemented with it. But it

doesn’t make sense to extrapolate the overhead of

StackFences comparing with gcc because some of

the optimisations used by the latter would also reduce

the overhead of StackFences if it was part of gcc.

For instance, tcc has many small inline functions.

Since our version of TCC ignores inline qualiﬁers,

that greatly increases the number of canReduce

calls and the average number of canaries checked by

canWalk. Such overhead would not happen if we

had integrated StackFences with gcc.

The results in Table 1 show that the overheads in-

troduced by StackFences are acceptable. With the de-

velopment policy, overheads are between 22% and

189% of the elapsed time with tcc and without

StackFences. With the production policy, overheads

are lower, as desired, between 3% and 41%. The pro-

duction policy greatly reduces the number of calls to

canWalk, as expected. But in some cases, namely

with ctags, results show that it is almost irrelevant

to use either checking policy. That happens because

the average number of canaries checked by canWalk

is low, making more relevant the cost of the process

switching between the application and its watchdog

in the production policy.

Considering the applications tested, the extra space

occupied by canaries in the stack is not an issue. In

the worst case there is a maximum of 252+10 canaries

(when tcc is compiled by itself using the develop-

ment policy), which represents a memory overhead

of about 1 KB.

A small comparison can be established between the

performance of ctags with StackGuard and with

ICETE 2004 - SECURITY AND RELIABILITY IN INFORMATION SYSTEMS AND NETWORKS

StackFences. According to (Cowan et al., 1998),

ctags with StackGuard has an overhead of 80%

when processing 78 ﬁles, 37,000 lines of code. Using

the same number of ﬁles and an approximated num-

ber of lines of code (37,188) we got for StackFences a

lower performance penalty: 31% with the production

policy and 41% with the debug policy.

According to Table 1, gcc and tcc generate

equally fast ctags executables. Therefore, for this

particular experience we can compare the overheads

of the two protection mechanisms independently of

the compilers implementing them. And the conclu-

sion is that StackFences is faster than StackGuard,

which makes less validation actions! This paradox is

probably explained by the fact that the performance

ﬁgures presented in (Cowan et al., 1998) were ob-

tained with a non-optimized version of StackGuard,

that added canaries and code for checking them to all

functions, instead of doing it only for functions with

vulnerable local variables. This fact is mentioned

in the performance optimisations for StackGuard de-

scribed in (Cowan et al., 1998). StackFences, on

the contrary, was optimized to add canaries and for

checking them locally with canReduce only in

functions with vulnerable variables.

8 CONCLUSIONS

We have presented StackFences, a run-time solution

for detecting buffer overﬂows affecting variables allo-

cated in stack frames. StackFences detects overﬂows

in all potentially vulnerable local variables instead of

detecting only overﬂows affecting known attack tar-

gets, like return addresses. To the best of our knowl-

edge this is the ﬁrst solution to perform such a de-

tailed run-time stack analysis.

For balancing detection accuracy and performance

we conceived two checking policies for StackFences:

a development policy, more detailed and allowing a

more accurate and timely detection of overﬂow occur-

rences, suitable for development scenarios; and a pro-

duction policy, lazier, checking only when absolutely

necessary, suitable for production environments.

In terms of effectiveness, StackFences detected and

halted all the 12 attacks of the test suite described

in (Wilander and Kambar, 2002). Although it may

appear that StackFences is minutely different from

propolice, which avoids 10 of those attacks, that

is not true because StackFences detects other over-

ﬂows that are not considered by the test suite and nei-

ther tackled by propolice. StackFences is much

better suited for assessing the correctness of applica-

tions in run-time than propolice, though the test

suite does not properly demonstrate that.

The performance of StackFences is acceptable but

depends a lot on the compiled application. We be-

lieve that optimizing canary checking code can fur-

ther reduce StackFences’ overheads. As desired, the

overhead of the production policy is lower than the

overhead of the debug policy. Finally, StackFences’

overheads are mainly compiler independent, though

they may depend on some compiling options.

REFERENCES

Aleph One (1996). Smashing The Stack For Fun And Proﬁt.

Phrack Magazine, 7(49).

Baratloo, A., Singh, N., and Tsai, T. (2000). Transparent

Run-Time Defense Against Stack Smashing Attacks.

In Proc. of the USENIX Annual Technical Conf., San

Diego, CA, USA.

Bellard, F. (2003). Tiny C Compiler, version 0.9.20.

http://fabrice.bellard.free.fr/tcc.

Chiueh, T.-C. and Hsu, F.-H. (2001). RAD: A Compile-time

Solution to Buffer Overﬂow Attacks. In IEEE Int.

Conf. on Distr. Computing Systems (ICDCS), Phoenix,

AZ, USA.

Cowan, C., Beattie, S., Johansen, J., and Wagle, P. (2003).

PointGuard: Protecting Pointers From Buffer Over-

ﬂow Vulnerabilities. In 12th USENIX Security Symp.,

Washington, D.C., USA.

Cowan, C., Pu, C., Maier, D., Hinton, H., Walpole, J.,

Bakke, P., Beattie, S., Grier, A., Wagle, P., and Zhang,

Q. (1998). StackGuard: Automatic Adaptive Detec-

tion and Prevention of Buffer-Overﬂow Attacks. In

Proc. 7th USENIX Security Conf., pages 63–78, San

Antonio, TX, USA.

Etoh, H. and Yoda, K. (2000). propolice: Im-

proved stack-smashing attack detection. IPSJ

SIGNotes Computer SECurity Abstract, 43(12).

http://www.ipsj.or.jp/members/

Journal/Eng/4312/article053.html.

Frantzen, M. and Shuey, M. (2001). StackGhost: Hard-

ware Facilitated Stack Protection. In Proc. of the 10th

USENIX Security Symp., Washington, D.C., USA.

Larochelle, D. and Evans, D. (2001). Statically Detecting

Likely Buffer Overﬂow Vulnerabilities. In Proc. of the

10th USENIX Security Symp., pages 177–190, Wash-

ington, D.C., USA.

McGraw, G. (2002). Building Secure Software. In

RTO/NATO Real-Time Intrusion Detection Symp., Es-

toril, Portugal. Invited Talk.

Spafford, E. H. (1989). The Internet Worm Incident. In

Ghezzi, C. and McDermid, J. A., editors, ESEC‘89

2nd European Software Engineering Conf., University

of Warwick, Coventry, United Kingdom. Springer.

Wagle, P. and Cowan, C. (2003). StackGuard: Simple Stack

Smash Protection for GCC. In Proc. of the GCC De-

velopers Summit, pages 243–255.

Wagner, D., Foster, J. S., Brewer, E. A., and Aiken, A.

(2000). A First Step towards Automated Detection of

STACKFENCES: A RUN-TIME APPROACH FOR DETECTING STACK OVERFLOWS

Buffer Overrun Vulnerabilities. In Proc. of the Inter-

net Soc. Symp. on Network and Distr. Systems Security

(NDSS 00), pages 3–17, San Diego, CA, USA.

Wilander, J. and Kambar, M. (2002). A Comparison of Pub-

licly Available Tools for static intrusion prevention. In

Proc. of the 7th Nordic Workshop on Secure IT Sys-

tems, pages 68–84, Karlstad, Sweden.

Xu, J., Kalbarczyk, Z., Patel, S., and Iyer, R. K. (2002).

Architecture Support for Defending Against Buffer

Overﬂow Attacks. In 2nd Works. on Evaluating and

Architecting System Dependability (EASY), San Jose,

CA, USA.

ICETE 2004 - SECURITY AND RELIABILITY IN INFORMATION SYSTEMS AND NETWORKS