AN IMPROVED MODEL FOR SECURE CRYPTOGRAPHIC

INTEGRITY VERIFICATION OF LOCAL CODE

Christian Payne

School of Information Technology, Murdoch University, Perth, Western Australia

Keywords:

Code integrity veriﬁcation, malicious code, one-way hash functions.

Abstract:

Trusted ﬁngerprinting is a new model for cryptographic integrity veriﬁcation of executables and related objects

to protect users against illicit modiﬁcations to system programs and attacks by malicious code. In addition

to a number of other novel features, trusted ﬁngerprinting improves upon previous designs by managing the

privileges assigned to processes based upon their veriﬁcation status. It also provides greater ﬂexibility as, in

addition to globally veriﬁed programs, each user can independently ﬂag for veriﬁcation software relevant to

their individual security requirements. Trusted ﬁngerprinting also allows for automatic updates to ﬁngerprints

of objects where these modiﬁcations are made by trusted code.

1 INTRODUCTION

Numerous vulnerabilities in software (Dowd et al.,

2007; US-CERT, 2006) and a proliferation of mali-

cious code (Gordon et al., 2006; Kalafut et al., 2006)

mean that compromise of computer systems is be-

coming increasingly difﬁcult to prevent. One ap-

proach to mitigating these risks is the use of crypto-

graphic techniques such as one-way hash functions or

digital signatures to support veriﬁcation of code in-

tegrity so that these intrusions may be detected and

subsequently repelled. While code signing is used ex-

tensively for remote and distributed code (Gong et al.,

1997; Microsoft Corporation, 2006), local code veri-

ﬁcation is far less common. Existing schemes have

both practical and architectural limitations and have

failed to recognize that code veriﬁcation is only one

piece of the puzzle. While veriﬁcation can detect

naive modiﬁcations, there are a number of ways that

attackers can bypass these security checks. Prevent-

ing this requires that the privileges assigned to a pro-

gram be determined based upon its veriﬁcation status.

This paper presents a new model for integrity ver-

iﬁcation of local system code. Like some previous

approaches it uses one-way hash functions to gener-

ate hash values or ‘ﬁngerprints’ for security-related

system objects. However, it improves upon previous

designs by also considering the ﬁles a program inter-

acts with at runtime and whether the program’s in-

tegrity has been veriﬁed or not. Based upon this a trust

level is assigned to the process and its privileges are

controlled by integrating the code veriﬁcation model

with a secure access control scheme. This allows

trusted, veriﬁed code to be granted higher privileges

than programs that have not been authenticated and

therefore defends against more sophisticated redirec-

tion attacks. It also resolves practical issues such as

veriﬁcation of non-native executables (such as scripts)

and allows the automatic updating of ﬁngerprints be-

longing to data objects where these modiﬁcations are

made by trusted code.

2 PREVIOUS APPROACHES

Use of cryptography to verify code integrity has long

been identiﬁed as a means for detecting substitution

or modiﬁcation of system programs. The Tripwire

software generates one-way hashes of important sys-

tem ﬁles and is then run on a regular basis to re-

calculate these hashes and detect if these ﬁles have

changed (Kim and Spafford, 1994a; Kim and Spaf-

ford, 1994b). However, Tripwire is a passive detec-

tion mechanism and its high detection latency means

that there is typically a considerable window of vul-

nerability between a compromise occurring and the

administrator being alerted to this event.

Consequently mechanisms which actively prevent

the execution of compromised code have since been

developed. CryptoMark involves embedding signa-

tures in the ELF binary header with the kernel veri-

fying this signature and potentially denying execution

Payne C. (2007).

AN IMPROVED MODEL FOR SECURE CRYPTOGRAPHIC INTEGRITY VERIFICATION OF LOCAL CODE.

In Proceedings of the Second International Conference on Security and Cryptography, pages 80-84

DOI: 10.5220/0002124300800084

 SciTePress

if this fails (Beattie et al., 2000). The I

FS scheme

uses a stacked ﬁlesystem that incorporates the storage

of hash values for data objects with veriﬁcation oc-

curring upon access (Patil et al., 2004). Yet another

approach utilises the code veriﬁcation features of the

TPM chip found in TCG-compliant computer systems

to cache known-good ‘measurements’ of system ob-

jects and verify these upon access (Sailer et al., 2004).

Each of these designs suffers from one or more

serious limitations. For example, the TCG-based in-

tegrity veriﬁcation scheme has no convenient way for

users to update hash values when legitimate changes

are made to the corresponding ﬁle object. This is

problematic from a security perspective as it means

objects that may change frequently are likely to be

omitted from veriﬁcation. This creates scope for a

veriﬁed program to be subverted by malicious modiﬁ-

cations to the data ﬁle inputs that control its behaviour

such as conﬁguration ﬁles.

A similar problem affects CryptoMark as its de-

pendence upon storing signatures inside the ELF bi-

nary executables themselves does not allow for ver-

iﬁcation of data ﬁles. This also precludes veriﬁca-

tion of non-natively executable scripts. Its designers

suggest that a solution to this is the use of a separate

table specifying signed objects and their signatures.

However, the table then becomes the target of attack

and leads to the more general problem of determining

which system objects require veriﬁcation and which

do not. This needs to be reliable and immutable in

the face of an attacker with superuser privileges oth-

erwise the system can be tricked into mis-identifying

a maliciously modiﬁed object as one that does not re-

quire veriﬁcation. If this is not the case then an at-

tacker can either insert new malicious code or redi-

rect users to unwittingly execute code other than the

intended trusted program. This problem is illustrated

in the I

FS design where a Unix inode value rather

than a path is used to identify each ﬁle. This allows

a privileged attacker to create a new program of the

same name as one that is cryptographically veriﬁed

but with a different inode number. A user executing

this program will believe they are running a trusted,

veriﬁed program but as the inode numbers will not

match, the redirection will not be detected.

This issue can be resolved by requiring all objects

(both code and data) to be veriﬁed but this is imprac-

tical (Reid and Caelli, 2005). Not only does it im-

pose an unnecessary performance penalty but dealing

with modiﬁed or newly created objects becomes prob-

lematic. An underlying cause of these limitations and

weaknesses is that all programs execute with the same

level of privilege, regardless of whether they have

been veriﬁed or not. A novel solution is therefore to

link the privilege with which a program executes with

its veriﬁcation status. This effectively limits the im-

pact of insertion or redirection attacks by constrain-

ing the privileges of non-veriﬁed code. A new model

for local code integrity veriﬁcation that uses this tech-

nique will now be described.

3 OVERVIEW OF TRUSTED

FINGERPRINTING

3.1 The Vaults Security Model

Vaults is an operating system security model that pro-

vides a key management infrastructure and a crypto-

graphic locks and keys based access control scheme

(Payne, 2003; Payne, 2004), in addition to the in-

tegrity veriﬁcation mechanism described here. The

access control model applies cryptographically-based

read and write protection to selected ﬁles which are

then accessed through tokens called ‘tickets’. Users

can also grant others access to their ﬁles by generat-

ing additional tickets for them. Note that these access

controls apply in addition to existing discretionary ac-

cess control lists or permissions. The model also fa-

cilitates the storage of a variety of items in secure

repositories known as ‘vaults’. Each user has their

own vault which they are responsible for administer-

ing using specially designated software. However,

there are also system vaults maintained by the ker-

nel and security administrator. Of these, the one most

relevant to trusted ﬁngerprinting is the Global Public

vault which can be read by all users but only modiﬁed

by the security administrator.

The items stored in these various vaults include

tickets and ﬁle encryption keys but users can also

store secret values such as passwords that are asso-

ciated with speciﬁc applications. These applications

can then request access to the keys that are bound to

them in a manner that is transparent to the user. Each

user’s vault is stored in encrypted form on the disk and

loaded into memory when the corresponding decryp-

tion key is supplied. Access to this vault is restricted

to trusted programs executed subsequent to vault de-

cryption.

Trusted ﬁngerprinting has been designed to in-

tegrate with the Vaults model and leverage the fea-

tures it provides. For example, the ﬁngerprints gen-

erated from system objects can be stored securely in

the appropriate vault. As access to a vault is crypto-

graphically controlled, this ensures the integrity of the

ﬁngerprint values. Fingerprints can then be veriﬁed

when the corresponding program or ﬁle is accessed.

AN IMPROVED MODEL FOR SECURE CRYPTOGRAPHIC INTEGRITY VERIFICATION OF LOCAL CODE

Additionally, the cryptographic access control archi-

tecture provides a convenient means for managing

process privilege and conﬁnement. However, trusted

ﬁngerprinting also improves the security of the Vaults

model since it allows applications to be authenticated

before keys are released to them.

3.2 Local and Global Dependencies

The existence of both the Global Public vault and

individual user vaults gives trusted ﬁngerprinting a

unique two-tier design. The security administrator

can specify programs with system-wide security sig-

niﬁcance to have their ﬁngerprints veriﬁed upon ex-

ecution by all system users. These ﬁngerprints are

stored in the Global Public vault and are readable

by all. However, in addition to this each user can

also store ﬁngerprints in their own vault for programs

relevant to them and their speciﬁc security require-

ments. This provides superior ﬂexibility compared

with previous approaches and recognizes that security

requirements differ from user to user. Furthermore, it

reduces the workload of the administrator as individ-

ual users can ﬂag programs for veriﬁcation without

requiring administrative intervention.

Trusted ﬁngerprinting also recognizes both that

not all system objects require veriﬁcation and that de-

pendency relationships exist between those that do.

For example, a program’s secure behaviour at runtime

may depend upon shared libraries, data ﬁles and even

other programs. These dependency relationships are

modelled using interconnected digraph structures that

reﬂect the shared dependencies of different programs.

This enables efﬁcient and secure runtime veriﬁcation

of objects.

3.3 Process Trust Levels

Programs are assigned a trust level which is stored

as part of their ﬁngerprint record and determines the

level of privilege that the process will normally have

when executed. Four hierarchical levels of trust are

deﬁned and these are designated Level 0 to 3 (L0–

3), with higher numbers indicating greater privileges.

Unveriﬁed code is classiﬁed as L0 and cannot access

a user’s vault. The effect of this is that L0 processes

cannot utilise any access tickets that the user pos-

sesses and are therefore unable to access cryptograph-

ically protected ﬁles where a ticket is required. Forc-

ing all unveriﬁed programs to be unprivileged limits

the damage from malicious code in the event of a suc-

cessful redirection or insertion attack. This resolves a

signiﬁcant limitation of previous schemes.

Beyond this, veriﬁed code can still run with lim-

ited privilege by being designated L1 which results in

the program being conﬁned to a limited set of specif-

ically identiﬁed ﬁle access tickets. This allows pro-

grams to execute with the minimum amount of priv-

ilege required and is particularly applicable to pro-

grams such as web browsers and e-mail clients that

are exposed to large amounts of untrusted code and

data. Applications labelled as L2 are considered to be

fully trusted and have access to all of the user’s tickets

while the L3 designation is reserved for the software

used to administer vault contents.

3.3.1 Determining Trust Levels

The trust level of a process at runtime depends pri-

marily upon the value assigned by the user and this is

the highest trust level it may hold. However, its trust

level can vary depending on other factors. Speciﬁ-

cally the trust level of a new child process is instanti-

ated as the minimum of that program’s user-speciﬁed

trust level and its parent’s current trust level

. With-

out this the secure operation of a trusted program

could be subverted if it were executed by malicious

code and supplied with dangerous parameters or in-

puts. This therefore prevents a malicious, untrusted

program from misusing a trusted one. As a result

it is not necessary to analyse the behaviour of each

individual program in response to all possible inputs

when assigning trust levels.

The manner in which the trust level of a program is

decided is designed to reﬂect both applicable security

semantics and the practical requirements of program

operation. For example, L0 applications are unable

to access cryptographically protected ﬁles where this

requires a ticket. However, where discretionary con-

trols allow, they may still access ﬁles that are only

protected with the opposite mode to that requested;

e.g., where read access is requested to a ﬁle that is

only write protected. This effectively minimises the

privileges of an L0 program by isolating it from any

sensitive data while not limiting its privileges to such

a degree that the user is forced to elevate its trust level

in order for the program to operate correctly. There-

fore, users need only increase an L0 program’s trust

level if there is a set of speciﬁc cryptographically pro-

tected objects that they wish the program to be able

to access. Furthermore, even in this scenario, unless

the program requires general access to cryptograph-

ically protected ﬁles, the user can set the trust level

The only exception to this rule is the special case where

an L3 program is executed and these can only be spawned

from L2 processes. Because of this, L3 programs must be

designed to prevent any subversion of their behaviour by

parameters or other external inputs.

SECRYPT 2007 - International Conference on Security and Cryptography

Attempt Match

Verify

Fingerprint

Execution

Initiated

Match Failed

Match

Succeeded

Fingerprint Valid

Fingerprint

Invalid

Deny

Execution

Execute

Trusted

Execute

Untrusted

Figure 1: Overview of process execution, matching and ver-

iﬁcation.

to L1, thereby conﬁning the program to these speciﬁ-

cally selected objects.

3.4 Matching, Veriﬁcation and

Execution of Programs

An overview of the process that occurs when a pro-

gram is executed is given in Figure 1. When a pro-

gram’s execution is initiated, a matching process oc-

curs where the user’s vault and Global Public vault are

searched for a ﬁngerprint entry corresponding to the

path of the ﬁle being executed. If no match is found

then the program will execute as untrusted (L0), oth-

erwise, if veriﬁcation is successful, the trust level is

set based upon the value in the matched ﬁngerprint

record and according to the rules described in the pre-

vious section. This ensures that the default behaviour

is to minimise the privileges assigned to a program

and this signiﬁcantly mitigates the risks due to inser-

tion or redirection attacks.

The local and global dependency digraphs identi-

ﬁed through the matching process are then merged to

construct a ‘runtime digraph’ for that program. This

is maintained for the lifetime of the process and is

used to identify and verify the relevant objects when

accessed. An abstracted example of a hypothetical

runtime digraph is given in Figure 2. In this exam-

ple, the program is linked with two different shared

libraries, which in turn have their own dependency ar-

rangements as shown by the direction of the arrows.

The program has both global and local (per-user) con-

ﬁguration ﬁles, both of which are identiﬁed in the run-

time digraph as a result of the merging process. The

program also has a helper application that it executes

at runtime.

Once matching is complete and the runtime di-

graph constructed, the program binary and the shared

libraries it depends upon are veriﬁed by calculating

Prog

Lib 1 Lib 2

Lib 3

Lib 4

Prog 2

(Helper)

Config

(Local)

Config

(Global)

Figure 2: Example runtime digraph.

one-way hashes and comparing these against the ﬁn-

gerprint values in the runtime digraph. The pro-

gram and shared library images are cached in mem-

ory between veriﬁcation and execution. This excludes

race condition-based attacks involving a privileged at-

tacker modifying a trusted object in the time between

veriﬁcation and use.

3.5 Data File Dependencies and

Automatic Updating

Non-executable data objects accessed by a trusted

program at runtime are checked against the runtime

digraph. If these are speciﬁed as a dependency for

that program then these are also loaded into memory

and veriﬁed. This allows for the ﬁles used by an ap-

plication which impact on security to be speciﬁcally

marked for veriﬁcation without the performance im-

pact of requiring all accessed ﬁles to be veriﬁed. This

design also facilitates veriﬁcation of non-natively ex-

ecutable programs such as scripts by specifying the

script source code as a dependency of the natively

executable interpreter or runtime environment. Trust

levels can also be assigned to individual scripts and

the effective trust level of the interpreter process is

modiﬁed to be the minimum of its current level and

that assigned to the data ﬁle dependency. This enables

interpreted scripts and other non-natively executable

programs to have different and independent trust lev-

els to that of their interpreter.

As is the case with programs and libraries, data

ﬁle dependencies are also cached at runtime. Be-

cause of this, modiﬁcations to these data objects by a

trusted process are captured by the system and used

to automatically update the user’s ﬁngerprint value

for that object. This is a convenient and secure ap-

proach to dealing with modiﬁcations to trusted data

that does not involve the level of administrative over-

head required in previous ﬁle integrity veriﬁcation

AN IMPROVED MODEL FOR SECURE CRYPTOGRAPHIC INTEGRITY VERIFICATION OF LOCAL CODE

schemes. Automatic ﬁngerprint updating can only be

performed by a program which is a parent of the data

ﬁle dependency in the program’s runtime digraph. In

this way the runtime digraph clearly deﬁnes the trust

relationships between a program and the libraries and

data ﬁles it depends on. However, users can also man-

ually update ﬁngerprint values in their vault using a

speciﬁcally designated L3 application.

4 CONCLUSION

Trusted ﬁngerprinting is a new model for local code

veriﬁcation which leverages the cryptographic infras-

tructure provided by the Vaults architecture. It re-

solves the weaknesses of previous approaches in that

it actively prevents execution of maliciously modi-

ﬁed objects, supports the veriﬁcation of non-natively

executable scripts, allows the automatic updating of

ﬁngerprints when modiﬁcations are made by trusted

code and limits the privileges assigned to unveriﬁed

processes. Other unique features include the use of

digraphs to represent the relationships between pro-

grams and the objects they depend upon for their se-

curity and a two-tier approach where users can inde-

pendently specify programs to be veriﬁed consistent

with their individual security requirements in addi-

tion to global defaults. This recognizes the need for

greater ﬂexibility in local code veriﬁcation architec-

tures in order to avoid administrative overheads that

otherwise represent a signiﬁcant impediment to their

adoption. Therefore, the adoption of the trusted ﬁn-

gerprinting code veriﬁcation technique, in conjunc-

tion with the other features of the Vaults model, has

the potential to signiﬁcantly mitigate the effects of

widespread malicious code and intrusion.

REFERENCES

Beattie, S. M., Black, A. P., Cowan, C., Pu, C., and Yang,

L. P. (2000). CryptoMark: Locking the stable door

ahead of the Trojan horse. White paper, WireX Com-

munications Inc.

Dowd, M., McDonald, J., and Schuh, J. (2007). The Art of

Software Security Assessment. Addison-Wesley.

Gong, L., Mueller, M., Prafullchandra, H., and Schemers,

R. (1997). Going beyond the sandbox: An overview of

the new security architecture in the Java Development

Kit 1.2. In Proceedings of the USENIX Symposium on

Internet Technologies and Systems.

Gordon, L. A., Loeb, M. P., Lucyshyn, W., and Richardson,

R. (2006). Eleventh annual CSI/FBI computer crime

and security survey. Technical report, Computer Se-

curity Institute (CSI).

http://GoCSI.com

Kalafut, A., Acharya, A., and Gupta, M. (2006). A study

of malware in peer-to-peer networks. In Proceedings

of the 6th ACM SIGCOMM on Internet Measurement.

ACM Press.

Kim, G. H. and Spafford, E. H. (1994a). The design and

implementation of Tripwire: A ﬁle system integrity

checker. In Proceedings of the 2nd ACM Conference

on Computers and Communication Security.

Kim, G. H. and Spafford, E. H. (1994b). Experiences with

Tripwire: Using integrity checkers for intrusion de-

tection. Technical Report CSD-TR-93-071, COAST

Laboratory, Purdue University, West Lafayette, IN

47907-1398.

Microsoft Corporation (2006). Introduction to code signing.

Online:

http://msdn.microsoft.com/workshop/

security/authcode/intro_authenticode.asp

Patil, S., Kashyap, A., Sivathanu, G., and Zadok, E.

(2004). I

FS: An in-kernel integrity checker and in-

trusion detection ﬁle system. In Proceedings of the

18th USENIX Large Installation System Administra-

tion Conference (LISA 2004).

Payne, C. (2003). Cryptographic protection for operating

systems. Research Working Paper Series IT/03/03,

School of Information Technology, Murdoch Univer-

sity, Perth, Western Australia.

Payne, C. (2004). Enhanced security models for operat-

ing systems: A cryptographic approach. In Proceed-

ings of the 28th Annual International Computer Soft-

ware and Applications Conference: COMPSAC 2004,

pages 230–235. IEEE Computer Society.

Reid, J. F. and Caelli, W. J. (2005). DRM, trusted comput-

ing and operating system architecture. In Proceedings

of the 2005 Australasian Workshop on Grid Comput-

ing and e-research, volume 44, pages 127–136.

Sailer, R., Zhang, X., Jaeger, T., and van Doorn, L. (2004).

Design and implementation of a TCG-based integrity

measurement architecture. In Proceedings of the 13th

USENIX Security Symposium, pages 223–238.

US-CERT (2006). Quarterly trends and analysis report,

volume 1, issue 2. Technical report, United States

Computer Emergency Readiness Team.

http://www.

us-cert.gov

SECRYPT 2007 - International Conference on Security and Cryptography