CLIENT-TIER VALIDATION OF DYNAMIC WEB APPLICATIONS

Hideo Tanida

, Masahiro Fujita

, Mukul Prasad

and Sreeranga P. Rajan

Dept. of Electrical Engg. & Information Systems, The University of Tokyo, Tokyo, Japan

VLSI Design & Education Center, The University of Tokyo, Tokyo, Japan

Trusted Systems Innovation Group, Fujitsu Laboratories of America, Sunnyvale, CA, U.S.A.

Keywords:

Dynamic analysis, Validation, Web application.

Abstract:

Web applications pervade all aspects of human activity today. Rapid growth in the scope, penetration and

user-base of web applications, over the past decade, has meant that web applications are substantially bigger,

more complex and sophisticated than ever before. This places even more demands on the validation process

for web applications. This paper presents a case study of the validation of Ajax web Applications, where a

combination of dynamic crawling-based model generation and back-end model checking is used to compre-

hensively validate the client-tier of the web application. Our experience shows that such an approach is not

only practical in the context of applications of such size and complexity but can provide greater automation

and better coverage than current industrial validation practices based on testing. A couple of experimental

results are presented to show the effectiveness of the proposed approach.

1 INTRODUCTION

Web applications are ubiquitous today. The last

decade has witnessed rapid growth in both the scope

and the penetration of web applications. On one

hand, the wide-scale adoption of web applications in

all spheres of business activity has brought validation

and quality assurance of such applications into focus.

On the other hand, the use of WEB 2.0 technologies

such as AJAX (Asynchronous JavaScript and XML)

results in feature-rich and highly interactive web ap-

plications, thereby further complicating the task of

validating such applications.

Current industrial practice for the functional val-

idation of web applications still continues to largely

rely on manually written test cases which exercise

and check the application behavior one trace at a

time. There is a growing gap between the cover-

age, automation and scalability of traditional testing-

based validation methodologies and the validation re-

quirements of modern WEB 2.0 applications, which

has been acknowledged by validation researchers and

practitioners alike. Research on automated model

generation (Mesbah et al., 2008), model-based testing

(Mesbah and Deursen, 2009; Marchetto et al., 2008b;

Marchetto et al., 2008a; Benedikt et al., 2002), and

model checking (Alfaro, 2001) offers the promise to

address this gap. Speciﬁcally, there has a been re-

cent work on automated model generation (Mesbah

et al., 2008) and model-based testing (Mesbah and

Deursen, 2009) of AJAX applications that looks es-

pecially promising.

This paper addresses the problem of developing a

better and practical validation solution for WEB 2.0

application development. We propose a methodology

for functional validation of WEB 2.0 AJAX applica-

tions that is based on an efﬁcacious combination of

some previously proposed techniques in the valida-

tion literature and our own novel extensions to these

techniques. We present case studies of applying this

methodology to the validation of a WEB 2.0 AJAX

web applications. The main objectives and contribu-

tions of this paper are as follows:

• We propose a solution for the validation of the

client-tier behavior of modern industrial-strength

dynamic web applications. This solution is a com-

bination of several previously proposed technolo-

gies as well as several novel extensions of our own

which we believe are critical to its applicability

in an industrial context. Further, this approach

is fairly complementary to current industrial prac-

tices of web application validation and at least in

some respects, superior to these approaches.

• We apply the proposed approach to the validation

of WEB 2.0 AJAX applications and report on both

the successes and short-comings of our proposed

Tanida H., Fujita M., Prasad M. and P. Rajan S..

CLIENT-TIER VALIDATION OF DYNAMIC WEB APPLICATIONS.

DOI: 10.5220/0003510500860095

In Proceedings of the 6th International Conference on Software and Database Technologies (ICSOFT-2011), pages 86-95

ISBN: 978-989-8425-77-5

 2011 SCITEPRESS (Science and Technology Publications, Lda.)

approach. We feel these lessons would be vital in

developing and delivering the next generation of

industrial practices for web application validation.

The rest of the paper is organized as follows. In

the next section we survey related works. Section 3

presents our proposed validation approach. In Sec-

tion 5 we describe the validation case study, including

s description of target applications, the experimental

setup, procedure and results. We summarize and con-

clude the paper in Section 6.

2 RELATED WORK

One of the key objectives of this paper is to serve as a

bridge between current industrial practice in web ap-

plication validation and the research literature on this

topic. Thus it is relevant to review related works in

these two broad areas. However, the following survey

will be restricted to works addressing validation of the

client-tier of web applications and speciﬁcally that of

the navigational aspects of the behavior (in contrast

to security or performance aspects). The vast body of

research on web application validation spans several

other interesting and important areas such as valida-

tion targeted at the server-tier(s) of web applications,

or the vast area of security validation. Nevertheless,

these areas are beyond the scope of this paper.

Current industrial practice for the functional vali-

dation of web applications, and speciﬁcally for val-

idating the client-tier of dynamic web applications

(such as those employing AJAX) primarily involves

the use of capture-replay tools such as Selenium

WebKing

and Sahi

. Using these frameworks, users

manually exercise the application through various test

scenarios, one at a time. These actions are recorded

by the tool and can be replayed back at a later time,

usually with user-deﬁned assertions regarding ex-

pected behavior, inserted at various steps. This mode

of validation, however, requires a substantial amount

of manual effort. Frameworks such as JsUnit

, can

be used to perform unit testing of the JavaScript code

in the application. However, this kind of testing is by

its very nature very localized, language speciﬁc and

also manually intensive.

Our proposed method for client-tier validation of

dynamic web applications involves creating a state-

based model (in our case, a ﬁnite state machine) of

http://seleniumhq.org/

http://www.parasoft.com/jsp/solutions/soa solution.jsp?

itemId=86

http://sahi.co.in/w/

http://www.jsunit.net/

the client-tier behavior of the web application by au-

tomatically and dynamically crawling the deployed

web application. This model is then checked against

a temporal logic speciﬁcation (represented as a set

of properties), using model checking (Clarke et al.,

1999). There are several works which overlap with

one or more aspects of our approach but differ in one

or more of the following respects.

2.1 The Target Applications

Many of these papers (Alfaro, 2001; Ricca and

Tonella, 2001; Benedikt et al., 2002; Andrews et al.,

2005) target traditional (WEB 1.0) web applications,

which differ substantially from the dynamic (e.g.

AJAX-based ) WEB 2.0 applications we target, espe-

cially in terms of client-tier behavior and its valida-

tion. Speciﬁcally, (Ricca and Tonella, 2001; Benedikt

et al., 2002; Andrews et al., 2005) do use auto-

matic crawlers to extract a navigational model for val-

idation. However, we concur with the assessment

of (Mesbah and Deursen, 2009; Marchetto et al.,

2008b) among others that these crawlers would not

be applicable to WEB 2.0 applications. VeriWeb

(Benedikt et al., 2002) does claim to have some sup-

port for client-side scripts but there is not sufﬁcient

detail in the paper to conclude if it would apply to

modern enterprise AJAX applications. VeriWeb also

uses a model checking based veriﬁcation back-end

(VeriSoft) similar to us, but the paper is scant on ex-

perimental results on real applications. Another class

of relevant work, relevant in this context is the work

on GUI Application testing (Strecker and Memon,

2009) where they reverse engineer a model of the

desktop GUI application, with the objective of gen-

erating test cases. However, while modern dynamic

web applications share the rich user-interface and in-

teractivity of desktop GUI applications, they have

several unique features as well (such as asynchronous

client-server communication and a DOM-based user

interface). Thus GUI testing techniques cannot be ap-

plied as such to the problem at hand i.e. the validation

of dynamic web applications.

2.2 The Model Generation Process

Several papers (e.g. (Marchetto et al., 2008b)) rely

on a completely or mostly manual speciﬁcation of

the behavioral model being checked. We submit that

this would place unreasonable speciﬁcation burden on

the designers and veriﬁcation would not be practi-

cal, especially in an industrial setting. An automatic

crawler for AJAX application which we have used as

a basis for model extraction is developed (Mesbah

CLIENT-TIER VALIDATION OF DYNAMIC WEB APPLICATIONS

et al., 2008). However, as explained in Section 3.1

the crawler needed several enhancements to yield a

comprehensive model for industrial applications.

2.3 The Veriﬁcation Methodology

Almost all previous papers rely on trace-by-trace

testing as the end means to validate the behavioral

model. The authors of (Mesbah and Deursen, 2009;

Marchetto et al., 2008b) automatically generate test-

benches which exercise one trace at a time from the

model. While this deﬁnitely increase the level of au-

tomation compared to the current industrial practice

of manually written test-cases, the underlying vali-

dation is still test-case driven and hence the require-

ments and their checking very trace-speciﬁc. We

submit that our approach, which is based on model

checking, is much more natural, given that we have

a pre-generated navigational model. Further, we can

pose and check more general and global properties of

the application. We present several instances of this

and the advantages it provides, in Section 5. The tool

MCWEB (Alfaro, 2001) is one of the few instances of

the direct application of model checking to web appli-

cation navigational behavior. However, the work was

targeted towards WEB 1.0 applications. Another key

difference is that in our approach the model genera-

tion step is distinctly separated from the subsequent

model checking (i.e. checking is not done on the ﬂy).

Thus, while MCWEB relies on contractive µ-calculus

we use considerably simpler speciﬁcation formalisms

(a ﬁnite state machine for the navigational model and

temporal logic model checking).

3 PROPOSED METHOD

This paper deals with the validation of modern dy-

namic WEB 2.0 applications. Such applications

are characterized by a feature-rich, client-side user-

interface and have a high degree of user interactivity,

typically by the use of technologies such as AJAX and

Flash. Further, this work focuses on validating the be-

havior of the client-tier of the web application, and

speciﬁcally the navigational aspects of the behavior.

Our overall approach is a two step process. The

ﬁrst step is to extract a behavioral model of the client-

tier of the web application. This is done by dynam-

ically and automatically exercising the web applica-

tion through a process called guided crawling, cap-

turing the resulting behavior and representing it as a

ﬁnite-state-machine model. Guided crawling is im-

plemented as extension to the CRAWLJAX tool (Mes-

bah et al., 2008) and is described in Section 3.1.2. The

second step is to use temporal logic model checking

(Clarke et al., 1999) to validate various functional re-

quirements of the application against this model.

This two-step approach has several advantages.

First, it allows us to isolate the relatively expensive

dynamic crawling from the actual validation and do

the crawling once (or a few times) in a mostly require-

ment and validation independent manner. Second, it

allows us to extract a compact model of the naviga-

tional behavior, achieving compaction both by a ju-

dicious choice of the model representation as well as

by discarding irrelevant application-level details dur-

ing crawling. This accelerates both of the crawling

process and the downstream model checking. Third,

since all the requirements to be checked are often not

known when the ﬁrst rounds of validation is done,

performing the validation ofﬂine obviates the need to

repeat the expensive model generation step.

3.1 Model Generation

This section describes generation of the model. As

mentioned earlier, the behavioral model of the web

application client-tier is generated by dynamically,

automatically and comprehensively executing the

web application and capturing the observed behavior

in a mathematical model.

3.1.1 The Navigational Model

The model extracted during the crawling process pri-

marily captures the navigational aspects of the web

application’s client-tier behavior (as opposed to, for

example, behavior speciﬁc to its security policies or

its performance). Hence we will refer to this model

as the navigational model in the sequel.

Navigational models take form of FSM (ﬁnite

state machine), with their internal states deﬁned by

content of DHTML (Dynamic HTML) document

and transitions representing changes to content of

DHTML document before and after user operations.

DHTML documents which reﬂect dynamic changes

made by on-browser script and page reloading, repre-

sent views in AJAX-based web applications, and are

therefore suitable for deﬁning states within our FSM

model to verify navigational aspects. The models in-

tended to represent navigational aspects of web appli-

cation, contain only inter-state transitions before and

after user operations. Further, while the web appli-

cation may theoretically allow for an inﬁnite number

of different screens or web pages it is neither practical

nor particularly useful to attempt to capture, represent

or check these inﬁnite screens. Thus we focus on au-

tomatically navigating (through our guided crawling)

ICSOFT 2011 - 6th International Conference on Software and Data Technologies

a rich but ﬁnite number of screens within the web ap-

plication, while using innovative state representation

and abstraction mechanisms to represent this informa-

tion. Thus our navigationalmodel can be summarized

as follows. We use a state representation to capture

the content of each DHTML page/screen visited by

the application and a ﬁnite state machine (FSM) to

represent the navigational behavior of the web appli-

cation.

Figure 1 shows a graphical representation for a

fragment of navigational model corresponding to a

simple web application. Each state (node) is an-

notated by reference to DHTML document it corre-

sponds to, and transition (edge) is annotated by input

which has triggered the transition. Note that the each

input may be an event on an element which is iden-

tiﬁed by index-number-based XPath or a sequence of

operations performed based on a guidance directive

(described in detail in Section 3.1.2). The edge la-

beled “user-guided crawling”, corresponds to a tran-

sition made by operations from a guidance directive.

3.1.2 Guided Crawling

The CRAWLJAX tool (Mesbah et al., 2008) proposes

a fully automated crawling of the web application un-

der test. The tool does allow for some customization

of the crawling to the given web application. For ex-

ample by specifying classes of DOM elements to ex-

ecute actions on (e.g. HTML

<a/>

el-

ements) or the set of actions to execute (e.g.

click,

mouseover

). However, while this level of customiza-

tion is sufﬁcient for automating a very basic level of

exploration, based on button-clicks and link naviga-

tion on simple web applications, it is usually not suf-

ﬁcient to produce meaningful comprehensive models

on real-life enterprise web applications. Some typical

scenarios where the above kind of “automatic” crawl-

ing falls short are the following:

1. User Authentication: The most common situa-

tion is where the web application requires some

kind of user authentication (i.e. login) in order to

proceed to the next step. This may require, for

example, ﬁlling in speciﬁc, valid username and

password information into speciﬁc input boxes on

the current page and clicking a speciﬁc login but-

ton. Further, this authentication step may not oc-

cur as the ﬁrst step of the navigation but at a later

point (e.g. some applications like

Amazon.com

not require a login till the checkout stage). An-

other variant on this scenario is that applications

typically have different behaviors (privileges, fea-

tures, actions) for administrative versus regular

users and a good behavioral model should encom-

pass both these use-cases. Thus the tool should do

one set of exploration with an administrative login

and another with a regular user login.

2. Processing Forms with Several Sets of Data:

HTML forms are commonplace in modern web

applications (a user authentication panel is a

special instance of this) and typical exploration

would require exploring these forms with differ-

ent data-sets (for example one erroneous data set

and one correct one) for checking the response of

the application to correct and incorrect user in-

puts.

3. Excluding Behavior from the Model: In many in-

stances practical resource constraints dictate that

only speciﬁc actions on speciﬁc pages be exer-

cised when crawling the application. One reason

for doing this could be that the excluded features

and actions may be outside the scope of the ensu-

ing validation step and thus need not be included

in the model.

The above scenarios necessitate that the tool have

a mechanism for specifying and executing (during

crawling) a speciﬁc sequence of actions, with speciﬁc

and possibly multiple sets of data, which are executed

at speciﬁc stage in the application evolution, i.e. based

on the state of the web application. There could be

several such sequences, based on the different scenar-

ios exempliﬁed above. Further, the behavior produced

by this kind of custom crawling should be seamlessly

and correctly integrated into the other parts of the

model produced by the blind crawling (such as fully

automatic link navigation).

Our solution to the above requirement is the mech-

anism of guided crawling. The guidance is based on a

set of Guidance Directives that are speciﬁc to the tar-

get application and supplied as input to our crawler.

Deﬁnition 3.1 (Guidance Directive). A Guidance Di-

rective G = (p, A ) is an ordered pair that consists of

a predicate p that is evaluated on the current web ap-

plication state and an action sequence A . The action

sequence A = (α

, α

. . . , α

) is a sequence of atomic

actions α

. Each atomic action α = (e, u, D ) is a triple

consisting of a DOM element e, a user-action u and a

set of data-instances D (potentially empty) associated

with u.

A guidance directive G , as per Deﬁnition 3.1 in-

cludes the predicate that determines when G should

be activated. The predicate is evaluated on the cur-

rent state of the web application during the crawl-

ing. In our case the state is essentially DHTML

document shown in current screen/web-page on the

browser. Therefore, predicate p can be supplied as

an expression which yields a binary value given a

CLIENT-TIER VALIDATION OF DYNAMIC WEB APPLICATIONS

index

state_5

user-guided crawling onclick /HTML/BODY[1]/A[1] onclick /HTML/BODY[1]/INPUT[3] onclick /HTML/BODY[1]/A[1]

state_6

onclick /HTML/BODY[1]/INPUT[1]

onclick /HTML/BODY[1]/A[1]

onclick /HTML/BODY[1]/INPUT[1]

Figure 1: Graphical view of a screen transition diagram.

DHTML document. The action sequence A repre-

sents the sequence of user actions that must be ex-

ecuted on the web application upon activation of G

(i.e. when p is true in the current state/screen), along

with the associated data, if applicable. Each atomic

action α in A is a simple (browser-based) user action

u on a particular DOM element e on the current web-

page/screen. For example, a typical action u would

be a click or mouse-over on a DOM element (such

as a link or button). Such actions have no associated

data. Hence the D component of the atomic action

would be the empty set

0. Another class of actions are

those that correspond to the choice or input of some

data string, for example selecting an option from a

element or assigning a string value to an

element etc. In these cases the D compo-

nent would represent the set of data values we would

like to test/exercise the element with. Upon activa-

tion of the guidance directive, the tool constructs a

concrete action sequence from A by picking speciﬁc

data-values from the set D corresponding to each α in

A and executes it on the web application. It does this

systematically, for each combination of data-values

represented in A .

Algorithms 1 and 2 present the pseudo code for

the overall model generation algorithm based on

guided crawling. The main procedure,

GuidedCrawl

(Algorithm 1) is supplied a web application, W and

a set of associated guidance directives, G

set

. It ini-

tializes the navigational model M, loads the web ap-

plication in the browser and invokes the inner pro-

cedure, GuidedCrawlFromState (Algorithm 2) on the

initial web-page (InitPage). GuidedCrawlFromState

does the actual crawling and recursively calls itself on

new successor states/screens. The various other func-

tions used in the algorithm are described as follows:

• IsVisited: Checks if the state S has been visited

by a previous invocation of GuidedCrawlFrom-

State. This check takes into account any state-

abstractions implemented for this step (explained

further in Section 3.1.3).

• MarkVisited: Marks state S as visited to exclude

it from future guided crawls

• AddState: Records the state S in the navigational

model M as a newly discovered state

Algorithm 1: GuidedCrawl(W, G

set

1: M =

2: InitPage ← LoadBrowser(W)

3: GuidedCrawlFromState(InitPage)

3: return M

Algorithm 2: GuidedCrawlFromState(S).

1: if IsVisited(S) then

1: return

2: end if

3: MarkVisited(S)

4: AddState(S, M)

5: Actions ← FindActions(S)

6: for all G (p, A ) ∈ G

set

7: if p(S) =

true

then

8: Actions ← Actions ∪

ComputeActionSequences(A )

9: end if

10: end for

11: for all a ∈ Actions do

12: nextState ← Execute(a, W, S)

13: AddTransition(nextState, S, M)

14: GuidedCrawlFromState(nextState)

15: UndoTransition(a, W, S)

16: end for

• FindActions: Computes the set of primitive (non-

guided) user actions (e.g. clicks, mouseovers etc.)

that can be executed on the current screen S.

• ComputeActionSequences: Computes concrete

sequences of actions from the guidance direc-

tive G by picking speciﬁc data values in its con-

stituent atomic actions α. Computes all possible

sequences that can be created by various choices

of the speciﬁed data-values.

• Execute: Executes the action (or action se-

quence) a on the web applicationW, which is cur-

rently in state/screen S.

• AddTransition: Records the transition from state

S to state nextState in the model M.

• UndoTransition: Functionally reverses the tran-

sition S → nextState on W to restore it to state S.

ICSOFT 2011 - 6th International Conference on Software and Data Technologies

3.1.3 Model Reduction

One of the principal advantages of our two-step ap-

proach, i.e. model generation followed by model

checking, is the ability to perform the validation on

a much more compact and abstracted model than the

full HTML content available at the browser-level dur-

ing model generation. This enables a much more efﬁ-

cient check. There are several sources of compression

we utilize in generating the ﬁnal model to be checked:

1. Validation-Directed Inclusion/Exclusion of

Events: It is obvious that not all of transitions

resulting from ﬁring a certain kind of event on

a certain class of element would be of relevance

from a validation perspective. Therefore, we

specify to the crawler a list of tuples of HTML el-

ements set and event types, that should deﬁnitely

be included and those that should be deﬁnitely

be excluded during crawling. The speciﬁcation is

done by the user on application-speciﬁc basis, as

an input to the model generation step. In each of

speciﬁcations above, set of HTML elements are

speciﬁed using XPath.

2. State Abstraction: In our proposed navigational

model, the determination of whether given two

states are equivalent or not is performed based on

content of DHTML document. While states corre-

sponding to DHTML documents with same con-

tent is considered to be equivalent in our naviga-

tional model, there are several instances of situa-

tions where two “similar-looking” DHTML pages

should be considered to be equivalent. The fol-

lowings are some of the scenarios where “exact”

determination of equivalence between DHTML

pages falls short.

(a) Current time (variable from run-time environ-

ment) included in page

(b) Server-side variables (which we do not explic-

itly reset), and are not of our interest exposed to

DHTML content

Therefore, we have introduced state abstraction

techniques which accepts user-given XPaths of el-

ements to be removed from DHTML documents

observed. When determining equivalence be-

tween two given states, document elements which

match the given XPaths and their descendants

are removed from DHTML document structures

corresponding to the states and thereafter com-

pared. This criteria for state-equivalence are im-

plemented within the IsVisited() function in Algo-

rithm 2.

3.2 Model Validation

As mentioned in Section 2 one of the key differences

between our approach and prior art in this area is that

while other approaches resort trace-by-trace check-

ing of the behavior a la traditional testing, we pro-

pose to check the navigational model as a whole us-

ing the formal technique of model checking (Clarke

et al., 1999). As pointed out in Section 3 the use of

a pre-generated navigational model, compactly rep-

resented as a ﬁnite state machine makes the applica-

tion of model checking both easy and very efﬁcient.

We further claim that several navigational and other

types of requirements that are typically checked on

web applications can be quite naturally formulated as

properties in temporal logic (Clarke et al., 1999), the

input language of model checkers. In the following

we present a few examples of such classes of require-

ments and speciﬁc instances in each class.

1. Screen Transition Requirements: The simplest

and most common check on web applications is of

the form: A user input i with the web application

on Screen A takes it to Screen B. Here, screens

A and B may be screens or pages of the web ap-

plication, identiﬁed by the presence or absence of

certain features, widgets or DOM elements, while

input i may be a simple input like a mouseover or

button/link click or a more complicated sequence

of such actions interspersed with data inputs to

various widgets on the screen. This kind of re-

quirement may be further generalized in checking

(for example) that Screen B follows A in one, all

or none of the valid execution sequences of the

web application. Some speciﬁc examples of this

class of requirements could be:

• In a web application with user authentication:

The LOGOUT screen is always preceded by the

• On a utilities web-site under the bill-payment

section: If the CONFIRM button is clicked on

the PAYMENT-DETAILS screen then the next

screen is always the

RECEIPT

screen.

2. Navigation Structure/Usability Requirements:

This kind of requirement would apply checks to

the overall structure of the navigational model,

with the intent of checking the ease with which

the user can access various functionalities offered

by the application. This class of requirements is

by its very intent, global and hence ideally suited

for model checking on a pre-extracted model

(versus real-time or trace-by-trace checking).

Some examples of requirements in this class are:

• All features of are accessible within 5 clicks,

starting from the home page

CLIENT-TIER VALIDATION OF DYNAMIC WEB APPLICATIONS

• The initial page is accessible from every screen

Since each screen of the web application and all

features/widgets or DOM elements on each screen are

captured as part of the state representation in our nav-

igational model, it is possible to formulate and check

the requirements above (and several other categories

of requirements typically checked on web applica-

tions) as properties in a temporal logic (Clarke et al.,

1999) (e.g. Computation Tree Logic (CTL) or Linear

Temporal Logic (LTL)) with the following steps.

3.2.1 Validation Scheme

First, we annotate each state within the navigational

model with values of “atomic propositions” used in

the properties. In our methodology, we allow atomic

propositions be any expression which hash same

DHTML document to same binary value. After the

annotation, there will be two kinds of variables within

the model: unique ID for each state and annotating

variables corresponding to value of atomic proposi-

tions in each state. Then we convert the FSM model

including annotating variables, into the format which

is accepted by external model checkers. While inter-

state transition relations are described using values of

ID variables, property can be described using any an-

notating variables containing values of atomic formu-

las which hashes equivalent state into same value.

Atomic propositions which are hash functions of

DHTML documents into binary output, can be given

by the user as a procedure in some programming lan-

guage. And they can take forms which are quite sim-

ilar to assertions used in existing automatic testing

tools for client-tier behavior of AJAX-based web ap-

plications such as Selenium. Therefore, it is quite

easy for developers in ﬁeld to migrate to our method-

ology. Examples of atomic expressions include avail-

ability of a node with given type and attribute, and

availability of text content which matches a given reg-

ular expression.

3.2.2 Temporal Property Templates

Still, writing complex temporal logic formula based

on those atomic propositions is troublesome work

for developers. So we have decided to introduce

template-based approach which is based on a few

temporal classes of properties in property speciﬁca-

tion. The idea is based on the notion from a study

that most veriﬁcation requirements observed in prac-

tice, can be captured by propertiesin a limited number

of temporal classes (Dwyer et al., 1999). Temporal

classes of properties we have targeted in our frame-

work are shown in Figure 2, where p,p

and p

are

atomic propositions which are evaluable in a given

1. G(p): Globally p is true

2. p

, i → p

: After transiting from a

state where p

is true, with an input or

a guidance-directive-driven input sequence

which matches i, or with any input (i =nil),

is always satisﬁed

3. p

→ F p

: After transiting from a state where

is true, eventually reaches a state where p

is true

4. p

→ Pp

: Need to reach a state where p

true, prior to reach a state where p

is true

Figure 2: Temporal property templates targeted.

1. G(p): A state S where p is false

2. p

, i → p

: A tuple (S

,T,S

), where S

is a

state in which p

is true, S

is a state in which

is false, and T is a transition from S

made by an input or a guidance-directive-

driven input sequence which matches i

3. p

→ F p

: A tuple (S

,T ,S

), where S

is a

state in which p

is true, S

is a state in which

is false, and T is a non-empty sequence of

transitions from S

to S

4. p

→ Pp

: A sequence of transitions T ,

which reaches a state where p

is true from

initial state, without going through any state

where p

is true

Figure 3: Counter examples for the property templates.

single state, based on DHTML page contents corre-

sponding to the state. In the second class of proper-

ties, inputs in i are speciﬁed as a tuple of a unique

ID in each DHTML document, and event type or in-

put data. Forms of counter example for the temporal

property templates are shown in Figure 3.

4 TOOL IMPLEMENTATION

We have implemented our model-based testing ap-

proach, in a system which is comprised of two com-

ponents. One of the components of the system is

an extended version of open-source CRAWLJAX tool,

and the other is a custom model checker called GO-

LIATH, which is dedicated for checking requirements

on screen transitions explained in Section 3.2.

4.1 Model Generation

We have extended CRAWLJAX-1.7, to include the fol-

ICSOFT 2011 - 6th International Conference on Software and Data Technologies

lowing features required for its application to our

ﬂow.

1. Guided Crawling (Section 3.1.2): The feature

was implemented so that predicate p in Deﬁni-

tion 3.1 can be supplied as a Java method which

accepts Document object and returns boolean

value. And each of actions α

in A , can be spec-

iﬁed as a tuple of XPath string specifying target

element (e), and event type string (u) or input data

string set (D ).

2. On-The-Fly Model Reduction (Section 3.1.3):

The feature was implemented in such a way, that

user can specify a set of strings each correspond-

ing to XPath of elements to abstract out, and all

of elements matching the XPath and their descen-

dants are removed from the document structure

corresponding to the state.

4.2 Model Validation

We have implemented a custom model checker GO-

LIATH, which is dedicated for checking properties

with atomic propositions and temporal forms de-

scribed in Section 3.2.1 and 3.2.2 respectively.

GOLIATH is implemented using Ruby program-

ming language, and atomic propositions (p, p

and

in Figure 2 and 3) are expected to take form of

Ruby expressions and may refer to DHTML docu-

ment object with identiﬁer

doc

with Nokogiri

HTML

parser API. The following is a very simple instance of

atomic proposition supported, which is true iff there

is some

<a/>

element with

attribute value

doc.xpath(’//a[@id="login"]’).any?

A little more complex and practical expression exam-

ple can be found in the following.

login_xpath = ’//a[@id="login"]’;

logout_xpath = ’//a[@id="logout"]’;

login_avail = doc.xpath(login_xpath);

logout_avail = doc.xpath(logout_xpath);

login_avail.any? != logout_avail.any?

The expression above (hereafter referred to as

not together

) is true only in states where only one of

logout

DOM object exists and they do not co-

exist (Note in Ruby, that sequences of expressions can

be evaluated as an expression which yields value of

their ﬁnal expression). We can write the following

property with p

not together

for example.

G(p

not together

) (1)

The property can be used to make sure that there is no

state, where both of

logout

exist or do not ex-

ist, in the application behavior extracted to the model.

http://nokogiri.org/

5 CASE STUDY

The proposed method has been implemented and

evaluated by applying them to examples including a

real industrial one. To assess the efﬁcacy and utility

of our approach and the corresponding implemented

tool, we have conducted a number of case studies fol-

lowing guidelines from (Kitchenham et al., 1995).

5.1 The Examples Targeted

We used two examples, the ﬁrst one (Example 1) is

taken from a textbook (Zammetti, 2006) on AJAX

based web application developments and the other

(Example 2) is a real industrial application.

• Example 1 has 8,004 lines of Java codes

• Example 2 is a business process manager com-

prised of 58,701 lines of Java code, 61,541 lines of

JavaScript codes, and 90,742 lines of JSP codes.

YUI

is the AJAX library used.

Typical screen images of user interface for the two

examples are shown in Figure 4 and Figure 5, respec-

tively. All experiments are performed on Intel Core2

Duo CPU E8400@3.00GHz.

Figure 4: Typical screen image of Example 1.

Figure 5: Typical screen image of Example 2.

5.2 Model Generation

Appropriate guidance directives, which are the keys

for meaningful and relative quick model generations,

http://developer.yahoo.com/yui/

CLIENT-TIER VALIDATION OF DYNAMIC WEB APPLICATIONS

are given to generate the corresponding screen tran-

sition diagrams for the two examples. The directives

include operations for logging in by providing user-

names and passwords whenever there is a screen im-

age having login prompts. Also, when generating the

screen transition diagrams, the maximum depth of op-

erations for each application is set to 9.

Table 1 shows the model (screen transition dia-

gram) generation results for the two examples. Please

note that although we also tried to generate tests using

the approach shown in (Mesbah and Deursen, 2009),

it did not ﬁnish generations for neither of the exam-

ples, simply because there are so many cases from its

exhaustive analysis even with the depth limit.

Table 1: Model generation results.

Example 1 Example 2

Time (sec) 3175 166114

#State 49 763

#Transition 360 4037

Avg. HTML size (kB) 11 2890

5.3 Properties for Model Checking

For Example 1, we extracted 7 properties, including

the one shown in Figure 6, from the textbook relating

to the manipulations of tabs shown in the top of Fig-

ure 4. The property in Figure 6 which is in the form

of p

, i → p

in Figure 2 says, if the tab “Day At A

Glance” in Figure 4 is clicked, the orange area which

shows the current screen name must display “Day At

A Glance”. The other properties also check if a tab is

clicked, its corresponding screen must be shown.

For Example 2, 14 properties each of which corre-

sponds to each test case that the program developers

have used for their testing, are prepared. As Example

2 is a real industrial application, their details cannot

be shown here, but they are somehow similar to the

ones for Example 1 in the sense of manipulations.

5.4 Model Checking Results

All 7 properties for Example 1 have been model

checked within one second. Only one of the 7 prop-

erties shown in Figure 6 generates a counter example

shown in Figure 7. The counter example is automati-

cally converted to a test program which performs in-

puts required to reach and activate it, by a helper tool.

Invoking the test program obtained actually generates

an error shown in Figure 8 within a few seconds. Also

in the log of the Java server software, an error mes-

sage which indicates “Null Pointer Exception” is re-

ported as shown in Figure 9. This is a real bug in a

program shown in the textbook.

For Example 2, all properties have been checked

in 337 seconds, and all 14 properties have been

proved to be correct. As the time for model checking

is rather short, many more properties can be checked

within practical time.

As you can see from the above, once the model

is generated, model checking is relatively very quick,

and various properties can be examined with reason-

able time even for large industrial examples.

doc.xpath(’//img[@id="dayAtAGlance"]’).any? ,

/HTML/BODY[1]/TABLE[1]/TBODY[1]/TR[1]/TD[1]/TABLE[1]/TBODY[1]/\

TR[1]/TD[1]/TABLE[1]/TBODY[1]/TR[1]/TD[1]/IMG[2]:onclick

doc.xpath(’//img[@src="http://localhost:8080/\

theorganizer/img/head_dayAtAGlance.gif"]’).any?

Figure 6: One of properties used for Example 1.

State:["state_534-raw.html"]

Transition: ["/HTML/BODY[1]/TABLE[1]/\

TBODY[1]/TR[1]/TD[1]/TABLE[1]/TBODY[1]/TR[1]/TD[1]/\

TABLE[1]/TBODY[1]/TR[1]/TD[1]/IMG[2]:onclick"]

State:["state_536-raw.html"]

(a) A counter example for the property

(A transition which does not meet the property).

State:["index-raw.html"]

Transition: ["/HTML/BODY[1]/CENTER[1]/ ... /INPUT[1]:onclick"]

State:["state_5-raw.html"]