Table 3: Faults found in Menu component.
Error Type W/E V1 V2
No Character encoding W 1 2
No attribute E 1 1
Element not allowed E 10 34
Cannot generate system identifier W 2 3
No system identifier could be generated E 3 8
Undefined entity E 2 3
Reference not terminated by REFC delimiter W 3 8
Reference to external entity in attribute value W 3 8
Missing end tag E 1 1
menu component. Interestingly, Mamoste found 407
HTML errors and 146 warnings in version 1. As seen
in the last column, only a small fraction of these er-
rors in the earlier version had been corrected by de-
velopers. The outcomes of this experiment are in
favor of our approach. First, Mamoste found more
HTML errors than found by maintenance team. Sec-
ond, Mamoste increased the ratio of error detection.
Table 3 shows the faults in menu component. This
component is used in all SGB.net pages in both ver-
sions. Unlike Table 2, the number of errors in version
2 is more than those of version 1. This increase is due
to a reimplementation of the menu component.
These experiments show the followings. First,
Mamoste improves efficiency of test in terms of using
less test inputs and collecting more HTML outputs.
Second, Mamoste increases code coverage by execut-
ing different branches of tested program than manual
testing. Finally, Mamoste increases the success ratio
of HTML error detection in ASP.NET sites.
6 RELATED WORK
Dynamic test generation is a promising research area.
DART (Godefroid et al., 2005) employs a combina-
tion of symbolic execution and random testing for C
programs. CUTE (Sen et al., 2005) employs concolic
execution for unit testing Java programs with memory
graphs as inputs. SAGE implements a whitebox fuzz
testing based on symbolic execution and dynamic test
generation. SAGE works on security testing of C
and C++ windows applications. Emmi et al. gen-
erate test inputs for database applications using con-
colic execution (Emmi et al., 2007). Recently, auto-
mated test generation is being applied to web applica-
tions. Wassermann et al. (Wassermann et al., 2008)
and Apollo (Artzi et al., 2010) use concolic testing
for automated test generation to web applications in
PHP. Wassermann et al. targets on SQL injection on
PHP applications. Apollo works on crashes and mal-
formed HTML outputs of PHP applications whereas
we focus on ASP.NET applications which are heavy-
weight compared to PHP.
As for ASP.NET web applications, SAFELI, a
static analysis framework (Fu et al., 2007), uses sym-
bolic execution to identify SQL injections whereas
Mamoste aims to detect execution errors and mal-
formed HTML outputs of these applications. Mi-
crosoft Pex (Pex) uses concolic execution to gener-
ate unit test for C#, Visual Basic and F# applications.
Working together with Moles for isolating units, Pex
creates unit tests for ASP.NET applications. Since
Pex works for public methods only, the modifier of
event handlers needs to be changed for testing. For
these tests, MS Research Team (MSR) simulates In-
ternet Information Services (IIS) with Moles. In con-
trast, Mamoste composes http requests as if a user
enters inputs and fires events; thus, working as both
users and browsers and do not need to simulate IIS.
7 CONCLUSIONS
In this paper we presented an automated tool called
Mamoste to detect execution errors and malformed
HTML pages generated in ASP.NET web applica-
tions. Our experiments revealed numerous bugs on
generated HTML files and a faulty component on a
highly used ASP.NET application. Mamoste detected
errors that lived through the versions of this applica-
tion and showed its effectiveness.
Our limitations are as follows. First, Mamoste
needs manual instrumentation to catch branch condi-
tions and related to web inputs. We plan to remove
this manual instrumentation in the future. Second,
due to the constraint solver’s limitation, currently,
only equality and inequality of strings are supported.
We plan to support other string operations in con-
straints, such as subset and prefix. Finally, Mamoste
does not supports no primitive types, which could be
avoided by linking custom types and their attributes
the web controls of tested page.
REFERENCES
(Pex). http://research.microsoft.com/en-us/projects/pex/.
(MSS). http://www.solverfoundation.com.
(MSR). http://research.microsoft.com.
Artzi, S., Kie˙zun, A., Dolby, J., Tip, F., Dig, D., Paradkar,
A., and Ernst, M. D. (2010). Finding bugs in web ap-
plications using dynamic test generation and explicit
state model checking. IEEE TSE, 36(4):474–494.
Emmi, M., Majumdar, R., and Sen, K. (2007). Dynamic test
input generation for database applications. In Proc. of
ISSTA.
DETECTING EXECUTION AND HTML ERRORS IN ASP.NET WEB APPLICATIONS
177