![](bg6.png)
(i.e. buttons representing ‘0’ – ‘9’), the AUT reacts
differently compared to what it should have reacted
according to its specifications. Therefore, the
Verification module of our framework detected the
difference between the modelled and the actual
implementation and reported it correctly.
Figure 5: Results of the Microsoft Calculator test.
Table 1 shows the average time needed to
perform the tests on the Microsoft® Calculator
application. These results clearly indicate the vast
increase of time when increasing the length of the
test case. The time limitations are posed by the
libraries of the Ranorex® Studio system, therefore
these limitations currently cannot be altered unless
new means that allow the interaction with the GUI
without user interference are brought to light.
Table 1: Experimental measurements of time.
Test case length (GUI events) Average Time (seconds)
5 12.069
10 23.758
15 35.438
20 46.112
50 116.854
5 CONCLUSIONS AND FUTURE
WORK
This paper presented a specification-based,
automatic GUI testing framework. A proof of
concept application has been developed. Two
different sets of tests were executed on the proposed
framework. The first set verified that the proposed
system can automate the GUI testing procedures,
whereas the second demonstrated that efficient
automated GUI testing may be achieved through the
use of specifications, by successfully detecting and
reporting erroneous GUI behaviour during actual
execution. Future research will attempt to automate
the whole process of constructing the specifications
since this is the only manual work that the tester has
to perform.
REFERENCES
Barnett, M., Grieskamp, W., Nachmanson, L., Schulte,
W., Tillmann, N., Veans, M., 2003. Model-Based
Testing with AsmL.NET. In Proceedings of the 1st
European Conference on Model-Driven Software
Engineering.
Briand, L., Labiche, Y., 2002. A UML-based approach to
system testing. Journal of Software and Systems
Modeling, vol. 1, no. 1, pp. 10-42. Springer.
Edwards, S. H., 2001. A framework for practical,
automated black-box testing of component-based
software. Journal of Software Testing, Verification
and Reliability, vol. 11, no. 2, pp. 97-111.
Kim, Y. G., Hong, H. S., Bae, D. H., Cha, S. D., 1999.
Test cases generation from UML state diagrams IEE
Proceedings – Software, vol. 146, no. 4, pp. 187-192.
Krichen, M., Tripakis, S., 2004. Black-box conformance
testing for real-time systems. Journal of Model
Checking Software, pp. 109-126. Springer.
Li, K., Wu, M,. 2004. Effective GUI Test Automation:
Developing an Automated GUI Testing Tool (Chapter
2). Sybex Inc.
Memon, A., Banerjee, I., Nagarajan, A., 2003. GUI
ripping: Reverse Engineering of graphical user
interfaces for testing. In Proceedings of The 10
th
Working Conference on Reverse Engineering.
Memon, A., Nagarajan, A., Xie, Q., 2005. Automating
regression testing for evolving GUI software. Journal
of Software Maintenance and Evolution: Research and
Practice vol.17, no. 1, pp.27-64.
Memon, A., Pollack, M., Soffa, M., 2001. Hierarchical
GUI test case generation using automated planning.
Journal of IEEE Transactions on Software
Engineering, vol. 27, no. 2, pp. 144-155.
Offutt, J., Liu, S., Abdurazik, A., Ammann, P., 2003.
Generating test data from state-based specifications.
Journal of Software Testing, Verification and
Reliability, vol. 13, no. 1, pp. 25-53. John Wiley &
Sons.
Paiva, A. C. R., Faria, J. C. P., Tillmann, N., Vidal, R. F.
A. M., 2005. A Model-to-implementation Mapping
Tool for Automated Model-based GUI Testing. In
Proceedings of ICFEM'05, Manchester, UK.
Ranorex GmbH, 2009. Ranorex – GUI Automation &
Automated Testing Tool. www.ranorex.com
Tahat, L. H., Bader, A., Vaysburg, B., Korel, B., 2001.
“Requirement-based automated black-box test
generation. In Proceedings of the 25th International
Computer Software and Applications Conference on
Invigorating Software Development, pp. 489-495.
SPECIFICATION-BASED AUTOMATED GUI TESTING
323