Performance Impact of Fuzz Testing Windows Embedded Handheld Applications

Nizam Abdallah, Sita Ramakrishnan

Abstract

Mobile systems are increasingly impacting our personal and business lives. It is crucial that we develop mobile software applications that are robust, efficient and deliver value to a wide range of users. As a result, appropriate software testing methodologies need to be employed during the development of these mobile applications to ensure high level of quality and robustness. Software test automation methodologies and fuzz testing techniques have proven to be successful in finding defects during the development lifecycle. However, due to the fact that mobile devices are resource constrained devices with limited memory and CPU, there are performance constraints that need to be considered when developing a test automation framework for mobile devices. This research introduces Torqueo. Torqueo is an automated fuzz testing tool designed specifically to target Windows Embedded Handheld GUI applications. It is capable of interacting with GUI applications using either Win32 API or .NET reflection API, and it is also capable of executing test scenarios from pre-generated test data and randomly generated test data at run time. The experiments described in this paper discuss the performance impact on memory usage of invoking GUI controls using the Win32 API vs. .NET reflection.

References

  1. Abdallah, N., 2010. Performance Impact of Using .NET Reflection in .NET Compact Framework Applications. Retrieved 12 22, 2010, from Monash University - Clayton School of Information Technology Publications: http://www.csse.monash.edu.au/ publications/2010/tr-2010-260-full.pdf
  2. Abdallah, N., & Ramakrishnan, S., 2009. Automated Stress Testing of Windows Mobile GUI Applications. International Symposium on Software Reliability Engineering (ISSRE). Mysore, India: IEEE, ISSRE.
  3. Alsmadi, I., 2008. Building a GUI Test Automation Framework Using the Data Model. Saarbrucken, Germany: VDM Verlag Dr. Muller Aktiengsellschaft & Co.
  4. Chen, T. Y., Kuo, F.-C., Merkel, R. G., & Tse, T., 2009. Adaptive Random Testing: the ART of Test Case Diversity. The University of Hong Kong, Department of Computer Science, Pokfulam, Hong Kong..
  5. Chong, W. H, 2006. iDEN Phones Automated Stress Testing. World Academy of Science, Engineering and Technology.
  6. Codenomicon. (n.d.). Codenomicon Defensics 3.0. Retrieved 11 16, 2010, from Codenomicon: http://www.codenomicon.com/defensics/
  7. Forrester, J. E., & Miller, B. P., 2000. An Empirical Study of the Robustness of Windows NT Applications Using Random Testing. Retrieved 05 16, 2010, from The University of Wisconsin Madison: http:// pages.cs.wisc.edu/bart/fuzz/fuzz-nt.html
  8. Google. (n.d.). Monkey Runner. Retrieved 01 13, 2012, from Android Developers Documentation: http:// developer.android.com/guide/developing/tools/monke yrunner_concepts.html
  9. Hammersland, R., & Snekkenes, E., 2008. Fuzz testing of web applications. Retrieved 07 16, 2010, from AquaLab Research in Distributed Computing: http:// www.aqualab.cs.northwestern.edu/HotWeb08/papers/ Hammersland-FTW.pdf
  10. Memon, A. M., Pollack, M. E., & Soffa, M. L., 2000. A Planning-Based Approach to GUI Testing. Proceedings of The 13th International Software/Internet Quality Week. San Francisco, California.
  11. Microsoft. (2010, 04 08). Hopper Test Tool. Retrieved 07 18, 2010, from MSDN (Microsoft Developer Network): http://msdn.microsoft.com/en-us/library/ bb158517.aspx
  12. Microsoft. (n.d.). Windows Embedded OS. Retrieved 11 18, 2010, from Microsoft Windows Embedded: http:// www.microsoft.com/windowsembedded/en-us/about/ what.mspx
  13. Miller, B. P., Cooksey, G., & Moore, F., 2006. An Empirical Study of the Robustness of MacOS Applications Using Random Testing. Retrieved 05 16, 2010, from The University of Wisconsin Madison: ftp://ftp.cs.wisc.edu/paradyn/technical_papers/FuzzMacOS.pdf
  14. Miller, B. P., Fredrikson, L., & So, B., 1990. An Empirical Study of the Reliability of UNIX Utilities. Retrieved 05 16, 2010, from The University of Wisconsin Madison: ftp://ftp.cs.wisc.edu/paradyn/technical_papers/fuzz.pdf
  15. Miller, B. P., Koski, D., Lee, C. P., Maganty, V., Murthy, R., Natarajan, A., et al., 1995. Fuzz Revisited: A Reexamination of the Reliability of UNIX Utilities and Services. Retrieved 05 16, 2010, from The University of Wisconsin Madison: ftp://ftp.cs.wisc.edu/paradyn/ technical_papers/fuzz-revisited.pdf
  16. Sutton, M., Greene, A., & Amini, P., 2007. Fuzzing Brute Force Vulnerability Discovery. New Jersey, United States: Pearson Education.
  17. Ye, M., Fneg, B., Lin, Y., & Zhu, L., 2006. Neural Networks Based Test Case Selection Strategy for GUI Testing. Proceeding of the 6th World Congress on Intelligent Control and Automation. Dalian, China: IEEE.
Download


Paper Citation


in Harvard Style

Abdallah N. and Ramakrishnan S. (2012). Performance Impact of Fuzz Testing Windows Embedded Handheld Applications . In Proceedings of the 14th International Conference on Enterprise Information Systems - Volume 2: NTMIST, (ICEIS 2012) ISBN 978-989-8565-11-2, pages 371-376. DOI: 10.5220/0004152103710376


in Bibtex Style

@conference{ntmist12,
author={Nizam Abdallah and Sita Ramakrishnan},
title={Performance Impact of Fuzz Testing Windows Embedded Handheld Applications},
booktitle={Proceedings of the 14th International Conference on Enterprise Information Systems - Volume 2: NTMIST, (ICEIS 2012)},
year={2012},
pages={371-376},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0004152103710376},
isbn={978-989-8565-11-2},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 14th International Conference on Enterprise Information Systems - Volume 2: NTMIST, (ICEIS 2012)
TI - Performance Impact of Fuzz Testing Windows Embedded Handheld Applications
SN - 978-989-8565-11-2
AU - Abdallah N.
AU - Ramakrishnan S.
PY - 2012
SP - 371
EP - 376
DO - 10.5220/0004152103710376