Coverage-guided Intelligent Test Loop - A Concept for Applying Instrumented Testing to Self-organising Systems

Jan Kantert, Sven Tomforde, Susanne Weber, Christian Müller-Schloer

2016

Abstract

Multi-agent systems typically consist of a large set of agents that act on behalf of different users. Due to inherent dynamics in the interaction patterns of these agents, the system structure is typically self-organising and appears at runtime. Testing self-organising systems is a severe challenge that has not received the necessary attention within the last decade. Obviously, traditional testing methods reach their limitations and are hardly applicable due to the runtime characteristics and dynamics of self-organisation. In this paper, we argue that we run into a paradoxon if we try to utilise self-organising testing systems. In order to circumvent parts of the underlying limitations, we propose to combine such an approach with instrumented testing.

References

  1. Chang, B., de Leos, R., Inverardi, P., and Magee, J., editors (2009). Software Engineering for Self-Adaptive Systems. Springer Verlag, New York, USA. ISBN 978-3-642-02161-9.
  2. Cheng, B. H., Sawyer, P., Bencomo, N., and Whittle, J. (2009). "a goal-based modeling approach to develop requirements of an adaptive system with environmental uncertainty". In Proceedings of the 12th International Conference on Model Driven Engineering Languages and Systems, MODELS 7809, pages 468-483, Berlin, Heidelberg. Springer-Verlag.
  3. Drysdale, D. (2016). Coverage-guided kernel fuzzing with syzkaller. https://lwn.net/Articles/677764/.
  4. Kantert, J., Scharf, H., Edenhofer, S., Tomforde, S., Hähner, J., and Müller-Schloer, C. (2014). A Graph Analysis Approach to Detect Attacks in Multi-Agent-Systems at Runtime. In 2014 IEEE Eighth International Conference on Self-Adaptive and Self-Organizing Systems, pages 80-89, London, UK. IEEE.
  5. McCaffrey, J. D. (2009). Software Testing. Fundamental Principles and Essential Knowledge. Samir Riad Press. ISBN-13: 978-1439229071.
  6. McKinley, P. K., Sadjadi, S. M., Kasten, E. P., and Cheng, B. H. C. (2004). "composing adaptive software". Computer, 37(7):56-64.
  7. Müller-Schloer, C., Schmeck, H., and Ungerer, T., editors (2011). Organic Computing - A Paradigm Shift for Complex Systems. Autonomic Systems. Birkhäuser Verlag.
  8. Prothmann, H., Tomforde, S., Branke, J., Hähner, J., MüllerSchloer, C., and Schmeck, H. (2011). Organic Traffic Control. In Organic Computing - A Paradigm Shift for Complex Systems, pages 431 - 446. Birkhäuser Verlag.
  9. Salehie, M. and Tahvildari, L. (2009). "self-adaptive software: Landscape and research challenges". ACM Trans. Auton. Adapt. Syst., 4(2):14:1-14:42.
  10. Sawyer, P., Bencomo, N., Whittle, J., Letier, E., and Finkelstein, A. (2010). "requirements-aware systems: A research agenda for re for self-adaptive systems". In Proceedings of the 2010 18th IEEE International Requirements Engineering Conference, RE 7810, pages 95-103, Washington, DC, USA. IEEE Computer Society.
  11. Schmeck, H., Müller-Schloer, C., Çakar, E., Mnif, M., and Richter, U. (2010). Adaptivity and Self-organisation in Organic Computing Systems. ACM Transactions on Autonomous and Adaptive Systems (TAAS), 5(3):1-32.
  12. Sutton, M., Greene, A., and Amini, P. (2007). Fuzzing: brute force vulnerability discovery. Pearson Education.
  13. Tomforde, S., Hähner, J., Seebach, H., Reif, W., Sick, B., Wacker, A., and Scholtes, I. (2014a). Engineering and Mastering Interwoven Systems. In ARCS 2014 - 27th International Conference on Architecture of Computing Systems, Workshop Proceedings, February 25-28, 2014, Luebeck, Germany, University of Luebeck, Institute of Computer Engineering, pages 1-8.
  14. Tomforde, S., Hähner, J., and Sick, B. (2014b). Interwoven Systems. Informatik-Spektrum, 37(5):483-487. Aktuelles Schlagwort.
  15. Tomforde, S. and Müller-Schloer, C. (2014). Incremental Design of Adaptive Systems. Journal of Ambient Intelligence and Smart Environments, 6:179 - 198.
  16. Tomforde, S., Prothmann, H., Branke, J., Hähner, J., Mnif, M., Müller-Schloer, C., Richter, U., and Schmeck, H. (2011). Observation and Control of Organic Systems. In Organic Computing - A Paradigm Shift for Complex Systems, pages 325 - 338. Birkhäuser Verlag.
  17. Tomforde, S., Steffen, M., Hähner, J., and Müller-Schloer, C. (2009). Towards an Organic Network Control System. In Proc. of the 6th Int. Conf. on Autonomic and Trusted Computing (ATC'09), held in Brisbane, Australia (Jul 7, 2009 - Jul 10, 2009), pages 2 - 16. Springer Verlag.
  18. Tomforde, S., Zgeras, I., Hähner, J., and Müller-Schloer, C. (2010). Adaptive control of Wireless Sensor Networks. In Proceedings of the 7th International Conference on Autonomic and Trusted Computing (ATC'10), held in Xi'an, China (October 26-29, 2010), pages 77 - 91.
  19. Welsh, K. and Sawyer, P. (2010). "understanding the scope of uncertainty in dynamically adaptive systems". In Wieringa, R. and Persson, A., editors, Requirements Engineering: Foundation for Software Quality, volume 6182 of Lecture Notes in Computer Science, pages 2-16. Springer Berlin Heidelberg.
  20. Weyuker, E. J. (1980). The oracle assumption of program testing. In Proceedings of the 13th International Conference on System Sciences (ICSS), pages 44 - 49, Honolulu, HI, USA. IEEE.
  21. Wooldridge, M. J. (1998). Agent Technology: Foundations, Applications, and Markets. Springer Science & Business Media, Berlin, Heidelberg, New York.
  22. Zalewski, M. (2015a). Bugs found by AFL. http://lcamtuf. coredump.cx/afl/.
  23. Zalewski, M. (2015b). Technical "whitepaper" for afl-fuzz. http://lcamtuf.coredump.cx/afl/technical_details.txt.
  24. Zhang, Z., Thangarajah, J., and Padgham, L. (2011). "automated testing for intelligent agent systems". In Gleizes, M.-P. and Gomez-Sanz, J., editors, Agent-Oriented Software Engineering X, volume 6038 of Lecture Notes in Computer Science, pages 66-79. Springer Berlin Heidelberg.
Download


Paper Citation


in Harvard Style

Kantert J., Tomforde S., Weber S. and Müller-Schloer C. (2016). Coverage-guided Intelligent Test Loop - A Concept for Applying Instrumented Testing to Self-organising Systems . In Proceedings of the 13th International Conference on Informatics in Control, Automation and Robotics - Volume 1: ICINCO, ISBN 978-989-758-198-4, pages 221-226. DOI: 10.5220/0005992702210226


in Bibtex Style

@conference{icinco16,
author={Jan Kantert and Sven Tomforde and Susanne Weber and Christian Müller-Schloer},
title={Coverage-guided Intelligent Test Loop - A Concept for Applying Instrumented Testing to Self-organising Systems},
booktitle={Proceedings of the 13th International Conference on Informatics in Control, Automation and Robotics - Volume 1: ICINCO,},
year={2016},
pages={221-226},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0005992702210226},
isbn={978-989-758-198-4},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 13th International Conference on Informatics in Control, Automation and Robotics - Volume 1: ICINCO,
TI - Coverage-guided Intelligent Test Loop - A Concept for Applying Instrumented Testing to Self-organising Systems
SN - 978-989-758-198-4
AU - Kantert J.
AU - Tomforde S.
AU - Weber S.
AU - Müller-Schloer C.
PY - 2016
SP - 221
EP - 226
DO - 10.5220/0005992702210226