Evaluating the Evaluators - An Analysis of Cognitive Effectiveness Improvement Efforts for Visual Notations

Dirk van der Linden, Irit Hadar

2016

Abstract

This position paper presents the preliminary findings of a systematic literature review of applications of the Physics of Notations: a recently dominant framework for assessing the cognitive effectiveness of visual notations. We present our research structure in detail and discuss some initial findings, such as the kinds of notations the PoN has been applied to, whether its usage is justified and to what degree users are involved in eliciting requirements for the notation before its application. We conclude by summarizing and briefly discussing further analysis to be done and valorization of such results as guidelines for better application.

References

  1. Bramer, W.M., Giustini, D., Kramer, B.M.R. and Anderson, P.F., 2013. The comparative recall of Google Scholar versus PubMed in identical searches for biomedical systematic reviews: a review of searches used in systematic reviews. Systematic reviews, 2, 1-9.
  2. Brereton ,P., Kitchenham, B.A., Budgen, D., Turner, M. and Khalil, M., 2007. Lessons from applying the systematic literature review process within the software engineering domain. Journal of Systems and Software, 80, 571-583.
  3. Da Silva, F.Q.B., Santos, A.L.M., Soares, S., Franca, A.C.C., Monteiro, C.V.F. and Maciel, F.F., 2011. Six years of systematic literature reviews in software engineering: An updated tertiary study. Information and Software Technology, 53, 899-913.
  4. Davies, I., Green, P., Rosemann, M., Indulska, M. and Gallo, S., 2006. How do practitioners use conceptual modeling in practice? Data & Knowledge Engineering, 58(3), pp.358-380.
  5. Engström, E. and Runeson, P., 2011. Software product line testing- a systematic mapping study. Information and Software Technology. 53, 2-13.
  6. Gehanno, J.-F., Rollin, L., Darmoni, S., 2013. Is the coverage of Google Scholar enough to be used alone for systematic reviews. BMC medical informatics and decision making, 13, 7.
  7. Granada, D., Vara, J.M., Brambilla, M., Bollati, V. and Marcos, E., 2013. Analysing the cognitive effectiveness of the WebML visual notation. Software & Systems Modeling, pp.1-33.
  8. Green, T.R.G. and Petre, M., 1996. Usability analysis of visual programming environments: a 'cognitive dimensions' framework. Journal of Visual Languages & Computing, 7(2), pp.131-174.
  9. Greenhalgh, T., 2014. How to read a paper: The basics of evidence-based medicine, John Wiley & Sons.
  10. Gulden, J. and Reijers, H.A., 2015. Toward advanced visualization techniques for conceptual modeling. In Proceedings of the CAiSE Forum 2015 Stockholm, Sweden, June 8-12.
  11. Khan, K.S., ter Riet, G., Glanville, J., Sowden, A.J. and Kleijnen, J., 2001. Undertaking systematic reviews of research on effectiveness: CRD's guidance for carrying out or commissioning reviews, NHS Centre for Reviews and Dissemination.
  12. Kitchenham, B., Brereton, O.P., Budgen, D., Turner, M., Bailey, J. and Linkman, S., 2009. Systematic literature views in software engineering - a systematic literature review. Information and software technology, 51, pp. 7-15.
  13. Kitchenham, B. and Charters, S., 2007. Guidelines for performing systematic literature reviews in software engineering. Tech Report EBSE-2007-01. School of Computer Science and Mathematics, Keele University.
  14. Krogstie, J., Sindre, G. and Jørgensen, H., 2006. Process models representing knowledge for action: a revised quality framework. European Journal of Information Systems, 15(1), pp.91-102.
  15. Moody, D. and van Hillegersberg, J., 2009. Evaluating the visual syntax of UML: An analysis of the cognitive effectiveness of the UML family of diagrams. In Software Language Engineering (pp. 16-34). Springer Berlin Heidelberg.
  16. Moody, D.L., 2009. The “physics” of notations: toward a scientific basis for constructing visual notations in software engineering. Software Engineering, IEEE Transactions on, 35(6), pp. 756-779.
  17. Schuette, R. and Rotthowe, T., 1998. The guidelines of modeling-an approach to enhance the quality in information models. In Conceptual ModelingER'98 (pp. 240-254). Springer Berlin Heidelberg.
  18. Störrle, H. and Fish, A., 2013. Towards an Operationalization of the “Physics of Notations” for the Analysis of Visual Languages. In Model-Driven Engineering Languages and Systems (pp. 104-120). Springer Berlin Heidelberg.
  19. van der Linden, D., 2015. An Argument for More UserCentric Analysis of Modeling Languages' Visual Notation Quality. In Advanced Information Systems Engineering Workshops (pp. 114-120). Springer International Publishing.
  20. van der Linden, D. and Hadar, I., 2015. Cognitive Effectiveness of Conceptual Modeling Languages: Examining Professional Modelers. In Proceedings of the 5th IEEE International Workshop on Empirical Requirements Engineering (EmpiRE). IEEE.
  21. Wiebring, J. and Sandkuhl, J., 2015. Selecting the “Right” Notation for Business Process Modeling: Experiences from an Industrial Case. In Perspectives in Business Informations Research (pp. 129-144). Springer International Publishing.
  22. Wohlin, C., 2014. Guidelines for snowballing in systematic literature studies and a replication in software engineering. Proceedings of the 18th Int. Conf. on Evaluation and Assessment in Software Engineering, ACM, 38.
  23. Zhang, H. and Babar, M.A., 2013. Systematic reviews in software engineering: An empirical investigation. Information and software technology, 55 1341-1354.
  24. Zhang, H., Babar, M.A. and Tell, P., 2011. Identifying relevant studies in software engineering. Information and Software Technology, 53, 625-637.
Download


Paper Citation


in Harvard Style

van der Linden D. and Hadar I. (2016). Evaluating the Evaluators - An Analysis of Cognitive Effectiveness Improvement Efforts for Visual Notations . In Proceedings of the 11th International Conference on Evaluation of Novel Software Approaches to Software Engineering - Volume 1: ENASE, ISBN 978-989-758-189-2, pages 222-227. DOI: 10.5220/0005895802220227


in Bibtex Style

@conference{enase16,
author={Dirk van der Linden and Irit Hadar},
title={Evaluating the Evaluators - An Analysis of Cognitive Effectiveness Improvement Efforts for Visual Notations},
booktitle={Proceedings of the 11th International Conference on Evaluation of Novel Software Approaches to Software Engineering - Volume 1: ENASE,},
year={2016},
pages={222-227},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0005895802220227},
isbn={978-989-758-189-2},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 11th International Conference on Evaluation of Novel Software Approaches to Software Engineering - Volume 1: ENASE,
TI - Evaluating the Evaluators - An Analysis of Cognitive Effectiveness Improvement Efforts for Visual Notations
SN - 978-989-758-189-2
AU - van der Linden D.
AU - Hadar I.
PY - 2016
SP - 222
EP - 227
DO - 10.5220/0005895802220227