Simulation-Based Evaluation of Recommendation Algorithms Assisting Business Process Modeling

Adam Styperek, Michal Ciesielczyk, Pawel Misiorek, Andrzej Szwabe

2015

Abstract

In this paper, we propose a methodology for objective and repeatable simulation-based evaluation of recommendation algorithms supporting the process of designing of a BPMN model. According to the methodology, an evaluation of the usability of recommendations is done entirely with the use of the dedicated software coupled by a predefined test set. In order to confirm the reliability of the methodology, an additional evaluation based on the user study has been performed.

References

  1. Activiti (2014). http://www.activiti.org/.
  2. Born, M., Brelage, C., Markovic, I., Pfeiffer, D., and Weber, I. (2009). Auto-completion for executable business process models. In Ardagna, D., Mecella, M., and Yang, J., editors, Business Process Management Workshops, volume 17 of Lecture Notes in Business Information Processing, pages 510-515. Springer Berlin Heidelberg.
  3. Cao, B., Yin, J., Deng, S., Wang, D., and Wu, Z. (2012). Graph-based workflow recommendation: On improving business process modeling. In Proceedings of the 21st ACM International Conference on Information and Knowledge Management, CIKM 7812, pages 1527-1531, New York, NY, USA. ACM.
  4. Card, S. K., Moran, T. P., and Newell, A. (1980). The keystroke-level model for user performance time with interactive systems. Commun. ACM, 23(7):396-410.
  5. Chan, N. N., Gaaloul, W., and Tata, S. (2011). Contextbased service recommendation for assisting business process design. In Huemer, C. and Setzer, T., editors, E-Commerce and Web Technologies, volume 85 of Lecture Notes in Business Information Processing, pages 39-51. Springer Berlin Heidelberg.
  6. Dijkman, R., Dumas, M., van Dongen, B., Käärik, R., and Mendling, J. (2011). Similarity of business process models: Metrics and evaluation. Information Systems, 36(2):498-516.
  7. Herlocker, J. L., Konstan, J. a., Terveen, L. G., and Riedl, J. T. (2004). Evaluating collaborative filtering recommender systems. ACM Transactions on Information Systems, 22(1):5-53.
  8. Holzinger, A. (2005). Usability engineering methods for software developers. Commun. ACM, 48(1):71-74.
  9. Hornung, T., Koschmider, A., and Lausen, G. (2008). Recommendation based process modeling support: Method and user experience. Conceptual ModelingER 2008, pages 265-278.
  10. Hornung, T., Koschmider, A., and Oberweis, A. (2009). A Recommender System for Business Process Models. 17th Annual Workshop on Information Technologies & Systems (WITS).
  11. ISO/IEC (2011). ISO/IEC 25010 - Systems and Software Engineering - Systems and Software Quality Requirements and Evaluation (SQuaRE) - System and Software Quality Models. Technical report.
  12. Kluza, K., Baran, M., Bobek, S., and Nalepa, G. J. (2013). Overview of recommendation techniques in business process modeling. In Nalepa, G. J. and Baumeister, J., editors, Proceedings of 9th Workshop on Knowledge Engineering and Software Engineering (KESE) , Koblenz, Germany.
  13. Koschmider, A., Hornung, T., and Oberweis, A. (2011). Recommendation-based editor for business process modeling. Data Knowl. Eng., 70(6):483-503.
  14. Leopold, H., Mendling, J., and Reijers, H. (2011). On the automatic labeling of process models. In Mouratidis, H. and Rolland, C., editors, Advanced Information Systems Engineering, volume 6741 of Lecture Notes in Computer Science, pages 512-520. Springer Berlin Heidelberg.
  15. Li, C., Reichert, M., and Wombacher, A. (2008). On measuring process model similarity based on high-level change operations. In Li, Q., Spaccapietra, S., Yu, E., and Oliv, A., editors, Conceptual Modeling - ER 2008, volume 5231 of Lecture Notes in Computer Science, pages 248-264. Springer Berlin Heidelberg.
  16. Li, Y., Cao, B., Xu, L., Yin, J., Deng, S., Yin, Y., and Wu, Z. (2014). An efficient recommendation method for improving business process modeling. Industrial Informatics, IEEE Transactions on, 10(1):502-513.
  17. Madhusudan, T., Zhao, J., and Marshall, B. (2004). A casebased reasoning framework for workflow model management. Data & Knowledge Engineering, 50(1):87- 115.
  18. Minor, M., Tartakovski, A., and Bergmann, R. (2007). Representation and structure-based similarity assessment for agile workflows. In Weber, R. and Richter, M., editors, Case-Based Reasoning Research and Development, volume 4626 of Lecture Notes in Computer Science, pages 224-238. Springer Berlin Heidelberg.
  19. Nielsen, J. and Molich, R. (1990). Heuristic evaluation of user interfaces. In Proceedings of the SIGCHI conference on Human factors in computing systems, pages 249-256. ACM.
  20. Smirnov, S., Weidlich, M., and Mendling, J. (2010). Business process model abstraction based on behavioral profiles. 6470:1-16.
  21. Smirnov, S., Weidlich, M., Mendling, J., and Weske, M. (2009). Action patterns in business process models. In Baresi, L., Chi, C.-H., and Suzuki, J., editors, ServiceOriented Computing, volume 5900 of Lecture Notes in Computer Science, pages 115-129. Springer Berlin Heidelberg.
  22. van der Aalst, W., de Medeiros, A., and Weijters, A. (2006). Process equivalence: Comparing two process models based on observed behavior. In Dustdar, S., Fiadeiro, J., and Sheth, A., editors, Business Process Management, volume 4102 of Lecture Notes in Computer Science, pages 129-144. Springer Berlin Heidelberg.
  23. Wharton, C., Rieman, J., Lewis, C., and Polson, P. (1994). Usability inspection methods. chapter The Cognitive Walkthrough Method: A Practitioner's Guide, pages 105-140. John Wiley & Sons, Inc., New York, NY, USA.
  24. Wieloch, K., Filipowska, A., and Kaczmarek, M. (2011). Autocompletion for business process modelling. In Abramowicz, W., Maciaszek, L., and Wcel, K., editors, Business Information Systems Workshops, volume 97 of Lecture Notes in Business Information Processing, pages 30-40. Springer Berlin Heidelberg.
  25. Zeng, R., He, X., and van der Aalst, W. (2011). A method to mine workflows from provenance for assisting scientific workflow composition. In Services (SERVICES), 2011 IEEE World Congress on, pages 169-175.
  26. Zhang, J., L. Q. and Xu, K. (2009). Flowrecommender: A workflow recommendation technique for process provenance. In Kennedy P. J., O. K. and P., C., editors, Australasian Data Mining Conference (AusDM'09),
Download


Paper Citation


in Harvard Style

Styperek A., Ciesielczyk M., Misiorek P. and Szwabe A. (2015). Simulation-Based Evaluation of Recommendation Algorithms Assisting Business Process Modeling . In Proceedings of the Fifth International Symposium on Business Modeling and Software Design - Volume 1: BMSD, ISBN 978-989-758-111-3, pages 231-240. DOI: 10.5220/0005887602310240


in Bibtex Style

@conference{bmsd15,
author={Adam Styperek and Michal Ciesielczyk and Pawel Misiorek and Andrzej Szwabe},
title={Simulation-Based Evaluation of Recommendation Algorithms Assisting Business Process Modeling},
booktitle={Proceedings of the Fifth International Symposium on Business Modeling and Software Design - Volume 1: BMSD,},
year={2015},
pages={231-240},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0005887602310240},
isbn={978-989-758-111-3},
}


in EndNote Style

TY - CONF
JO - Proceedings of the Fifth International Symposium on Business Modeling and Software Design - Volume 1: BMSD,
TI - Simulation-Based Evaluation of Recommendation Algorithms Assisting Business Process Modeling
SN - 978-989-758-111-3
AU - Styperek A.
AU - Ciesielczyk M.
AU - Misiorek P.
AU - Szwabe A.
PY - 2015
SP - 231
EP - 240
DO - 10.5220/0005887602310240