Straight to the Point - Evaluating What Matters for You: A Comparative Study on Playability Heuristic Sets

Felipe Manzoni, Tayana Conte, Milene Silveira, Simone Barbosa

2020

Abstract

Background: Playability is the degree by which a player can learn, control, and understand a game. There are many and different Playability evaluation techniques that can evaluate different and numerous game aspects. However, there is a shortage of comparative studies between these proposed evaluation techniques. These comparative studies can assess whether the evaluated techniques can identify playability problems with a better cost-benefit ratio. Also, these studies can show game developers the evaluation power that a technique has in comparison to others. Aim: This paper aims to present and initially evaluate CustomCheck4Play, a configurable heuristic-based evaluation technique that can be suited for different game types and genres. We evaluate CustomCheck4Play, assessing its efficiency and effectiveness in order to verify if CustomCheck4Play performs better than the compared heuristic set. Method: We have conducted an empirical study comparing a known literature heuristic set and CustomCheck4Play. The study had 54 participants, who identified 49 unique problems in the evaluated game. Results: Our statistical results comparing both evaluation techniques have shown that there was a significant statistical difference between groups. Efficiency (p-value = 0.030) and effectiveness (p-value = 0.004) results represented a statistically significant difference in comparison to the literature heuristic set. Conclusions: Overall, statistical results have shown that CustomCheck4Play is a more cost-beneficial solution for the playability evaluation of digital games. Moreover, CustomCheck4Play was able to guide participants throughout the evaluation process better and showed signs that the customization succeeded in adapting the heuristic set to suit the evaluated game.

Paper Citation

in Harvard Style

Manzoni F., Conte T., Silveira M. and Barbosa S. (2020). Straight to the Point - Evaluating What Matters for You: A Comparative Study on Playability Heuristic Sets.In Proceedings of the 22nd International Conference on Enterprise Information Systems - Volume 2: ICEIS, ISBN 978-989-758-423-7, pages 499-510. DOI: 10.5220/0009381304990510

in Bibtex Style

@conference{iceis20,
author={Felipe Manzoni and Tayana Conte and Milene Silveira and Simone Barbosa},
title={Straight to the Point - Evaluating What Matters for You: A Comparative Study on Playability Heuristic Sets},
booktitle={Proceedings of the 22nd International Conference on Enterprise Information Systems - Volume 2: ICEIS,},
year={2020},
pages={499-510},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0009381304990510},
isbn={978-989-758-423-7},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 22nd International Conference on Enterprise Information Systems - Volume 2: ICEIS,
TI - Straight to the Point - Evaluating What Matters for You: A Comparative Study on Playability Heuristic Sets
SN - 978-989-758-423-7
AU - Manzoni F.
AU - Conte T.
AU - Silveira M.
AU - Barbosa S.
PY - 2020
SP - 499
EP - 510
DO - 10.5220/0009381304990510