Using a Study to Assess User eXperience Evaluation Methods from the Point of View of Users

Luis Rivero, Tayana Conte

2015

Abstract

User eXperience (UX) refers to a holistic perspective and an enrichment of traditional quality models with non-utilitarian concepts, such as fun, joy, pleasure or hedonic value. In order to evaluate UX, several methods have been proposed that range from using questionnaires to employing biometrics to evaluate the users’ emotions. However, few of these UX evaluation methods are comfortable or easy to use from the point of view of users. This paper presents a study in which 10 users applied the Expressing Emotions and Experiences (3E) and EmoCards methods. While 3E provides a template for reporting the experience, the EmoCards provides a set of cards illustrating emotions as helping material. We have analyzed the features that make it easy or difficult for users to employ these methods, the users’ preference and the number of identified problems. Besides showing an application example of the methods to aid software practitioners in future evaluations, we identified that the EmoCards allowed users to identify more problems, but 3E was preferred do to its ease of use and freedom when describing an emotion and its causes.

References

  1. Bargas-Avila, J., Hornbaek, K.: “Old wine in new bottles or novel challenges: a critical analysis of empirical studies of user experience”. Proc. SIGCHI Conference on Human Factors in Computing Systems, 2011.
  2. Basili, V., Rombach, H.: “The tame project: towards improvement-oriented software environments”. IEEE Transactions on Software Engineering, vol. 14, 1988.
  3. Bradley, M. Lang, P.: “Measuring emotion: the selfassessment manikin and the semantic differential”. Journal of behavior therapy and experimental psychiatry, 25(1), 1994.
  4. Desmet, P., Overbeeke, K., Tax, S.: “Designing Products with Added Emotional Value: Development and Application of an Approach for Research Through Design”. The design journal, 4(1), 2001.
  5. Fernandez, A., Insfran, E., Abrahão, S.: “Usability evaluation methods for the web: A systematic mapping study”. Information and Software Technology, 53(8), 2011.
  6. Hassenzahl, M.: “The thing and I: understanding the relationship between user and product”. In Funology, Springer Netherlands, 2005.
  7. Isbister, K., Höök, K., Sharp, M., Laaksolahti, J.: “The sensual evaluation instrument: developing an affective evaluation tool”. In Proc. of the SIGCHI conference on Human Factors in computing systems, 2006.
  8. ISO DIS 9241-210:2010.: “Ergonomics of human system interaction - Part 210: Human-centered design for interactive systems”. International Standardization Organization (ISO), 2010.
  9. Lavie, T., Tractinsky, N.: “Assessing dimensions of perceived visual aesthetics of web sites”. International journal of human-computer studies, 60(3), 2004.
  10. Mandryk, R., Inkpen, K., Calvert, T.: “Using psychophysiological techniques to measure user experience with entertainment technologies”. Journal of Behaviour & Information Technology, 25(2), 2006.
  11. Read, J., MacFarlane, S.: “Using the fun toolkit and other survey methods to gather opinions in child computer interaction”. In Proc. of the 2006 conference on Interaction design and children, 2006.
  12. Rivero, L., Conte, T.: “Employment of 3E and EmoCards in the Evaluation of a Web Application” Technical Report USES TR-USES-2015-0002, 2015. Available at: http://uses.icomp.ufam.edu.br/index. php/ publicacoes#5-relat%C3%B3rios-t%C3%A9cnicos.
  13. Shapiro, S., Wilk, M.: “An analysis of variance test for normality (complete samples).” Biometrika, 1965.
  14. Ståhl, A., Höök, K., Svensson, M., Taylor, A., Combetto, M.: “Experiencing the affective diary”. Personal and Ubiquitous Computing, 13(5), 2009.
  15. Tähti, M., Niemelä, M.: “3E-expressing emotions and experiences”. WP9 Workshop on Innovative Approaches for Evaluating Affective Systems, 2006.
  16. Vermeeren, A., Law, E., Roto, V., Obrist, M., Hoonhout, J., Väänänen-Vainio-Mattila, K.: “User experience evaluation methods: current state and development needs”. In Proceedings of the 6th Nordic Conference on Human-Computer Interaction, 2010.
  17. Wohlin, C., Runeson, P., Host, M., Ohlsson, M., Regnell B., Wessl, A.: “Experimentation in software engineering: an introduction,” (Book), Kluwer Academic Publishers, 2000.
  18. Yong, L.: “User experience evaluation methods for mobile devices”. In Proc. of the III International Conference on Innovative Computing Technology, 2013.
Download


Paper Citation


in Harvard Style

Rivero L. and Conte T. (2015). Using a Study to Assess User eXperience Evaluation Methods from the Point of View of Users . In Proceedings of the 17th International Conference on Enterprise Information Systems - Volume 3: ICEIS, ISBN 978-989-758-098-7, pages 88-95. DOI: 10.5220/0005377300880095


in Bibtex Style

@conference{iceis15,
author={Luis Rivero and Tayana Conte},
title={Using a Study to Assess User eXperience Evaluation Methods from the Point of View of Users},
booktitle={Proceedings of the 17th International Conference on Enterprise Information Systems - Volume 3: ICEIS,},
year={2015},
pages={88-95},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0005377300880095},
isbn={978-989-758-098-7},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 17th International Conference on Enterprise Information Systems - Volume 3: ICEIS,
TI - Using a Study to Assess User eXperience Evaluation Methods from the Point of View of Users
SN - 978-989-758-098-7
AU - Rivero L.
AU - Conte T.
PY - 2015
SP - 88
EP - 95
DO - 10.5220/0005377300880095