![](bg9.png)
that is primarily intended for private, leisure-time
use when people can be expected to be somewhat
relaxed. Social pressure, of course, is relevant for all
kinds of systems used in public, including public
information systems, vending and teller machines,
and also mobile devices. Also, people working in
open-plan offices or generally with other colleagues
or customers around might experience social
pressure that should be considered when planning
usability tests. Likewise, the occurrence of noise,
interruptions and other kinds of disturbances can be
derived from use-cases and scenarios.
Furthermore, interindividual differences
regarding the level of stress should be considered
when trying to assess the effect of stressors on
usability tests. While some people might be hardly
affected, others experience profound stress and
anxiety. Simple and short questionnaires like the one
we used in our study can help to judge the impact
that such feelings had on test results.
One might argue that our results indicate that we
should leave the lab altogether and conduct usability
evaluations in the field instead to achieve really
meaningful results. Indeed, on-site investigations
and observations of actual work processes and user
experiences are particularly valuable, especially
regarding usability engineering and socio-technical
design: When developing or implementing a new
system in an organization it is crucial to involve real
users in their real environments. Nevertheless, the
lab might be preferred in several situations; e.g. in
early stages of product design when actual users are
not yet available, for test cases where sophisticated
observation and recording technologies are
desirable, to simulate certain occurrences, or simply
because on-site testing is not possible for
administrative or other reasons. In these cases
inducing stressors in lab tests can enhance results.
However, the question of external validity should
always be asked when working in the lab.
Of course the deliberate use of stressors raises
ethical issues and considerations. We were surprised
about the strong effects the stressors used in our
study had on the participants. Many of them
experienced a substantial amount of stress, anger
and frustration. How to handle these feelings, e.g. by
clarifying the goals and intentions of the study in a
follow-up interview as we did in our study, needs to
be carefully planned in advance.
REFERENCES
Andrzejczak, C., Liu, D. (2010). The effect of testing
location on usability testing performance, participant
stress levels, and subjective testing experience.
Journal of System Software 83, (7), 1258-1266.
Ayyagari, R., Grover, V., Purvis, R. (2011). Technostress:
technological antecedents and implications. MIS
Quarterly 35, (4), 831-858.
Bakker, A. B., Demerouti, E. (2007). The job demands-
resources model: State of the art. Journal of
Managerial Psychology, 22 (3), 309-328.
Bosenick, T., Kehr, S., Kühn, M., Nufer, S. (2007).
Remote usability tests: an extension of the usability
toolbox for online-shops. In Proceedings of
UAHCI'07, Berlin: Springer, 392-398.
Driskell, J., Salas, E. (Eds) (1996). Stress and human
performance. Hillsdale: Lawrence Erlbaum
Associates.
Dumas, J.S., Loring, B.A. (2008). Moderating Usability
Tests: Principles and Practices for Interacting. San
Francisco: Morgan Kaufmann.
Dumas, J.S., Redish, J.C. (1999). A practical guide to
usability testing (revised). Willmington: Intellect.
Faulkner, L. (2003). Beyond the five-user assumption:
Benefits of increased sample sizes in usability testing.
Behavior Research Methods, Instruments, and
Computers, 35 (3), 379-383.
Frese, M., Zapf, D. (1994). Action as the core of work
psychology: A German approach. In H. C. Triandis,
M. D. Dunnette & L. M. Hough (Eds.), Handbook of
industrial and organizational psychology (pp. 271-
340). Palo Alto: Consulting Psychologists Press, Inc.
Greifeneder, E. (2011). The impact of distraction in
natural environments on user experience research. In
Proceedings of TPDL'11, Berlin: Springer, 308-315.
Hassenzahl, M. (2008). The interplay of beauty, goodness,
and usability in interactive products. Human-
Computer Interaction 19, (4), 319-349.
International Standards Organization (1998): Ergonomic
requirements for office work with visual display
terminals, part 11 (ISO 9241-11).
Janssen, P. P. M., Bakker, A. B., De Jong, A. (2001). A
test of the demand-control-support model in
construction industry. International Journal of Stress
Management, 8 (4), 315-332.
Kanner, A.D., Coyne, J.C., Schaefer, C., Lazarus, R.S.
(1981). Comparison of two modes of stress
measurement: Daily hassles and uplifts versus major
life events. Journal of Behavioral Medicine, 4, 1-39.
Laugwitz, B., Held, T., Schrepp, M. (2008). Construction
and evaluation of a user experience questionnaire. In
Proceedings of USAB 2008, Berlin: Springer, 63-76.
Lin, T., Omata, M., Hu, W., Imamiya, A. (2005). Do
physiological data relate to traditional usability
indexes?. In Proceedings of OZCHI '05, Narrabundah,
Australia, 1-10.
Ogden, J. (2007). Health Psychology: a textbook (4th ed.).
New York: McGraw-Hill.
Reason, J. (1990). Human Error. Cambridge, UK:
Cambridge University Press.
Robertson, D. (2012). Build your Resilience. London:
Hodder.
TheInfluenceofStressorsonUsabilityTests-AnExperimentalStudy
589