Pragmatic versus Hedonic: Determining the Dominant Quality in User
Experience for Professional and Leisure Collaboration Tools
Lisa Eidloth
1 a
, Anna-Lena Meiners
2 b
, Jörg Thomaschewski
1 c
and Andreas Hinderks
3 d
1
Faculty of Technology, University of Applied Science Emden/Leer, Emden, Germany
2
Karlsruhe Institute of Technology, Karlsruhe, Germany
3
Computer Languages and Systems, University of Sevilla, Sevilla, Spain
Keywords:
UX, User Experience Factors, Collaboration Tools, Professional and Leisure Contexts.
Abstract:
As collaborative technologies become integral in both professional and leisurely settings, especially during
the rise of remote work and digital communities due to COVID-19, understanding the user experience (UX)
factors is critical. This study aims to explore the differential importance of these UX factors across profes-
sional and leisure contexts, leveraging the widespread use of collaboration tools for an in-depth analysis. The
objective of the study is to identify and assess key UX factors in collaboration tools, and to quantify their
differential impact in professional and leisure settings. Our research underscores the nuanced role of context
in evaluating User Experience (UX) factors’ importance in collaboration tools, with significant variances ob-
served across professional and leisure settings. While some UX factors, including accessibility, clarity, and
intuitive use, maintained universal importance across contexts and tools, others—specifically dependability
and efficiency—contradicted assumptions of being universal "hygiene factors", demonstrating the complexity
of UX evaluations. This complexity necessitates a differentiated approach for each context and collaboration
tool type, challenging the possibility of a singular evaluation or statement.
1 INTRODUCTION
Collaborative technologies, encompassing online
platforms and specialized software, have emerged as
integral components of the contemporary digital land-
scape. These tools facilitate seamless communication
and synergistic collaboration among team members,
regardless of geographical barriers. The labor market
is in a state of evolution towards spatially dispersed
teams and adaptable work environments, underpin-
ning an augmented relevance of distributed collabo-
ration tools.
In the wake of the recent COVID-19 pandemic,
the necessity and potential of remote collaboration
has been heightened, with implications stretching be-
yond the confines of professional domains to perme-
ate private spheres. Digital communities within the
hobby and leisure sector have witnessed considerable
growth, and personal interactions have increasingly
a
https://orcid.org/0000-0001-5356-3307
b
https://orcid.org/0000-0002-9803-1555
c
https://orcid.org/0000-0001-6364-5808
d
https://orcid.org/0000-0003-3456-9273
transitioned to virtual spaces.
User experience is widely recognized as a mul-
tifaceted concept (Boy, 2017). A quality user expe-
rience typically demands that a product be easy to
learn, efficient in its use, and provide a sense of con-
trol, supplemented by attributes such as aesthetics,
enjoyment in usage, novelty, and appeal. The first
set of criteria is commonly classified as aspects of
pragmatic quality, while the latter set represents he-
donic quality aspects (Hassenzahl, 2001). Alterna-
tive terminology often employed to distinguish these
two classes of quality criteria includes usability goals
as opposed to user experience goals (Rogers et al.,
2023).
ISO 9241-210 provides a notable definition of
user experience (ISO9241-210, 2020). In this con-
text, user experience is characterized as “a person’s
perceptions and responses that result from the use
or anticipated use of a product, system or service”.
Consequently, user experience is viewed as a com-
prehensive concept, inclusive of all emotional, cog-
nitive, or physical responses related to the actual or
even merely the presumed use of a product, shaped
before, during, and after the utilization. An alter-
Eidloth, L., Meiners, A., Thomaschewski, J. and Hinderks, A.
Pragmatic versus Hedonic: Determining the Dominant Quality in User Experience for Professional and Leisure Collaboration Tools.
DOI: 10.5220/0012205700003584
In Proceedings of the 19th International Conference on Web Information Systems and Technologies (WEBIST 2023), pages 391-398
ISBN: 978-989-758-672-9; ISSN: 2184-3252
Copyright © 2023 by SCITEPRESS Science and Technology Publications, Lda. Under CC license (CC BY-NC-ND 4.0)
391
nate approach involves defining user experience as
a collection of distinct quality criteria (Rogers et al.,
2023). These encompass traditional usability aspects
such as efficiency, controllability, and learnability, as
well as non-goal-oriented or hedonic quality aspects
(Hassenzahl, 2001) such as stimulation, enjoyment in
use, novelty, emotions (Norman, 2007), and aesthet-
ics (Tractinsky, 1997).
The significance of UX factors can notably dif-
fer contingent upon the particular use case (Meiners
et al., 2021) (Hinderks et al., 2020) (Schrepp et al.,
2023). In professional contexts, it is often perceived
that pragmatic quality holds greater importance than
hedonic quality. To scrutinize the validity of this no-
tion, we selected collaboration tools as our object of
evaluation due to their pivotal role across both pro-
fessional and leisure contexts. These tools present
a unique opportunity to discern the comparative in-
fluence of pragmatic and hedonic qualities in diverse
use-case scenarios.
In this paper, we conducted a study with five dis-
tinct software systems used in both professional and
leisure settings. The objective of this study revolves
around the research question:
RQ: How does the importance of pragmatic and
hedonic factors of User Experience vary between pro-
fessional and leisurely contexts in software systems?
This paper is structured as follows: Section 2
briefly summarizes the related work. Section 3
presents the research methodology of our study. Sec-
tion 4 outlines the results and key findings of our
study, as well as the answers to our research ques-
tion. Section 5 discusses the meaning of the findings
and the limitations of our study. The paper ends with
Section 6, with conclusions and ideas for future work.
2 RELATED WORK
Hassenzahl et al. (Hassenzahl et al., 2010) expanded
the understanding of User Experience (UX) by in-
vestigating pleasurable experiences with technology
from the perspective of universal psychological needs
such as competence, relatedness, popularity, stimula-
tion, meaning, security, and autonomy. In their study,
they collected over 500 positive experiences with in-
teractive products like mobile phones and computers.
As expected, a significant relationship was found be-
tween the fulfillment of these psychological needs and
positive affect. Stimulation, relatedness, competence,
and popularity emerged as particularly salient needs.
The experiences collected were categorizable
based on the primary need they fulfilled, and qualita-
tive differences were observed among the categories
in terms of the emotions involved. Interestingly, need
fulfillment was strongly linked to perceptions of he-
donic quality (the pleasure derived from product use),
but not as strongly to pragmatic quality (perceived us-
ability). This supported their notion of hedonic qual-
ity as a ’motivator’ and pragmatic quality as a ’hy-
giene factor.
An important aspect of their study was the consid-
eration of attribution, i.e., the belief that the product
was responsible for the experience. They found that
the perception of hedonic quality as a reflection of
need fulfillment depended on this attribution. These
findings suggest that user experiences are not solely
about functionality or usability, but also hinge signif-
icantly on the extent to which products fulfill users’
psychological needs.
Tuch et al. (Tuch et al., 2016) conducted a thor-
ough investigation into the impact of need fulfillment
on the perception of technology in both leisure and
professional contexts, utilizing Hassenzahl model’s as
a foundation (Hassenzahl et al., 2010). They found
that the hedonic quality (pleasure derived from tech-
nology use) of both work and leisure experiences is
influenced by need fulfillment. However, pragmatic
quality (usability) was notably influenced by need ful-
fillment, primarily in leisure experiences. This ob-
servation deviates from Hassenzahl’s claim that prag-
matic quality simply acts as a barrier remover for need
fulfillment, suggesting that usability can indeed con-
tribute to need fulfillment, especially in leisure con-
texts (Hassenzahl et al., 2010).
Regarding context, leisure experiences frequently
took place with a partner or friend and at home, while
work experiences were more common with acquain-
tances or colleagues and at work. The narrative con-
tent corroborated these observations. The study re-
inforces Tinsley and Tinsley’s (Tinsley and Tinsley,
1986) suggestion that leisure experiences lead to aes-
thetic rewards and stimulation, with higher hedonic
quality and beauty being associated with leisure ex-
periences.
In our review of existing literature, we noted some
degree of discrepancy between the assertions made by
Hassenzahl et al. (Hassenzahl et al., 2010) and Tuch
(Tuch et al., 2016). It is important to note, however,
that in their respective studies, there was no evalua-
tion of test objects that spanned both leisure and pro-
fessional contexts. Our investigation seeks to provide
an initial insight into the influence of both pragmatic
and hedonic qualities on software systems in the con-
text of leisure and professional use.
WEBIST 2023 - 19th International Conference on Web Information Systems and Technologies
392
3 RESEARCH METHODOLOGY
In this study, our focus was the evaluation of collabo-
ration tools, investigated through the lens of hedonic
and pragmatic factors. We wanted to understand the
relative importance of these factors, as rated by the
participants, using the list of factors from the User
Experience Questionnaire Plus (UEQ+) (Schrepp and
Thomaschewski, 2019). Additionally, we explored
the possibility of further hedonic and pragmatic as-
pects not captured in the questionnaire. Our guiding
hypothesis was that perceptions of the importance of
hedonic and pragmatic factors would differ between
professional and leisure contexts.
In this study, products with a high level of aware-
ness were evaluated to ensure the subjects could as-
sess the products. Our selection criterion for these
products allowed for freedom of choice, but they had
to be classified as collaboration tools.
The study was conducted with master’s degree
students at the University of Applied Sciences Em-
den/Leer (Germany) as part of the university course
"User Experience". We had a total of 44 respondents
participating in the study. Our approach is divided
into three main steps:
Step 1: Identification of the most important UX
factors for different collaboration tools.
Step 2: Evaluation of the importance of each UX
factor in specific use cases.
Step 3: Quantitative determination of the differ-
ences in the evaluation of products utilized in pro-
fessional and leisure contexts.
The different steps are explained in greater detail
in the next three subsections.
3.1 Step 1: Identification of the Most
Important UX Factors
In the first phase, participants were asked to select the
UX factors from the UEQ+ they considered intrin-
sically important when evaluating the UX factors of
collaboration tools. They also had the option to sug-
gest other UX factors not included in the UEQ+ list.
These chosen factors were then compiled and ranked
based on importance through a web conference dis-
cussion among the participants. This way, a validated
list of UX factors could be included but new, possibly
overlooked UX factors could be introduced as well.
3.2 Step 2: Evaluation of the
Importance of Each UX Factor
Informed by the list of UX factors deduced from Step
1, the participants were then asked to rate the im-
portance of selected products on a seven-point Lik-
ert scale, ranging from -3 (extremely unimportant) to
+3 (extremely important). They were instructed to se-
lect a product of their choice from the following cat-
egories: Collaboration Platform, Document Manage-
ment System, Instant Messaging Service, Task Man-
agement System, and Web Conferencing.
3.3 Step 3: Determination of the
Distinction
In the final phase of our study, we used a two-tailed t-
test to determine differences in the importance of UX
factors between product groups and contexts of use.
The objective of this analysis was to identify UX fac-
tors that demonstrated significant differences in the
ratings when used in either a professional or leisure
setting.
4 RESULTS
In this section, we will present the results from the
three steps as described in Section 3.
4.1 Results from Step 1
A web conference was organized with the partici-
pants, during which the most pertinent UX factors
were discussed. As a result of this interactive pro-
cess, the list of UX factors was reduced to 18 factors.
Each of these UX factors is detailed below, complete
with succinct descriptions for clarity. Notably, those
factors that were initially part of the User Experience
Questionnaire Plus (UEQ+) have been appropriately
marked.
Accessibility: Does the product allow the largest
possible number of users to be used without re-
strictions?
Adaptability (UEQ+): Can the product be adapted
to personal preferences or personal working
styles?
Aesthetics (UEQ+): Does the product look beau-
tiful and appealing?
Attachment: Is it possible to establish a connec-
tion with other individuals through the use of the
product?
Pragmatic versus Hedonic: Determining the Dominant Quality in User Experience for Professional and Leisure Collaboration Tools
393
Attractiveness (UEQ+): Overall impression of the
product. Do users like or dislike it?
Clarity (UEQ+): Impression towards order, struc-
ture and visual complexity of a graphical user in-
terface.
Dependability (UEQ+): Does the user feel in
control of the interaction? Is it secure and pre-
dictable?
Efficiency (UEQ+): Can users solve their tasks
without unnecessary effort? Does it react fast?
Functionality: Does the product provide an ade-
quate array of functionalities to facilitate the suc-
cessful completion of the tasks in question?
Interaction: Does the product support interaction
between users?
Intuitive Use (UEQ+): Can the product be used
immediately without any training or help?
Novelty (UEQ+): Is the design of the product cre-
ative? Does it catch the interest of users?
Perspicuity (UEQ+): Is it easy to get familiar with
the product and to learn how to use it?
Self-Presentation: Is it easy to present yourself
with the product?
Stimulation (UEQ+): Is it exciting and motivating
to use the product? Is it fun to use?
Trust (UEQ+): Are the users’ data in safe hands
and not misused to harm him or her?
Usefulness (UEQ+): Does using the product bring
advantages?
Value (UEQ+): Does the product design look pro-
fessional and of high quality?
Utilizing the identified list of UX factors, we pro-
ceeded to implement Step 2 of the study.
4.2 Results from Step 2
In the subsequent step of the study, participants were
directed to select one software product from each cat-
egory (see 3.2). They were then required to assign
an importance value to each UX factor for the chosen
product, employing a numerical scale ranging from -3
(extremely unimportant) to +3 (extremely important).
The derived values are visually represented in a bar
chart, as illustrated in Figure 1.
We’ve used these values for our next set of calcu-
lations. The results are presented in the next section.
4.3 Results from Step 3
In the final step, we examined how closely related the
professional and leisure contexts are for each UX fac-
tor in regard to their perceived importance and group
by doing a t-test (Williams, 1959). The results are
shown in Table 1.
In the group of Collaboration Platform products,
significant differences were observed in the UX fac-
tors of Aesthetics, Dependability, Efficiency, Novelty,
Stimulation, and Usefulness. Within the Document
Management group, Functionality emerged as the dis-
tinguished UX factor exhibiting a significant differ-
ence. For products within the Instant Messaging Ser-
vice category, the UX factors of Aesthetics, Attrac-
tiveness, Dependability, Efficiency, Novelty, Stimula-
tion, and Usefulness were recognized as significantly
different. With regard to the Task Management Sys-
tem group, Efficiency, Functionality, Usefulness, and
Value were identified as factors of considerable vari-
ance. Lastly, in the category of Web Conferencing
Systems, Aesthetics, Attachment, Attractiveness, De-
pendability, Efficiency, Functionality, Novelty, Stim-
ulation, Trust, Usefulness, and Value were the UX
factors showcasing significant differences.
5 DISCUSSION
Our research reveals the complexity and nuances in
evaluating the importance of UX factors for collab-
oration tools. It has been shown that the context
in which these tools are used, whether professional
or leisure, significantly impacts the perceived impor-
tance of individual UX factors.
The following conclusions can be drawn from the
t-test:
1. A higher level of significance suggests an equiva-
lence in importance ascribed to both professional
and leisure contexts, indicating no apparent differ-
ences.
2. Conversely, a lower level of significance implies
a disparity in the importance attributed to pro-
fessional and leisure contexts, highlighting dis-
cernible differences.
Interestingly, our study shows that not all col-
laboration tools are evaluated similarly. In particu-
lar, for tools such as document management and task
management systems, no significant differences were
found in evaluating the importance of UX factors be-
tween professional and recreational contexts. This
suggests that these tools are used similarly in both
WEBIST 2023 - 19th International Conference on Web Information Systems and Technologies
394
Figure 1: Importance ratings per group and UX factor. Left: Leisure context; Right: Professional. Error Bars: 5% Confidence
Interval.
Pragmatic versus Hedonic: Determining the Dominant Quality in User Experience for Professional and Leisure Collaboration Tools
395
Table 1: T-test between professional and leisure context for each UX factor and Group. Significance is coded as follows: p <
.05*, p < .01**, p < .001***.
UX Factor Collaboration
Plattform
Document
Management
Instant
Messaging
Service
Task
Management
System
Web
Conferencing
System
Accessibility 0,903 0,559 0,180 0,110 0,239
Adaptability 0,412 0,121 0,886 0,402 0,081
Aesthetics *0,030 0,411 ***0,000 0,094 **0,001
Attachment 0,529 0,863 0,083 0,738 *0,026
Attractiveness 0,322 0,209 ***0,000 0,359 **0,001
Clarity 0,137 0,749 0,156 0,445 0,117
Dependability *0,030 0,192 *0,041 0,086 **0,005
Efficiency ***0,000 0,067 *0,014 *0,043 *0,011
Functionality 0,086 *0,019 0,246 ***0,000 **0,007
Interaction 0,390 0,177 0,613 0,099 0,597
Intuitive Use 0,240 0,210 0,534 0,896 0,680
Novelty *0,039 0,157 ***0,000 0,086 ***0,001
Perspicuity 0,068 0,619 0,912 0,362 0,377
Self-Presentation 0,306 0,877 0,156 0,510 0,377
Stimulation **0,009 0,421 ***0,000 0,127 **0,002
Trust 0,403 0,890 0,365 0,081 *0,012
Usefulness *0,012 0,437 ***0,000 *0,035 ***0,000
Value 0,426 0,321 0,775 *0,039 **0,009
contexts. On the other hand, collaboration tools, in-
stant messaging and web conferencing systems show
significant differences in importance ratings, suggest-
ing that their use and perception are highly context-
dependent.
Despite these differences, there is a group of UX
factors whose importance remains the same in all con-
texts and for all tool types. These include:
Accessibility
Adaptability
Clarity
Interaction
Intuitive Use
Perspicuity
Self-Presentation
These factors seem to be universal and independent of
the specific application of the tool.
It is also worth noting that our study does not con-
firm the assumption that pragmatic qualities such as
Dependability and Efficiency are universally applica-
ble "hygiene factors" (Hassenzahl et al., 2010). In-
stead, we found significant differences in the evalua-
tion of these aspects. Moreover, our results support
the idea that more hedonic factors, such as novelty
and stimulation, serve as motivators.
In contrast to Tuch’s (Tuch et al., 2016) findings,
our study did not demonstrate that pragmatic qualities
are particularly important in the leisure context. This
raises questions about the role and importance of UX
factors in different contexts that require further inves-
tigation.
Our study emphasizes the need for a differentiated
approach when evaluating the importance of UX fac-
tors, depending on context and specific type of collab-
oration tool. A general statement or evaluation does
not seem possible due to the heterogeneous results.
This underlines the complexity of UX factors and the
need to consider specific contexts of use and require-
ments in the evaluation and design of collaboration
tools.
5.1 Limitations
Despite the findings garnered from this research, it’s
essential to acknowledge certain limitations that may
WEBIST 2023 - 19th International Conference on Web Information Systems and Technologies
396
influence the interpretability and generalizability of
our results. Firstly, the scope of our research was con-
fined to collaboration tools exclusively, therefore lim-
iting the broad applicability of our findings across all
software systems or technological domains.
Secondly, our study population comprised only
students, potentially introducing a bias in our results.
Given the distinct demographics, usage patterns, and
expectations of this specific group, caution must be
exercised when extending the results to a broader,
more heterogeneous user base.
Thirdly, a bias towards the importance of factors
may be given, as steps one and two of the study were
performed by the same group of individuals: UX
factors identified as being of relevance in step one
may be more likely rated as relevant in step two of
the study. This may be prevented by conducting the
named steps in a single-blind manner, where partici-
pants of step two did not preselect potentially relevant
factors.
Lastly, the set of User Experience (UX) factors
we considered may not be exhaustive. Though we
endeavored to encompass a comprehensive array of
factors, there may still be critical, yet overlooked as-
pects that could influence users’ perceptions and ex-
periences with collaboration tools.
6 CONCLUSIONS AND FUTURE
WORK
In our study of collaboration tools from a User Expe-
rience (UX) perspective, we sought to discern the rel-
ative significance of UX factors in various usage con-
texts. The study used a three-step approach to iden-
tify, evaluate, and ascertain distinctive UX factors and
their perceived importance across collaboration tools.
Significant variations were observed in the per-
ceived importance of several UX factors across dif-
ferent collaboration tool types, underscoring the intri-
cate role of professional and leisure context in such
evaluations. For instance, certain UX factors like De-
pendability and Efficiency, contrary to being univer-
sally applicable "hygiene factors," exhibited signif-
icant variations, illuminating the nuanced nature of
these evaluations. Similarly, more hedonic elements
like novelty and stimulation emerged as motivators,
thereby refuting the established notion that pragmatic
qualities primarily dominate the leisure context.
However, our study also identifies certain UX fac-
tors that remain universally important, regardless of
the tool type or context, such as accessibility, adapt-
ability, clarity, interaction, intuitive use, perspicuity,
and self-presentation. This discovery suggests an en-
during baseline of factors pertinent to any successful
UX design across contexts.
The findings highlight the need for a differenti-
ated, context-specific approach in assessing the im-
portance of UX factors. A homogeneous evalua-
tion or universal statement about the importance of
UX factors appears untenable given the diverse out-
comes. Consequently, our study emphasizes the in-
tricate complexity of UX evaluations and the impor-
tant necessity of considering specific contexts and re-
quirements when designing and evaluating collabora-
tion tools.
These findings do not only underscore the impact
of context on the significance of UX factors but also
generate new research avenues exploring the role and
importance of such factors in different settings. Fu-
ture work could strive to expand these initial findings
and continue to unravel the complexities of UX in
a world increasingly dependent on digital collabora-
tion.
REFERENCES
Boy, G. A. (2017). The Handbook of Human-Machine
Interaction: A Human-Centered Design Approach.
CRC Press, Milton, 1st ed. edition.
Hassenzahl, M. (2001). The effect of perceived he-
donic quality on product appealingness. Inter-
national Journal of Human-Computer Interaction,
2001(13(4)):481–499.
Hassenzahl, M., Diefenbach, S., and Göritz, A. (2010).
Needs, affect, and interactive products – facets of user
experience. Interacting with Computers, 22(5):353–
362. Modelling user experience - An agenda for re-
search and practice.
Hinderks, A., Winter, D., Schrepp, M., and
Thomaschewski, J. (2020). Applicability of user
experience and usability questionnaires. Journal of
Universal Computer Science, (25):1717–1735.
ISO9241-210 (2020). Ergonomics of human-system inter-
action - part 210: Human-centred design for interac-
tive systems.
Meiners, A.-L., Kollmorgen, J., Schrepp, M., and
Thomaschewski, J. (2021). Which UX Aspects Are
Important for a Software Product?: Importance Rat-
ings of UX Aspects for Software Products for Mea-
surement with the UEQ+. In Mensch und Computer
2021, pages 136–139, Ingolstadt Germany. ACM.
Norman, D. A. (2007). Emotional Design: Why We Love
(or Hate) Everyday Things. Basic Books, New York.
Rogers, Y., Sharp, H., and Preece, J. (2023). Interaction
design: Beyond human-computer interaction. Wiley,
Chichester, 6. ed. edition.
Schrepp, M., Kollmorgen, J., Meiners, A.-L., Hinderks, A.,
Winter, D., and Thomaschewski, J. (2023). On the Im-
portance of UX Quality Aspects for Different Product
Pragmatic versus Hedonic: Determining the Dominant Quality in User Experience for Professional and Leisure Collaboration Tools
397
Categories. International Journal of Interactive Mul-
timedia and Artificial Intelligence, page 15. Citation-
Key: Schrepp2023.
Schrepp, M. and Thomaschewski, J. (2019). Design and
Validation of a Framework for the Creation of User
Experience Questionnaires. International Journal
of Interactive Multimedia and Artificial Intelligence,
5(7):88.
Tinsley, H. E. A. and Tinsley, D. J. (1986). A theory of the
attributes, benefits, and causes of leisure experience.
Leisure Sciences, 8(1):1–45.
Tractinsky, N. (1997). Aesthetics and apparent usability. In
Pemberton, S., editor, the SIGCHI conference, pages
115–122.
Tuch, A. N., Schaik, P. V., and Hornbæk, K. (2016). Leisure
and work, good and bad: The role of activity domain
and valence in modeling user experience. ACM Trans.
Comput.-Hum. Interact., 23(6).
Williams, E. J. (1959). The Comparison of Regression Vari-
ables. Journal of the Royal Statistical Society: Series
B (Methodological), 21(2):396–399.
WEBIST 2023 - 19th International Conference on Web Information Systems and Technologies
398