UX4ALL: A Repository of User Experience Evaluation Methods

Ana Clarissa Barroso Beleza

1 a

, Suzan Evellyn Nascimento da Mota

1 b

arcia Sampaio Lima

2 c

and Tayana Uch

oa Conte

1 d

Institute of Computing, Universidade Federal do Amazonas, Manaus, Amazonas, Brazil

Computer Center, Universidade do Estado do Amazonas, Manaus Amazonas, Brazil

{ana.beleza, suzan.nascimento}@ufam.edu.br, msllima@uea.edu.br, tayana@icomp.ufam.edu.br

Keywords:

User Experience, UX, UX Methods, UX Repository.

Abstract:

The evolution of the User Experience (UX) area is crucial for the success of any design or system develop-

ment process. Although numerous UX evaluation methods exist, understanding and utilizing these methods

can be challenging for interested parties. In this context, the Experience Research Society (EXPRESSO)

platform aims to support the comprehension of UX by providing data on over 80 UX evaluation methods.

However, the platform’s content has limitations that hinder understanding and application. In this sense, this

research proposes to create a new repository model for UX evaluation methods called UX4ALL. We intend to

democratize access to UX evaluation knowledge. We developed UX4ALL using data collected from the EX-

PRESSO platform, which underwent analysis, selection, enrichment, and classiﬁcation before being included

in the UX4ALL prototype. Furthermore, we used the System Usability Scale (SUS) method to evaluate the

prototype. In addition, a UX expert assessed the UX4ALL. We used the results to support the development

of the second version of UX4ALL. The main contributions of this research are: (1) The democratization of

understanding UX evaluation practices; and (2) The creation of an easy-to-use repository prototype for UX

evaluation methods, named UX4ALL Future studies aim to evolve the prototype into a ﬁnal product, making

it accessible to all interested parties and contributing to the popularization of UX practices.

1 INTRODUCTION

In the ﬁeld of Human-Computer Interaction (HCI),

User Experience (UX) emerged from a desire to un-

derstand how individuals feel when interacting with

technology (Hassenzahl, 2018). According to ISO

9241-110:2020 (Din, 2020), UX is deﬁned as the

combination of user perceptions and responses that

arise from using or anticipating the use of a system,

product, or service. Researchers within the HCI com-

munity argue that UX has subjective, holistic, sit-

uational, and temporal characteristics, strongly em-

phasizing design (Bargas-Avila and Hornbæk, 2011;

Roto et al., 2011). However, each of these attributes

brings speciﬁc requirements for both design and eval-

uation (Lallemand, 2015).

UX evaluation includes various methods designed

to assess the emotional, cognitive, and sociocultural

aspects of user interactions with products and systems

https://orcid.org/0000-0001-6478-9870

https://orcid.org/0000-0002-4653-0950

https://orcid.org/0000-0002-4913-7513

https://orcid.org/0000-0001-6436-3773

(Maia et al., 2020). However, ﬁnding a comprehen-

sive and clear resource on the application of UX eval-

uation methods can be challenging for researchers and

professionals. This difﬁculty arises when they attempt

to select the most appropriate approach for evaluating

the quality of user interactions with digital products

and systems.

Collaboration between the academic and profes-

sional communities is essential for disseminating

knowledge and information in the ﬁeld, fostering its

continuous development. Aiming to promote aware-

ness and understanding of UX, the Experience Re-

search Society platform (EXPRESSO

) was created

to provide various resources for user experience re-

search. The platform is maintained by academics and

professionals in the ﬁeld and is supported by an exec-

utive committee from several countries. EXPRESSO

offers information on 81 UX evaluation methods.

However, in a prior experience using the platform,

it was deemed “inefﬁcient” due to signiﬁcant short-

comings found in the tool. One of the main limita-

tions observed is the restricted information provided

https://experienceresearchsociety.org/

614

Beleza, A. C. B., Nascimento da Mota, S. E., Lima, M. S. and Conte, T. U.

UX4ALL: A Repository of User Experience Evaluation Methods.

DOI: 10.5220/0013435100003929

Paper published under CC license (CC BY-NC-ND 4.0)

In Proceedings of the 27th International Conference on Enterprise Information Systems (ICEIS 2025) - Volume 2, pages 614-621

ISBN: 978-989-758-749-8; ISSN: 2184-4992

for each UX evaluation method (Oliveira et al., 2023).

When navigating the platform’s catalog of methods,

it is common to encounter incomplete texts, vague

descriptions, and references with unavailable or re-

stricted links. This creates a barrier for those in-

terested in ﬁnding detailed and reliable information

about UX evaluation methods.

A diagnostic analysis of the EXPRESSO platform

revealed that the method descriptions, intended to

provide crucial information for their application, lack

consistency. Some methods are presented with de-

tailed descriptions, while others offer only a superﬁ-

cial summary. Moreover, the quality of the descrip-

tions is insufﬁcient to enable someone intending to

conduct a UX evaluation to apply the method based

solely on the information provided. In this way, EX-

PRESSO becomes only an intermediary for users to

identify different UX evaluation methods, failing to

effectively clarify their application process. To ob-

tain such information, users must consult additional

resources. This lack of uniformity creates an uneven

experience for EXPRESSO users, leaving them un-

certain about the quality and amount of knowledge

available for each method.

Motivated by the practical problem described and

aiming to address potential knowledge gaps regarding

UX evaluation methods, this research seeks to create

and evaluate a new data repository model for UX eval-

uation methods, called UX4ALL (UX for All). This

repository aims to provide clear and objective infor-

mation to support researchers, students, and indus-

try professionals in applying UX evaluation methods

within their respective contexts.

Thus, the following research question (RQ) guides

the execution of this work:

• RQ: Is it possible to propose a repository of

UX evaluation methods that present relevant and

easy-to-understand information to democratize

UX practices?

To achieve the objective of this work and address

the RQ, we used the EXPRESSO platform as an in-

put data source to extract and ﬁlter the UX evaluation

methods. In the data extraction phase, the 81 UX eval-

uation methods available on the platform were char-

acterized and classiﬁed. During the ﬁltering phase,

we selected some UX methods based on predeﬁned

quality criteria. Finally, we developed UX4ALL, a

new UX evaluation method repository prototype. The

new repository aims to ensure the standardization and

quality of the information provided.

A study of the ﬁrst version of the UX4ALL pro-

totype and an evaluation carried out by an expert in

the ﬁeld of UX showed that UX4ALL provides rele-

vant and easy-to-understand information. These eval-

uations provide evidence positioning UX4ALL as a

support tool for users interested in selecting and ap-

plying UX evaluation methods, democratizing knowl-

edge of UX practices.

The main contributions of this research are: (1)

The democratization of understanding UX evalua-

tion practices; and (2) The creation of an easy-to-

use repository prototype for UX evaluation methods,

named UX4ALL

(UX for All).

2 BACKGROUND

The concept of User Experience (UX) emerged in the

1990s when Don Norman recognized that traditional

Human Interface and Usability principles did not en-

compass all aspects of a user’s interaction with a sys-

tem (Norman, 2008). According to Hassenzahl, UX

distinguishes itself by utilizing a holistic and multi-

disciplinary approach to product development (Has-

senzahl, 2008). From an industrial perspective, Apple

emphasizes the importance of UX in the product de-

sign process, noting that even seemingly trivial issues

can become quite complex. For instance, determin-

ing the location and function of the power button on

Apple computers can present signiﬁcant challenges

(Norman et al., 1995).

Ara

ujo (2024) states that “with the constant emer-

gence of new technologies, products are becoming in-

creasingly interactive, and, consequently, their devel-

opment must also focus on user experiences resulting

from these interactions.” Thus, to ensure that products

meet user expectations and provide a positive experi-

ence, UX evaluation methods become indispensable

tools (Lachner et al., 2017). By gathering data on how

users interact with a system, we can gain valuable in-

sights into user behavior, difﬁculties, frustrations, and

satisfaction points (Nakamura et al., 2023).

UX has become a crucial quality attribute to con-

sider in software projects (Marques et al., 2019).

Identifying the various aspects of UX evaluation and

navigating the diverse range of available methods

poses signiﬁcant challenges (Nakamura et al., 2019).

This complexity is further heightened by the wide

range of evaluation methods available (D

ıaz-Oreiro

et al., 2019), which range from questionnaires to bio-

physiological measures aimed at assessing different

dimensions such as enjoyment, pleasure, and emo-

tions. Moreover, the challenges of UX evaluation are

intensiﬁed by the ﬁeld’s evolving nature. As high-

lighted by Hussain et al., UX research is characterized

by its emergent qualities, which come with difﬁculties

https://tinyurl.com/UX4ALL

UX4ALL: A Repository of User Experience Evaluation Methods

615

in developing effective evaluation methods (Hussain

et al., 2018).

According to Saad et al., UX methods are highly

signiﬁcant as they ensure that the development phase

of a system progresses in a timely manner (Saad et al.,

2021). In every element where user interaction oc-

curs, there are usage experiences. Thus, UX methods

vary depending on the type of interaction and are in-

tended for evaluating products, prototypes, concepts,

or design aspects (Vermeeren et al., 2010).

Many UX methods demand a centralized source,

such as a repository, where researchers, professionals,

or enthusiasts can consult them. To address this need,

Oliveira et al. developed a web platform (Oliveira

et al., 2023). As a database, the authors used a pre-

vious version of the EXPRESSO platform, known as

AllAboutUX

. The authors addressed the problem

faced by users in selecting the most suitable UX eval-

uation method for their projects due to the large va-

riety of available methods. In this sense, Oliveira et

al. introduced UXNator, a tool designed to recom-

mend UX evaluation methods (Oliveira et al., 2023).

While using AllAboutUX as a data source, the au-

thors observed that some methods cataloged on the

EXPRESSO platform lacked adequate documentation

to guide their application or were not directly aligned

with the scope of UXNator, which focuses on soft-

ware evaluation.

This work, similar to that of Oliveira et al.

(Oliveira et al., 2023), uses data from EXPRESSO

(formerly known as AllAboutUX). However, our goal

is to create a repository of UX methods that extends

beyond the context of software evaluation. We aim to

include methods applicable to various areas and prod-

ucts, such as software and games.

3 METHODOLOGY

This work outlines the initial phase of a research

project designed to assist researchers, students, and

industry professionals in evaluating user experience.

The project aims to guide them in selecting the ap-

propriate UX evaluation methods and effectively ap-

plying these methods in their respective projects. Fig-

ure 1 illustrates the steps involved in creating the

UX4ALL repository, as detailed below:

Diagnostic Analysis of the EXPRESSO Plat-

form — This step aimed to identify the strengths and

weaknesses of the EXPRESSO platform. Initially, the

ﬁrst two authors of this research used the platform and

explored its functionalities, enabling an initial evalua-

https://experience.aalto.ﬁ/all-about-ux-blog/

Figure 1: UX4ALL repository build steps.

tion of the tool. Subsequently, to validate the authors’

perceptions, an interview was conducted with 17 par-

ticipants, who were asked two open-ended questions

about their experiences using the EXPRESSO plat-

form. These participants engaged with the platform

under the same usage context as the authors. Each

comment was analyzed and categorized into speciﬁc

codes, differentiating between positive points, nega-

tive points, and suggestions for improvement. Fol-

lowing the initial coding, the codes were reﬁned by

grouping and quantifying mentions of each theme.

Data Extraction — In this step, we characterized

and classiﬁed the 81 UX evaluation methods avail-

able on the EXPRESSO platform. As a result, we

developed an artifact called the Method Characteri-

zation Model for each UX evaluation method. This

model includes the following information: method

name, deﬁnition, application criteria, references, and

year of publication. We extracted the data from the

information provided by the EXPRESSO platform.

When information on the platform was unavailable,

we conducted additional research using other sources.

We also highlighted any missing data in the Method

Characterization Model to indicate the absence of in-

formation on the EXPRESSO platform.

Filtering and Classiﬁcation — In this step,

we selected the UX methods for inclusion in the

UX4ALL repository. We selected the UX methods

based on selection criteria created by the ﬁrst two au-

thors of this research and validated by a UX expert.

The selection process relied on the criteria established

during the “Data Extraction” step. In total, four crite-

ria were deﬁned: Guidance, Scope, Free Access, and

Reproducibility. The deﬁnition of each criterion is

outlined as follows: (1) Guidance – The method must

include instructions for its application and the evalua-

tion of results; (2) Scope – The method must be used

by individuals with different proﬁles; (3) Free Access

– The method must be accessible without associated

costs; and (4) Replicability – The evaluation using

the method should be replicable.

After the ﬁltering process, ten methods were se-

lected and classiﬁed according to their respective cat-

egories, resulting in three distinct groupings: “Obser-

vation,” “Scale,” and “Self-report” (Rivero and Conte,

2017). The observation category includes those meth-

ICEIS 2025 - 27th International Conference on Enterprise Information Systems

616

ods used to analyze the user within the context of

use. The scale category includes questionnaires, rat-

ings, and other quantitative metrics designed to mea-

sure speciﬁc aspects of the user experience. Lastly,

self-report methods allow users to provide qualitative

feedback, enabling them to express their opinions and

feelings openly (Rivero and Conte, 2017). We as-

signed a speciﬁc color to each category for clarity:

green for Observation, blue for Scale, and brown for

Self-report. This color-coded classiﬁcation provides

visual cues that facilitate the identiﬁcation of avail-

able methods in the UX4ALL repository.

Examples of Use — Once the methods were se-

lected, the next step involved creating use case exam-

ples for each method. During this phase, all chosen

methods were applied with users to generate practical

examples of their application. The outcomes of this

step provide valuable information that enhances the

descriptions of the methods available in the UX4ALL

repository.

Prototyping — In this step, we developed the

ﬁrst version of the UX4ALL prototype. We estab-

lished a standardized format for presenting the meth-

ods in UX4ALL, ensuring consistency in the infor-

mation available in the repository. We deﬁned a cat-

alog record format with the following ﬁelds: Method

Name, Author, Method Type, Number of Participants,

Advantages, Limitations, How to Apply, Use Case

Example, Available Material (if any), and Biblio-

graphic References. These records were completed

and validated by experts. Finally, we created the ﬁrst

version of the UX4ALL repository prototype using

the Figma prototyping tool

Repository Evaluation — We conducted an ex-

ploratory experiment involving six participants who

were knowledgeable about UX methods and had prior

experience with the EXPRESSO platform. The par-

ticipants, comprising students from Computing and

Design courses, were engaged to assess the usability

of the ﬁrst version of UX4ALL. We collected qual-

itative data during the evaluation by observing how

participants interacted with the repository. Addition-

ally, we distributed a System Usability Scale (SUS)

questionnaire to gather their feedback. We also pre-

sented the ﬁrst version of the UX4ALL repository to

a UX and Human-Computer Interaction expert, who

provided valuable insights into the repository.

It is important to emphasize that the entire pro-

cess was carried out ethically, with the consent of the

participants. Each participant signed a Free and In-

formed Consent Form (FICF) before the study, which

ensured they understood and agreed to the procedures

and objectives of the research purposes, thereby safe-

https://www.ﬁgma.com/

guarding the ethics and integrity of the research.

Prototype Evolution — After analyzing the re-

sults from user observations during the exploratory

experiment, the feedback from the SUS method, and

the expert’s opinions, we implemented improvements

in the repository, developing a second version of the

UX4ALL repository.

4 UX4ALL REPOSITORY

4.1 Diagnostic Analysis

The ﬁrst two authors of this paper conducted the ini-

tial diagnostic analysis of the EXPRESSO platform.

The ﬁrst two authors had their ﬁrst contact with the

EXPRESSO platform during an assessment activity

in the HCI course. The assessment required students

to use UX evaluation methods to assess an application

of their choice. The course professor introduced EX-

PRESSO as a platform speciﬁcally designed to assist

in selecting appropriate UX evaluation methods.

As a result of the initial diagnostic evaluation, the

two ﬁrst authors of this research identiﬁed that, de-

spite providing information on different UX methods,

the EXPRESSO platform did not provide the neces-

sary detailed information for applying the UX evalu-

ation methods. The authors also noticed that data on

the methods repeated in different information ﬁelds

provided by the platform or were incomplete. In addi-

tion, the authors observed a superﬁcial description of

the methods. The data ﬁeld contents were considered

confusing and incomplete. The analysis showed the

platform’s positive points, for example, information

on the advantages and disadvantages of the methods.

However, not all methods provided this information.

To gain a deeper understanding and obtain spe-

ciﬁc, relevant information, users often need to search

for the method’s name in external sources. Addition-

ally, there is a notable lack of helpful resources that

could aid in understanding and applying these meth-

ods, such as printable questionnaires or practical ex-

amples. Moreover, the initial diagnostic analysis in-

dicated that the page for evaluation methods on the

platform can be unintuitive for ﬁrst-time users. The

pathway to access this page is somewhat unclear. For

example, users must deduce that they must click on

the “Resources” section in the header to ﬁnd a button

directing them to the repository.

To further support the diagnostic analysis of the

EXPRESSO platform, we invited 17 HCI students

who participated in an HCI assessment to share their

opinions about their experience in using EXPRESSO.

To do so, the students answered two questions: “Did

UX4ALL: A Repository of User Experience Evaluation Methods

617

using the EXPRESSO platform help or hinder you in

selecting UX evaluation methods?” and “Do you have

any suggestions for improving the EXPRESSO plat-

form?” The students’ responses were analyzed qual-

itatively. We performed an open coding process to

identify and categorize the students’ responses into

themes that highlight the strengths and weaknesses of

the EXPRESSO platform.

The qualitative analysis enabled a deeper analy-

sis of the student’s answers, showing that the EX-

PRESSO platform is valuable for supporting UX eval-

uation methods selection and presenting a variety of

available methods. However, only 5 participants (P4,

P5, P6, P9, and P13) reported a positive experience

using EXPRESSO, while the remaining 12 partici-

pants provided feedback ranging from mixed (posi-

tive and negative) to entirely negative experiences.

Despite offering a wide range of methods, the di-

agnostic analysis showed the platform has signiﬁcant

limitations. Many methods lack the content and in-

formation necessary for their application, particularly

those requiring questionnaires or other speciﬁc mate-

rials (P17). Even when external reference links are

available, they are often inaccessible (P15 and P16).

Consequently, we believe only the simplest methods

can be fully applied using the platform alone. This

limitation forces users to seek additional information

from other sources (P1), potentially impacting the

overall user experience. The analysis and processing

of the data are publicly available

4.2 Data Extraction

To organize the data extracted from the EXPRESSO

platform, we cataloged the 81 UX evaluation meth-

ods. The cataloging artifact contains the name, a brief

description of the method, the context of application

for each method, the references, and the year of pub-

lication. The artifact resulting from this step is avail-

able for consultation

We could note that some of the cataloged meth-

ods lack bibliographic references detailing the origins

of the evaluation methods. To ensure a clear charac-

terization of these methods, we supplemented miss-

ing information through searches on digital libraries

that index scientiﬁc research. During the data extrac-

tion phase, we also observed a lack of essential de-

tails, such as guidance on how to apply the methods

with users, the required number of participants, and

the process for analyzing the results. This missing in-

formation poses challenges in understanding how to

effectively use the methods.

https://ﬁgshare.com/s/73f85d7378bb84cba4ae

4.3 Filtering and Classiﬁcation

We selected ten UX evaluation methods that satisﬁed

the established selection criteria in this step. These

methods included AttrackDiff, Audio Narrative, Co-

discovery, EmoCards, Experiential Contextual In-

quiry, Geneva Appraisal Questionnaire, NASA-TLX,

Positive and Negative Affect Schedule, System Usabil-

ity Scale, and 3E. According to Rivero and Conte

(Rivero and Conte, 2017), it is possible to system-

atically categorize these methods. In this research,

we classiﬁed the UX methods into three categories:

observation methods, scale methods, and self-report

methods, based on the “Filtering and Classiﬁcation”

step described in Section 3.

We assigned a corresponding color to each cate-

gory: green for observation methods, blue for scale-

based methods, and brown for self-report methods.

The color-coded ﬁlter allows users to visualize and

select the most appropriate UX evaluation method for

their context.

4.4 Examples of Use

We used the UX evaluation methods on real case stud-

ies to collect examples of use for the methods se-

lected during the ﬁltering step. This process required

research beyond the EXPRESSO platform to under-

stand each method fully and to establish the neces-

sary procedures for their application. Some methods,

such as the Geneva Appraisal Questionnaire and 3E,

already had examples published in previous studies.

The conﬁgurations used to create the examples of

use for the UX evaluation methods are available

We highlight the UX evaluation method employed,

the context of the application being evaluated, the ac-

tivities conducted during the evaluation process, the

characterization of the participants, and the number

of participants involved.

4.5 Prototyping

After creating examples of how to utilize the methods,

we established a standardized set of information to be

provided for each method in the UX4ALL repository.

Since the data presented by the EXPRESSO platform

had numerous inconsistencies, it was critical to en-

sure that the newly developed repository model did

not inherit these ﬂaws. To achieve this, UX4ALL of-

fers a comprehensive and uniform set of information

for all cataloged methods. This includes the method

name, author, method type, minimum number of par-

ticipants, application process, usage examples, advan-

tages, limitations, and references.

ICEIS 2025 - 27th International Conference on Enterprise Information Systems

618

The standardized presentation of data is designed

to ensure the integrity and quality of the information

provided, creating a more consistent and reliable ex-

perience for users of UX4ALL. By adopting a stan-

dardized approach, we aim to address the shortcom-

ings observed in the current model, ultimately offer-

ing a more comprehensive and useful resource for the

UX evaluation community.

The graphic design of the interface for the

UX4ALL repository aims to facilitate ease of use for

interested users. Methods are presented in a card for-

mat and categorized using a color scheme that enables

quick identiﬁcation of their respective categories.

The ﬁrst version of the UX4ALL repository com-

prises three main functionalities

: the ’About UX,’

’Methods,’ and ’Categories’ pages. Additionally, a

search bar is positioned at the top of the screen as

a navigation component. The search bar provides a

quick way to ﬁnd information within the repository.

The “About UX” page includes a mind map that

outlines key concepts and terminology within the UX

ﬁeld

. The mind map was developed based on the

white paper by Roto et al. (Roto et al., 2011). This

model was selected for its effectiveness as a visual

representation, which simpliﬁes complex ideas by or-

ganizing information in a connected structure cen-

tered around a central theme (Buzan, 1994). Addi-

tionally, mind maps facilitate information retention,

making them a versatile and efﬁcient tool for under-

standing and organizing content.

On the “Methods” page

, ten selected evaluation

methods are displayed. UX4ALL shows the meth-

ods using color-coded cards to indicate their respec-

tive categories. In UX4ALL, a grid layout has been

employed to showcase the cards, providing a more

visually appealing presentation compared to a tradi-

tional list format.

Grid navigation is intuitive for users, particularly

those who are familiar with e-commerce platforms

and social media. Additionally, the grid layout en-

ables the inclusion of images, icons, and other visual

elements, making the interface more engaging and

aesthetically pleasing.

On the “Categories” page

, the three main cat-

egories encompassing the methods in UX4ALL are

described: scale-based methods (blue), observation

methods (green), and self-report methods (brown).

5 ASSESSMENT OF UX4ALL

We conducted an exploratory experiment to evaluate

the initial version of the repository UX4ALL. The ex-

periment involved six participants, all undergraduate

students in Computing and Design with prior knowl-

edge of UX and experience using the EXPRESSO

website. These participants were invited to explore

the prototype without speciﬁc guidance or a prede-

ﬁned time limit, allowing for a natural immersion in

the proposed interface. During this exploration, their

experiences were carefully documented and recorded

to capture nuances and spontaneous reactions.

A qualitative analysis of the data collected through

observations revealed both positive aspects and areas

for improvement. One of the most signiﬁcant ﬁndings

was the inconsistency in the information presented on

the cards, particularly the use of the term “disadvan-

tages” rather than “limitations.” This discrepancy was

noted by a participant during their interaction with the

prototype, underscoring the importance of consistent

language throughout the content.

An additional issue raised during the experiment

was the lack of clarity concerning the colors of the

cards used to present UX methods. One partici-

pant questioned whether the colors were intention-

ally related, as there was no clear indication of their

meaning. Furthermore, participants reported difﬁ-

culty scrolling back to the methods page, which high-

lighted certain prototyping issues that need to be ad-

dressed to ensure smooth and intuitive navigation.

Users expressed appreciation for the information

presented on the site, ﬁnding it relevant and useful

for their needs. Feedback like “Can I take notes?”

and “I’ll save this link; it has very useful information

for me” highlights the value of the content provided

by UX4ALL. While some participants initially noted

the extensive amount of text, they later recognized the

relevance of the information after reading it.

Participants were invited to assess their experi-

ence using the well-known System Usability Scale

(SUS) questionnaire (Brooke et al., 1996). This in-

strument provided a quantitative evaluation that com-

plemented the qualitative observations.The results of

the SUS evaluation indicated an acceptable usability

score, with an average of 93.75 points. This suggests

that the participants encountered no major issues with

the system, successfully achieved their goals, and had

an overall satisfactory experience.

A UX specialist reviewed the UX4ALL ﬁrst ver-

sion and provided several key recommendations: (1)

enhance the platform’s UI by incorporating images re-

lated to the methods to attract future users, (2) create

a dedicated section to present the project and its au-

thors, (3) integrate a form that allows users to con-

tribute additional methods, thereby transforming the

platform into a collaborative environment, and (4) add

new evaluation methods. These suggestions aim to

improve user experience and strengthen the commu-

UX4ALL: A Repository of User Experience Evaluation Methods

619

nity around the project, promoting participation and

knowledge sharing.

In response to our research question, we conclude

that it is indeed possible to create a repository of UX

evaluation methods that offers relevant and easily un-

derstandable information. This repository should be

educational, provide materials and application exam-

ples, and be reviewed and standardized, ultimately

democratizing UX practices. Feedback from partici-

pants indicates that UX4ALL contains useful and rel-

evant content, with thorough and complete descrip-

tions of the methods. It offers resources that facil-

itate searches and adds value through its collabora-

tive approach. However, challenges remain, including

the limited number of available methods, the diver-

sity of method categories, and the language used for

UX4ALL. These issues need to be addressed in future

versions of the repository.

6 UX4ALL EVOLUTION

Based on the results obtained from the exploratory

study and the feedback from the UX specialist, we

improved the repository UX4ALL. We also added ﬁve

new UX evaluation methods to the repository, aiming

to expand the range of methods available

Moreover, a new tab was added to the UX4ALL

homepage, the “About Us.” This tab is designed to

provide information about the repository’s purpose,

as well as include a form for the community to con-

tribute to the repository’s growth

. The new “About

Us” tab aims to create opportunities for dialogue be-

tween the repository and users interested in collabo-

rating with this initiative.

The repository interface has been enhanced. The

changes made UX4ALL more visual and interactive,

making it easier to ﬁnd the materials needed to under-

stand and apply UX evaluation methods. The layout

of the “Methods” page was redesigned. Images cor-

responding to each presented method were added

Finally, a “Available Material” section was included

on the pages describing the methods

, providing the

necessary instruments for applying the methods

7 CONCLUSION

This research presents the initial design and evalu-

ation of a new repository model for UX evaluation

methods, named UX4ALL. We created the reposi-

tory prototype to provide comprehensive information

about UX evaluation methods and help democratize

UX practices for all interested individuals. Using

UX4ALL, users can access pages that explain the

concept of UX and the categories of each UX eval-

uation method. In addition, users can contribute by

adding information about new UX methods to expand

the repository’s offerings.

The motivation behind developing UX4ALL

stemmed from the challenges encountered while us-

ing the EXPRESSO platform, which was described as

frustrating by the ﬁrst two authors of this research. A

diagnostic evaluation of EXPRESSO revealed several

signiﬁcant limitations, including the lack of detailed

method descriptions, insufﬁcient practical use exam-

ples, broken links to bibliographic references, and

missing materials necessary for applying the methods.

In response, UX4ALL aims to address these short-

comings and meet the needs of individuals seeking

to understand and implement UX evaluation methods.

This effort ultimately aims to promote the democrati-

zation of UX practices. The new repository addresses

the most common issues highlighted by participating

students and the ﬁrst two authors, such as updating

references, adding application materials, and provid-

ing practical examples.

The UX4ALL repository currently has limita-

tions, especially in terms of the number of available

UX evaluation methods. To address this issue, we

will conduct a new cataloging of UX evaluation meth-

ods. Additionally, the community can help expand

the repository by suggesting new methods through the

form available in the “About Us” section of the cur-

rent UX4ALL version

The initial evaluation of UX4ALL focused on us-

ability and user satisfaction. To gain deeper insights

into its effectiveness, comparative evaluations with

other platforms will be necessary. Additionally, the

familiarity of participants with the methods used in

UX4ALL and the inﬂuence of individual preferences

should be considered as limitations in assessing the

repository’s overall effectiveness.

For future work, we envision the following goals:

(1) validation in an academic context to conduct a

deeper analysis of UX4ALL use, ensuring that its

content meets user needs and addresses gaps found

in EXPRESSO; (2) addition of new UX evaluation

methods in collaboration with the academic and pro-

fessional communities; (3) evolution of the prototype

into a software product; and (4) exploration of the re-

liability of information about UX evaluation methods

produced by large-scale language models (LLMs),

considering the current technological trends.

ICEIS 2025 - 27th International Conference on Enterprise Information Systems

620

ACKNOWLEDGEMENTS

We thank all the participants in the empirical study

and USES Research Group for their support. The

present work is the result of the Research and De-

velopment (R&D) project 001/2020, signed with Fed-

eral University of Amazonas and FAEPI, Brazil,

which has funding from Samsung, using resources

from the Informatics Law for the Western Ama-

zon (Federal Law nº 8.387/1991), and its disclo-

sure is in accordance with article 39 of Decree No.

10.521/2020. Also supported by CAPES - Financing

Code 001, CNPq process 314797/2023-8, CNPq pro-

cess 443934/2023-1, CNPq process 445029/2024-2,

Amazonas State Research Support Foundation - FA-

PEAM - through POSGRAD 24-25, and Amazonas

State University through Academic Productivity Pro-

gram 01.02.011304.026472/2023-87.

REFERENCES

Ara

ujo, F. S. (2014). User experience assessment: a pro-

posal for systematization of the product development

process (in Portuguese). 2014. 238 p. PhD thesis,

Universidade Federal de Santa Catarina, Centro Tec-

nol

ogico.

Bargas-Avila, J. A. and Hornbæk, K. (2011). Old wine in

new bottles or novel challenges: a critical analysis of

empirical studies of user experience. In Proceedings

of the SIGCHI conference on human factors in com-

puting systems, pages 2689–2698.

Brooke, J. et al. (1996). Sus-a quick and dirty usability

scale. Usability evaluation in industry, 189(194):4–7.

Buzan, T. (1994). The Mind Map Book. Dutton.

ıaz-Oreiro, I., L

opez, G., Quesada, L., and Guerrero, L. A.

(2019). Standardized questionnaires for user experi-

ence evaluation: A systematic literature review. In

Proceedings, volume 31, page 14. MDPI.

Din, E. (2020). Ergonomie der mensch-system-interaktion–

teil 110: Interaktionsprinzipien (iso 9241-110:

2020)[ergonomics of human-system interaction-part

110: Interaction principles]. Deutsche Fassung EN

ISO, pages 9241–110.

Hassenzahl, M. (2008). User experience (ux) towards an ex-

periential perspective on product quality. In Proceed-

ings of the 20th Conference on l’Interaction Homme-

Machine, pages 11–15.

Hassenzahl, M. (2018). The thing and i (summer of’17

remix). Funology 2: From usability to enjoyment,

pages 17–31.

Hussain, J., Ali Khan, W., Hur, T., Muhammad Bilal, H. S.,

Bang, J., Ul Hassan, A., Afzal, M., and Lee, S. (2018).

A multimodal deep log-based user experience (ux)

platform for ux evaluation. Sensors, 18(5):1622.

Lachner, F., Fincke, F., and Butz, A. (2017). Ux metrics:

Deriving country-speciﬁc usage patterns of a web-

site plug-in from web analytics. In Human-Computer

Interaction–INTERACT 2017: 16th IFIP TC 13 Inter-

national Conference, Mumbai, India, September 25–

29, 2017, Proceedings, Part III 16, pages 142–159.

Springer.

Lallemand, C. (2015). Towards consolidated methods for

the design and evaluation of user experience.

Maia, M. A. Q., Barbosa, R. R., and Williams, P. (2020).

Usabilidade e experi

encia do usu

ario de sistemas de

informac¸

ao: em busca de limites e relac¸

oes. Ci

encia

da Informac¸

ao em Revista, 6(3):34–48.

Marques, L., Matsubara, P., Nakamura, W., Wiese, I.,

Zaina, L., and Conte, T. (2019). Ux-tips: A ux evalua-

tion technique to support the identiﬁcation of software

application problems. In Proceedings of the XXXIII

Brazilian Symposium on Software Engineering, pages

224–233.

Nakamura, W. T., de Oliveira, E. H., and Conte, T. (2019).

Negative emotions, positive experience: What are we

doing wrong when evaluating the ux? In Extended

abstracts of the 2019 chi conference on human factors

in computing systems, pages 1–6.

Nakamura, W. T., Marques, L. C., Redmiles, D.,

de Oliveira, E. H., and Conte, T. (2023). Investigat-

ing the inﬂuence of different factors on the ux evalua-

tion of a mobile application. International Journal of

Human–Computer Interaction, 39(20):3948–3968.

Norman, D., Miller, J., and Henderson, A. (1995). What

you see, some of what’s in the future, and how we go

about doing it: Hi at apple computer. In Conference

companion on Human factors in computing systems,

page 155.

Norman, D. A. (2008). Design emocional: por que

adoramos (ou detestamos) os objetos do dia-a-dia.

Rocco.

Oliveira, S., Cristo, A., Geovane, M., Xavier, A., Silva, R.,

Rocha, S., Marques, L. C., Gomes, G., Gadelha, B.,

and Conte, T. (2023). Uxnator: A tool for recom-

mending ux evaluation methods. In ICEIS (2), pages

336–343.

Rivero, L. and Conte, T. (2017). A systematic mapping

study on research contributions on ux evaluation tech-

nologies. In Proceedings of the XVI Brazilian sympo-

sium on human factors in computing systems, pages

1–10.

Roto, V., Law, E.-C., Vermeeren, A. P., and Hoonhout, J.

(2011). User experience white paper: Bringing clarity

to the concept of user experience.

Saad, J., Martinelli, S., Machado, L. S., de Souza, C. R., Al-

varo, A., and Zaina, L. (2021). Ux work in software

startups: A thematic analysis of the literature. Infor-

mation and Software Technology, 140:106688.

Vermeeren, A. P., Law, E. L.-C., Roto, V., Obrist,

M., Hoonhout, J., and V

anen-Vainio-Mattila, K.

(2010). User experience evaluation methods: current

state and development needs. In Proceedings of the

6th Nordic conference on human-computer interac-

tion: Extending boundaries, pages 521–530.

UX4ALL: A Repository of User Experience Evaluation Methods

621