Rethinking Usability Assessment: Integrating UX and Information

Architecture

Jo Santos

1 a

, M

arcia Lima

2 b

and Tayana Conte

1 c

Instituto de Computac¸

ao, Universidade Federal do Amazonas, Manaus, Brazil

Universidade do Estado do Amazonas, Manaus, Brazil

Keywords:

Usability, User Experience, Information Architecture, Heuristic Evaluation.

Abstract:

Evaluating software usability in the early stages of development is important, thus avoiding costs associated

with future changes and dissatisﬁed users. Current usability inspection techniques may be limited in scope as

they do not embrace concepts such as User Experience (UX) and Information Architecture (IA). This paper

presents a new set of heuristics based on Garrett’s elements of UX to be used in usability inspections, aiming

to create systems that prioritize UX and IA aspects as an alternative to existing heuristic sets. The set was

developed based on Garrett’s planes of UX, resulting in the creation of 14 heuristics organized according to

each UX plane. We conducted an empirical study to analyze the technique’s feasibility. The results indicate

that the new heuristics set can detect a reasonable amount of defects within an appropriate time frame. Ad-

ditionally, we received feedback on the heuristics themselves, allowing for slight modiﬁcations. Finally, the

paper concludes by discussing future directions for the new heuristics set.

1 INTRODUCTION

Usability can be deﬁned as the ease of use of a system

and acceptance from a speciﬁc group of users, and it

is important to evaluate it in the early stages of the

project, thus avoiding possible costs and difﬁculties

in redesigning the software (Holzinger, 2005). Taking

this into consideration, one low-cost way of assessing

system usability is through usability inspection meth-

ods, which rely on interface review by experts rather

than direct interaction with users (Novick, 2007).

Heuristic Evaluation (HE) is an inspection method

introduced by Nielsen and Molich (1990) . In this

technique, a group of usability experts analyzes a sys-

tem’s interface based on a set of principles known as

heuristics. During the evaluation, the experts iden-

tify usability issues and assess their severity (Novick,

2007). This method is widely used and is commonly

based on the set of 10 heuristics proposed and tested

by Nielsen and Molich in 1990 .

To conduct a HE, it is recommended that the eval-

uation not be done by just one specialist, but rather

by 3 to 5 assessors individually (Nielsen, 1990). Fur-

https://orcid.org/0009-0009-9298-4724

https://orcid.org/0000-0002-4913-7513

https://orcid.org/0000-0001-6436-3773

thermore, HE offers several advantages, such as being

cost-effective, intuitive, and not requiring prior plan-

ning (Nielsen and Molich, 1990).

Although HE provides a valuable approach to de-

tecting usability issues and certain advantages, its ap-

plicability may be limited in scenarios where the sys-

tem emphasizes user interaction (Masip et al., 2011).

Similar to usability, User Experience (UX) is con-

sidered a quality attribute of software, broadening the

aspects of the user’s interaction with the system. Nor-

man (1995) was one of the ﬁrst researchers to use the

term UX, with the intention of encompassing all as-

pects of the user’s experience with a system. Norman

coined the term UX because he believed that usability

was too narrow or limited to represent a comprehen-

sive view of human-computer interactions (Norman

et al., 1995).

UX is a multifaceted concept encompassing the

interaction between users and products or services,

emphasizing both the immediate and evolving nature

of these interactions. Hassenzahl highlights UX as

a temporal, evaluative feeling driven by the fulﬁll-

ment of fundamental human needs such as auton-

omy, competency, stimulation, and relatedness (Has-

senzahl, 2008). The Nielsen Norman Group expands

this deﬁnition by framing UX as the holistic experi-

ence, integrating various disciplines to ensure usabil-

Santos, J., Lima, M. and Conte, T.

Rethinking Usability Assessment: Integrating UX and Information Architecture.

DOI: 10.5220/0013363900003929

Paper published under CC license (CC BY-NC-ND 4.0)

In Proceedings of the 27th International Conference on Enterprise Information Systems (ICEIS 2025) - Volume 2, pages 589-596

ISBN: 978-989-758-749-8; ISSN: 2184-4992

589

ity, satisfaction, and efﬁciency beyond just the visual

interface (Norman and Nielsen, 1998). Garrett (2011)

further complements this perspective by focusing on

the real-world behavior and context of use, emphasiz-

ing that UX extends beyond functionality and aesthet-

ics to include the broader experiential and situational

aspects.

In addition to deﬁning UX, Garrett (2011) intro-

duces the concept of Information Architecture (IA),

which focuses on how users cognitively process in-

formation by creating organizational and navigational

structures that facilitate efﬁcient content navigation.

Building on these ideas, Garrett presents a ﬁve-plane

framework, each addressing speciﬁc purposes and el-

ements to guide the design process, ensuring a system

that provides a positive user experience.

In this sense, conducting research to evaluate UX

and address IA issues in the early stages of system de-

sign is crucial for creating user-centered systems that

deliver meaningful and efﬁcient interactions. That

way, designers can ensure the system aligns with user

expectations and emotional responses, reducing the

risk of costly redesigns later in the process. Since IA

focuses on how users process and navigate informa-

tion, enabling the creation of intuitive organizational

and navigational schemes, the early-stage attention to

IA ensures that the system’s structure supports clear

and efﬁcient user interactions, minimizing cognitive

overload and enhancing usability(Garrett, 2011).

Considering the need to evaluate UX and address

IA issues early in system design, we propose UXIA,

a new set of heuristics based on Garrett’s UX ele-

ments. These heuristics cover content, IA, naviga-

tion, and layout, aiming to provide a framework for

assessing both UX and IA during the initial design

phases. UXIA was developed to help product teams

create a cohesive and well-structured user experience

from the outset, ensuring that key design elements are

integrated early in the process.

We also conduct a feasability study, attempting

to assess UXIA’s initial viability. The evaluation re-

vealed several usability issues, such as difﬁcult nav-

igation and inconsistent icons, which led to reﬁne-

ments in the heuristics, including clearer descriptions

and the addition of a new heuristic focused on media

format, enhancing UXIA’s ability to evaluate both UX

and IA effectively.

From this study, we aim to develop a more com-

prehensive and effective heuristics set, capable of

guiding the design of system interfaces from the early

stages of the project, prioritizing UX and IA.

2 BACKGROUND

2.1 Garrett Framework: The Elements

of User Experience

Garrett (2011) proposed a framework of UX elements

planes that presents an organizational structure for the

web application design process. It aims to divide UX

into ﬁve distinct and complementary planes, with the

goal of providing a comprehensive and holistic under-

standing of UX, as shown in Figure 1.

Figure 1: Garrett’s planes framework(Garrett, 2011).

Each plane of the framework represents a unique

perspective of UX, being used to ensure that no as-

pect of users’ experience when using the product is

neglected. The framework’s structure is conceived in

a bottom-up approach, in which each plane increases

concreteness, providing a more detailed and speciﬁc

view at each stage. Figure 1 illustrates the ﬁve planes

of the framework. The following are each of the

planes in detail:

The ﬁrst plane is the Strategy, which concerns the

objectives of the application, both internally and ex-

ternally. Internally, these objectives are aligned with

the product goals and business objectives, while ex-

ternally, they address the needs of the application’s

users(Garrett, 2011).

The second plane is the Scope, which addresses

the application’s functional speciﬁcations and content

requirements. It involves a detailed description of the

product features, as well as the media contents that

will be shown in the application(Garrett, 2011).

The third plane is the Structure, which focuses

on interaction design and IA. Interaction design refers

to the system’s behavior in dealing with user inter-

actions, while IA involves arranging media content

and interface elements to facilitate user understand-

ing(Garrett, 2011).

The fourth plane is the Skeleton, which is divided

into three parts: information design, interface design,

and navigation design. Information design deals with

ICEIS 2025 - 27th International Conference on Enterprise Information Systems

590

the presentation of data to facilitate user understand-

ing, interface design involves the elements that al-

low users to interact with the application interface,

and navigation design refers to the elements that al-

low users to move through the application through

IA(Garrett, 2011).

Finally, the ﬁfth plane is the Surface, which

stands out for the importance of the user’s sensory

experience. At this plane, content, functionality, and

aesthetics come together to produce the ﬁnal design

of the product, fulﬁlling the objectives established in

the four previous planes.

Garrett’s framework provides a thorough ap-

proach to UX design, covering all aspects of user ex-

perience, from strategy to sensory design. Its bottom-

up structure helps create cohesive designs by aligning

business goals with user needs, offering a solid foun-

dation in more structured design processes.

2.2 Different Approaches to Usability

Heuristics

Masip et. al. (2011) note that Nielsen’s heuristics

(Nielsen, 1993) are insufﬁcient for evaluating interac-

tive systems. They present two cases where standard

heuristics failed. In the ﬁrst case, the evaluation of

Berta, a virtual assistant for the Lleida City Council,

focused on dialog and facial expression issues, which

were not addressed by traditional heuristics. In the

second case, the evaluation of public kiosks (PICs) re-

quired heuristics that considered the physical environ-

ment and hardware features, which standard heuris-

tics did not cover. These examples demonstrate the

need for alternative evaluation approaches for inter-

active systems.

One of the various approaches to adding the UX

perspective to usability inspection is attempted by

Choma et al. (2016), in which a set of usability as-

pects is selected when inspecting a system. Then, a

group of typical users is selected, indicating the de-

gree of importance of each aspect. The idea is that,

through these steps, a relative usability degree can be

generated to identify the usability degree of the evalu-

ated application. Despite the approach adding the UX

perspective to usability inspection, the technique used

presents results that are inﬂuenced by the high degree

of subjectivity of the experts.

On the other hand, Bolchini et al. (2009) propose

a usability inspection technique that addresses various

design dimensions, including content, IA, navigation,

and layout. This technique consists of a set of heuris-

tics analogous to traditional usability heuristics, but

all related to semiotics, with the aim of identifying

problems in interfaces. Although the initial focus of

the work is to assist designers and testers in conduct-

ing a semiotic inspection on a speciﬁc website, it is

important to note that the set of heuristics does not

explicitly focus on evaluating the UX.

Islam et al. (2020) conduct an experiment to eval-

uate the creation and application of the SIDE frame-

work, a new set of heuristics designed for the semi-

otic analysis of web and mobile interfaces. The

study compares the effectiveness of this new method

with the traditional HE (Nielsen and Molich, 1990),

demonstrating that each approach identiﬁes different

types of usability issues. While Nielsen’s heuristics

(Nielsen, 1993) are more effective in detecting struc-

tural problems, such as navigation and layout, the

proposed heuristics framework prove to be more pre-

cise in analyzing the intuitiveness and clarity of visual

signs. The results indicate that Nielsen’s set of heuris-

tics can be complemented by other frameworks, such

as the SIDE framework, to achieve a more compre-

hensive and detailed evaluation. In this case, the com-

plementarity allowed for a more in-depth examination

of the semiotic dimensions of the interface.

Parente Da Costa et al. (2019) conduct a sys-

tematic literature review to identify usability heuris-

tics for mobile applications, highlighting gaps such

as the lack of heuristics that consider usage context,

mobility, and dynamic environments. It also empha-

sizes the importance of heuristics that address cogni-

tive load by reducing interface complexity, minimiz-

ing memory demands, and providing clear feedback

to improve efﬁciency and user experience, especially

in multitasking scenarios.

In addition to the previously mentioned studies

and approaches, it is important to emphasize the role

of IA in system design. Morville and Rosenfeld

(2006) highlight that incorporating IA early in the

design process offers signiﬁcant beneﬁts. These in-

clude reducing costs related to locating and identify-

ing incorrect information. Furthermore, the authors

note that integrating IA enhances the user experience,

leading to a more efﬁcient and intuitive application

design.

The cited papers propose new heuristics encom-

passing UX, AI, and other contexts, analyze exist-

ing heuristics, and conduct systematic reviews to re-

ﬁne usability evaluation and enhance interface design.

However, none of them presents an approach that in-

tegrates IA and UX concepts into a single compre-

hensive and effective heuristic set. The objective of

the work presented in this paper is to create a set of

heuristics that addresses these identiﬁed gaps by inte-

grating IA and UX concepts into a uniﬁed framework.

Rethinking Usability Assessment: Integrating UX and Information Architecture

591

3 DEVELOPMENT OF THE UXIA

HEURISTICS

3.1 Concept and Methodology

In this section, we describe the UXIA Heuristics, a

heuristics set developed to integrate UX and IA con-

cepts into the system design process. Based on Gar-

rett’s UX elements(Garrett, 2011), the UXIA heuris-

tics aim to support usability evaluation by ensuring

the inclusion of UX and IA perspectives in the devel-

opment process. UXIA heuristics should be applied

during usability inspections and can be used at any

phase of development, as long as prototypes are avail-

able for evaluation by usability experts. This ﬂexibil-

ity allows the technique to be adapted throughout the

entire project lifecycle.

The approach for the use of UXIA opts for a HE

as a usability inspection, aiming to provide a simple

assessment(Novick, 2007) that can be quickly con-

ducted without user participation, making it easier to

implement.

To create this set of new heuristics, we followed

the methodology proposed by Rusu et al.(Rusu et al.,

2011), which involves initial research through ex-

ploratory, descriptive, and correlational steps. This

process moves on to describing the heuristics them-

selves and, ﬁnally, validating and reﬁning the created

set of heuristics.

3.2 Initial Research

The initial research steps arose from the observation

that applications focused on user interaction were not

fully covered by evaluations using Nielsen’s heuris-

tics(Nielsen, 1993), as seen in related works(Masip

et al., 2011). Therefore, it was necessary to develop

a set of heuristics that considered the UX perspective

when assessing such applications. In this regard, an

in-depth study of Garrett’s work (Garrett, 2011) was

conducted to create UXIA heuristics for each of the

planes proposed by Garrett.

3.3 Describing UXIA Heuristics

After the initial research step, the next step(Rusu

et al., 2011) involved developing the heuristics them-

selves. Based on the study of Garrett’s work, ini-

tially, we developed UXIA comprising 14 heuristics

that cover each of the planes of UX elements pro-

posed by the author (Garrett, 2011). Subsequently,

the heuristics are presented and organized according

to Garrett’s planes.

Below, we present the heuristics that have been

developed.

Strategy Plane

Garrett (2011) highlights the importance of user

segmentation and its impact on the overall user ex-

perience of an application. He also points out that

users in different roles have distinct needs. With this

in mind, we developed H1:

H1: Application for Multiple Types of Users -

Attention should be paid to ensure that, according to

the objectives, the application is designed for more

than just one type of user.

H2, on the other hand, was developed based on

Garrett’s insights(Garrett, 2011) that suggest that to

create a good UX, every decision must be rooted in

an understanding of its consequences, closely aligned

with the intended goal.

H2: Clear Application Objective - It is neces-

sary to evaluate whether the application’s objective is

clearly deﬁned and evident within the application, en-

suring it aligns with its intended purpose.

Scope Plane

When discussing the scope of an application, Gar-

rett (2011) emphasizes the importance of including

essential functionalities that are directly aligned with

the proposed objectives. He underscores the need to

avoid incorporating unnecessary features that could

potentially complicate the product without adding

signiﬁcant value. Based on these considerations, we

developed heuristics H3 and H4:

H3: Essential Features - The application must

have the essential features to achieve the business ob-

jectives.

H4: Inconsistent Features - Evaluate the pres-

ence of features that are not part of the application’s

scope to prevent the loss of the objective.

Based on Garrett’s observations(Garrett, 2011),

aligning content with required functionalities is more

than just creating text or visuals. It’s essential to

understand how this content will be used to achieve

strategic goals and meet user expectations. From

these insights, we developed H5:

H5: Cohesive Content with Features - Ensure

that the content, texts, photos, and media in general,

are related to the features in a cohesive manner.

Structure Plane

Garrett (2011) suggests that a signiﬁcant aspect

of interaction design involves handling user errors,

which includes considering how the system behaves

when errors occur and how to prevent them from hap-

pening in the ﬁrst place. From this perspective, we

developed H6:

H6: Effective Error Management - Applica-

tions need to handle errors in a way that allows the

ICEIS 2025 - 27th International Conference on Enterprise Information Systems

592

user to understand what is going wrong, along with a

possible solution, if applicable.

Garrett (2011) underscores the importance of cre-

ating navigation schemes that enable users to navigate

site content efﬁciently, ensuring they can easily ﬁnd

the information and functions they seek. With this in

mind, we developed H7 and H8 heuristics:

H7: Navigation Control - It should always be

possible for the user to navigate wherever possible,

avoiding creating screens where the user can only exit

using the browser or operating system navigation but-

tons.

H8: Easy Access to Functions - Functions are

the core of an application; therefore, they should be

easily accessible to users, enabling them to ﬁnd them

without much difﬁculty.

Continuing his discussion on architectural

schemes, Garrett (2011) emphasizes the importance

of ensuring that the number of steps or clicks re-

quired to complete a task makes sense to the user,

and mainly if each step logically follows the previous

one. This underscores the critical importance of a

well-thought-out and intuitive structure. Based on

these principles, we developed H9:

H9: Organized and Ordered Functionalities - It

is necessary to assess whether the application’s func-

tionalities present a logical and cohesive ﬂow with the

business objectives.

Skeleton Plane

Garrett (2011) emphasizes the critical importance

of choosing interface elements that truly enhance user

understanding and facilitate interaction with the ap-

plication. He also highlights the signiﬁcance of using

navigation elements to help users orient themselves

and know where they can navigate. Keeping these

principles in mind, H10 was developed:

H10: Clear and Structured Navigation - Inter-

face navigation items should be visible and easily ac-

cessible to users, allowing them to know where they

came from and where they can go.

Revisiting concepts of IA, Garrett (2011) under-

scores the signiﬁcance of grouping application infor-

mation and functionalities based on shared charac-

teristics, ensuring that organizational principles align

with both user objectives and the application’s goals.

From this perspective, H11 is formulated:

H11: Appropriate Organization - The appli-

cation should present some form of organization,

whether alphabetical, temporal, by category, etc., but

always in line with the business objectives. With eas-

ily accessible menus and without overlapping.

Surface Plane

When discussing the surface plane, Garrett (2011)

makes it clear that he is referring to the sensory design

plane, speciﬁcally more connected to the senses, par-

ticularly vision and hearing and also emphasizes that

hearing plays a fundamental role in the experience of

using an application. From this arises H12:

H12: Necessary Sounds - It is important to as-

sess whether the sounds used in the application are

truly necessary and if they align with the rest of the

application.

Addressing the perspective of vision, Garrett

(2011) emphasizes the importance of applications

having adequate contrast to prevent eye strain, while

also maintaining internal and external consistency.

From this, we developed heuristics H13 and H14:

H13: Color Contrast and Uniformity - The col-

ors of an application need to have a certain level of

contrast so that the user knows where to ”stop” their

eyes instead of seeing everything at once, in addition

to uniformity so that the color palette is consistent and

does not overwhelm the user.

H14: Proper Icon Usage - The application’s

icons must make sense of the functionalities and al-

low users to recognize a feature easily.

Among the proposed UXIA heuristics, some cor-

respond to Nielsen’s heuristics (Nielsen, 1993), while

others represent novel contributions. In particular,

the heuristics addressing the application for multiple

types of users (H1), the clarity of the application’s ob-

jective (H2), the evaluation of essential features (H3),

and the cohesion between content and features (H5),

as well as guidelines for sound usage (H12), stand out

as novel. These new propositions bring a more strate-

gic approach focused on product coherence and user

experience, complementing Nielsen’s classic heuris-

tics.

The complete description of the initially devel-

oped UXIA heuristic set, including positive and neg-

ative examples as well as some illustrative images, is

available at the following link: https://ﬁgshare.com/

articles/ﬁgure/UXIA/28355546

3.4 Validation of UXIA Heuristics

According to the methodology outlined by Rusu et

al. (2011) , the next step in heuristic creation in-

volves validating the developed set of heuristics. In

this study, we conducted the initial validation in two

steps: ﬁrst, by searching for real-life examples of each

heuristic, and second, through a feasibility study.

The search for examples aimed to verify whether

the heuristics could be applied to existing applica-

tions. Both positive and negative examples were con-

sidered. During this process, we identiﬁed the need to

combine some heuristics, modify existing ones, and

create new heuristics. These changes led to the cur-

Rethinking Usability Assessment: Integrating UX and Information Architecture

593

rent set of heuristics, along with the examples pro-

vided for each.

This iterative approach resulted in the set of

heuristics presented here, each supported by illustra-

tive examples (see section 3.3).

Meanwhile, during the feasibility study, two par-

ticipants were tasked with performing a series of ac-

tivities within a speciﬁc application, evaluating it in

a manner similar to a conventional HE, but using the

UXIA heuristics. The following section outlines the

details of this inspection process.

3.4.1 Instrumentation and Planning

The methodology for the feasibility study involved

selecting the two participants, introducing the key

concepts and ideas necessary to understand the use

of UXIA heuristics, and presenting the complete set

of heuristics. Next, a script of activities was pro-

vided, outlining the tasks to be completed within a

speciﬁc application. Along with this script, materials

for recording any defects discovered during the eval-

uation were also provided, following the guidelines

of HE (Novick, 2007). Finally, the participants were

asked to participate in an interview to share their ex-

perience with the UXIA heuristics, offering feedback

and suggestions for improvement.

The instruments used during the study were: (i)

Activity script, (ii) HE form, (iii) Feedback interview,

(iv) Informed Consent Form, and (v) Training on UX

and Garrett’s Framework.

The activity script contained a series of tasks de-

signed to guide the participants, acting as inspectors,

through activities commonly performed by applica-

tion users. The goal was for the participants to ex-

plore the application thoroughly in search of non-

conformities. When a non-conformity was found, the

participants ﬁlled out the HE form, indicating the de-

fect, the context in which it occurred, and any addi-

tional observations.

The feedback interview aimed to gather the partic-

ipants’ impressions of using the UXIA heuristics, in-

cluding ease of use, challenges encountered, and both

positive and negative aspects. The participants also

assessed the names and descriptions of the heuristics,

the issues they identiﬁed, and the relevance of UXIA

heuristics for evaluating UX.

The informed consent form was presented and

signed by the participants to ensure they were aware

of how their data would be used for research purposes.

The participants were informed that they could with-

draw their data at any time. Since the study was con-

ducted with postgraduate students, the process was

simpliﬁed, as the participants were already familiar

with the academic and research environment. This

familiarity facilitated open communication about the

details and implications of the study.

3.4.2 Execution

The inspection took place at a university with post-

graduate students who had prior knowledge of Usabil-

ity Inspection and HE, as well as a training on UX and

Garrett’s Framework. The training was conducted re-

motely via the Google Meet platform two days before

the inspection, which, on the other hand, was carried

out in person.

Upon starting the inspection, the participants re-

ceived the activity script and a computer with access

to the application to be inspected, along with the HE

form to note down the defects found. The partici-

pants were given as much time as necessary to com-

plete all activities and identify defects individually. At

the end of the inspection, an interview was conducted

to provide feedback on the use of the UXIA heuris-

tics. The script used, as well as the raw data concern-

ing the inspection conducted by the participant during

the inspection, are available in: https://ﬁgshare.com/

articles/ﬁgure/FeasibilityStudy/28355519

3.5 Results

After the system inspection, one of the participants,

Inspector 1, identiﬁed 8 issues within approximately

40 minutes. These included problems such as an

overly large banner that made navigating the site dif-

ﬁcult, trouble ﬁnding essential functions due to in-

consistent labels, complex navigation with too many

steps or disorderly ﬂow, icons that didn’t match their

functions, and titles/labels that hindered usability,

among other ﬁndings.

Inspector 2, on the other hand, identiﬁed 10 issues

in roughly the same amount of time. These defects

included inadequate error handling on the page, over-

lapping menus, difﬁculty locating essential features,

overly large icons and images that affected naviga-

tion, among others.

In the interview, the participants pointed out that

UXIA heuristics were interesting, easy to use, and

also relevant - “I found it interesting and very easy

to use, the examples and the description of the heuris-

tics helped me a lot to know how to classify [...], [...]

and I also thought that yes, it was relevant to evalu-

ate UX.”. But also pointed out that some heuristics

were very similar - “Some are very similar and you

need to read carefully to understand that they are dif-

ferent things, so you classify better.”. Participants also

suggested that one improvement would be to create a

heuristic related to the format of media on pages, as it

was a problem encountered - “[. . . ] it would also be

ICEIS 2025 - 27th International Conference on Enterprise Information Systems

594

good perhaps to create a new heuristic to evaluate the

size of images and media because sometimes it gets in

the way.”.

After completing the feasibility study, we proceed

to the last stage of the heuristic creation process(Rusu

et al., 2011), the reﬁnement stage. In this stage, feed-

back from the previous stage was analyzed, and pos-

sible changes were considered if they were within the

objectives proposed by UXIA set.

Based on the issues identiﬁed by the participants

and their ﬁndings, it appears, as an initial observation,

that UXIA can effectively identify usability prob-

lems. Therefore, it demonstrates reasonable feasibil-

ity. However, UXIA can be reﬁned to enhance their

effectiveness. The highlighted alterations, based not

only on the defects identiﬁed but also on the insights

gathered during the interview, were implemented as

follows:

Change in the title or description of the following

heuristics:

H7: User Navigation control - It should always

be possible for the user to navigate wherever pos-

sible, avoiding creating screens where the user can

only exit using browser or operating system naviga-

tion buttons, while also providing the possibility for

users to perform searches in the system, at least on

the main screen.

The inclusion of the term ”user” in the heuristic

title was implemented to assist inspectors in under-

standing its focus on user-speciﬁc navigation. Fur-

thermore, incorporating the capability for users to

perform searches within the system in the description

enhances the integration of IA navigation principles

within this heuristic.

H10: Clear and structured navigation items -

Interface navigation items should be visible and easily

accessible to users, allowing them to know where they

came from and where they can go.

In this case, the title was adjusted to include

”items” to clarify that this heuristic speciﬁcally eval-

uates navigation items within the interface.

H11: Appropriate Organization - The appli-

cation should present some form of organization,

whether alphabetical, temporal, by category, etc. But

always in line with the business objectives and with

labels and titles that are meaningful and help users

understand the purpose of the menu or functionality.

The adjustment in this heuristic serves to consoli-

date concepts of IA related to organization and label-

ing within a single heuristic.

And based on the insights gathered from the in-

terview and the inspection, a new heuristic has been

developed at the Surface level:

H15: Media and content with appropriate for-

mat - The media and content of the application need

to have a format that aligns with the functionalities

and does not compromise the UX.

Examples related to H15:

• Gov.br

displays a very large photo in certain

menus that occupies more than half of the screen,

which can hinder access to other functions. It is

necessary to zoom out the screen to view the im-

age as a clickable menu.

• Netﬂix

demonstrate an example of H15 compli-

ance, as all the media and images on the website

are appropriately sized and aligned with the plat-

form’s features as a movie streaming service.

4 FINAL CONSIDERATIONS AND

FUTURE WORK

Developing a set of heuristics to evaluate system us-

ability from multiple perspectives, such as UX and

AI, is essential.

Therefore, this research proposes a new set of

heuristics aimed at providing a comprehensive eval-

uation of systems, with a focus on UX. The initial

validation results demonstrate the feasibility of this

UXIA set, showing that it can be successfully applied

in such cases and has signiﬁcant potential to con-

tribute to the software development process. UXIA is

designed to be utilized to ensure continuous improve-

ment and user-centric design throughout the entire de-

velopment lifecycle.

Based on the feedback received, the heuristic set

was slightly adjusted, resulting in its second version.

These were the ﬁrst steps of this research. However,

it is important to note the limitations of this research.

The initial validation process was limited to a small

sample size, and the results may not be fully repre-

sentative of broader contexts. Additionally, the set of

heuristics was tested only in speciﬁc scenarios, which

may affect its generalizability to other types of sys-

tems or user groups.

Future work related to UXIA set of heuristics in-

cludes conducting a controlled experiment, in which

the UXIA heuristics set will be compared with other

existing sets, such as Nielsen’s heuristics. The aim

is to assess and compare the effectiveness of the pro-

posed technique, with a particular focus on evaluat-

ing the efﬁcacy through the average number of defects

found by each technique.

Other possible future outcomes include evaluating

the set of heuristics in different contexts, potentially

https://www.gov.br/

https://www.netﬂix.com/

Rethinking Usability Assessment: Integrating UX and Information Architecture

595

through more qualitative assessments of its use. This

could involve an observational study of students using

the heuristics, providing insights into their practical

application and usability.

ACKNOWLEDGEMENTS

We thank all the participants in the empirical study

and USES Research Group for their support. The

present work is the result of the Research and De-

velopment (R&D) project 001/2020, signed with Fed-

eral University of Amazonas and FAEPI, Brazil,

which has funding from Samsung, using resources

from the Informatics Law for the Western Ama-

zon (Federal Law nº 8.387/1991), and its disclo-

sure is in accordance with article 39 of Decree No.

10.521/2020. Also supported by CAPES - Financing

Code 001, CNPq process 314797/2023-8, CNPq pro-

cess 443934/2023-1, CNPq process 445029/2024-2,

Amazonas State Research Support Foundation - FA-

PEAM - through POSGRAD 24-25, and Amazonas

State University through Academic Productivity Pro-

gram 01.02.011304.026472/2023-87.

REFERENCES

Bolchini, D., Chatterji, R., and Speroni, M. (2009). Devel-

oping heuristics for the semiotics inspection of web-

sites. In Proceedings of the 27th ACM international

conference on Design of communication, SIGDOC

’09, pages 67–, Bloomington, Indiana. ACM Press.

Choma, J., Zaina, L. A., and da Silva, T. S. (2016). Work-

ing beyond technical aspects: an approach for driving

the usability inspection adding the perspective of user

experience. In Proceedings of the 34th ACM Interna-

tional Conference on the Design of Communication,

number 11 in SIGDOC ’16, pages 1–10, New York,

NY. ACM Press.

Garrett, J. J. (2011). The Elements of User Experience:

User-Centered Design for the Web and Beyond. Pear-

son, Berkeley, 2nd. edition.

Hassenzahl, M. (2008). User experience (ux): towards

an experiential perspective on product quality. In

Proceedings of the 20th International Conference of

the Association Francophone d’Interaction Homme-

Machine, IHM ’08, pages 11–, Metz, France. ACM

Press.

Holzinger, A. (2005). Usability engineering methods for

software developers. Communications of the ACM,

48(1):71–74.

Islam, M. N., Bouwman, H., and Islam, A. K. M. N. (2020).

Evaluating web and mobile user interfaces with semi-

otics: An empirical study. IEEE Access, 8:84396–

84414.

Masip, L., Granollers, T., and Oliva, M. (2011). A heuristic

evaluation experiment to validate the new set of us-

ability heuristics. In Eighth International Conference

on Information Technology: New Generations, ITNG,

pages 429—-434, Las Vegas, NV. IEEE 2011.

Morville, P. and Rosenfeld, L. (2006). Information Archi-

tecture for the World Wide Web: Designing Large-

Scale Web Sites. O’Reilly Media, Inc., Sebastopol,

3rd. edition.

Nielsen, J. (1990). Finding usability problems through

heuristic evaluation. In Proceedings of the SIGCHI

conference on Human factors in computing systems,

CHI ’92, page 373–380, Monterey, California. ACM

Press.

Nielsen, J. (1993). Usability Engineering. Morgan Kauf-

mann Publishers, San Francisco, 1st. edition.

Nielsen, J. and Molich, R. (1990). Heuristic evaluation of

user interfaces. In Proceedings of the SIGCHI con-

ference on Human factors in computing systems Em-

powering people, CHI ’90, pages 249–265, Seattle,

Washington. ACM Press.

Norman, D., Miller, J., and Henderson, A. (1995). What

you see, some of what’s in the future, and how we go

about doing it. In Conference Companion on Human

Factors in Computing Systems, CHI ’95, Denver, Col-

orado.

Norman, D. and Nielsen, J. (1998). The deﬁnition of user

experience (ux). Accessed on July 15, 2024.

Novick, D. G. (2007). Usability inspection methods after 15

years of research and practice. In Proceedings of the

25th annual ACM international conference on Design

of communication, SIGDOC ’07, pages 249–, El Paso,

Texas. ACM Press.

Parente Da Costa, R., Canedo, E. D., De Sousa, R. T.,

De Oliveira Albuquerque, R., and Garc

ıa Villalba,

L. J. (2019). Set of usability heuristics for quality

assessment of mobile applications on smartphones.

IEEE Access, 7:116145–116161.

Rusu, C., Roncagliolo, S., Rusu, V., and Collazos, C.

(2011). A methodology to establish usability heuris-

tics. In 4th International Conference on Advances

in Computer-Human Interactions, ACHI 2011, pages

59–62, Guadeloupe, France. ACHI 2011.

ICEIS 2025 - 27th International Conference on Enterprise Information Systems

596