Accessibility of e-Government Websites in Italy: The User Experience
of People with Disabilities
Maria Claudia Buzzi
1a
, Marina Buzzi
1b
, Giuseppe Della Penna
2c
,
Barbara Leporini
3d
and Francesca Ricci
2
1
CNR-IIT, via Moruzzi 1, 56124 Pisa, Italy
2
Department of Information Engineering, University of L'Aquila, L’Aquila, Italy
3
CNR-ISTI, via Moruzzi 1, 56124 Pisa, Italy
Keywords: Human-Computer Interaction, Web Interfaces, Accessibility, People with Disabilities, e-Government.
Abstract: Public Administration services must be accessible for anyone, including people with disabilities who interact
via assistive technology. In 2016, the European Union published Directive 2016/2102 with the aim of making
such services more accessible to any citizen, regardless of its abilities. This paper investigates the accessibility
of e-Government services in Italy from the point of view of people with disabilities: seventy-six users
participated in an online survey, and the collected answers have been further refined through semi-structured
interviews. Results have been compared with a previous study, showing that the number of services has
increased but no substantial improvement in terms of accessibility has been recorded. Simplified interaction
and increased efficiency are still lacking, even if global user satisfaction seems to have slightly improved.
1 INTRODUCTION
E-Government is the use of technology to enhance the
access to and delivery of government services to
benefit citizens, business partners and employees
(Silcock, 2001). Thanks to the Internet and ICT, e-
Government services have been spreading out
worldwide, allowing citizens to access data and
information or perform online services 24 hours a day
in an efficient and cost-effective way, saving time and
reducing burden of the Public Administration (PA)
offices. More delivering e-Government services,
more transparency is reached, reducing corruption,
especially in developing countries (Alam et al.,
2023).
According to the EU Strategy for the rights of
persons with disabilities 2021-2030 (EU, 2021), to
guarantee an equal access e-Government services
must be accessible to all, including people with
disabilities. Applying accessibility (to guarantee
access to all) and usability (to make the interaction
a
https://orcid.org/0000-0001-7818-0601
b
https://orcid.org/0000-0003-1725-9433
c
https://orcid.org/0000-0003-2327-9393
d
https://orcid.org/0000-0003-2469-9648
easy and satisfactory) guidelines from early design
improves service effectiveness and efficiency, thus
increasing also user satisfaction. Although public
administrations gained an increased awareness on
usability and accessibility, also due to legal
obligations, some issues can still obstacle e-
Government services access by everyone despite
numerous guidelines have been available in the
literature for many years. Automatic tools can be
applied for testing web pages and services for
machine-detectable accessibility and usability issues;
however, according to W3C Web Accessibility
Initiative (WAI), they cannot catch severe
accessibility problems, so knowledgeable human
evaluation is also required
(W3C, 2022). Directly
involving and understanding the point of view of
people with disabilities can shed a light on interaction
issues of e-Government services.
This study investigates accessibility of PA
electronic services as perceived by people with
disabilities. An online questionnaire with 18 items
Buzzi, M., Buzzi, M., Della Penna, G., Leporini, B. and Ricci, F.
Accessibility of e-Government Websites in Italy: The User Experience of People with Disabilities.
DOI: 10.5220/0012177400003584
In Proceedings of the 19th International Conference on Web Information Systems and Technologies (WEBIST 2023), pages 59-70
ISBN: 978-989-758-672-9; ISSN: 2184-3252
Copyright © 2023 by SCITEPRESS – Science and Technology Publications, Lda. Under CC license (CC BY-NC-ND 4.0)
59

has been filled in by 76 citizens with disabilities.
Semi-structured interviews were performed with a
small set of participants to understand the
encountered problems more in depth. Results have
been discussed compared with previous studies.
The paper is organized as follows: Section 2
introduces the related work, Section 3 presents the
study methodology, the results of the questionnaire
and insights from the semi-structured interviews. In
Section 4 a discussion with a comparison with
previous studies is presented. Conclusion and future
work are described in Section 5.
2 RELATED WORK
Technology may play a crucial role in promoting the
participation and inclusion of people with disabilities
in society (Ferri and Favalli, 2018). Accessibility of
websites and services is important for everyone, but
it is crucial for people with disabilities, since their
interaction may require more time and cognitive
effort and may encounter more obstacles (Yesilada &
Harper, 2019). If not designed with accessibility in
mind, the Internet may increase the marginalization
of people with disability in society (Jaeger, 2022).
Different kind of barriers due to competences and
skills, geographic position and resources, income,
education and special needs, create the so called
“digital divide” and without urgent countermeasures,
in the future this gap could even increase (Seljan et
al., 2020). Several countries have published laws to
favor and guarantee equal rights for all to access
digital resources, web sites and services. In 2016, the
European Union published the Directive (EU)
2016/2102 on the accessibility of the websites and
mobile applications of public sector bodies. Such
requirements are mostly based on accessibility web
standards defined by the World Wide Web
Consortium (W3C), the Web Content Accessibility
Guidelines (WCAG), that evolved for over 25 years
to better incorporate usability into accessibility
guidelines (W3C, 2023).
E-Government accessibility is a widely
investigated topics, due to the importance to ensure
equal opportunities for all citizens. Attentions to
accessibility seems also bound to the transparency of
the public administrations. Alcaraz-Quiles et al.
investigated relations between transparency,
accessibility and usability of Spanish Regional
Government websites, showing that the transparency
of analyzed websites is inversely related to
accessibility since without visibility retrieving
information can require a considerable time (Alcaraz-
Quiles et al., 2018).
The assessment of the degree of accessibility of e-
Government services has been a subject of
investigations all over the world. Galvez and
Youngblood (2016) analysed more than one hundred
national and local e-government website sites in
Rhode, using a combination of code inspection,
heuristic evaluation, and automated analysis. Results
suggest that best-practice templates may be helpful in
improving usability, accessibility, and mobile
readiness, while it is critical for designers to receive
training and for governments to monitor Web sites
compliance with standards.
Kous et al. (2021) performed an accessibility
evaluation (compliance with Standard EN 301549) of
189 Slovenian municipalities’ websites using an
automatic tool (Achecker). The results of the
statistical analysis showed that the website home
pages in 2018 (after the adoption of European
Standard EN 301 549) have significantly improved
compared to 2017.
Paul (2022) presented the evaluation of the
accessibility of Indian e-government websites using a
sample of 65 websites of various ministries based on
the WCAG 2.1 standard, founding the majority of e-
government websites do not meet Level A
conformance with WCAG 2.1.
Al-Sakran and Alsudairi (2021) investigated how
well the Saudi mobile e-government websites comply
with usability standards and accessibility guidelines
recommended in the WCAG. Websites assessments
were conducted using manual evaluation and
complemented by different automated analysis tools.
The findings revealed usability and accessibility
problems that affect the performance of government
websites. Several recommendations for improving
the usability and accessibility of websites in Saudi
Arabia were also suggested.
Agrawal et al. (2022) investigated the usability,
accessibility, and mobile readiness of Indian
government websites. The analysis covered 164
government websites delivering e-services. The
evaluation of websites compliance with the WCAG
2.0 was done on various quality parameters using
automatic online tools. The results indicate that many
websites had low usability, most of the website does
not follow WCAG 2.0 guidelines, while none was
usable and fully accessible on mobile devices.
Siquiera et al. (2022) investigated whether public
prosecutors' websites are following web accessibility
guidelines. The authors evaluated the websites of
each of the 27 states of Brazil using the WCAG 2.0.
WEBIST 2023 - 19th International Conference on Web Information Systems and Technologies
60

Results indicated that the websites violated between
16 and 33 different success criteria out of 61 criteria.
Very recently, Lynn et al. (2023) examined the
web accessibility of local authority websites in the
Republic of Ireland specifically referring the Web
Accessibility Directive, Results confirmed that most
local government websites examined present
significant accessibility issues.
Similar studies were performed also in Italy.
Barricelli et al (2018) exploited automatic tools for
assessing the degree of accessibility of PA web sites
in Italy, showing that on more than 8000 websites of
the Italian municipalities only the 1% was fully
conform to the Italian legislation that required PA to
implement a subset of the WCAG guidelines. This
issue was confirmed by several successive studies.
Buzzi et al. (2019) presented a study that sheds light
on Italian public administration (PA) accessibility via
an online survey answered by 68 people with
disabilities from Tuscany, accessing PA services in
Italy. Results from the sample highlighted the need to
improve service accessibility and usability, and the
request for increasing their number and set of
functions. Furthermore, Valtolina and Fratus (2022)
investigated accessibility issues of local government
websites in Italy proposing an evaluation strategy via
two validators (AChecker and VaMolà) and
analysing 7,713 homepages against WCAG 2.0 and
Italian Law recommendations. Results confirmed a
low degree of web accessibility of municipal websites
However, most studies exploited only automated
analysis tools (validators). A recent systematic
literature review on automated tool utilization in web
accessibility research from 2002 to 2021 showed an
increasing trend in its use year by year (Macakoglu
and Peker, 2022). Considering the limits of automatic
evaluation of web accessibility (due to their inability
to check all aspects of accessibility automatically),
our study investigates the current degree of
accessibility of Italian PA services by the point of
view of users with disabilities, who can highlight
actual problems via an online survey further deepened
through semi-structured interviews.
3 THE STUDY
3.1 Methodology
This study investigates the accessibility of digital
services provided by PAs for people with disabilities
in Italy. At this aim, the study exploits two tools for
gathering participants’ feedback: an online survey
and semi-structured interviews.
The online survey was administered via Google
Forms, which supporting screen readers and keyboard
shortcuts offers a good degree of accessibility for
screen reading users (Leporini et al., 2018). The
questionnaire focuses on interaction aspects:
authentication, effectiveness in successfully
completing the service, ease to use of the interface,
overall user satisfaction.
To recruit potential end-users, an invitation letter
was defined, describing the study aim and the
process. The main Italian organizations of people
with disabilities (visual, motor, hearing, intellectual)
were contacted first by phone, and next via email
asking them to distribute the invitation letter (with the
questionnaire link) to their associates. This assures
the authors not having any direct contact with
participants.
The purpose of the semi-structured interviews was
to better understand the accessibility problems
encountered in interacting with government websites.
The interviews are designed to reveal the participants'
personal narratives, leaving them free to express their
opinions, considerations, and suggestions, that could
not emerge in closed questions.
Participants were recruited via e-mail or phone by
the associations for people with disabilities. Then, the
contacts of users available for further information
were provided by the associations to the authors.
Interviews were conducted via a video conferencing
tool (Google Meet) or by phone (preferred by visual
impaired users) by two of the authors: one conducted
the interview and the other one annotated answers,
comments and observations. After each interview,
both authors immediately checked and integrated the
annotated answers.
3.2 Questionnaire
The questionnaire, in Italian language, is composed
of eighteen questions organized in 3 sections. The
first section is composed of six questions to
characterize the sample, their use of the Internet and
their experience with most-used e-Government
services. Section 2 includes eight questions to
investigate the participants’ usability experience
when interacting with e-Government services: access,
effectiveness, ease to use, user satisfaction, support,
errors and customization. Last Section 3 proposes
four questions concerning authentication and citizen
rights.
The questionnaire’s content and language were
assessed by two accessibility experts and checked by
a totally blind person who verified its accessibility via
screen reader. The list of questions is presented in the
Accessibility of e-Government Websites in Italy: The User Experience of People with Disabilities
61

Appendix. Last, the heading of the questionnaire
describes:
• Who performed the study
• The research purpose: to understand the level
of accessibility and any issues encountered by
users with disabilities when interacting with
Public Administration websites
• The target users: persons with legal age (≥18)
and at least one type of disability
• Information about data treatment (no personal
data collected, no transfer to third parties).
3.3 Participants
A total of 76 participants filled out the questionnaire:
50 males and 26 females. Participants are aged
between 18 and more than 70 years: 2 (2,6%)
participants in the 18-29 range, 8 (10,5%) in the 30-
39 range, 20 (26,3%) in the 40-49 range, 30 (39,5%)
aged 50-59, 8 (10,5%) aged 60-69, and 8 (10,5%)
participants were 70 or older.
All participants indicated their gender: 2/3 are
males (50 out of 76) and 1/3 are female (25 out of 76).
Regarding the type of disability, most of
participants, are visually impaired (65 out of 76),
followed by people with motor disability (8 out of 76)
and three people having a mild cognitive impairment.
Last, two hearing impaired people who communicate
through the Italian sign language (signists) and one
with a behavioral disability complete the sample. It
was allowed to provide more than one answer in order
to correctly identify also multiple disabilities.
The sample composition suffers from an
imbalance in the representation of disability types,
with 81,25% of respondents being individuals with
visual disabilities. However, it should be taken into
consideration that several studies have shown that
users with visual impairment, and more specifically
blind users, encounter greater difficulties than users
with other disabilities when interacting with websites.
Visually-impaired users searching the Web content
took an average of 2,5 times longer than sighted users
(Craven and Brophy, 2003). An efficiency gap was
also confirmed by Ivory et al. (2004): when blind
people executed a search task, they took twice as long
as sighted users to explore search results and three
times as long to explore the corresponding web pages.
Petrie et al. in 2004 performed an accessibility test of
one hundred websites with users with visual, motor
and perceptual disabilities. Results showed a mean
task success rate of 76% which fall to 53%
considering only the totally blind (the lowest score of
all categories). Besides, regarding user satisfaction,
blind users confirmed the lowest score of all the user
categories (Petrie et al., 2004). Overall, blind users
interacting on the Web lose, on average, 30.4% of
time due to frustrating situations, such as page layout
causing confusing screen reader feedback, conflict
between screen reader and application, poorly
designed/unlabeled forms, no alternative text for
pictures, misleading links, inaccessible PDFs, and
screen reader crashes (Lazar et al., 2007). For these
reasons, we believe this study’s sample can offer
interesting insights on the accessibility of PA
services, including a high percentage of visually
impaired users, who experience the greatest
difficulties in web navigation.
The distribution of participants by their
occupation/job depicts that more than half of
participants (~54%) are employees of a PA (Italian
law requires PAs to recruit a percentage of people
with disabilities). Twenty-three participants (30,2%)
are retired, two participants are students (2,6%), five
participants (6,6%) are practitioners and five
unemployed people complete the sample.
Concerning the frequency of Internet use, most
(68 out of 76) participants use the Internet daily, three
weekly and two sometimes (Figure 1). Three
participants declared they use the Internet rarely. Two
of them are 70+ aged and one 50+.
In most cases, the Internet is an opportunity for
people with disabilities: accessible PA online services
can be performed autonomously, without the need for
an accompanying person, greatly empowering
personal autonomy. Autonomy is one of the most
important conditions for people with disability and
therefore it is very important to progressively increase
the portfolio of available services: investigating the
main reasons why users who could access such
services avoid using them could provide interesting
information helping to understand if there are barriers
or problems that can be solved.
Figure 1: Sample characterization: Participants’ Internet
use (Q5).
The users interact with several services of both
local and central public administrations (Figure 2).
Most frequently accessed services are job or pension,
Health, Local Administrations and taxation.
68
3
2
3
Daily
Weekly
Sometimes
Rarely
Internet Usage
WEBIST 2023 - 19th International Conference on Web Information Systems and Technologies
62

3.4 Results
Results are organized by analysing the answers to
sections 2 and 3 of the questionnaire.
Figure 2: Frequently accessed PA services (Q6).
3.4.1 Accessibility and Usability
Section 2 contains eight questions related to
participants’ usability experience when interacting
with e-Government services. Answers to question
Q7: “Have you had problems accessing the PA's
online services?” offer indications on the access
issues. It is remarkable that 89% of the sample
reported access problems (‘Sometimes’ for 46 out of
76 users, and ‘Often’ for 22). Only 11% of the users
(8 out of 76) never had an access problem (Fig. 3).
Figure 3: Frequency of problems reported/encountered by
participants when accessing e-Government services (Q7).
Regarding the effectiveness of interaction with
PA services (Q8), more than half participants
reported they were able to complete the requested
services always (6 users, 8%) or often (37 users,
49,3%), while 27 users (36%) only sometimes and 3
never, as shown in Figure 4.
Considering how frequently the interaction with
e-Government services was easy (Q9), most
participant were positive: 4 users (5,3%) found it
always simple and 37 users (48,7%) found it simple
in most cases, but it is relevant also the percentage of
participants that found it difficult in most cases
(32,9%, 25 users) or very difficult in all cases (13,2%,
10 users), as shown in Figure 5.
Figure 4: Effectiveness of PA service interaction (Q8).
Figure 5: Ease of the Interaction with PA services (Q9).
User satisfaction reported a consistent number of
participants (the majority) with a neutral position (27
users, 35,5%), as shown in Figure 6. Positive rating is
30,3% (satisfied, 23 users) and 7,9% (very satisfied,
6 users), while negative rating is 15,8% unsatisfied
(12 users) and 10,5 very unsatisfied (8 users).
Figure 6: User satisfaction about the Interaction with PA
service (Q10).
Three additional questions concern three usability
aspects: presence and clarity of documentation and
support, information, errors and recovery, and
possibility of customization. For evaluating the PA
website documentation and support (documentation
and manuals, pop-up messages, etc.) a 5-item Likert
scale (1 = Totally dissatisfied, to 5 = Truly satisfied)
has been administered.
3
49
41
53
35
10
17
13
19
0 102030405060
Others
Health
Municipality
Job/Pensions
Tax
Justice
Official Documents
Finance/Economy
School & Education
Frequently Accessed Sites
22
46
8
Yes, often
Sometimes
No, never
Access Problems
6
37
27
3
3
0 10203040
Yes, always
Often
Sometimes
Never
I've never tried
Effectiveness of PA service interaction
4
37
25
10
0 10203040
Always simple
Simple (in most cases)
Difficult
Very difficult
Ease of the interaction
8
12
27
23
6
0 102030
Very unsatisfied
Unsatisfied
Neutral
Satisfied
Very Satisfied
User satisfaction
Accessibility of e-Government Websites in Italy: The User Experience of People with Disabilities
63

Most users, 31 out 76, (i.e. 40,8%) chose the
neutral option (neither satisfied nor dissatisfied),
while the percentage of satisfied (22,4% satisfied and
7,9 very satisfied) is only slightly higher than the one
of dissatisfied (19,7% somewhat dissatisfied and
9,2% totally dissatisfied), as shown in Figure 7.
Figure 7: User satisfaction about documentation and help
(Q11).
Another question was about errors: if you made
mistakes, were you able to fix them easily (and were
the error messages clear and helpful)? Most of users
(40%) were able to correct the errors, but with
difficulties, 12% were unable to correct them, while
16% corrected the errors with ease (Figure 8).
Next question was: “If you made an error, were
you able to manage it and move forward?” and it aims
to understand if the management of errors is provided
through clear messages and interaction mechanisms.
Most users (40%, 30 users) reported that in case of
errors they could hardly correct them, reaching half
of the sample if we also consider the ones who said
they could not correct errors (12%, 9 users). The
neutral option (I don’t know, 17,3%, 13 users), “I
made mistakes, but I could easily correct them” (16%,
12 users) and “I don’t made mistakes” (14,7%, 11
users) complete the answers.
Figure 8: Errors visibility and easy of recovery (Q12).
Finally, concerning the ability to customize or
adapt the interface to the user's needs (Q13), one-fifth
of the sample (20,3%) was unable to adapt the
interface to their needs, 12,2 % were able in general,
while 31,1% only sometimes. One-fifth of the
participants (20,3%) did not try to modify the
interface (Figure 9).
Figure 9: Usability: interface customization (Q13).
About aspects to be improved urgently (Q14),
most of the users (45 out 76, i.e. 59,2%) suggested to
make the UIs easier to use and simplify the
interaction, while others suggested to add new online
services (25%) and shortening the service response
time (5,3%) as shown in Figure 10. Some skilled
users also suggested more specific improvements,
e.g., better structuring of the content, use of logical
templates to make it easier to find what you are
looking for, addition of instant chat for immediate
assistance, subtitling and possibly translation in the
sign language of videos.
Figure 10: Aspect to Improve. Q14.
3.4.2 Authentication & Citizen Rights
Section 3 of the questionnaire contains four questions
concerning authentication & citizen rights. The first
question (Q15) is about authentication via SPID
(Public Digital Identity System). Overall, 63
participants (82,9%) use SPID for accessing PA
services, 4 users (5,3%) have the authentication
credentials but do not use them, 6 users (7,9%) know
7
15
31
17
6
0 10203040
Totally dissatisfied
Dissatisfied
Neutral
Satisfied
Totally satisfied
Documentation and help
12%
40%
17%
16%
15%
I was not able to correct errors
I correct errors with difficulty
I don't know
I correct errors easily
I did not made errors
Errors
15
23
12
9
15
0 5 10 15 20 25
No, in general
Only in some sites
I don't know
Yes, generally
I have not tried
UI Customization
6
2
4
19
45
0 1020304050
Others
None, everything is fine
Response times
Increase number of services
Ease of use
Aspects to improve
WEBIST 2023 - 19th International Conference on Web Information Systems and Technologies
64

about the credentials but never requested them, and
only 3 users (3,9%) still do not know anything about
the SPID credentials.
Figure 11: Authentication: SPID credentials.
The other three questions regard citizen rights.
Almost all participants (72, 95%) know their right to
easily access all PA services (Q16).
Figure 12: Knowledge of citizen rights.
Q17 investigates the awareness of users on the
possibility of reporting accessibility problems to
AgID (Agency for digital Italy), the authority in
charge of monitoring and fuelling the digitalization of
the Italian PAs (Q17). Most of the participants
(48,7%) are aware of this possibility but did not report
any accessibility issues while 42,1% was not aware,
and only 9,2% sent some reports (Figure 13).
Figure 13: Reporting Accessibility feedback to AgID
(Q17).
Q18 was about the knowledge of the role of the
Public Advocate (Ombudsman) for the Digital Rights
and the possibility to provide feedback for
contributing to improve PA services. Most
participants (47 out of 76, 61,8%) are interested in
contributing, while 19 users (25%) do not consider
themselves able to do it, one has no time, and 9 are
not interested (11,8%).
Figure 14: Reporting Accessibility feedback to the Public
Advocate (Q18).
3.5 The Semi-Structured Interviews
The interviews started with four questions about
Difficulties, Effectiveness, Efficiency, Satisfaction of
use and Improvements:
1. What obstacles have you encountered in
accessing PA Web sites?
2. Were you able to do what you wanted, or did you
need help?
3. Was the interaction challenging, did it require
effort and/or a lot of time?
4. Were you satisfied or what would you like to
improve?
The sample consisted of 8 users: 3 visually
impaired (2 blind and one visually impaired that
navigates via screen reader), 2 motor impaired, and 3
old persons with mild cognitive and motor
impairment. Tab. 1 summarizes the key findings from
the interviews.
A common obstacle encountered in accessing PA
websites among all the users interviewed is the
difficulty in finding what they are looking for, due to
the lack of clarity in the contents and the complex
structure of the websites.
A totally blind user, who uses PA services daily,
reported two major problems: the presence of
advertisements on the sites that interfere with blind
users' listening to the content, and the difficulty in
using downloaded documents, as they are not
3
6
4
63
0
20
40
60
80
I don't know
what they
are
I know it, but
I don't own it
I own it, but I
don't use it
I own it and I
use it
SPID Credentials
Yes
95%
No
5%
Citizen rights
32
37
7
0
10
20
30
40
No, I didn't know Yes, but I didn't
report it
Yes, and I made
some reports
Reporting Accessibility issues to Agid
9
19
47
1
0 1020304050
No, I'm not interested
No, I wouldn't be able
Yes, I would like to
participate
I don't have time to do it.
Feedback to improve PA services
Accessibility of e-Government Websites in Italy: The User Experience of People with Disabilities
65

Table 1: Key findings from the interviews.
Issues Affected tasks
Lack of clarity in the
contents
Exploring, searching
Complex structure of the
websites
Exploring, searching
Audio of the commercial
advertisements overlaps the
reading of the screen reader
Listening the vocal
synthesizer
Documents created as an
image (unreadable and
unstructured)
Reading a document
Request to change the
password frequently
Access to website areas
with authentication
Tight and insufficient time
for strong authentication
Access to website areas
with authentication
readable by the screen reader being not in text format
(e.g., PDF documents containing scans of paper
documents).
Most notably, many of the interviewed users
reported difficulties in accessing PA sites due to the
SPID authentication or the need to periodically
change their passwords, as required by the current
national security guidelines. In some cases,
participants were even unable to complete the service.
Such users often require the assistance of family
members or friends to accomplish these tasks.
An experienced blind user declared that the SPID
authentication was accessible for him only through a
specific provider. Indeed, such provider grants the
user with 1 minute and 30 seconds to enter and
confirm (SPID uses a two-factor authentication) the
user credentials, as opposed to the 30 seconds granted
by other providers.
4 DISCUSSION
To understand the accessibility of e-Government
services as perceived by participants with disability,
the proposed survey investigates the following
aspects: a) Ability to access to the PA service; b)
Effectiveness (to successfully complete the service);
c) Ease of use and user interaction; d) Adaptability
and help. The survey results have been also
corroborated by semi-structured interviews, as
presented in Section 3.5.
There are several previous studies investigating
accessibility of PA services in Italy, mostly exploiting
automatic tools for assessing the degree of
accessibility of PA web sites. It is interesting to
compare the results of the current survey with an
analogous one administered by the authors of this
paper in late 2017 on a sample of 68 participants with
disability (about 61% blind users, 20% motor
disabilities, 9% auditory, less than 5% cognitive and
behavioral) described in (Buzzi, 2019). Indeed, the
two studies have a similar number of participants (68
vs. 76) and both recruited mainly blind and visually
impaired (62% in the 2019 study and 85,5% in this
study). To make the results comparable, this study
only extends the number of questions of the original
questionnaire to collect more data (adding Q11, Q12,
and Q13) but keeps the original ones unchanged.
The previous study contains data collected in
2017, so in the following, we write “2017” to indicate
the results of (Buzzi et al, 2019) and “2023” for the
results of the current study.
Figure 15: Frequency of problems encountered when
accessing online PA services: 2017 vs. 2023.
Figure 16: Effectiveness of online PA Services: 2017 vs.
2023.
A main aspect concerns the participants’
experience in accessing PA websites. Most users
(89,4%) still are experiencing issues: 60,5% (vs
67,2% of 2017) of participants had problems
sometimes and 28,9 % (vs 19,7% of 2017) often, as
shown in Figure 15. The situation seems to have
gotten worse: the number of people who never
experienced problems has decreased by 2,6% (10,5 in
19,7%
67,2%
13,1%
28,9%
60,5%
10,5%
Yes, often Sometimes No, never
Access Problems
2017 2023
10,6%
51,5%
30,3%
3,0%
4,50%
8,0%
49,3%
36,0%
4%
2,70%
Yes, always Often Sometimes Never I never tried
Effectiveness
2017 2023
WEBIST 2023 - 19th International Conference on Web Information Systems and Technologies
66

2023 vs 13,1 in 2017) and the total number of people
experiencing some issues has increased (89,4% of the
sample in 2023 vs 86,9% in 2017). This could be
caused by the new requirement of strong
authentication for accessing the PA services,
introduced by the European eIDAS Regulation on
electronic Identification, Authentication and trusts
services
1
. Indeed, in Italy, access to PA services
exploits digital identification through an Identity
Service Provider (SPID) or via an Electronic Identity
Card (CIE). As discovered in the interviews, strict
temporal limits can obstacle the access by blind users
interacting with such authentication mechanisms via
a screen reader, as discussed in the following.
Also regarding effectiveness, i.e. the ability to
complete a service, the situation seems slightly worse
than in 2017: only 8% of the participants (in 2023),
compared to 10,6% (in 2017), has always achieved
their goal, while the majority, i.e., 49,3% (in 2023),
compared 51,5% (in 2017), achieved it only often and
36% (in 2023) vs 30,3% (in 2017) only sometimes
(Figure 16). The total number of participants
experiencing effectiveness issues increased to 89,3%
in 2023 vs 84,8% in 2017. However, we have to
consider the narrow samples do not allow a
generalization of such results.
Figure 17: Ease of interaction in accessing online PA
Services: 2017 vs. 2023.
A light worsening was highlighted also regarding
the ease of use in the interaction with online PA
websites (Figure 17). Participants who evaluated the
interaction as simple decreased from 54,9% in 2017
(9,7% always simple + 45,2% simple) to 54,2% in
2023 (5,5% always simple + 48.7% simple).
Furthermore, 46,1% of 2023 participants found the
interaction quite difficult (32,9% difficult + 13,2%
very difficult) compared to 45,2% in 2017 (35,5%
difficult + 9,7% very difficult) of 2017.
1
https://digital-strategy.ec.europa.eu/en/policies/eidas-
regulation)
To measure the user satisfaction, we used a 5-item
Likert scale (1 = Very unsatisfied to 5 = Very
satisfied). Figure 18 shows that most participants
were not completely satisfied, 30,3% (vs 26,5% in
2017) were satisfied and only 7,9% (vs 6,3% in 2017)
very satisfied. However, since the percentage of
unsatisfied increased to 26,3% in 2023 vs 22,9% in
2017 the overall improvement is very little. Very
often, inexperienced people having problems
accessing online services believe that it is their fault,
being unable to correctly interact with the interface,
while the cause could be a poor usability design.
Figure 18: User satisfaction in accessing PA website: 2017
vs. 2023.
Overall, the collected users’ feedback concerning
the accessibility of PA websites, although on a small
sample, seems to indicate a worsening, even if only
slightly, compared to those of 2017, suggesting that,
despite the regulation’s intent, the problems seem not
under resolution.
Concerning the Authentication & citizen rights
session, with respect to the result of the 2017
questionnaire, the situation improved: 82,9% of
participants use SPID for accessing PA services
respect to only 5,9% in 2017, 5,3% have the
authentication credentials but do not use them vs.
8,8% in 2017, 7,9% know about the credentials are
but never requested them (vs 47,1 in 2017), and only
3,9% still do not know anything about the SPID
credentials (vs 37,2 in 2017). Analogously, most of
participants (96% vs 89,7 % in 2017) know their right
to easily access all PA services.
Concerning reports on accessibility issues of PA
services to AGID, awareness increased a lot. Most of
the participants (48,7%) are aware of this possibility
but did not report any accessibility issues (vs 29,4%
in 2017) while 42,1% were not aware in comparison
to 61,8% in 2017. However, few users exploit this
9,7%
45,2%
35,5%
9,7%
5,5%
48,7%
32,9%
13,2%
Always simple Simple Difficult Very difficult
Interaction
2017 2023
4,7%
17,2%
45,3%
26,5%
6,3%
10,5%
15,8%
35,5%
30,3%
7,9%
Very
unsatisfied
Unsatisfied Neutral Satisfied Very
Satisfied
2017 2023
User Satisfaction
Accessibility of e-Government Websites in Italy: The User Experience of People with Disabilities
67

possibility: only 9,2% in 2023 and 8,8 % in 2017.
Last, most of the participants would contribute by
sending feedback to the Public Advocate: 61,8% vs
47,1% in 2017, while 25% do not consider
themselves able to do it, respect 29,4% in 2017.
5 CONCLUSION
This study investigates the experience of people with
disabilities when accessing PA websites in Italy.
Results show that, although user awareness is
significantly increased and more users with disability
are able to benefit from PA digital services, user
satisfaction is only very slightly increased over the
last 6 years, effectiveness has slightly decreased, and
interaction seems more difficult, e.g., due to the
increased security and authentication constraints (as
the multi-factor authentication mechanisms)
introduced in the last years.
Furthermore, while the increasing number of
services offered through the PA websites is generally
a positive aspect for all users, often this implies
complex user interfaces that, if not suitably
structured, are more difficult to understand and
navigate via screen readers. Clearly, both the current
authentication processes and the user interfaces need
further design simplification efforts to make users
with disabilities able to successfully access the PA
services, asking for urgent improvement of usability
such as ease of use of the user interfaces that should
be logically organized for easier navigation, offering
more help and a simplified interaction (also for
authentication purposes).
The path for a truly accessible PA is still in
progress and needs considerable endeavour. New
initiatives inside the National Recovery and
Resilience Plan, which is part of the Next Generation
EU program, can place crucial resources and
accelerate this process in Europe, but it is important
to involve users with disabilities in these efforts since
their feedback must be at the heart of a truly inclusive
design process.
As future work, we intend to expand our survey
in two directions: a) involving a more significant and
balanced sample of users, especially with regard to
the types of disability; b) investigating why some
people do not use the internet (or just very rarely) in
order to understand if it is due to specific barriers
(technological, cultural, accessibility, etc.), or
personal issues (privacy concerns, lack of interest,
etc.) or other causes such a psychological technology
rejection.
ACKNOWLEDGEMENTS
We thank all the study participants for their valuable
contribution.
REFERENCES
Agrawal, G., Kumar, D. & Singh, M. Assessing the usability,
accessibility, and mobile readiness of e-government
websites: a case study in India. Univ Access Inf Soc 21,
737–748, 2022. https://doi.org/10.1007/s10209-021-
00800-8
Akram, M., Ali, G.A., Sulaiman, A. et al. Accessibility
evaluation of Arabic University websites for compliance
with success criteria of WCAG 1.0 and WCAG 2.0. Univ
Access Inf Soc, 2022. https://doi.org/10.1007/s10209-
022-00921-8
Alam, T.; Aftab, M.; Abbas, Z.; Ugli, K.M.M.; Bokhari,
S.A.A. Impact of E-Government Initiatives to Combat
Corruption Mediating by Behavioral Intention: A
Quantitative Analysis from Emerging Economies.
Sustainability 2023, 15, 2694.
https://doi.org/10.3390/su15032694
Alajarmeh, N. Evaluating the accessibility of public health
websites: An exploratory cross-country study. Univ
Access Inf Soc 21, 771–789 (2022).
https://doi.org/10.1007/s10209-020-00788-7
Alcaraz-Quiles, F. J., Urquia-Grande, E., Muñoz-Colomina,
C. I., & Rautiainen, A. E-Government Implementation:
Transparency, Accessibility and Usability of
Government Websites. In International E-Government
Development, 291-306, Palgrave Macmillan, Cham,
2018.
Al-Sakran, H. O., Alsudairi, M. A.: Usability and
Accessibility Assessment of Saudi Arabia Mobile E-
Government Websites, IEEE Access, vol. 9, 2021, DOI:
10.1109/ACCESS.2021.3068917
Barricelli, B. R., Sciarelli, P., Valtolina, S., & Rizzi, A.: Web
accessibility legislation in Italy: a survey 10 years after
the Stanca Act. Universal Access in the Information
Society, 17, 211-222, 2018.
Buzzi, M. C., Buzzi, M., & Ragni, F.: Accessibility of Italian
E-Government services: The Perspective of Users with
Disabilities. In Electronic Governance and Open Society:
Challenges in Eurasia, EGOSE 2018, Communications
in Computer and Information Science, vol 947. Springer,
2019 https://doi.org/10.10 07/978-3-030-13283-5_21
Craven, J., and Brophy, P.: “Non-visual access to the digital
library: the use of digital library interfaces by blind and
visually impaired people.” Technical report, Manchester:
Centre for Research in Library and Information
Management (CERLIM), 2003,
http://www.cerlim.ac.uk/pubs/index.php
European Union (2021). European Commission. Union of
equality: Strategy for the rights of persons with
disabilities 2021-2030. https://ec.europa.eu/social/
main.jsp?catId=1484
WEBIST 2023 - 19th International Conference on Web Information Systems and Technologies
68

Ferri, D., & Favalli, S. Web accessibility for people with
disabilities in the European Union: Paving the road to
social inclusion. Societies, 8(2), 40, 2018.
Galvez, R. A., Youngblood, N. E.: e-Government in Rhode
Island: what effects do templates have on usability,
accessibility, and mobile readiness? UAIS, June 2016,
Volume 15, Issue 2, pp 281–296
Ivory, M. Y., Yu, S., and Gronemyer, K.: “Search result
exploration: a preliminary study of blind and sighted
users' decision making and performance”. Extended
abstracts of CHI 2004, pp. 453-1456
Jaeger, P. T. (2022). Disability and the Internet. In Disability
and the Internet. Lynne Rienner Publishers.
Kous, K., Kuhar, S., Pavlinek, M., Heričko, M., & Pušnik,
M. (2021). Web accessibility investigation of Slovenian
municipalities’ websites before and after the adoption of
European Standard EN 301 549. Universal Access in the
Information Society, 20, 595-615.
Lazar, J., Allen, A., Kleinman, J., Malarkey, K. (2007) What
Frustrates Screen Reader Users on the Web: A Study of
100 Blind Users, International Journal of Human–
Computer Interaction, 22:3, 247-269, DOI:
10.1080/10447310709336964
Leporini, B., & Buzzi, M. (2018, April). Home automation
for an independent living: investigating the needs of
visually impaired people. In Proceedings of the 15th
International Web for All Conference (pp. 1-9).
Lynn, T., Kennedy, J., Rosati, P., Fox, G., O’Gorman, C.,
Curran, D. and Hynes, K. Web Accessibility of Irish
Local Government Websites, ICDS 2023 (Seventeenth
International Conference on Digital Society), 2023
Macakoglu, S.S. and Peker, S.: "Web accessibility
performance analysis using web content accessibility
guidelines and automated tools: a systematic literature
review," 2022 International Congress on Human-
Computer Interaction, Optimization and Robotic
Applications (HORA), Ankara, Turkey, 2022, pp. 1-8,
doi: 10.1109/HORA55278.2022.9799981.
Paul, S. Accessibility analysis using WCAG 2.1: evidence
from Indian e-government websites. Univ Access Inf
Soc, 2022. https://doi.org/10.1007/s10209-021-00861-9
Petrie, H., Hamilton, F., and King, N.: “Tension, what
tension?: Website accessibility and visual design”, Proc.
2004 International Cross-disciplinary Workshop on Web
Accessibility (W4A), 2004, pp. 13 – 18.
Seljan, S., Miloloža, I., & Pejić Bach, M.: e-Government in
European countries: gender and ageing digital divide.
Interdisciplinary Management Research, 16, 1563-1584,
2020.
Silcock, R.: What is e-government. Parliamentary Aff. 54(1),
88–101 2001
Siqueira, M.S.S., Dias, F.S., Rigatto, S.H., Carvalho, M.C.N.,
Marques, T.A.M. and Freire, A.P.: Who watches the
watchers? Accessibility of the public prosecutor's office
websites in Brazil and implications for e-government
accessibility surveillance policies, Electronic
Government Journal, Vol. 19, No. 1, pp 72-94, 2022,
https://doi.org/10.1504/EG.2023.127576
Valtolina, S. and Fratus, D.: Local Government Websites
Accessibility: Evaluation and Finding from Italy. Digit.
Gov.: Res. Pract. 3, 3, Article 17 (July 2022), 16 pages,
2022. https://doi.org/10.1145/3528380
W3C. Introduction to Web Accessibility. 31 March 2022
https://www.w3.org/WAI/fundamentals/accessibility-
intro/#evaluate
W3C. Web Content Accessibility Guidelines (WCAG) 2.2.
25 January 2023 https://www.w3.org/TR/WCAG22/
Yesilada, Y., & Harper, S. (2019). Web Accessibility.
London: Springer
APPENDIX
Survey on the Accessibility of e-Government
Websites in Italy
The purpose of this research conducted by the
University of L’Aquila and CNR is to understand the
level of accessibility and issues encountered by users
with disabilities when interacting with Public
Administration websites. The questionnaire is
addressed to adults and it is completely anonymous.
Data collected will be processed in aggregate form for
only research purposes. They will not be transferred
to third parties. Consent to data processing: I confirm
that I have read and understood the above information
and I understand that:
a) my participation is voluntary, and I can withdraw
at any time without giving any reason
b) my data are anonymous and no identifying
information will be made available to anyone.
I accept, I give my consent and I participate
in the questionnaire
I do not accept (exit the questionnaire)
Section 1: Sample Characterization
Q1 Gender:
Male
Female
I don’t want to specify.
Q2 Age:
18-29
30-39
40-49
50-59
60-69
≥70
Q3 Occupation/Job:
Student
Employee
Professional
Unemployed
Retired
Q4 Disability – multiple choice
Visual
Motor
Hearing
Accessibility of e-Government Websites in Italy: The User Experience of People with Disabilities
69

Intellectual
Behavioural
Q5 Internet Usage - How often do you use the
internet?
Almost Never
Sometimes
Weekly
Daily
Q6 Which Online PA Services do you Access most
Frequently? - Multiple choice:
Health
Municipality
Job/Pension
Tax
Justice
Official Documents
Finance/Economy
School & Education
Section 2: Interaction with e-Government services
Q7 Access Problems – Have you ever had problems
accessing the PA's online services?
Yes, often
Yes, sometimes
No, never
Q8 Effectiveness of PA Service Interaction - Were
you able to get what you wanted and to complete the
requested services?
Yes, always
Often
Sometimes
Never
I've never tried
Q9 Ease of the Interaction
Always simple
Simple (in most cases)
Difficult (It was quite difficult in most cases)
Very difficult (No, it was difficult in all cases and
sometimes I was unable to complete the service)
Q10 User satisfaction
Very unsatisfied
Unsatisfied
Neutral
Satisfied
Very Satisfied
Q11 Documentation and Help - Rate documentation
and help (manuals, interface messages, contacts, etc.)
1 = Totally dissatisfied
2 = Dissatisfied
3 = Neutral
4 = Satisfied
5 = Totally satisfied
Q12 Errors – Were you able to manage it and
move forward if you made mistakes? (clear
messages and interaction mechanisms)
I made mistakes and could not correct them
I made mistakes and could hardly correct them
I don't know
I made mistakes, but I could easily correct them
I did not make mistakes
Q13 User Interface Customization - Could the
interface be adapted to your needs? (e.g. increase font
size, change contrast, increase line spacing)
No, in general
Only in some sites
I don't know
Yes, generally
I have not tried
Q14 Aspects To Be Improved Urgently
None, everything is fine
Response times
Ease of use
Increase the number of online services to avoid face-
to-face services
Others
Section 3 -Authentication & Citizen Rights
Q15 Do you have SPID credentials?
I don't know what they are
I know what SPID is, but I don't own it
I own it, but I don't use it
I own it and I use it
Q16 Did you know that every citizen has the right
to easily access all PA services?
YES
NO
Q17 Did you know that AgID (Agency for digital
Italy) is the authority in charge of receiving
reports on difficulties in accessing PA services?
NO, I didn't know
YES, but I didn't report it
YES, and I made some reports
Q18 Each Administration must identify a Digital
Public Advocate. Would you like to participate by
sending them your personal suggestions?
No, I'm not interested
No, I wouldn't be able
Yes, I would like to participate
I don't have time to do it.
WEBIST 2023 - 19th International Conference on Web Information Systems and Technologies
70