Using a Self-Assessment Tool (SAT) to Review National Health

Information Systems in Ireland

Sarah Craig

Health Research Board, Dublin, Ireland

Keywords: Health Information Systems, Self-Assessment, National Standards, Ireland.

Abstract: This paper presents a review of four national health information systems in Ireland using a nationally agreed

self-assessment tool (SAT) developed in Ireland by the health information regulatory body there. The review

was undertaken using documentary analysis of written materials about the systems from both primary and

secondary data sources. The findings show high levels of compliance with the standards identified but that

there is still some work to do to ensure that all aspects of the standards are met.

1 INTRODUCTION

This paper presents a review of four national health

information systems using a self-assessment tool

(SAT) which was developed by Ireland’s health

information regulatory body, the Health Information

and Quality Authority (HIQA, 2017a) and which

have since become an agreed set of information

management standards for evaluating national health

and social care data collections (HIQA, 2017b).

The value of having key standards for monitoring

data quality is well-recognised internationally (see

Canadian Institute for Health Information, 2009;

Australian Institute of Health Welfare, 2014) and

several tools have been developed to assist the

managers of health information systems to ensure

good information governance is at the heart of their

processes (e.g. American Health Information

Management Association, 2011; Health and Social

Care Information Centre, 2015) in their work.

Building on the international evidence, HIQA

developed a self-assessment tool (SAT) for use on

national health information systems and this is in

process of being applied to the major Irish health and

social care data collections currently (e.g HIQA,

2019).

The research questions for the study were:

a) To what extent, if at all, do the Health Research

Board’s (HRB) systems comply with the HIQA

standards?

b) What does the HRB need to do to ensure full

compliance with the standards?

The paper highlights the value of having

standards that are informed by international best

practice and that create targets for national bodies in

achievement of this best practice. The overall benefits

are better quality data which in turn results in better

service planning at the national and local level, safer,

better care for patients and service users and

improved population health (HIQA, 2017b).

2 BACKGROUND

The Health Information and Quality Authority

(HIQA) is an independent statutory authority in

Ireland that was established to promote safety and

quality in the provision of health and social care

services for the benefit of the health and welfare of

the public in Ireland. Its role in health information is

focused on advising on the efficient and secure

collection and sharing of health information, setting

standards, evaluating information resources and

publishing information on the delivery and

performance of Ireland’s health and social care

services. In recent years HIQA’s health information

function has been particularly focused on producing

resources for the managers of health information

systems that are designed to support the

implementation of common standards in relation to

information governance and data quality.

The SAT devised by HIQA is based on six broad

themes which are then broken down into ten

standards (see Table 1).

Craig, S.

Using a Self-Assessment Tool (SAT) to Review National Health Information Systems in Ireland.

DOI: 10.5220/0009166302490253

In Proceedings of the 13th International Joint Conference on Biomedical Engineering Systems and Technologies (BIOSTEC 2020) - Volume 5: HEALTHINF, pages 249-253

ISBN: 978-989-758-398-8; ISSN: 2184-4305

 2022 by SCITEPRESS – Science and Technology Publications, Lda. All rights reser ved

249

Table 1: Themes and National Standards.

Theme Standard

Person-centred

1. Arrangements to protect

rivac

Governance,

leadership and

management

2. Effective governance

3. Publicly available statement of

purpose

4. Compliance with relevant

islation

Use of data

5. Compliance with health

information standards

6. Monitors quality of data

7. Effective and appropriate

dissemination strategies

Information

governance

8. Effective information

governance

Workforce

9. Workforce to deliver

objectives

Use of resources

10. Effective allocation of

resources

The themes and standards were agreed nationally

through expert collaboration and public consultation

and are based on robust international evidence in this

area (see American Health Information Management

Association, 2011, Australian Institute of Health

Welfare, 2014). The themes, standards and metrics

for measuring compliance are all set out in the self-

assessment tool (HIQA, 2017a) which is available on

the HIQA website (www.hiqa.ie ).

Following the publication of the self-assessment

tool for health and social care data collections in

2017, the HRB committed to undertaking an

evaluation of its four national systems using the tool.

The HRB is a public service body under the remit of

the Department of Health in Ireland. It collects data

in the areas of drugs and alcohol (the National Drug

Treatment Reporting System (NDTRS) and the

National Drug-related Deaths Index (NDRDI)); in

relation to disability (the National Ability Supports

System (NASS)) and mental health (the National

Psychiatric In-patient Reporting System (NPIRS)).

Data collection began as far back as the 1960s in

the mental health area, followed by drug treatment

and disability data systems in the 1990s and drug-

related deaths in 2005. Appendix 1 presents a

summary overview of the four systems. Twenty-five

people are employed across the four systems and the

annual budget is around €1.2 million. All of the

systems generate timely and accurate data at a

national level to assist with service planning and

monitoring of key policies in the areas identified as

well as reporting at national, EU and international

levels.

3 METHODS

During 2018, a project team was formed in the HRB

consisting of staff members working in each of the

four systems along with the HRB’s Head of National

Health Information Systems. The project team was

chaired by a HIQA representative. The method

adopted was largely documentary where key

protocols for each of the systems were reviewed and

the SAT was completed based on this written

evidence as well as staff knowledge about practices

within each system. Staff worked in pairs so as to

ensure peer-review of the completed SAT which

improved the quality of the information recorded.

Analysis was undertaken of written materials for

each of the Health Research Board’s (HRB) four

national health information systems including

primary sources such as protocols for data collection

and collation, data validation and reporting as well as

secondary sources such as evaluations and reviews.

The work took place over a period of 3 months.

4 FINDINGS OF THE

SELF-ASSESSMENT

Overall, the HRB’s health information systems

displayed high levels of compliance with the themes

and standards set out by HIQA. The specific findings

in relation to each of the themes and standards are set

out below.

4.1 Person-centredness

A person-centred approach focuses on the need to

protect the privacy of the individuals about whom the

data are collected (HIQA, 2017b). One of the

fundamental principles in this regard is that the data

subject is aware of how their data is being used. Ways

in which this can be achieved include having a

statement of information practice and undertaking

privacy impact assessments on a regular basis.

In relation to how the HRB’s information systems

rate on effective arrangements in to protect the

privacy of people about whom it holds information,

compliance is high as there are privacy policies

implemented for each system. However, there is a

need to make statements of information practice

publicly available on the organisation’s website to

ensure greater transparency.

HEALTHINF 2020 - 13th International Conference on Health Informatics

250

4.2 Governance, Leadership and

Management

This theme requires bodies that manage health

information to have good governance, leadership and

management arrangements in at the level of the

organization. Ways in which this can be achieved is

through clarity in relation to the objectives of the data

collection, having an identified individual with

overall responsibility for the data collection, a

statement of purpose and a well-documented

organizational chart and an identified governance

group consisting of key stakeholders.

In relation to the standards captured in this theme

the HRB’s national health information systems rate

highly but there is variability in the practices across

systems. Two of the four systems have formalised

arrangements in place (Table 2) to bring together key

stakeholders on a regular basis to consider the

strategic direction of the system and to make

decisions about any required changes.

Table 2: Governance Arrangements for the HRB’s health

information systems.

System Governance arrangement

NDTRS

None

NDRDI

National Steering Committee

comprising all stakeholders

NASS

National Committee comprising all

stakeholders

NPIRS

None

Source: Health Research Board, 2014.

However, the HRB does not currently publish

reports on the effectiveness of the national data

collections it holds although all publications from

these systems refer to the quality of the data contained

within. The assessment also found that there should

be more key performance indicators in place to

measure and report on the effectiveness of the HRB’s

information systems.

Within the HRB, some policies are in place on

privacy and confidentiality but more transparency is

needed on how and how often statements of purpose

are reviewed.

The assessment also showed high levels of

compliance with legislation, particularly data

protection legislation but proactive identification of

risks and issues in upcoming legislation was seen to

be needed.

4.3 Use of Data

HIQA recommends in its information management

standards (HIQA, 2017b) that all organisations

involved in the collection of health information

should ensure that the use of the information is

optimized to achieve the best value for money and to

maximize social gain. This includes ensuring

accuracy, completeness, legibility, relevance,

reliability and timeliness of the data. It also includes

having published data dictionaries for each data

collection and a framework for data quality that is

agreed at the level of all stakeholders as well as the

incorporation of international classifications.

In relation to this theme, all but one of the HRB’s

systems have incorporated international

classifications. Whilst a data dictionary has been

developed for each system it is not currently made

publicly available.

Similarly, within the HRB, there is a recognition

of the need for quality data in all the information

systems and considerable time and effort go in to

ensuring the quality of the data. However, there is no

formalized data quality framework in place. Some

work has been undertaken on audit internally and two

of the systems have invested in external audit and

evaluation, but these are outdated, and further work is

required in this regard. The SAT findings suggest

that the HRB needs to plan for further internal and

external audit of the data and consider the use of the

data quality framework dimensions as an audit

framework.

With regard to dissemination of data there is a

high level of compliance in the HRB’s information

systems. The assessment found that more could be

done to support users of the data, data quality

statements and notifying, in advance, on an annual

basis what publications will be available. In

particular, the findings suggested that more efforts

were needed to provide training to data users around

the use of the data and greater regard needed to be

given to recording metrics in relation to the timeliness

of responding to requests for the data. The SAT also

found that a simple calendar of when publications

from the information systems, if disseminated to

stakeholder, would improve timeliness and raise

awareness about when reports would come on stream.

4.4 Information Governance

This theme is about ensuring that those involved in

the collection of health data build in processes to

promote security and privacy in the collection and

reporting of data. It includes reliable information

governance practices in areas such as ensuring

consent is obtained from the data subject where it is

necessary, having statements of information

Using a Self-Assessment Tool (SAT) to Review National Health Information Systems in Ireland

251

practices, arrangements around the appropriate

sharing of information and ongoing audit on

information governance practice.

There are high levels of compliance in the HRB

with the standard on information governance but the

assessment found that more work is required on

information governance audits and training for staff

on information governance issues.

Each of the four systems managed by the HRB has

a publicly available statement of purpose, setting out

how the objectives of each system are achieved (see

Table 3). The statement of purpose has been agreed

by those involved in the governance structures that

are in place to oversee the systems (where they exist).

The statement is maintained by the staff responsible

for managing each system and is reviewed regularly

to ensure that it is fit-for-purpose. All statements are

published in HIQA’s catalogue of health and social

care data collections (HIQA, 2017c).

Table 3: Statements of Purpose for the HRB’s health

information systems.

NASS

The NASS is a service-planning tool

designed to capture data on the usage

of and need for specialist disability

services among people with a

disability.

NDTRS

The NDTRS was established as an

epidemiological database on treated

drug and alcohol misuse in Ireland. It

records incidence of drug and alcohol

treatment.

NDRDI

The NDRDI provides epidemiological

monitoring of cases of drug- and

alcohol-related deaths, and deaths

among drug users and among

alcoholics in Ireland.

NPIRS

The NPIRS is a psychiatric database,

which provides detailed information

on all admissions and discharges to in-

patient psychiatric services in Ireland.

4.5 Workforce

HIQA defines the workforce standard to include all of

those who work in or for the national health

information system and recommends that staff with

specialist skills and qualifications are needed to

ensure that aspects of quality are factored in to the

work and that there is good workforce planning to

deal with expected and unexpected events. It is about

having the ‘right people with the right knowledge’

(HIQA, 2017b: 41).

In relation to the HRB’s workforce there are high

levels of compliance with the organisation of its

workforce and its ongoing development. Workforce

planning is undertaken regularly to anticipate any

likely changes in staffing levels.

4.6 Use of Resources

HIQA defines resources as including human,

physical, financial and ICT resources and

recommends that organisations involved in the

collection and reporting of health data should strive

to ensure that its resources are adequate to ensure the

sustainability, continual relevance and maximum

impact of the data for which they are responsible.

The SAT exercise showed that in relation to the

allocation and use of resources within the HRB, there

are high levels of compliance with planning and

management of the necessary resources. This is

particularly the case in relation to skills audit,

succession planning and the organisation’s training

and development fund.

5 DISCUSSION

Following the completion of the SAT, the HRB

reviewed its findings with HIQA and devised a set of

actions which were then presented to senior

management in the organisation as an improvement

plan. Overall, the review of the HRB’s health

information systems shows that there is a broad range

of processes at play within and between the four

systems in achieving their goals and objectives in the

delivery of key national health data. The systems

vary significantly in how they collect, organise,

manage and disseminate data but, for the most part,

there is good practice in a number of the standards

and themes that constituted the SAT. Areas for

improvement are as follows:

Publish statements of information practice on the

HRB website.

Set up oversight committees for the remaining

two systems, the NPIRS and the NDTRS.

Put in place formalised agreements with data

providers.

Consider adding more detail on

performance/effectiveness of HRB systems in

publications.

Develop an annual process to review statements

of purpose.

HEALTHINF 2020 - 13th International Conference on Health Informatics

252

Put in place a process for reviewing upcoming

legislation relevant to the HRB data collections.

Publish data dictionaries for each system.

Formalise a data quality framework for the in-

formation systems that incorporates the elements

of good practice already applied to the data.

Plan for further internal and external audit of the

data and consider the use of the data quality

framework dimensions as an audit framework.

Use the organisation’s website monitoring to

record access to health information publications

Add more metrics to monitor the use of the data

such as tracking the number of times data are

accessed on the HRB’s website.

Provide training to data users around the value of

the HRB’s data.

Add some performance metrics to the existing

log of data requests to include timeliness of

response to these requests.

Develop an annual calendar on publication of

reports from the HRB’s systems.

Include more detailed data quality statements in

each HRB publication.

Devise a schedule for internal and external audit

of the HRB systems.

Devise a training programme for staff on

information governance.

6 CONCLUSION

The study set out to address two research questions:

a) To what extent, if at all, do the Health Research

Board’s (HRB) systems comply with the HIQA

standards?

b) What does the HRB need to do to ensure full

compliance with the standards?

The findings indicate that there are high levels of

compliance with the standards across all four systems

but that there is variability in the level of this

compliance. The areas for improvement identified as

part of the study show a pathway towards full

compliance. A follow-up review will be undertaken

to track progress in this regard.

Overall, the study highlights the value of having

agreed national standards for health information

systems such as those managed by the HRB.

REFERENCES

American Health Information Management Association,

2011. Health Data Analysis Toolkit. AHIMA. Chicago.

Australian Institute of Health Welfare, 2014. Data

Governance Framework. AIHW. Canberra.

Canadian Institute for Health Information, 2009. The CIHI

Data Quality Framework. CIHI. Ottawa.

Health and Social Care Information Centre, 2015.

Information Governance Toolkit. HSCIC. London.

Health Research Board, 2016. Research, Evidence, Action.

HRB Strategy 2016-2020, HRB. Dublin.

HIQA, 2013. Guiding Principles for National Health and

Social Care Data Collections, HIQA. Dublin.

HIQA, 2017a. Self-Assessment Tool for National Health

and Social Care Data Collections, HIQA, Dublin.

HIQA 2017b. Information management standards for

national health and social care data collections. HIQA.

Dublin.

HIQA 2017c. HIQA Catalogue for national health and

social care data collections. HIQA. Dublin.

HIQA 2018. Guidance on a data quality framework for

health and social care. HIQA. Dublin.

HIQA 2018. Review of information management practices

in the HSE Primary Care Reimbursement Service.

HIQA. Dublin.

APPENDICE

Appendix 1: Overview of HRB Systems.

Objectives Activity

National

Drug

Treatment

Reporting

System

(NDTRS)

To gather data on

treated drug and

alcohol misuse in

Ireland that can be

used by policy

makers and

service providers

• 30,000 records

annually

• Data items: 64

National

Drug-

Deaths

Index

(NDRDI)

To collect

information on

drug and alcohol-

related deaths and

deaths among

drug and alcohol

users in Ireland

• 600 deaths

annually

• Data items: 70

• 15,000

coroners files

consulted each

National

Ability

Supports

System

(NASS)

To capture details

of current service

provision and the

future service

requirements of

individuals with a

disability

• 30,000

records

• Data items:

200

National

Psychiatric

In-patient

Reporting

System

(NPIRS)

To collect and

report on all

admissions and

discharges to

inpatient

psychiatric units

both public and

private. A regular

psychiatric census

is carried out.

• 38,000 records

annually -

19,000

admissions

and 19,000

discharges

• 67 sites

• Number of

data items: 28

Using a Self-Assessment Tool (SAT) to Review National Health Information Systems in Ireland

253