QUALITY IMPROVEMENTS FOR READING CENTRES

Methods to Improve Quality and Compliance using Computerised Systems

Gunnar Lotz, Torsten Strasser, Eric Tröger, Eberhart Zrenner

Institute for Ophthalmic Research, Centre for Ophthalmology, University of Tuebingen

Paul-Ehrlich-Strasse 17, Tuebingen, Germany

Tobias Peters

STC Biomed, Tuebingen, Germany

Robert Wilke

Institute for Ophthalmic Research, Centre for Ophthalmology, University of Tuebingen, Germany

Keywords: Clinical trials, Electronic workflows, Reading centres, Reading quality improvements, Sop compliance.

Abstract: Reading centres are a concept to enhance the results and the acceptance of clinical trials. Computerised

systems provide the possibility to improve reading centres in a significant way. However ophthalmic

reading centres often do not fully use the potential of computerised systems. In this paper we show some of

the advantages sophisticated software can provide over traditional methods like e-mail. The improvements

can be structured in the following categories: Decreasing the chance of human mistakes; optimizing data

exchange and data flow; enforcing SOPs and complex workflows; further improvements. Large parts of the

proposed methods were successfully implemented in a proof of concept system for the Tuebingen ERG

Reading Centre that can serve as a reading centre in large ophthalmologic multi-centre clinical trials.

1 INTRODUCTION

The need for constantly high quality results in

clinical trials leveraged the application of reading

centres. In a reading centre especially trained experts

analyze technical examinations. This offers several

advantages for the analysis of clinical trials, like

high quality results, invariably interpretation of the

measurements and a wide acceptance of the results.

Furthermore the concept comes with more general

advantages like omitting the need for especially

trained reading personnel at the local sites, fast

turnaround times, highly tested and broadly accepted

standard operating procedures. Although those

advantages are already present in single-centre trials,

the effects of most of them are significantly higher

in multi-centre clinical trials. Reading centres can

therefore be seen as a quality improvement

instrument for the conduction of clinical trials.

Computerised systems changed the conduction

of clinical trials radically, nevertheless at present

many existing reading centres in ophthalmology still

rely on traditional concepts, e.g. the ubiquitous

communication via e-mail as the electronic

equivalent to communication by mail. Unfortunately

e-mail is much more insecure which is especially a

concern if source data and reading results are

exchanged in this way. Oftentimes the workflows

and processes in the centre are still manually

coordinated which not only causes much effort for

the organisers but also boosts the risk of human

mistakes.

We believe that with modern software

technology it is possible to enhance reading centres

in a significant way. All presented methods either

improve the reading results or contribute to maintain

study protocol compliance and hence improve the

reading centres’ quality considerably. In order to

bail out the full potential it is assumed that all source

data is submitted in electronic form.

450

Lotz G., Strasser T., Tröger E., Zrenner E., Peters T. and Wilke R. (2009).

QUALITY IMPROVEMENTS FOR READING CENTRES - Methods to Improve Quality and Compliance using Computerised Systems.

In Proceedings of the International Conference on Health Informatics, pages 450-453

DOI: 10.5220/0001538904500453

 SciTePress

The methods we propose are structured in

categories as follows: Decreasing the chance of

human mistakes; optimizing data exchange and data

flow; enforcing SOPs and complex workflows;

further improvements. We will close with a short

conclusion.

2 QUALITY IMPROVEMENTS

2.1 Decreasing the Chance of Human

Mistakes

Great potential regarding quality improvement lies

in the elimination of "human factors". Of course a

reading centre is not possible without human

participation but the personnel should be able to

concentrate on the core task - that is the reading of

technical examinations - and not on support

processes.

One example for such a support process is the

anonymization of patients during the reading

process. To guarantee the objectiveness of the

actual rating it is necessary to hide not only the

name of the local site which submitted the data but

also to provide the examination data initially without

any patient context to the reader. After this first

reading the data can be provided again with the

patient context, hence with past examinations and

reading results. The patient ID within the reading

centre naturally has to differ from the patient ID the

local site uses. Therefore a pseudo ID which is only

valid within the reading centre has to be provided for

the history check. This de-personalization is very

error-prone if done by humans. Furthermore it is

questionable at which point in the reading process it

should be done. If nobody at the reading centre is

allowed to know the real IDs of the patients and the

software does not support this process it must occur

at the local site. However this may be confusing for

those persons who coordinate the exchange between

the local site and the reading centre, because they

have to remember at least two IDs per patient. This

becomes even worse if the local site submits source

data to more than one reading centre. This implies

the danger of mixing the IDs up during

communication with the reading centre. If this part is

handled by the reading centre software, it can be

absolutely transparent to local sites and the reading

centre personnel. The local sites can use their well

known IDs when submitting or querying data. The

software generates new pseudo IDs for the use in the

reading centre in the background without human

action. Furthermore the software can show or hide

this ID to readers dependent on the process context.

Thus the de-personalization by the reading centre

software does not only eliminate a source of possible

human mistakes but also enhances the comfort for

all participants.

Other examples for human mistakes are simple

spelling mistakes which are as trivial as common.

However they can be automatically identified up to a

certain level and the user can be notified

immediately and the mistake can be corrected just in

time. This does not only improve the data quality but

also reduces unneeded iterations of input processes

and thus enables faster turnaround times.

2.2 Optimizing Data Exchange

and Data Flow

Many reading centres still rely on e-mail

communication. This is problematic for several

reasons. First and foremost unencrypted e-mail

communication is very insecure and not comparable

to traditional mail. Before an e-mail reaches its

recipient it is usually processed by many servers.

Unfortunately an e-mail usually has, if it is not

encrypted, no envelope like a letter and everybody

on its way from the sender to the recipient can read

subject and body of the e-mail. More badly this

happens unconsciously. If the e-mail has no digital

signature, it would be even possible that the content

of that e-mail changed unconsciously. Both

problems can be addressed with modern

cryptography, but this introduces great complexity

and effort for the users and therefore is hardly used

in practice.

Another problem of e-mail communication is

that it is not observable and therefore not tracked in

an audit trail. However the traceability of all actions

during the study is a common requirement. E-mail

communication and even more data exchange via e-

mail are therefore often inapplicable.

Since the e-mail content is innately not

structured the reading centres’ standard operating

procedures (SOPs) have to define some kind of

template which is to be applied by the users. But this

way automatic parsing at recipient side becomes

very difficult because excessive exception handling

is needed to catch user-made deviations from the

defined format.

Beside these technical reasons e-mail

communication is uncomfortable for the users at the

local sites as well as at the reading centre. For e-

mails usually programs are used, which are not

dedicated or customized for clinical trials. Not only

can the reading centres’ e-mails mix up with

unrelated e-mails, but also the possible support for

QUALITY IMPROVEMENTS FOR READING CENTRES - Methods to Improve Quality and Compliance using

Computerised Systems

451

Local SiteSoftwareReader #1Senior Reader Reader #2

Capture

Source Data

Upload

Source Data

Read

Source Data

Save Results

Read

Source Data

Save Results

Compare

Results

Download

Results

Correct

Results

Check

Results

Anonymization

De-

Anonymization

No Major

Discrepancies

No Correct

Result

Major

Discrepancies

Choose

Correct Result

One Correct Result

the users is underachieved compared to a specialised

program.

If the reading centre is implemented as a web

application with secured and encrypted access via

https these disadvantages can be completely

eliminated. In this case e-mail is hardly necessary

for communication and in no case for data exchange.

Instead local sites get a special account for

uploading source data and querying reading results;

of course all encryption and digital signing takes

place in the background without explicit user action

and all actions are fully tracked in an audit trail. The

software also takes care of transferring the data to

the correct recipient within the reading centre which

disburdens human coordinators and thus eliminates

risks of human mistake. Since the web application is

tailored for the requirements of the reading centre,

every member is supported in an optimal way in

accomplishing their tasks. Additionally the reading

centre business is kept separated from unrelated

work.

2.3 Enforcing SOPs and Complex

Workflows

Every study protocol defines a set of SOPs

describing the workflows before, during and after

the trial conduction. The reading centre personnel

have to follow all these rules strictly and without the

slightest deviation. Unfortunately it is hard to track

in detail if these rules are always followed as they

should. In an electronic reading centre parts of these

SOPs can be digitally modelled as workflows. Then

the reading centre software can take care of

enforcing the processes at all time, thus enhancing

the compliance with the SOPs by eliminating

deviations.

Also workflows become possible which would

be too much effort to implement without electronic

support. Especially those complex workflows have

great potential to raise and maintain overall results

quality.

One example for such a workflow is a parallel

independent double reading of each examination by

two readers seen in Figure 1. The software can

compare both results afterwards and notify a senior

reader if they differ significantly. This reading

workflow greatly reduces the possibility of

subjective reading results. If the work introduced by

double reading is too expensive, it is also possible to

let the software assign this sophisticated workflow

only to random examinations or to directly assign

the senior reader check to random examinations

without double reading.

As mentioned before it is indispensible to read

the examinations first without the context of a

patient in order to guarantee the most objective

reading result possible. However in this case the

reader does not know the history of the patient and

has no possibility to compare the examination to

past measurements and it is possible that slight

deterioration of measured values is overlooked first.

To eliminate this disadvantage without losing

objectiveness the old examinations and reading

results can be shown only after the reading has been

completed and closed. This history check can easily

be implemented electronically but would be much

effort to be ensured manually.

Since the goal of reading centres are constantly

high quality results, the result of one particular

examination should be invariant. In order to assure a

constant quality it is necessary to track whether it

makes a difference when they are read. For this

purpose an automatic re-reading, either by the same

Figure 1: Workflow for parallel independent double reading.

HEALTHINF 2009 - International Conference on Health Informatics

452

or by a different reader, of old cases after some time

is suitable. This enables the tracking of the reading

quality over time and is a great quality assurance

method which would be hard to do without

electronic support.

2.4 Further Improvements

Great potential of electronic software support lies in

the possibility to automate routine tasks like

notifications and thus disburden human

coordinators.

The software can automatically notify a reader

when a local site submitted an examination for

reading or the local site when an examination has

been read. Every actor in the reading process is

notified at the right point of time, so there is no need

for manually checking to-do lists. This enables

higher turnaround times because of minimal waiting

time.

For high quality reading results the data quality

of the source data is crucial. Therefore it is

mandatory to approve all local sites prior to their

first submission of real data. Usually this is done by

submitting several sample examinations which then

are tested for eligibility. Great parts of this

certification can be automated, e.g. the check for

completeness of the submissions.

Not only is the source data quality important but

also the data quality at every point of the reading

process. Wherever possible the software should

observe it. As already mentioned spelling mistakes

are quite common and to some level the software

can automatically notify the user about them. There

are more mistakes of that kind. For example it can

happen that the source data does not suffice the

needed quality standards, e.g. because of missing

measurements or being in an invalid format. If the

source data format is well defined the reading centre

software can automatically reject the submission and

notify the local site. Reading results can be wrong

due to various reasons, too. Again the software can

inform the reader in the case of implausible results.

Beside that the reading centre software can

observe the examination and reading values over

time and notify the senior reader if they drop out of

predefined ranges or worsen by a predefined

percentage rate. This helps to ensure that all adverse

events are noticed.

Furthermore it is possible to enforce an ongoing

training of all reading centre members. The software

system can be used for this purpose, e.g. by

discussing sample cases in a group. Not only can the

requirement of continuous training be fulfilled this

way. Also a common way to rate examinations is

created, which asserts a constant quality regardless

of the particular reader. This becomes even more

relevant if difficult real-world cases are used for this

training. The acceptance of the training among the

personnel may be enhanced by integrating the

training cases via e-learning into the normal day-to-

day work of the reading centre.

3 CONCLUSIONS

Reading centres are a good way to enhance the

quality and acceptance of clinical trials. Although

computerised systems have the potential to

drastically improve the delivered quality, in

ophthalmic reading centres specialised software is

rarely used. Instead many centres still rely on e-mail

communication and manual coordination of the

reading process. In this paper we proposed several

possibilities how a specialised software system can

improve the quality and compliance of a reading

centre.

Large parts of the proposed methods were

successfully implemented in a proof of concept

system for the Tuebingen ERG Reading Centre at

the Institute for Ophthalmic Research, University of

Tuebingen. For the actual implementation Nuxeo, an

Enterprise Content Management (ECM) System

which provides a reliable basis for data storage,

workflow management and permission management,

was used. This ECMS was heavily customized to

fulfil the requirements of modern reading centres.

The result is a sophisticated software system, which

not only ensures high quality results and study

protocol compliance, but also greatly improves the

comfort for both the local sites and the reading

centre members.

Further advantages will emerge if the software

manages not only one single reading centre but the

whole study like proposed in (Strasser et al., 2008).

This way all information is kept in a single

repository, superseding the need for data exchange

between different systems used during the trial.

REFERENCES

Nuxeo: open source ECM project based on modern Java

EE standards, http://www.nuxeo.org, accessed July

16, 2008.

Strasser et al., 2008. An integrated System for Workflow

and Data Management in Clinical Trials, Invest.

Ophthalmol. Vis. Sci. 2008 49:E-Abstract 5216, 2008.

Peters et al., 2008. The Steinbeis ERG Reading Centre for

Electrophysiological Data in Tuebingen, ISOT

Congress 2008. Philadelphia, September 25, 2008 –

September 28, 2008.

QUALITY IMPROVEMENTS FOR READING CENTRES - Methods to Improve Quality and Compliance using

Computerised Systems

453