QUALITY IMPROVEMENTS FOR READING CENTRES
Methods to Improve Quality and Compliance using Computerised Systems
Gunnar Lotz, Torsten Strasser, Eric Tröger, Eberhart Zrenner
Institute for Ophthalmic Research, Centre for Ophthalmology, University of Tuebingen
Paul-Ehrlich-Strasse 17, Tuebingen, Germany
Tobias Peters
STC Biomed, Tuebingen, Germany
Robert Wilke
Institute for Ophthalmic Research, Centre for Ophthalmology, University of Tuebingen, Germany
Keywords: Clinical trials, Electronic workflows, Reading centres, Reading quality improvements, Sop compliance.
Abstract: Reading centres are a concept to enhance the results and the acceptance of clinical trials. Computerised
systems provide the possibility to improve reading centres in a significant way. However ophthalmic
reading centres often do not fully use the potential of computerised systems. In this paper we show some of
the advantages sophisticated software can provide over traditional methods like e-mail. The improvements
can be structured in the following categories: Decreasing the chance of human mistakes; optimizing data
exchange and data flow; enforcing SOPs and complex workflows; further improvements. Large parts of the
proposed methods were successfully implemented in a proof of concept system for the Tuebingen ERG
Reading Centre that can serve as a reading centre in large ophthalmologic multi-centre clinical trials.
1 INTRODUCTION
The need for constantly high quality results in
clinical trials leveraged the application of reading
centres. In a reading centre especially trained experts
analyze technical examinations. This offers several
advantages for the analysis of clinical trials, like
high quality results, invariably interpretation of the
measurements and a wide acceptance of the results.
Furthermore the concept comes with more general
advantages like omitting the need for especially
trained reading personnel at the local sites, fast
turnaround times, highly tested and broadly accepted
standard operating procedures. Although those
advantages are already present in single-centre trials,
the effects of most of them are significantly higher
in multi-centre clinical trials. Reading centres can
therefore be seen as a quality improvement
instrument for the conduction of clinical trials.
Computerised systems changed the conduction
of clinical trials radically, nevertheless at present
many existing reading centres in ophthalmology still
rely on traditional concepts, e.g. the ubiquitous
communication via e-mail as the electronic
equivalent to communication by mail. Unfortunately
e-mail is much more insecure which is especially a
concern if source data and reading results are
exchanged in this way. Oftentimes the workflows
and processes in the centre are still manually
coordinated which not only causes much effort for
the organisers but also boosts the risk of human
mistakes.
We believe that with modern software
technology it is possible to enhance reading centres
in a significant way. All presented methods either
improve the reading results or contribute to maintain
study protocol compliance and hence improve the
reading centres’ quality considerably. In order to
bail out the full potential it is assumed that all source
data is submitted in electronic form.
450
Lotz G., Strasser T., Tröger E., Zrenner E., Peters T. and Wilke R. (2009).
QUALITY IMPROVEMENTS FOR READING CENTRES - Methods to Improve Quality and Compliance using Computerised Systems.
In Proceedings of the International Conference on Health Informatics, pages 450-453
DOI: 10.5220/0001538904500453
Copyright
c
SciTePress
The methods we propose are structured in
categories as follows: Decreasing the chance of
human mistakes; optimizing data exchange and data
flow; enforcing SOPs and complex workflows;
further improvements. We will close with a short
conclusion.
2 QUALITY IMPROVEMENTS
2.1 Decreasing the Chance of Human
Mistakes
Great potential regarding quality improvement lies
in the elimination of "human factors". Of course a
reading centre is not possible without human
participation but the personnel should be able to
concentrate on the core task - that is the reading of
technical examinations - and not on support
processes.
One example for such a support process is the
anonymization of patients during the reading
process. To guarantee the objectiveness of the
actual rating it is necessary to hide not only the
name of the local site which submitted the data but
also to provide the examination data initially without
any patient context to the reader. After this first
reading the data can be provided again with the
patient context, hence with past examinations and
reading results. The patient ID within the reading
centre naturally has to differ from the patient ID the
local site uses. Therefore a pseudo ID which is only
valid within the reading centre has to be provided for
the history check. This de-personalization is very
error-prone if done by humans. Furthermore it is
questionable at which point in the reading process it
should be done. If nobody at the reading centre is
allowed to know the real IDs of the patients and the
software does not support this process it must occur
at the local site. However this may be confusing for
those persons who coordinate the exchange between
the local site and the reading centre, because they
have to remember at least two IDs per patient. This
becomes even worse if the local site submits source
data to more than one reading centre. This implies
the danger of mixing the IDs up during
communication with the reading centre. If this part is
handled by the reading centre software, it can be
absolutely transparent to local sites and the reading
centre personnel. The local sites can use their well
known IDs when submitting or querying data. The
software generates new pseudo IDs for the use in the
reading centre in the background without human
action. Furthermore the software can show or hide
this ID to readers dependent on the process context.
Thus the de-personalization by the reading centre
software does not only eliminate a source of possible
human mistakes but also enhances the comfort for
all participants.
Other examples for human mistakes are simple
spelling mistakes which are as trivial as common.
However they can be automatically identified up to a
certain level and the user can be notified
immediately and the mistake can be corrected just in
time. This does not only improve the data quality but
also reduces unneeded iterations of input processes
and thus enables faster turnaround times.
2.2 Optimizing Data Exchange
and Data Flow
Many reading centres still rely on e-mail
communication. This is problematic for several
reasons. First and foremost unencrypted e-mail
communication is very insecure and not comparable
to traditional mail. Before an e-mail reaches its
recipient it is usually processed by many servers.
Unfortunately an e-mail usually has, if it is not
encrypted, no envelope like a letter and everybody
on its way from the sender to the recipient can read
subject and body of the e-mail. More badly this
happens unconsciously. If the e-mail has no digital
signature, it would be even possible that the content
of that e-mail changed unconsciously. Both
problems can be addressed with modern
cryptography, but this introduces great complexity
and effort for the users and therefore is hardly used
in practice.
Another problem of e-mail communication is
that it is not observable and therefore not tracked in
an audit trail. However the traceability of all actions
during the study is a common requirement. E-mail
communication and even more data exchange via e-
mail are therefore often inapplicable.
Since the e-mail content is innately not
structured the reading centres’ standard operating
procedures (SOPs) have to define some kind of
template which is to be applied by the users. But this
way automatic parsing at recipient side becomes
very difficult because excessive exception handling
is needed to catch user-made deviations from the
defined format.
Beside these technical reasons e-mail
communication is uncomfortable for the users at the
local sites as well as at the reading centre. For e-
mails usually programs are used, which are not
dedicated or customized for clinical trials. Not only
can the reading centres’ e-mails mix up with
unrelated e-mails, but also the possible support for
QUALITY IMPROVEMENTS FOR READING CENTRES - Methods to Improve Quality and Compliance using
Computerised Systems
451
Local SiteSoftwareReader #1Senior Reader Reader #2
Capture
Source Data
Upload
Source Data
Read
Source Data
Save Results
Read
Source Data
Save Results
Compare
Results
Download
Results
Correct
Results
Check
Results
Anonymization
De-
Anonymization
No Major
Discrepancies
No Correct
Result
Major
Discrepancies
Choose
Correct Result
One Correct Result
the users is underachieved compared to a specialised
program.
If the reading centre is implemented as a web
application with secured and encrypted access via
https these disadvantages can be completely
eliminated. In this case e-mail is hardly necessary
for communication and in no case for data exchange.
Instead local sites get a special account for
uploading source data and querying reading results;
of course all encryption and digital signing takes
place in the background without explicit user action
and all actions are fully tracked in an audit trail. The
software also takes care of transferring the data to
the correct recipient within the reading centre which
disburdens human coordinators and thus eliminates
risks of human mistake. Since the web application is
tailored for the requirements of the reading centre,
every member is supported in an optimal way in
accomplishing their tasks. Additionally the reading
centre business is kept separated from unrelated
work.
2.3 Enforcing SOPs and Complex
Workflows
Every study protocol defines a set of SOPs
describing the workflows before, during and after
the trial conduction. The reading centre personnel
have to follow all these rules strictly and without the
slightest deviation. Unfortunately it is hard to track
in detail if these rules are always followed as they
should. In an electronic reading centre parts of these
SOPs can be digitally modelled as workflows. Then
the reading centre software can take care of
enforcing the processes at all time, thus enhancing
the compliance with the SOPs by eliminating
deviations.
Also workflows become possible which would
be too much effort to implement without electronic
support. Especially those complex workflows have
great potential to raise and maintain overall results
quality.
One example for such a workflow is a parallel
independent double reading of each examination by
two readers seen in Figure 1. The software can
compare both results afterwards and notify a senior
reader if they differ significantly. This reading
workflow greatly reduces the possibility of
subjective reading results. If the work introduced by
double reading is too expensive, it is also possible to
let the software assign this sophisticated workflow
only to random examinations or to directly assign
the senior reader check to random examinations
without double reading.
As mentioned before it is indispensible to read
the examinations first without the context of a
patient in order to guarantee the most objective
reading result possible. However in this case the
reader does not know the history of the patient and
has no possibility to compare the examination to
past measurements and it is possible that slight
deterioration of measured values is overlooked first.
To eliminate this disadvantage without losing
objectiveness the old examinations and reading
results can be shown only after the reading has been
completed and closed. This history check can easily
be implemented electronically but would be much
effort to be ensured manually.
Since the goal of reading centres are constantly
high quality results, the result of one particular
examination should be invariant. In order to assure a
constant quality it is necessary to track whether it
makes a difference when they are read. For this
purpose an automatic re-reading, either by the same
Figure 1: Workflow for parallel independent double reading.
HEALTHINF 2009 - International Conference on Health Informatics
452
or by a different reader, of old cases after some time
is suitable. This enables the tracking of the reading
quality over time and is a great quality assurance
method which would be hard to do without
electronic support.
2.4 Further Improvements
Great potential of electronic software support lies in
the possibility to automate routine tasks like
notifications and thus disburden human
coordinators.
The software can automatically notify a reader
when a local site submitted an examination for
reading or the local site when an examination has
been read. Every actor in the reading process is
notified at the right point of time, so there is no need
for manually checking to-do lists. This enables
higher turnaround times because of minimal waiting
time.
For high quality reading results the data quality
of the source data is crucial. Therefore it is
mandatory to approve all local sites prior to their
first submission of real data. Usually this is done by
submitting several sample examinations which then
are tested for eligibility. Great parts of this
certification can be automated, e.g. the check for
completeness of the submissions.
Not only is the source data quality important but
also the data quality at every point of the reading
process. Wherever possible the software should
observe it. As already mentioned spelling mistakes
are quite common and to some level the software
can automatically notify the user about them. There
are more mistakes of that kind. For example it can
happen that the source data does not suffice the
needed quality standards, e.g. because of missing
measurements or being in an invalid format. If the
source data format is well defined the reading centre
software can automatically reject the submission and
notify the local site. Reading results can be wrong
due to various reasons, too. Again the software can
inform the reader in the case of implausible results.
Beside that the reading centre software can
observe the examination and reading values over
time and notify the senior reader if they drop out of
predefined ranges or worsen by a predefined
percentage rate. This helps to ensure that all adverse
events are noticed.
Furthermore it is possible to enforce an ongoing
training of all reading centre members. The software
system can be used for this purpose, e.g. by
discussing sample cases in a group. Not only can the
requirement of continuous training be fulfilled this
way. Also a common way to rate examinations is
created, which asserts a constant quality regardless
of the particular reader. This becomes even more
relevant if difficult real-world cases are used for this
training. The acceptance of the training among the
personnel may be enhanced by integrating the
training cases via e-learning into the normal day-to-
day work of the reading centre.
3 CONCLUSIONS
Reading centres are a good way to enhance the
quality and acceptance of clinical trials. Although
computerised systems have the potential to
drastically improve the delivered quality, in
ophthalmic reading centres specialised software is
rarely used. Instead many centres still rely on e-mail
communication and manual coordination of the
reading process. In this paper we proposed several
possibilities how a specialised software system can
improve the quality and compliance of a reading
centre.
Large parts of the proposed methods were
successfully implemented in a proof of concept
system for the Tuebingen ERG Reading Centre at
the Institute for Ophthalmic Research, University of
Tuebingen. For the actual implementation Nuxeo, an
Enterprise Content Management (ECM) System
which provides a reliable basis for data storage,
workflow management and permission management,
was used. This ECMS was heavily customized to
fulfil the requirements of modern reading centres.
The result is a sophisticated software system, which
not only ensures high quality results and study
protocol compliance, but also greatly improves the
comfort for both the local sites and the reading
centre members.
Further advantages will emerge if the software
manages not only one single reading centre but the
whole study like proposed in (Strasser et al., 2008).
This way all information is kept in a single
repository, superseding the need for data exchange
between different systems used during the trial.
REFERENCES
Nuxeo: open source ECM project based on modern Java
EE standards, http://www.nuxeo.org, accessed July
16, 2008.
Strasser et al., 2008. An integrated System for Workflow
and Data Management in Clinical Trials, Invest.
Ophthalmol. Vis. Sci. 2008 49:E-Abstract 5216, 2008.
Peters et al., 2008. The Steinbeis ERG Reading Centre for
Electrophysiological Data in Tuebingen, ISOT
Congress 2008. Philadelphia, September 25, 2008 –
September 28, 2008.
QUALITY IMPROVEMENTS FOR READING CENTRES - Methods to Improve Quality and Compliance using
Computerised Systems
453