TELEMEDICINE TEST-BED
A Tool for Determination of Accuracy in Asynchronous Collaborative Method
Josceli Tenório, Cristina Lucia Feijó Ortolani, Paulo Roberto de Lima Lopes and Ivan Torres Pisa
Federal University of São Paulo (UNIFESP), Brazil
Keywords: Teledermatology, telemedicine.
Abstract: Store-and-forward method has contributed for telemedicine as an efficient method to the clinical diagnosis
acquisition. Studies in teledermatology demonstrated important outcomes, such as the use of computer
systems based in asynchronous method for long-distance diagnosis, categorization, triage, or clinical
outcomes agreements. However, for some issues there is no consensus between studies, such as the image
parameters or the data that must be submitted to analyze. The aim of this study is propose the development
of a web system that establishing patterns to construct web asynchronous system. A prototype was
constructed to test a structure (clinical data and images) to obtain clinical diagnosis concordance in
teledermatology. The preliminary outcomes showed that rate clinical diagnosis is nearly when we compare
face-to-face method (76.6%) and store-and-forward (70.3%). The most effectiveness occurs to some
diseases, who present easy clinical diagnosis in traditional consult. We expect that this system could
introduce a numerical method, established in statistical data, to construct store-and-forward system.
1 INTRODUCTION
The telemedicine was developed and based under
the concept of the access to health-care using
information technology in situations where the long
distance is a critical factor (Craig and Patterson,
2005). The Internet extended this concept under the
paradigm of the widely collaboration between health
professionals.
Due to traditional face-to-face method
consultation, using visual observation of the patient
and method to obtain date to realize an initial
diagnosis hypothesis, specialties such as
dermatology and ophthalmology (Finamor et al.,
2005) are further adjusted to these requirements,
widely documented (Ferguson, 2006).
Several studies based in store-and-forward
teledermatology have been performed and the
outcomes are:
• higher concordance rates in comparison to
outcomes histological diagnosis (Du Moulin et
al., 2003; Moreno-Ramirez et al., 2005; Chao et
al., 2003);
• categorization of outcomes (Piccolo et al.,
2004);
• clinical case triage, according to urgency level
(Oakley et al., 2006);
• fast reply (80% of submitted consultations in
one day) to requisitions submitted to experts
using a non-commercial network based in use of
easy platform to teleconsultations services
(Massone, 2006);
• number of patients (51%) reduction referral for
the consultation with the expert, when
teleconsultations (data and image) are submitted
from general practioner by email, to expert.
This value is similar to the real time cases (Knol
et al., 2006);
• similar clinical outcomes in comparison to
virtual teleconsultation (Pak et al., 2007).
However, there is no agreement in several issues
as the influence and minimum level of images
quality (resolution and compression) for a compose
efficient clinical diagnosis, minimum data set to be
submitted, the composition of the patients history
clinical and reply, the adequate specialties to
asynchronous method, the restrictions to use, the
level of complexity of the clinical cases that must be
diagnostics efficiently, and the influence of expert’s
experience.
It’s necessary to implement a structure and
vocabulary to the establishment of a computation
system to realize experiments that allows the
231
Ten
´
orio J., Lucia Feij
´
o Ortolani C., Roberto de Lima Lopes P. and Torres Pisa I. (2008).
TELEMEDICINE TEST-BED - A Tool for Determination of Accuracy in Asynchronous Collaborative Method.
In Proceedings of the First International Conference on Health Informatics, pages 231-234
Copyright
c
 SciTePress

evaluation of the necessary and enough parameters
for efficient clinical diagnosis.
The aim of this study is to propose the basis to
development of system web multi-specialty and to
show the outcomes of a prototype web system,
applied to teledermatology, which allows to
establishment of a virtual collaborative research
environment, to realize tests for the evaluation of the
telemedicine accuracy, an important subject for the
reliability of the asynchronous method.
2 MATERIALS AND METHODS
The model is the reproduction of a real experimental
environment, where a clinical data set and images,
of diagnostic performed by face-to-face
consultations can be available and evaluated by
specialists, whom opinion could be validated
according to gold-standard.
The physical architecture will contemplate the
client-server model. According to established aims,
the use of open source software is necessary.
To assessment of platform will realize
experiments: a control group composed for clinical
cases recorded in clinical archives with data and
images obtained in a face-to-face consultation,
selected by the manager health professional for test
control, with previously known clinical diagnosis,
what it will compose the standard-gold of this tests.
The initial focus of study will be dermatology and
ophthalmology.
The proposed web platform will provide the
manager health professional a computational system
to deliberate data set and diagnosis options,
including false options (tricks). Besides, it will be
able to change images size, resolution, and
compression rate. To further reliability, the manager
health professional will be able available to
randomize for choosing the clinical cases.
Potential data sets to be tried are composed of
minimum personal data, clinical history, and clinical
data from face-to-face consultation, image quality
parameters (minimum resolution, and compression).
The combination of these factors will compose other
experimental data sets.
2.1 Application
A prototype was performed to application in
teledermatology. The aim was the determination of
rate diagnosis concordance among initial diagnosis
hypothesis face-to-face consultations and store-and-
forward in comparison of both cases with gold-
standard. The literature (Bowns et al., 2006) reveals
that the diagnosis concordance rate will must be
more than 60 %.
To conduct this study, it was defined two
standard users: evaluator (a dermatologist
responsible for a test control and access to statistical
outcomes) and specialist (two dermatologists
analyzed the 64 cases together, according to real
environment).
To realize the trials, a control group was
performed for 64 randomized clinical cases
registered in paper, composed by personal and
clinical data, dermatoscopic images, a initial
diagnoses hypothesis and a result of clinical
analysis (gold-standard), selected by evaluator.
Figure 1: Web environment to dermatologist.
This prototype allowed trial a minimum data set
to teleconsultation in dermatology. Personal data
(gender and age), clinical (body’s part and time of
occurrence of lesion) and dermatological images (2-
6 images, mean score 2.6). There were two options
to resolution: standard (640 x 480) or the captured
image. The figure 1 shows the web system
environment available to specialist.
The specialist task was choice the option of a
diagnoses hypothesis, image’s quality, difficulty
degree for diagnosis and available data in a list.
Commentaries can be performed if necessary.
HEALTHINF 2008 - International Conference on Health Informatics
232

3 RESULTS
The diagnostic concordance rate in face-to-face
consultation (initial diagnoses hypothesis) was
76.6% (49/64) and using web system (asynchronous
method) was 70.3% (45/64). This outcome is
according to literature (Bowns, 2006) to web system.
The trial allowed to obtain the diagnosis
concordance among consultation face-to-face and
using web system according to diagnoses
hypothesis, showed in table 1. The most significant
results occurs in absolutely success in basal cell
carcinoma diagnosis (face-to-face: 11 match/11
choices; web: 11 match/11 choices) and atypical
nevus failure (face-to-face: 2 match/10 choices;
web: 0 match/6 choices).
Table 1: Diagnoses concordance rates in comparison to
gold-standard.
Diagnoses Hypothesis
Face-to-face
(%)
Web
(%)
Basal Cell Carcinoma 100.0 100.0
Haematoma 100.0 100.0
Malignant Lentigo
Melanoma or Malignant
Lentigo 77.8 75.0
Simple Lentigo 0.0 0.0
Solar Lentigo or Solar
Melanosys 100.0 100.0
Malignant Melanoma 70.0 75.0
Atypical Nevus 20.0 0.0
Blue Nevus 100.0 100.0
Congenital Nevus 100.0 71.4
Reed’s Nevus 0.0 0.0
Melanocytic Nevus 88.9 80.0
Actinic Keratosis 0.0 0.0
Seborrhoeic Keratosis 100.0 83.3
The specialist’s opinion about the available data
using the web system was showed in table 2.
This trial allowed us to measure of a specialist
opinion about the diagnosis difficulty. It was
possible compares the face-to-face and web analysis.
Table 3 shows this comparison.
The table 4 shows comparison among the
specialist opinion about the diagnosis difficulty and
the success of diagnoses hypothesis. This is
according our forecast, because the rate for easy
diagnosis is major.
Table 2: Specialists opinion rates about data available in
web system.
Data was enough to realize a
diagnostic?
Rate (%)
No, it’s very important includes
more data. 1.6
Yes, certain. 70.3
Yes, but I had doubt. 28.1
Image’s Quality
Very Nice 47.1
Nice 6.5
Bad 5.3
No important 0.0
Table 3: Diagnosis difficulty degree rates.
Diagnosis
Difficulty Degree
Face-to-face
(%)
Web
(%)
Easy 46.9 43.8
Medium 21.9 28.1
Difficult 31.3 28.1
Sum 100.0 100.0
Table 4: Success diagnosis hypothesis in relation with the
diagnosis difficulty degree.
Diagnosis
Difficulty Degree
Face-to-face
(%)
Web
(%)
Easy 96.7 96.4
Medium 64.3 55.6
Difficult
55.0 44.4
These trials conduct us to basic structure to web
system to remote diagnosis web system in
dermatology:
• Personal data minimum: gender and age.
• Clinic data minimum: body’ part and time.
• Image’s quantity: 2.6 (mean-score).
This structure presents more efficiency (around
70%) to diseases with easy diagnosis when
traditional consultation.
4 CONCLUSIONS
Studies based on the asynchronous method has not
been enough yet to establish it as an greatest choice
for the resolution of difficulty access health-care to
populations in isolated areas geographically, triage
of clinical cases for reduction of specialist access
delay, available of the second opinion or discussions
TELEMEDICINE TEST-BED - A Tool for Determination of Accuracy in Asynchronous Collaborative Method
233

of difficult clinical cases. There is only a potential
indication for the use of asynchronous method and
the success factors are being established yet, but
only for specific specialties (Finamor et al., 2005
and Ferguson, 2006). The studies of the diagnosis
agreement performed still not presented a consensus.
This prototype showed us a basic structure to
have success in remote telediagnosis in dermatology.
We expect that a web system based in these
characteristics must have rate success nearly face-to-
face consult. This is the aim of test-bed: trial a
structure, to obtain data to analyse the outcomes and
construct the web systems based in tested values.
In the experiments, found in literature, that use
email to submit data and attach images files, it there
is not a pattern that it indicates that set data must be
submitted. The text is based only on professional
common-sense and medical practice. Bergus et al.
(2006), demonstrated that the quality of information
is very important to referral reduce the need for
clinic consultation. In this prototype data
standardization was used to facility the data input
and establish a vocabulary.
In the next trials, we expected that test-bed
shows efficiently to obtain a list of another standard
indicators that will answer previously to the raised
issues: what size, minimum image resolution and
compression will be able to realize a diagnosis,
minimum personal data, what data will must be
compose recent clinical history, the minimum
amount of images to diagnosis, that images are
essential, the efficiency rate if images and data will
be analyzed separately. Indications to the use for
specific clinical cases also could be known.
We expect that this study will contribute for the
establishment of the asynchronous flexible
collaboration as efficient, reliable, comfortable, and
economic method for health professionals in specific
medical specialty.
ACKNOWLEDGEMENTS
Sérgio Hirata, Sérgio Yamada and Priscila Yshioka,
Department of Dermatology - Federal University of
São Paulo (UNIFESP), Brazil.
REFERENCES
Bowns, I.R., Collins, K., Walters, S.J., McDonagh, A.J.G.
Telemedicine in dermatology: a randomised
controlled trial. Health Technology Assessment 2006;
10; 43:
Bergus, George R., Myra E., Reed, David A. e Attaluri, A.
Email teleconsultations: well formulated clinical
referrals reduce the need for clinic consultation.
Journal of Telemedicine and Telecare 2006; 12: 33–
38.
Chao, L. W., Cestari, T. F., Bakos, L. et al. Evaluation of
an Internet-based teledermatology system. Journal of
Telemedicine and Telecare 2003; 9: Suppl 1:S9-12.
Craig, J., Patterson, V. Introduction to the pratice of
telemedicine. Journal of Telemedicine and Telecare
2005; 11: 3-9.
Du Moulin, M. F. M. T., Bullens-Goessens, Y. I. J. M.,
Henquet, C. J. M. et al. The reliability of diagnosis
using store-and-forward teledermatology. Journal of
Telemedicine and Telecare 2003: 9: 249-252.
Ferguson, J. How to do a telemedical consultation. Journal
of Telemedicine and Telecare 2006; 12: 220-227.
Finamor, L. P. dos S, Martins, M C, Muccioli, C, Sigulem,
D. Lopes, P R L, Belfort, R Teleoftalmologia como
auxílio diagnóstico nas doenças infecciosas e
inflamatórias oculares. Revista da Associação Médica
Brasileira, 2005; 51(5):279-84.
<http://www.scielo.br/scielo.php?pid=S010442302005
000500020&script=sci_arttext>. Accessed:
03/20/2006.
Knol, A., Van der Akker, T.W., Damstra, R.J., Haan, J de.
Teledermatology reduces the number of patient
referrals to a dermatologist. Journal of Telemedicine
and Telecare 2006: 12; 75-78.
Massone, C., Soyer, H.P., Hofmann-Wellenhof, R. et al.
Two years’ experience with web-based teleconsulting
in dermatology. Journal of Telemedicine and Telecare
2006: 12; 83-87
Moreno-Ramirez, D., Ferrandiz, L., Galdeano, R.,
Camacho, F. M. Teledermatoscopy as a triage system
for pigmented lesions: a pilot study. Clinical and
Experimental Dermatology 2005; 31: 13-18.
Oakley, A.M.M, Reeves, F., Bennet, J., Holmes, S.H.,
Wickham, H. Diagnostic value of written referral
and/or images for skin lesions. Journal of
Telemedicine and Telecare 2006; 12: 151-158.
Pak, H., Crystal, A.T., Lindquist, J.H., Grambow, S.C.,
Whited, J.D. Store-and-forward teledermatology
results in similar clinical outcomes to convencional
clinic-based care. Journal of Telemedicine and
Telecare 2007: 13; 26-30.
Piccolo, D., Soyer, H. P., Chimenti, S. et al. Diagnosis
and categorization of acral melanocytic lesions using
teledermatoscopy. Journal of Telemedicine and
Telecare 2004; 10): 346-350.
HEALTHINF 2008 - International Conference on Health Informatics
234