Mobile-based Risk Assessment of Diabetic Retinopathy using a

Smartphone and Adapted Ophtalmoscope

Sim

ao Felgueiras, Jo

ao Costa, Jo

ao Gonc¸alves and Filipe Soares

Fraunhofer Portugal AICOS, Rua Alfredo Allen 455/461, 4200-135 Porto, Portugal

Keywords:

Diabetic Retinopathy, Retinal Image Acquisition, Automated Detection, Exudates, Microaneuryms, Decision

Support System.

Abstract:

The large prevalence of diabetes in the global population is associated with an increasing number of Diabetic

Retinopathy cases. This disease is associated with a progressive risk of blindness, due to physiological changes

that affect the retina. Since most of the progression is asymptomatic and late stage damage is often irreversible,

there is a large incentive to implement effective methodologies that allow large scale screening of the diabetic

population. In this work, a research study of a mobile approach for the assessment of Diabetic Retinopathy

was conducted, by analyzing 80 patients already being followed for ophthalmological care. A smartphone-

based fundus imaging system was used to acquire images of the retina during the normal clinical workﬂow in

a Central Hospital in Portugal. Relevant images were automatically analyzed by a Decision Support System

(DSS) based on computer vision methods. The results were obtained for ground-truth correlation as well

as time impact of this novel system. Our conclusions support that the DSS is highly sensitive in detecting

pathological information on images, after a dedicated quality image ﬁltering, and the acquisition procedure

has minimal adverse impact in the clinical setting.

1 INTRODUCTION

The increasing prevalence of diabetes in the popula-

tion is associated with several health issues, one of

which is the development of Diabetic Retinopathy.

This disease causes several cumulative lesions to the

patient retina through microvascular changes and, if

left untreated, may lead to blindness.

Diabetic retinopathy affects a large proportion of

the diabetic population: around 40% of all the people

with type 2 diabetes in the United States suffer from

some stage of the disease (Cheung et al., 2010), and a

particularly problematic aspect is that its progression

is mostly asymptomatic until the later stages, often

already associated with some degree of vision loss.

Even though effective treatment is possible in

early stages of the disease, vision loss is often irre-

versible, motivating the need for implementation of

screening programs. In these screening programs, im-

ages of the patient retinas are acquired by trained per-

sonnel and analyzed by experts in opthalmology, but

this requires the use of expensive equipment to per-

form retinal image acquisition as well as the manual

time-consuming analysis of those images.

In order to tackle the limitations associated with

traditional retinal imaging equipment, several mobile

solutions have been proposed. Volk inView is a re-

cent development which allows low cost acquisition

of retinal images with iPhone devices, but it is not

suitable for non-mydriatic acquisition, thus requiring

the administration of pharmacological agents for en-

abling mydriasis.

More recently, a solution for telemedicine screen-

ing was developed to detect retina diseases with a

portable non-mydriatic fundus camera. However, it

only performs image acquisition and the camera is

incorporated into the device. By having an embed-

ded camera, these kind of approaches do not take ad-

vantage of the widely available and expansible smart-

phones with increasingly enhanced cameras (Jin et al.,

2017). Besides reducing the cost for retinal image

acquisition, the use of smartphone exploits the con-

siderable computing power of these devices to per-

form automatic analysis of the acquired retinal im-

ages. Other solutions are the Volk Pictor and Pictor

Plus cameras, which allow non-mydriatic acquisition

but at very high costs (Zhang et al., 2015).

In a previous research work by (Costa et al.,

2016), it was proposed a methodology for mobile-

based computer-aided detection of Diabetic Retinopa-

168

Felgueiras, S., Costa, J., Gonçalves, J. and Soares, F.

Mobile-based Risk Assessment of Diabetic Retinopathy using a Smartphone and Adapted Ophtalmoscope.

DOI: 10.5220/0006599701680175

In Proceedings of the 11th International Joint Conference on Biomedical Engineering Systems and Technologies (BIOSTEC 2018) - Volume 5: HEALTHINF, pages 168-175

ISBN: 978-989-758-281-3

thy signs in retinal images. This approach was tested

using images from tabletop fundus cameras while

achieving a sensitivity of 87% in the detection of Di-

abetic Retinopathy, running in smartphones with rel-

atively low computational requirements and fast pro-

cessing.

In this work, we introduce a smartphone compat-

ible system for acquisition of retinal images which,

by building upon the work of (Costa et al., 2016),

allows an end-to-end low cost solution for Diabetic

Retinopathy screening and detection, without requir-

ing pupil dilation. The main goal is to evaluate auto-

matically the quality of the acquired images obtained

during the normal clinical workﬂow and, at the same

time, measure the performance of a Decision Support

System (DSS) on the previously ﬁltered images.

2 METHODOLOGY

2.1 Retinal Image Acquisition Device

In order to acquire retinal images without using cum-

bersome equipment, a mobile prototype was devel-

oped, based in the commercial Welch Allyn PanOp-

tic ophthalmoscope, which provides a relatively large

Field of View of 25

◦

. The ophthalmoscope enables

easy access to the patient retina without requiring

pupil dilation, which is of major importance in a

screening setting, even if not necessarily so in spe-

cialized ophthalmological care.

Welch Allyn commercializes a mechanical

adapter for the iPhone 4/4s smartphone, but the An-

droid ecosystem was preferred for our work, due to

the higher ﬂexibility offered. As such, we developed

a 3D printed mechanical case (see Figure 1) for the

Samsung S6 smartphone, which aligns the optical

axis of the ophthalmoscope with the smartphone

camera.

The unmodiﬁed ophthalmoscope uses a standard

halogen lamp for illumination of the retina, whose in-

tensity is manually adjusted through a rheostat. Since

the retina has low light reﬂectance, it is generally de-

sirable to use a relatively high light intensity in or-

der to obtain images with good quality. However, this

makes the examination process extremely uncomfort-

able for the patient, since he needs to withstand this

light for one or two minutes while the operator aligns

and acquires the retinal images. This also hinders the

operator, since alignment is more difﬁcult due to the

inevitable blinking of the patient and pupil constric-

tion (or myosis). Our preliminary experiments re-

vealed that it was very difﬁcult to achieve a satisfy-

ing trade-off between image quality and patient com-

fort/operator ease of use and so an alternative was re-

searched.

The alignment of the ophthalmoscope with the pa-

tient retina is crucial for obtaining meaningful images

for analysis, but image quality is not a priority for this

task. As such, it is possible to improve patient com-

fort by using a very low light intensity, offsetted by

using a high camera sensor sensitivity (ISO). Since

image acquisition is very brief, taking less than 50

milliseconds, we can minimize exposure of the pa-

tient to the high light intensity (see Figure 2 and 3).

However, the process of using different light in-

tensities for alignment and acquisition is not feasible

with the manual light intensity adjustment through the

rheostat. As such, we developed a Custom Light Con-

troller hardware module, directly powered and con-

trolled by the smartphone through USB On-The-Go

(OTG). The Custom Light Controller adjusts the light

output of a white high powered LED, at the command

of a Android smartphone application. This applica-

tion manages the process of acquiring the retinal im-

ages, by accommodating several camera parameters

such as focus, sensor exposure time, sensitivity, etc.

The applications sets the light output extremely low

while the user is aligning the ophthalmoscope, and,

when the user presses a physical button associated

with the Custom Light Controller, an order to acquire

an image is triggered, and the application synchro-

nizes the camera shutter with a high intensity light

pulse, similar to a ﬂash.

Figure 1: Assembled EyeFundusScope prototype.

Mobile-based Risk Assessment of Diabetic Retinopathy using a Smartphone and Adapted Ophtalmoscope

169

Figure 2: Example of an EyeFundusScope acquisition pro-

cedure.

Figure 3: Example of an acquired image.

2.2 Quality Image Algorithm

The quality of the acquired retinal image is the main

focus of mobile image acquisition systems (Jin et al.,

2017) for ocular diseases. Therefore, it is important

to verify if the EyeFundusScope prototype acquires

an image with enough quality and details about the

presence of Diabetic Retinopathy. In order to evalu-

ate the images collected in this study, a Quality Image

Algorithm was implemented with the goal of select-

ing images with good quality for further processing,

as shown in Figure 4. This algorithm is not only based

on image luminosity and color, but also in entropy and

most relevantly in the tissue area. If the area is higher

than a predeﬁned minimum value, the current image

is further analyzed by the next stages of quality eval-

uation. The minimum area value is an absolute value

and it was calculated using the dimensions of the full

image capture. To determine the tissue area, the algo-

rithm calculates the median ﬁlter of the image using

the red channel and applies a region threshold on the

median ﬁltered image. All these transformations are

done with downsized images, with the objective of re-

ducing computational costs.

Figure 4: Images with good quality, according to the Qual-

ity Image Algorithm, despite minor reﬂections.

2.3 Panoramic Image

To increase the ﬁeld-of-view obtained by the pro-

posed acquisition system using the commercial oph-

thalmoscope, which can be a problem for clinical

use when compared with the gold standard of table-

top fundus cameras, the EyeFundusScope software

application performs stitching of the acquired im-

ages. There are two main steps when generating a

panoramic image, one of which is to extract the blood

vessels from the green channel and use them for im-

age registration. The other step aims to create the mo-

HEALTHINF 2018 - 11th International Conference on Health Informatics

170

saic image with the original images and the registra-

tion calculated with the blood vessels. This allows the

stitching of various images from different quadrants

of the eye to get a greater ﬁeld-of-view. The objective

is to provide to the prototype operator the most com-

plete and easy to use information from the acquisition

that is performed, as shown in Figure 5.

Figure 5: Example of a stitched image, as a result of merg-

ing two overlapping retinal images.

2.4 Decision-Support System

After the ﬁrst phases of image acquisition and qual-

ity ﬁltering, the Decision-Support System is exe-

cuted, aiming at the separation between pathology-

free and positive cases. When Diabetic Retinopa-

thy is detected earlier, the success of the treatment

will increase. Therefore, in early stages, an auto-

mated computer-aided tool may increase the success

of the diagnosis, particularly in remote scenarios. The

Decision-Support System receives as input the re-

sults from microaneurysms detection and from the

decision-tree that classiﬁes the exudates, based on

the approaches developed in (Costa et al., 2016) and

(Felgueiras et al., 2016). Having this information, the

DSS outputs if the image is pathology-free or not.

2.4.1 Microaneurysms Detection

Microaneurysms formation is the ﬁrst evident (and

visible) sign of Diabetic Retinopathy. Histologically,

microaneurysms present a loss of capillary cells (the

perycites) on its walls. This leads to acellular cap-

illaries, thus enabling the pouching of these capillar-

ies, which originate the microaneurysms. The mech-

anisms that causes the decellularization are not well

understood, but they include release of a vasoprolifer-

ative factor and an increase in capillary pressure (Adal

et al., 2014).

Automated microaneurysm detection on the ac-

quired retinal images follows a similar approach to

(Costa et al., 2016). This approach uses the character-

istics of microaneurysms, such as small size and low

grayscale intensity to extract them from the rest of

the image. An example of detected microaneurysms

is shown in Figure 6.

Template matching using an inverted 2D Gaussian

kernel is applied to the green channel of the image

and the result is thresholded, with each component

in the resulting binary image constituting a detected

microaneurysm. In order to account for variation in

microaneurysm size, several σ values are used for the

Gaussian kernel.

Since retinal vessels are a common source of false

positives, the microaneurysm extraction method also

relies on the segmentation of vessels in the image, in

order to exclude microaneurysms located on those re-

gions. As a ﬁrst step, the green channel of the origi-

nal image is subtracted to its median ﬁltered version,

which is followed by a top hat transform. The result

of this operation is thresholded, and small connected

components are discarded as noise.

Figure 6: Retinal image with microaneurysms.

2.4.2 Exudate Detection

There are two types of exudates, the soft exudates,

also known as cotton wool spots, and the hard exu-

dates. Soft exudates are accumulations of axioplasm

and hard exudates accumulations of lipid and protein

in the retina (Ravishankar et al., 2009). Their typical

Mobile-based Risk Assessment of Diabetic Retinopathy using a Smartphone and Adapted Ophtalmoscope

171

characteristics are bright, reﬂective, white or cream

colored lesions found in the retina (Ravishankar et al.,

2009). These lesions indicate increased vessel perme-

ability and an associated risk of retinal edema. If they

appear close to the macula center, they are considered

as sight threatening lesions. Exudates are often asso-

ciated with microaneurysms.

EyeFundusScope performs automatic detection

of soft and hard exudates based on the work of

(Felgueiras et al., 2016) comprising image pre-

processing, feature extraction and candidate classiﬁ-

cation.

Figure 7: Retinal image with exudates. Right image high-

lights the area where the exudates are located.

3 STUDY SETUP

The ophthalmology department at Hospital Santo

Ant

onio, Centro Hospitalar do Porto, Portugal, hosted

this study with patients in a real scenario. Written in-

formed consents were obtained from the patients as

well as the approval from the institution’s research

ethics committee. The main objective of the study

was the validation of the mobile acquisition in di-

abetic patients without intrusive procedures. In the

sense, the acquisition was planned for the duration of

2 minutes per eye. Some patient data had to be reg-

istered: age, type and duration of diabetes, and pres-

ence of Diabetic Retinopathy according to a previous

diagnosis.

An Android application was developed to allow

the enhanced hardware control of the camera. Ad-

ditionally, the application was responsible for the

anonymization of the data by generating a unique ran-

domized key for each patient. The images acquired

are divided in two groups, images from the left and

the right eye. A form to insert information about the

patient status was part of the acquisition workﬂow.

To perform image acquisition the room should

comply with some basic guidelines. The luminosity

should be low and a chair for the patient should be

available. The acquisition should also follow a pro-

tocol. The patient should sit on the chair, remove the

glasses if necessary, and look to a ﬁxed point. While

acquiring images the operator will try to obtain im-

ages around the macula region. After performing this

action on both eyes the acquisition is ended.

The acquisitions took place after the ophthalmolo-

gist appointment and the specialist rated the acquired

images in terms of quality and gave a diagnosis about

Diabetic Retinopathy.

4 RESULTS

A total of 80 patients were analyzed, 68 of which

suffer from Diabetic Retinopathy. Diabetes type 2

is the predominant condition, and there is no infor-

mation about 4 patients. The acquired database in-

cludes 16 patients with pharmacologic dilation of the

pupil. Based on clinical expert analysis, no cases

were found with the presence of soft exudates. Only

4 cases do not have been analyzed by the Doctor on

both eyes, due to physical conditions of the patients.

The youngest patient acquired was 43 years old and

the oldest was 85 years old. The longest acquisition

with EyeFundusScope took 9 minutes and the fastest

one 4 minutes, being the average 5 minutes per case.

Photocoagulation treatment was already applied

to 30 patients, 12 of which had visible marks that

are noticeable in the acquired images. Analysing

these acquisitions, 7 exams have hard exudates and

4 present microaneurysms. From these photocoagu-

lation cases, 10 have good quality and no presence of

lesions on the acquired quadrants of the eye.

From the complete dataset, 31 acquisitions pre-

sented bad quality, 23 due to acquisition conditions

and 8 due to wrong prototype conﬁguration. The

database has 13 exams where the acquisition difﬁcul-

ties were originated from various patient conditions.

Problems like cataracts, retinal detachment, strabis-

mus and intermittent eyelids movement have a nega-

tive impact on the acquisition process.

The quality image algorithm evaluated the ac-

quired images from all acquisitions, resulting in a to-

tal of 277 good quality images for classiﬁcation pur-

poses using the DSS.

As expected, the images considered as having bad

quality by the Quality Image Algorithm were auto-

matically excluded. Figure 8 represents the number

of cases with medical diagnosis for which at least

one image was selected after quality ﬁltering. The

class of Other Problems is related with other eye prob-

lems presented on patient eyes, as already mentioned

above.

HEALTHINF 2018 - 11th International Conference on Health Informatics

172

Exudates

Microaneurysms

Photocoagulation

Other Problems

No Lesion

Number of cases

Figure 8: Medical diagnosis for the patients from which the

images were selected after quality ﬁltering.

4.1 Quality of the Acquisitions

From a specialist perspective, a good image should al-

ways contain retinal tissue with minimal reﬂections,

vessels, macula and optic disc. However, during the

analysis of the exams, some images were found con-

taining hard exudates and vessels, but without mac-

ula or optic disc. For this reason, the quality met-

ric that provided the best results was the tissue area.

The other group of metrics already mentioned in sec-

tion 2.2, ensures that the acquired image has relevant

structures to be detected, as shown in Figure 9.

Figure 9: Image with vessels and hard exudates only.

4.2 Performance of the Detection

Algorithms

The exudates detection was tested on the selected im-

ages by the quality image algorithm and a classiﬁ-

cation accuracy of 91% was achieved using the DSS

(see Table 1).

Table 1: Exudates classiﬁcation obtained.

Predicted

Exudates

Predicted

No Exudates

Exudates 31 11

No Exudates 12 223

From the quality image algorithm analysis, 277

images were considered adequate, 235 of which with-

out the presence of lesions. A Sensitivity of 74% and

a Speciﬁcity of 95% was achieved by the proposed

system on a per image-based analysis, as shown in

Table 2. Without considering the 16 images acquired

with a dilated pupil (removing mydriatic cases) as part

of the dataset, the Speciﬁcity is 95% and the Sen-

sitivity is reduced to 67%. Therefore, it is relevant

to state that the performance of EyeFundusScope is

naturally affected in non-mydriatic cases, due to the

reduced pupil size which complicates pupil and oph-

talmoscope alignment. However, the results show no

variance in the high values of Speciﬁty performance

which is relevant for screening use cases. The sys-

tem has the advantage of acquiring multiple images

and the Quality Image Algorithm works as a ﬁlter to

achieve the best images. For this reason, it is feasible

to acquire images with relevant information without

pupil dilation.

Table 2: Classiﬁcation Performance.

Full

dataset

Non-mydriatic

dataset

Sensitivity 74% 67%

Speciﬁcity 95% 95%

Before this study, the EyeFundusScope automated

screening algorithms were tested with Messidor and

E-Ophta databases. Knowing if the analyzed eye is

the left or the right, we can predict if the macula is to

the right or left of the optic disc, since anatomically

it is located temporally. In order to detect the op-

tic disc, EyeFundusScope uses a template matching

algorithm, using a circular shape as a kernel. Since

the macula is usually located at a distance of approx-

imately 2.5 times the diameter of the optic disc, and

knowing the macula direction, it is possible to locate

the macular region. If the acquired image contains

the temporal quadrants of the retina and the optic disc

is visible, the developed methodology is then able to

ﬁnd the location of the macula, as shown in Figure 10.

Mobile-based Risk Assessment of Diabetic Retinopathy using a Smartphone and Adapted Ophtalmoscope

173

Figure 10: The probabilistic risk zone near to the macula.

4.3 Clinical Impact

As found in this study, the adverse impact of inte-

grating the retinal image acquisition with the Eye-

FundusScope prototype is minimal. The appointment

generally last from 10 to 15 minutes per patient and

the EyeFundusScope acquisition has an average du-

ration of 5 minutes, but without requiring specialist

time.

The obtained images have enough information to

provide feedback about Diabetic Retinopathy. How-

ever, sometimes it is complicated to provide feedback

about all quadrants of the eye. Regarding this aspect,

the solution needs to be improved in order to cover all

the retinal regions.

The pupil dilation improves the quality and the

area acquired in each image, but the objective of the

solution is to avoid requiring this procedure. To en-

able a similar result without performing dilation will

allow more ﬂexibility for the retinal image acquisi-

tion, since the patient will have full visual capabilities

after the examination, being able to walk without as-

sistance and even drive on its own.

5 CONCLUSIONS

In this paper, a mobile approach is evaluated for

fundus image acquisition, together with quality and

pathology assessment, by software suitable for run-

ning in smartphone devices. The time taken for

the proposed acquisition procedure in clinical context

was very low, even facing old adults with severe opti-

cal conditions, which ﬁts the use case of a mobile sys-

tem in remote places with operation by non-experts in

opthalmology.

The Quality Image Algorithm works as a ﬁlter and

ensures each exam performed has acceptable images,

so that the detection algorithms are able to extract

valuable information.

The classiﬁcation results has shown that the usage

of EyeFundusScope may contribute for the reduction

of the time-consuming task of image interpretation by

specialists.

This study provided valuable data to improve the

research in the scope of the EyeFundusScope soft-

ware, hardware and trigger new acquisition perspec-

tives and use cases.

In the future, the authors intend to improve the

classiﬁcation performance by employing state-of-art

Deep Learning approaches to the problem of Dia-

betic Retinopathy risk assessment, considering the

speciﬁcities of smartphone-based image acquisition.

Moreover, improving the usability of operation with

minimal training of the health professional is ex-

pected by leveraging real-time quality image assur-

ance to provide feedback to the prototype operator,

improving human-machine interaction.

ACKNOWLEDGEMENTS

We would like to acknowledge the ﬁnancial support

obtained from North Portugal Regional Operational

Programme (NORTE 2020), Portugal 2020 and the

European Regional Development Fund (ERDF) from

European Union through the project Symbiotic tech-

nology for societal efﬁciency gains: Deus ex Machina

(DEM), NORTE-01-0145-FEDER-000026.

A special acknowledgement to Doctor Ant

onio

Friande from Hospital Santo Ant

onio, Centro Hospi-

talar do Porto, Portugal, for providing the access to

patients, anonymised data and diagnosis.

HEALTHINF 2018 - 11th International Conference on Health Informatics

174

REFERENCES

Adal, K. M., Sidib, D., Ali, S., Chaum, E., Karnowski,

T. P., and Mriaudeau, F. (2014). Automated detection

of microaneurysms using scale-adapted blob analysis

and semi-supervised learning. Computer Methods and

Programs in Biomedicine, 114(1):1–10.

Cheung, N., Mitchell, P., and Wong, T. Y. (2010). Diabetic

retinopathy. The Lancet, 376(9735):124 – 136.

Costa, J., Sousa, I., and Soares, F. (2016). Smartphone-

based decision support system for elimination of

pathology-free cases in diabetic retinopathy screen-

ing.

Felgueiras, S., Costa, J., Soares, F., and Monteiro, M. P.

(2016). Cotton wool spots in eye fundus scope.

Jin, K., Lu, H., Su, Z., Cheng, C., Ye, J., and Qian, D.

(2017). Telemedicine screening of retinal diseases

with a handheld portable non-mydriatic fundus cam-

era. BMC Ophthalmology, 17.

Ravishankar, S., Jain, A., and Mittal, A. (2009). Auto-

mated feature extraction for early detection of diabetic

retinopathy in fundus images. In IEEE Conference

on Computer Vision and Pattern Recognition, 2009.

CVPR 2009, pages 210–217.

Zhang, W., Nicholas, P., Schuman, S., Allingham, M.,

Faridi, A., Tushar, S., Cousins, S. W., and Prakala-

pakorn, S. G. (2015). Screening for diabetic retinopa-

thy using the hand-held pictor camera. Investigative

Ophthalmology & Visual Science, 56(7):1426–1426.

Mobile-based Risk Assessment of Diabetic Retinopathy using a Smartphone and Adapted Ophtalmoscope

175