Automatic Segmentation Methodology for Dermatological Images

Acquired via Mobile Devices

ıs Rosado and Maria Jo

ao M. Vasconcelos

Fraunhofer Portugal AICOS, Porto, Portugal

Keywords:

Mobile Devices, Segmentation, Teledermatology.

Abstract:

Nowadays, skin cancer is considered one of the most common malignancies in the Caucasian population, thus

it is crucial to develop methodologies to prevent it. Because of that, Mobile Teledermatology (MT) is thriving,

allowing patients to adopt an active role in their health status while facilitating doctors to early diagnose skin

cancers. Skin lesion segmentation is one of the most important and difﬁcult task in computerized image

analysis process, and so far the attention is mainly turned to dermoscopic images. In order to turn MT more

accurate, it is therefore fundamental to develop simple segmentation methodologies speciﬁcally designed for

macroscopic images or images acquired via smartphones, which is the main focus of this work. The proposed

method was applied in 80 images acquired via smartphones and promising results have been achieved: a mean

Jaccard index of 81%, mean True Detection Rate of 96% and mean Accuracy around 98%. The major goal of

this work is to develop a mobile application easily accessible for the general population, with the aim of raise

awareness and help both patients and doctors in the early diagnosis of skin cancers.

1 INTRODUCTION

The early detection of malignant skin lesions is fun-

damental for a successful treatment. The melanoma-

based mortality rates are as high as 23%, where

the majority are due to missed or late diagnosed

melanomas. In this context, Mobile Teledermatology

has the potential to improve efﬁciency and quality as-

pects of care at lower costs and empowers patients to

adopt and active role in managing their own health

status while facilitating the early diagnosis of skin

cancers.

Segmentation of skin lesions is one of the most

important and difﬁcult task in computerized image

analysis process and its success considerably inﬂu-

ences the accuracy of the subsequent steps. However,

up until now the majority of the available skin lesion

segmentation methods are optimal for dermoscopic

images. While for dermatological or macroscopic im-

ages, like images obtained by mobile phones or cam-

eras, there is still the need to evolve on the develop-

ment of methods for segmenting this type of images.

The present study investigates the acquisition and seg-

mentation of skin lesion images acquired via mobile

devices.

Considering that a pigmented skin lesion is a

depigmentation of the skin, many of the segmen-

tation methods start by converting the input image

from color to grayscale and try to distinguish be-

tween skin mole and surrounding skin pixels. Sev-

eral methods have been proposed for the segmen-

tation of dermoscopic images. The Otsu’s thresh-

olding method (Otsu, 1979) has been widely used

for this purpose (Manousaki et al., 2006; Tabatabaie

et al., 2009). Later, (Cavalcanti et al., 2010) em-

ployed Otsu’s method only in the Red channel from

the RGB color space obtaining good segmentation re-

sults. Other researchers proposed to use Snakes (or

Active-Contours) for skin lesion segmentation, like

in (Mahmoud and Al-Jumaily, 2011) that the authors

also use the grayscale image, apply Wiener and Me-

dian ﬁlters to remove noise and hairs, threshold the

ﬁltered image and propose a Gradient Vector Flow

snake to obtain the ﬁnal contour, or as in (Ivanovici

and Stoica, 2012) where a multiscale approach for

active contours is proposed for color images. Oppo-

sitely, (Cavalcanti et al., 2011) observed that when

independent component analysis (ICA) is applied to

the image, one of the resultant ICA component corre-

sponds mainly to the lesion area and proposed deter-

mining the lesion boundary more precisely using the

Chan-Vese Active contours method.

The rest of the paper is organized as follows: In

section 2, the dataset used in the study is presented as

246

Rosado L. and Vasconcelos M..

Automatic Segmentation Methodology for Dermatological Images Acquired via Mobile Devices.

DOI: 10.5220/0005178302460251

In Proceedings of the International Conference on Health Informatics (HEALTHINF-2015), pages 246-251

ISBN: 978-989-758-068-0

 2015 SCITEPRESS (Science and Technology Publications, Lda.)

well as the mobile application developed to obtain the

best possible quality images during the image acqui-

sition process. Our proposed segmentation method-

ology for dermatological images acquired via mobile

devices is described in section 3. Section 4 presents

and discusses the obtained results. Finally, in section

5 conclusions are made and future directions drawn.

2 DATASET

Our study intends to assess the segmentation qual-

ity of dermatological images acquired with a smart-

phone. Based on what is known, there is no publicly

available image database that includes dermatologi-

cal images with the ground truth for segmentation.

So we have used the database collected at the Por-

tuguese Institute of Oncology of Porto (IPO), under

the scope of the project Melanoma Detection (Fraun-

hofer, 2014). The images were acquired during 4 ap-

pointments with 2 dermatologists, where the project

was previously explained to the patients and the state-

ment of agreement obtained. The database contains a

total of 80 images, corresponding to 80 different skin

moles, obtained from 31 subjects (14 males and 17 fe-

males) with ages between 28 and 70 years (mean age

around 43 years). They are 24-bit color images with

652x652 pixels of resolution, acquired with a mobile

phone HTC One S.

With the purpose of helping the image acquisition

process of the referred dataset, a mobile application

was developed for the Android OS that performs real-

time detection of the square region of interest of the

target skin lesion. As shown in Figure 1.a, the user

must place the circular red target located at the cen-

ter of the screen inside the skin lesion. Only when

the skin lesion is detected, the circular target becomes

green (see Figure 1.a and 1.b) and the application al-

lows to acquire a image with the maximum resolu-

tion supported by the smartphone. Since this process

guarantees that the skin lesion is at the center of the

circular target, the acquired image at maximum reso-

lution is then cropped to the black square shown on

Figure 1.a, being this the square region of interest im-

age of the target skin lesion (see Figure 1.c). It is

worth noting that the real-time skin lesion detection

uses a preview image with a signiﬁcantly lower res-

olution when compared with the acquired image, in

order to decrease processing time per frame. With the

HTC One S smartphone, the preview image has a res-

olution of 960x544 pixels (Figure 1.a and 1.b ), the ac-

quired image has 3264x1840 pixels, while the result-

ing square region of interest is an image of 652x652

pixels (Figure 1.c). The resolution of this last image is

imposed by the minimum focus distance of the cam-

era (around 5.5 cm for HTC One S), which guarantees

the maximum resolution for the skin mole image.

Figure 1: Android application screenshots of: (a) skin le-

sion not detected; (b) skin lesion detected, which allows the

image acquisition at maximum resolution; (c) Square region

of interest image acquired at maximum resolution.

The real-time detection of the skin lesion on the

preview images uses the same segmentation method-

ology that is later applied to the acquired square re-

gion of interest image (described in the following sec-

tion). Furthermore, the developed mobile application

respected the image acquisition protocol proposed on

(Rosado et al., 2012):

1. The smartphone camera should be kept directly

perpendicular to the target skin area during the im-

age acquisition.

2. The smartphone camera should be at the mini-

mum focus distance from the skin mole, in order

to ensure maximum image resolution.

AutomaticSegmentationMethodologyforDermatologicalImagesAcquiredviaMobileDevices

247

Original image

GT1

GT2

Figure 2: Examples of original images (ﬁrst row) and man-

ual border from two the authors (border contour in white).

3. The smartphone intrinsic ﬂash light should be

used in the image acquisition.

4. The autofocus should be used in the image ac-

quisition process. In particular, the macro focus

mode should be used if available, and the skin le-

sion must be in the center of the image to ensure

a better automatic focusing.

The IPO Mobile dataset was then acquired using

the described mobile application, and each acquired

image was later manually segmented by the authors,

generating two different ground truth datasets (see

Figure 2).

3 SEGMENTATION

METHODOLOGY

The proposed methodology comprises six main

blocks, as shown in Figure 3.

The original image is ﬁrstly transformed Red-

Green-Blue image (RGB) into a grayscale image and

Figure 3: Block diagram for the segmentation methodology

of skin lesion images acquired via mobile devices.

an adaptive thresholding is applied, as explained next.

Considering an original image I

, the correspond-

ing segmented I

obtained by adaptive thresholding is

given by the following equation:

(x, y) =



0 if I

(x, y) > T

(x, y)

255 otherwise

(1)

where T

is the mean intensity value of the square re-

gion centered on the pixel location (x,y) with a side

value of R

minus the constant C. In the proposed

approach, it is used C=10, deﬁned empirically, and:

max{Image

width

, Image

height

}

(2)

Afterwards, a sequence of three opening morpho-

logical operations (erosion followed by dilatation)

is applied to the binary image with the purpose of

smoothing the object contours, eliminating narrow

extensions and breaking thin connections between the

objects. The subsequent step consists on ﬁnding the

largest object in the segmented image and consider

it as the region that represents the skin lesion, being

all the other regions discarded. Since the detection of

the biggest blob demands comparing all objects areas

in the image, a median ﬁlter is previously applied to

eliminate small objects, thus, signiﬁcantly decreasing

the processing time of this step. At last, all the holes

inside the selected object are ﬁlled, and the ﬁnal seg-

mented image is obtained.

4 RESULTS

As previously referred, each image was manually seg-

mented by the authors and two different ground truth

datasets were generated. The suggested method was

implemented in C++ and the average computational

time for the segmentation at 134 miliseconds using an

Intel

 Core

2 Quad CPU Q9400 with 2.66GHz.

Figure 4 depicts some examples of the segmen-

tation results obtained with our methodology over-

lapped with the ground truth datasets. From the im-

ages observation it can be seen that the proposed

method achieves good segmentation results, with the

HEALTHINF2015-InternationalConferenceonHealthInformatics

248

segmentation result being extremely close to both

ground truth borders.

To quantify the discrepancy between manual and

automatic segmentation, three distances were calcu-

lated: Jaccard index (J), True Detection Rate (TDR)

and Accuracy.

The Jaccard index (Jaccard, 1912) is used to eval-

uate the overlap between the segmentation results and

the ground truth:

J =

#(X ∩Y )

#(X ∪Y )

, (3)

where X and Y are the binary representation of seg-

mented object of the automatic method and the spe-

cialist, respectively and the operator # returns the

number of pixels belonging to the object. This metric

takes values between 0 and 1, where 1 corresponds to

a perfect match between images and 0 when they are

completely dissimilar.

The True Detection Rate (Silveira et al., 2009) is

given by:

T DR =

#(X ∩Y )

#(Y )

. (4)

While, Jaccard index and the TDR only consider

the segmented regions, the Accuracy (Fraz et al.,

2012) takes into consideration the whole image and

is calculated by the formula:

Accuracy =

#(T N) + #(T P)

#(T N) + #(FP) + #(FN) + #(T P)

, (5)

where TN is the number of true negative cases (num-

ber of pixels correctly classiﬁed as background), TP

is the number of true positive cases (number of pix-

els correctly classiﬁed as object), FP is the number

of false positive cases and FN is the number of false

negative cases. An Accuracy of 1 means optimal seg-

mentation.

For each segmentation image obtained through the

proposed method, the Jaccard index, TDR and accu-

racy was calculated, taking into consideration both

ground truth datasets separately. In addition, the Jac-

card index between the ground truth datasets was also

determined, with the purpose of comparing the dis-

crepancy between the manual segmentations. Table 1

shows the resulting mean and standard deviation (std)

values of the previously referred metrics. Analysing

the results for the Jaccard index, the obtained mean

error for the automatic segmentation was around 81%

for both ground truth datasets (SvsG1 and SvsG2),

and 88.36% for the comparison between the ground

truth datasets (G1vsG2). This result (88.36%) indi-

cates that exists a signiﬁcant variability between the

ground truth datasets, which should be taken into

consideration when analyzing the error obtained for

Table 1: Jaccard, TDR and Accuracy mean and standard

deviation values segmentation errors calculated for the 80

images - proposed method (S) and each ground truth dataset

(G1, G2).

Jaccard (%)

SvsG1 SvsG2 G1vsG2

Mean 81.58 81.41 88.36

Std 8.68 8.10 4.73

TDR (%)

SvsG1 SvsG2

Mean 97.38 95.56

Std 3.93 6.14

Accuracy (%)

SvsG1 SvsG2

Mean 97.38 98.95

Std 0.30 0.41

the automatic segmentation (81%). The mean TDR

around 97% and 96% with std of 4% and 6%, respec-

tively, as well as the mean Accuracy of 97% and 99%,

corroborate the results of the Jaccard index and con-

ﬁrms the quality of the present automatic segmenta-

tion methodology suggested.

Figure 5 presents the Jaccard distribution for the

considered segmented images combinations, while

Figures 6 and 7 present the TDR and accuracy dis-

tribution errors. It is possible to see that the re-

ferred metrics for the automated classiﬁcation are not

so different when comparing with both ground truth

datasets separately. As expected, the distribution and

mean error values are inferior for the combination

that compares the ground truth datasets with each

other. However, the results are only slightly worse,

meaning that the automatic segmentation method per-

forms well. Figures 6 and 7 and show that TDR

and Accuracy errors are very close to each other and

with means near the optimal result (100%), where the

worst (minimal values) are around 80% for TDR and

98% for Accuracy.

5 CONCLUSIONS AND FUTURE

WORK

Most of available segmentation methods on the liter-

ature are directed to dermoscopic images. The need

to promote the usage of Mobile Teledermatology to

facilitate the early diagnosis of skin cancers led us to

explore and develop methodologies orientated to der-

matological images acquired via mobile devices.

A mobile application for the Android OS was de-

veloped to help the image acquisition process, per-

AutomaticSegmentationMethodologyforDermatologicalImagesAcquiredviaMobileDevices

249

forming real-time detection of the region of interest

of the target skin lesion. In this work we also present

a methodology to automatically segment skin lesions

from dermatological images acquired via mobile de-

vices. The method was applied in 80 smartphone-

acquired images, achieving a mean Jaccard index

result of 81%, mean True Detection Rate of 96%

and mean Accuracy around 98%, conﬁrming the

adequacy of the suggested automatic segmentation

methodology.

In order to expand this study in the near future,

we consider that is important to have a testing dataset

with more skin lesion images acquired via mobile de-

vices, manually segmented by different specialists in

the area and also investigate if the methodology is ro-

bust for different brands of mobile devices.

Above all, it is our goal to develop a mobile ap-

plication easily accessible for the general population,

with the aim of raise awareness and help both patients

and doctors in the early diagnosis of skin cancers.

ACKNOWLEDGEMENTS

This work was done under the scope of the project

“SMARTSKINS: A Novel Framework for Super-

vised Mobile Assessment and Risk Triage of Skin

Lesion via Non-invasive Screening” with reference

PTDC/BBB-BMD/3088/2012 ﬁnancially supported

by Fundac¸

ao para a Ci

encia e a Tecnologia in Por-

tugal.

REFERENCES

Cavalcanti, P., Scharcanski, J., Di Persia, L., and Milone,

D. (2011). An ICA-based method for the segmenta-

tion of pigmented skin lesions in macroscopic images.

In 2011 Annual International Conference of the IEEE

Engineering in Medicine and Biology Society,EMBC,

pages 5993–5996.

Cavalcanti, P., Yari, Y., and Scharcanski, J. (2010). Pig-

mented skin lesion segmentation on macroscopic im-

ages. In 25th International Conference of Image and

Vision Computing New Zealand, pages 1–7.

Fraunhofer, P. A. (2014). Melanoma detection,

internal project. http://www.fraunhofer.pt/

en/fraunhofer aicos/projects/internal research/

melanoma detection.html.

Fraz, M. M., Remagnino, P., Hoppe, A., Uyyanonvara, B.,

Rudnicka, A. R., Owen, C. G., and Barman, S. A.

(2012). Blood vessel segmentation methodologies in

retinal images–a survey. Computer methods and pro-

grams in biomedicine, 108(1):407–433.

Ivanovici, M. and Stoica, D. (2012). Color diffusion model

for active contours-an application to skin lesion seg-

mentation. In 2012 Annual International Conference

of the IEEE, Engineering in Medicine and Biology So-

ciety (EMBC), pages 5347–5350. IEEE.

Jaccard, P. (1912). The distribution of the ﬂora in the alpine

zone. 1. New phytologist, 11(2):37–50.

Mahmoud, M. and Al-Jumaily, A. (2011). Segmentation

of skin cancer images based on gradient vector ﬂow

(GVF) snake. In 2011 International Conference on

Mechatronics and Automation (ICMA), pages 216–

220.

Manousaki, A. G., Manios, A. G., Tsompanaki, E. I.,

Panayiotides, J. G., Tsiftsis, D. D., Kostaki, A. K.,

and Tosca, A. D. (2006). A simple digital image

processing system to aid in melanoma diagnosis in

an everyday melanocytic skin lesion unit. a prelimi-

nary report. International Journal of Dermatology,

45(4):402–410.

Otsu, N. (1979). A threshold selection method from gray-

level histograms. IEEE Transactions on Systems, Man

and Cybernetics, 9(1):62–66.

Rosado, L., Castro, R., Ferreira, L., and Ferreira, M. (2012).

Extraction of ABCD rule features from skin lesions

images with smartphone. Studies in health technology

and informatics, 177:242–247.

Silveira, M., Nascimento, J. C., Marques, J. S., Marc¸al,

A. R., Mendonc¸a, T., Yamauchi, S., Maeda, J., and

Rozeira, J. (2009). Comparison of segmentation meth-

ods for melanoma diagnosis in dermoscopy images.

IEEE Journal of Selected Topics in Signal Processing,

3(1):35–45.

Tabatabaie, K., Esteki, A., and Toossi, P. (2009). Extraction

of skin lesion texture features based on independent

component analysis. Skin research and technology,

15(4):433–439.

HEALTHINF2015-InternationalConferenceonHealthInformatics

250

Figure 4: Examples of segmentation results: the ground

truth borders are showed in white and yellow, while the

border obtained using the proposed methodology appears

in black, respectively.

Jaccard errors

Figure 5: Jaccard distribution errors.

TDR

Figure 6: Distribution errors for True Detection Rate of the

automated segmentation, considering the two ground truth

datasets.

Accuracy

Figure 7: Distribution errors for Accuracy of the automated

segmentation, considering the two ground truth datasets.

AutomaticSegmentationMethodologyforDermatologicalImagesAcquiredviaMobileDevices

251