Automatic Segmentation Methodology for Dermatological Images
Acquired via Mobile Devices
ıs Rosado and Maria Jo
ao M. Vasconcelos
Fraunhofer Portugal AICOS, Porto, Portugal
Mobile Devices, Segmentation, Teledermatology.
Nowadays, skin cancer is considered one of the most common malignancies in the Caucasian population, thus
it is crucial to develop methodologies to prevent it. Because of that, Mobile Teledermatology (MT) is thriving,
allowing patients to adopt an active role in their health status while facilitating doctors to early diagnose skin
cancers. Skin lesion segmentation is one of the most important and difficult task in computerized image
analysis process, and so far the attention is mainly turned to dermoscopic images. In order to turn MT more
accurate, it is therefore fundamental to develop simple segmentation methodologies specifically designed for
macroscopic images or images acquired via smartphones, which is the main focus of this work. The proposed
method was applied in 80 images acquired via smartphones and promising results have been achieved: a mean
Jaccard index of 81%, mean True Detection Rate of 96% and mean Accuracy around 98%. The major goal of
this work is to develop a mobile application easily accessible for the general population, with the aim of raise
awareness and help both patients and doctors in the early diagnosis of skin cancers.
The early detection of malignant skin lesions is fun-
damental for a successful treatment. The melanoma-
based mortality rates are as high as 23%, where
the majority are due to missed or late diagnosed
melanomas. In this context, Mobile Teledermatology
has the potential to improve efficiency and quality as-
pects of care at lower costs and empowers patients to
adopt and active role in managing their own health
status while facilitating the early diagnosis of skin
Segmentation of skin lesions is one of the most
important and difficult task in computerized image
analysis process and its success considerably influ-
ences the accuracy of the subsequent steps. However,
up until now the majority of the available skin lesion
segmentation methods are optimal for dermoscopic
images. While for dermatological or macroscopic im-
ages, like images obtained by mobile phones or cam-
eras, there is still the need to evolve on the develop-
ment of methods for segmenting this type of images.
The present study investigates the acquisition and seg-
mentation of skin lesion images acquired via mobile
Considering that a pigmented skin lesion is a
depigmentation of the skin, many of the segmen-
tation methods start by converting the input image
from color to grayscale and try to distinguish be-
tween skin mole and surrounding skin pixels. Sev-
eral methods have been proposed for the segmen-
tation of dermoscopic images. The Otsu’s thresh-
olding method (Otsu, 1979) has been widely used
for this purpose (Manousaki et al., 2006; Tabatabaie
et al., 2009). Later, (Cavalcanti et al., 2010) em-
ployed Otsu’s method only in the Red channel from
the RGB color space obtaining good segmentation re-
sults. Other researchers proposed to use Snakes (or
Active-Contours) for skin lesion segmentation, like
in (Mahmoud and Al-Jumaily, 2011) that the authors
also use the grayscale image, apply Wiener and Me-
dian filters to remove noise and hairs, threshold the
filtered image and propose a Gradient Vector Flow
snake to obtain the final contour, or as in (Ivanovici
and Stoica, 2012) where a multiscale approach for
active contours is proposed for color images. Oppo-
sitely, (Cavalcanti et al., 2011) observed that when
independent component analysis (ICA) is applied to
the image, one of the resultant ICA component corre-
sponds mainly to the lesion area and proposed deter-
mining the lesion boundary more precisely using the
Chan-Vese Active contours method.
The rest of the paper is organized as follows: In
section 2, the dataset used in the study is presented as
Rosado L. and Vasconcelos M..
Automatic Segmentation Methodology for Dermatological Images Acquired via Mobile Devices.
DOI: 10.5220/0005178302460251
In Proceedings of the International Conference on Health Informatics (HEALTHINF-2015), pages 246-251
ISBN: 978-989-758-068-0
2015 SCITEPRESS (Science and Technology Publications, Lda.)
well as the mobile application developed to obtain the
best possible quality images during the image acqui-
sition process. Our proposed segmentation method-
ology for dermatological images acquired via mobile
devices is described in section 3. Section 4 presents
and discusses the obtained results. Finally, in section
5 conclusions are made and future directions drawn.
Our study intends to assess the segmentation qual-
ity of dermatological images acquired with a smart-
phone. Based on what is known, there is no publicly
available image database that includes dermatologi-
cal images with the ground truth for segmentation.
So we have used the database collected at the Por-
tuguese Institute of Oncology of Porto (IPO), under
the scope of the project Melanoma Detection (Fraun-
hofer, 2014). The images were acquired during 4 ap-
pointments with 2 dermatologists, where the project
was previously explained to the patients and the state-
ment of agreement obtained. The database contains a
total of 80 images, corresponding to 80 different skin
moles, obtained from 31 subjects (14 males and 17 fe-
males) with ages between 28 and 70 years (mean age
around 43 years). They are 24-bit color images with
652x652 pixels of resolution, acquired with a mobile
phone HTC One S.
With the purpose of helping the image acquisition
process of the referred dataset, a mobile application
was developed for the Android OS that performs real-
time detection of the square region of interest of the
target skin lesion. As shown in Figure 1.a, the user
must place the circular red target located at the cen-
ter of the screen inside the skin lesion. Only when
the skin lesion is detected, the circular target becomes
green (see Figure 1.a and 1.b) and the application al-
lows to acquire a image with the maximum resolu-
tion supported by the smartphone. Since this process
guarantees that the skin lesion is at the center of the
circular target, the acquired image at maximum reso-
lution is then cropped to the black square shown on
Figure 1.a, being this the square region of interest im-
age of the target skin lesion (see Figure 1.c). It is
worth noting that the real-time skin lesion detection
uses a preview image with a significantly lower res-
olution when compared with the acquired image, in
order to decrease processing time per frame. With the
HTC One S smartphone, the preview image has a res-
olution of 960x544 pixels (Figure 1.a and 1.b ), the ac-
quired image has 3264x1840 pixels, while the result-
ing square region of interest is an image of 652x652
pixels (Figure 1.c). The resolution of this last image is
imposed by the minimum focus distance of the cam-
era (around 5.5 cm for HTC One S), which guarantees
the maximum resolution for the skin mole image.
Figure 1: Android application screenshots of: (a) skin le-
sion not detected; (b) skin lesion detected, which allows the
image acquisition at maximum resolution; (c) Square region
of interest image acquired at maximum resolution.
The real-time detection of the skin lesion on the
preview images uses the same segmentation method-
ology that is later applied to the acquired square re-
gion of interest image (described in the following sec-
tion). Furthermore, the developed mobile application
respected the image acquisition protocol proposed on
(Rosado et al., 2012):
1. The smartphone camera should be kept directly
perpendicular to the target skin area during the im-
age acquisition.
2. The smartphone camera should be at the mini-
mum focus distance from the skin mole, in order
to ensure maximum image resolution.
Original image
Figure 2: Examples of original images (first row) and man-
ual border from two the authors (border contour in white).
3. The smartphone intrinsic flash light should be
used in the image acquisition.
4. The autofocus should be used in the image ac-
quisition process. In particular, the macro focus
mode should be used if available, and the skin le-
sion must be in the center of the image to ensure
a better automatic focusing.
The IPO Mobile dataset was then acquired using
the described mobile application, and each acquired
image was later manually segmented by the authors,
generating two different ground truth datasets (see
Figure 2).
The proposed methodology comprises six main
blocks, as shown in Figure 3.
The original image is firstly transformed Red-
Green-Blue image (RGB) into a grayscale image and
Figure 3: Block diagram for the segmentation methodology
of skin lesion images acquired via mobile devices.
an adaptive thresholding is applied, as explained next.
Considering an original image I
, the correspond-
ing segmented I
obtained by adaptive thresholding is
given by the following equation:
(x, y) =
0 if I
(x, y) > T
(x, y)
255 otherwise
where T
is the mean intensity value of the square re-
gion centered on the pixel location (x,y) with a side
value of R
minus the constant C. In the proposed
approach, it is used C=10, defined empirically, and:
, Image
Afterwards, a sequence of three opening morpho-
logical operations (erosion followed by dilatation)
is applied to the binary image with the purpose of
smoothing the object contours, eliminating narrow
extensions and breaking thin connections between the
objects. The subsequent step consists on finding the
largest object in the segmented image and consider
it as the region that represents the skin lesion, being
all the other regions discarded. Since the detection of
the biggest blob demands comparing all objects areas
in the image, a median filter is previously applied to
eliminate small objects, thus, significantly decreasing
the processing time of this step. At last, all the holes
inside the selected object are filled, and the final seg-
mented image is obtained.
As previously referred, each image was manually seg-
mented by the authors and two different ground truth
datasets were generated. The suggested method was
implemented in C++ and the average computational
time for the segmentation at 134 miliseconds using an
2 Quad CPU Q9400 with 2.66GHz.
Figure 4 depicts some examples of the segmen-
tation results obtained with our methodology over-
lapped with the ground truth datasets. From the im-
ages observation it can be seen that the proposed
method achieves good segmentation results, with the
segmentation result being extremely close to both
ground truth borders.
To quantify the discrepancy between manual and
automatic segmentation, three distances were calcu-
lated: Jaccard index (J), True Detection Rate (TDR)
and Accuracy.
The Jaccard index (Jaccard, 1912) is used to eval-
uate the overlap between the segmentation results and
the ground truth:
J =
#(X Y )
#(X Y )
, (3)
where X and Y are the binary representation of seg-
mented object of the automatic method and the spe-
cialist, respectively and the operator # returns the
number of pixels belonging to the object. This metric
takes values between 0 and 1, where 1 corresponds to
a perfect match between images and 0 when they are
completely dissimilar.
The True Detection Rate (Silveira et al., 2009) is
given by:
T DR =
#(X Y )
#(Y )
. (4)
While, Jaccard index and the TDR only consider
the segmented regions, the Accuracy (Fraz et al.,
2012) takes into consideration the whole image and
is calculated by the formula:
Accuracy =
#(T N) + #(T P)
#(T N) + #(FP) + #(FN) + #(T P)
, (5)
where TN is the number of true negative cases (num-
ber of pixels correctly classified as background), TP
is the number of true positive cases (number of pix-
els correctly classified as object), FP is the number
of false positive cases and FN is the number of false
negative cases. An Accuracy of 1 means optimal seg-
For each segmentation image obtained through the
proposed method, the Jaccard index, TDR and accu-
racy was calculated, taking into consideration both
ground truth datasets separately. In addition, the Jac-
card index between the ground truth datasets was also
determined, with the purpose of comparing the dis-
crepancy between the manual segmentations. Table 1
shows the resulting mean and standard deviation (std)
values of the previously referred metrics. Analysing
the results for the Jaccard index, the obtained mean
error for the automatic segmentation was around 81%
for both ground truth datasets (SvsG1 and SvsG2),
and 88.36% for the comparison between the ground
truth datasets (G1vsG2). This result (88.36%) indi-
cates that exists a significant variability between the
ground truth datasets, which should be taken into
consideration when analyzing the error obtained for
Table 1: Jaccard, TDR and Accuracy mean and standard
deviation values segmentation errors calculated for the 80
images - proposed method (S) and each ground truth dataset
(G1, G2).
Jaccard (%)
SvsG1 SvsG2 G1vsG2
Mean 81.58 81.41 88.36
Std 8.68 8.10 4.73
TDR (%)
SvsG1 SvsG2
Mean 97.38 95.56
Std 3.93 6.14
Accuracy (%)
SvsG1 SvsG2
Mean 97.38 98.95
Std 0.30 0.41
the automatic segmentation (81%). The mean TDR
around 97% and 96% with std of 4% and 6%, respec-
tively, as well as the mean Accuracy of 97% and 99%,
corroborate the results of the Jaccard index and con-
firms the quality of the present automatic segmenta-
tion methodology suggested.
Figure 5 presents the Jaccard distribution for the
considered segmented images combinations, while
Figures 6 and 7 present the TDR and accuracy dis-
tribution errors. It is possible to see that the re-
ferred metrics for the automated classification are not
so different when comparing with both ground truth
datasets separately. As expected, the distribution and
mean error values are inferior for the combination
that compares the ground truth datasets with each
other. However, the results are only slightly worse,
meaning that the automatic segmentation method per-
forms well. Figures 6 and 7 and show that TDR
and Accuracy errors are very close to each other and
with means near the optimal result (100%), where the
worst (minimal values) are around 80% for TDR and
98% for Accuracy.
Most of available segmentation methods on the liter-
ature are directed to dermoscopic images. The need
to promote the usage of Mobile Teledermatology to
facilitate the early diagnosis of skin cancers led us to
explore and develop methodologies orientated to der-
matological images acquired via mobile devices.
A mobile application for the Android OS was de-
veloped to help the image acquisition process, per-
forming real-time detection of the region of interest
of the target skin lesion. In this work we also present
a methodology to automatically segment skin lesions
from dermatological images acquired via mobile de-
vices. The method was applied in 80 smartphone-
acquired images, achieving a mean Jaccard index
result of 81%, mean True Detection Rate of 96%
and mean Accuracy around 98%, confirming the
adequacy of the suggested automatic segmentation
In order to expand this study in the near future,
we consider that is important to have a testing dataset
with more skin lesion images acquired via mobile de-
vices, manually segmented by different specialists in
the area and also investigate if the methodology is ro-
bust for different brands of mobile devices.
Above all, it is our goal to develop a mobile ap-
plication easily accessible for the general population,
with the aim of raise awareness and help both patients
and doctors in the early diagnosis of skin cancers.
This work was done under the scope of the project
“SMARTSKINS: A Novel Framework for Super-
vised Mobile Assessment and Risk Triage of Skin
Lesion via Non-invasive Screening” with reference
PTDC/BBB-BMD/3088/2012 financially supported
by Fundac¸
ao para a Ci
encia e a Tecnologia in Por-
Cavalcanti, P., Scharcanski, J., Di Persia, L., and Milone,
D. (2011). An ICA-based method for the segmenta-
tion of pigmented skin lesions in macroscopic images.
In 2011 Annual International Conference of the IEEE
Engineering in Medicine and Biology Society,EMBC,
pages 5993–5996.
Cavalcanti, P., Yari, Y., and Scharcanski, J. (2010). Pig-
mented skin lesion segmentation on macroscopic im-
ages. In 25th International Conference of Image and
Vision Computing New Zealand, pages 1–7.
Fraunhofer, P. A. (2014). Melanoma detection,
internal project.
en/fraunhofer aicos/projects/internal research/
melanoma detection.html.
Fraz, M. M., Remagnino, P., Hoppe, A., Uyyanonvara, B.,
Rudnicka, A. R., Owen, C. G., and Barman, S. A.
(2012). Blood vessel segmentation methodologies in
retinal images–a survey. Computer methods and pro-
grams in biomedicine, 108(1):407–433.
Ivanovici, M. and Stoica, D. (2012). Color diffusion model
for active contours-an application to skin lesion seg-
mentation. In 2012 Annual International Conference
of the IEEE, Engineering in Medicine and Biology So-
ciety (EMBC), pages 5347–5350. IEEE.
Jaccard, P. (1912). The distribution of the flora in the alpine
zone. 1. New phytologist, 11(2):37–50.
Mahmoud, M. and Al-Jumaily, A. (2011). Segmentation
of skin cancer images based on gradient vector flow
(GVF) snake. In 2011 International Conference on
Mechatronics and Automation (ICMA), pages 216–
Manousaki, A. G., Manios, A. G., Tsompanaki, E. I.,
Panayiotides, J. G., Tsiftsis, D. D., Kostaki, A. K.,
and Tosca, A. D. (2006). A simple digital image
processing system to aid in melanoma diagnosis in
an everyday melanocytic skin lesion unit. a prelimi-
nary report. International Journal of Dermatology,
Otsu, N. (1979). A threshold selection method from gray-
level histograms. IEEE Transactions on Systems, Man
and Cybernetics, 9(1):62–66.
Rosado, L., Castro, R., Ferreira, L., and Ferreira, M. (2012).
Extraction of ABCD rule features from skin lesions
images with smartphone. Studies in health technology
and informatics, 177:242–247.
Silveira, M., Nascimento, J. C., Marques, J. S., Marc¸al,
A. R., Mendonc¸a, T., Yamauchi, S., Maeda, J., and
Rozeira, J. (2009). Comparison of segmentation meth-
ods for melanoma diagnosis in dermoscopy images.
IEEE Journal of Selected Topics in Signal Processing,
Tabatabaie, K., Esteki, A., and Toossi, P. (2009). Extraction
of skin lesion texture features based on independent
component analysis. Skin research and technology,
Figure 4: Examples of segmentation results: the ground
truth borders are showed in white and yellow, while the
border obtained using the proposed methodology appears
in black, respectively.
Jaccard errors
Figure 5: Jaccard distribution errors.
Figure 6: Distribution errors for True Detection Rate of the
automated segmentation, considering the two ground truth
Figure 7: Distribution errors for Accuracy of the automated
segmentation, considering the two ground truth datasets.