Fully Automatic Deformable Model Integrating Edge, Texture and Shape
Application to Cardiac Images Segmentation
Cl
´
ement Beitone
1,3
, Christophe Tilmant
1,3
and Fr
´
ed
´
eric Chausse
2,3
1
Clermont Universit
´
e, Univ. Blaise Pascal, Institut Pascal, BP 10448, F-63000 Clermont-Fd, France
2
Clermont Universit
´
e, Univ. d’Auvergne, Institut Pascal, BP 10448, F-63000 Clermont-Fd, France
3
CNRS, UMR 6602, Institut Pascal, F-63171 Aubi
`
ere, France
Keywords:
Fully Automatic Segmentation, Deformable Model, MRI, Weibull Model, Monogenic Signal.
Abstract:
This article presents a fully automatic left ventricle (LV) segmentation method on MR images by means of
an implicit deformable model (Level Set) in a variational context. For these parametrizations, the degrees of
freedom are: initialization and functional energy. The first is often delegated to the practician. To avoid this
human intervention, we present an automatic initialisation method based on the Hough transform exploiting
spatio-temporal information. Generally, energetic functionals integrate edges, regions and shape terms. We
propose to bundle an edge-based energy computed by feature asymmetry on the monogenic signal, a region-
based energy capitalizing on image statistics (Weibull model) and a shape-based energy constrained by the
myocardium thickness. The presence of multiple tissues implies data non-stationarity. To best estimate dis-
tribution parameters over the regions and regarding anatomy, we propose a deformable model maximizing
locally and globally the log-likelihood. Finally, we evaluate our method on MICCAI 09 Challenge data.
1 INTRODUCTION - STATE OF
THE ART
Cardiovascular diseases are the main cause of death
on earth. According to the World Health Organiza-
tion in 2008, they are responsible for 30% of the to-
tal number of deaths. Systolic function impairment
and especially the left ventricle (LV) is one of the
main characteristics reflecting that the heart is dam-
aged. Quantitative analysis provides important car-
diac functional parameters for heart disease diagnosis,
for example the strain showed is reliable prognostic
value.
The evaluation of these parameters requires,
among other things, to have an accurate segmentation
result. This step has been the subject of a large num-
ber of studies: a review and an evaluation of segmen-
tation methods applied to MR images can be found in
(Petitjean and Dacher, 2011).
Authors propose the following segmentation
methods classification regarding the level of exter-
nal information they require (strong/weak) and their
methodological approach:
Weak Priors Strong Priors
(i) Image-based (a) AAM
(ii) Pixel classification (b) Atlas-based
(iii) Deformable models (c) Shape Prior
We propose a variational method, belonging to the
deformable model category with weak priors (iii), in
order to segment LV. There are three degrees of free-
dom: parametrization, initialization and energy func-
tional.
We decided to represent our deformable model us-
ing the level set framework because it allows topolog-
ical changes. These issues arise, for example, when
there are papillary muscles inside the endocardium
cavity.
Deformable model evolution is linked to an op-
timisation problem. The functional minimization is
Figure 1: Overall scheme of the proposed method.
517
Beitone C., Tilmant C. and Chausse F..
Fully Automatic Deformable Model Integrating Edge, Texture and Shape - Application to Cardiac Images Segmentation.
DOI: 10.5220/0005304005170522
In Proceedings of the 10th International Conference on Computer Vision Theory and Applications (VISAPP-2015), pages 517-522
ISBN: 978-989-758-089-5
Copyright
c
2015 SCITEPRESS (Science and Technology Publications, Lda.)
achieved using a descent method, and leads to a lo-
cal minimum dependent on the initialization. Usu-
ally, this step is delegated to the practician (Lu et al.,
2009). Nevertheless, this tedious intervention can
be automated using the geometrical properties of the
left cavity (quasi-conic shape); which can be approx-
imated by a circular shape in a basal short axis (SAX)
slice. Heart motion is also discriminant to initialize
the model (Pednekar et al., 2006).
Using a deformable model requires to design an
energy functional which depends on internal informa-
tion (regularization) and external information (image,
prior). For example, the geodesic model (Caselles
et al., 1997) uses a functional dependent on the image
gradient intensity. The key idea behind edge-based
energy is to force the model to converge on areas
where the gradient intensity is high, i.e where edges
are. The authors also proposed to work on image tex-
ture by integrating region-based energy. The seminal
Chan and Vese method (Chan and Vese, 2001) mini-
mizes inter-class variance on pixels inside/outside the
model. Some generalizations exploit statistics linked
to the physical process of the image formation. For
example, the Weibull model (Ayed et al., 2006) al-
lows to model different distributions which can be
used to finely characterise regions. To solve the non-
stationarity issue induced by region response, Lank-
ton et al. (Lankton and Tannenbaum, 2008) propose a
local region energy. In this method, statistics are eval-
uated along the deformable model. However in real
images, noise and missing data require to incorpo-
rate geometrical constraints in the method, known as
shape-based energy, to solve these issues. This prior
information can be integrated in different ways into
the variational method. For example, Foulounneau
(Foulonneau et al., 2003) uses a mean square process
over Legendre moments between the current model
shape and a reference shape. One of the main interests
of the variational context is its ability to glue different
constraints (edge, region and shape-based). This cou-
pling allows to strengthen the method by merging in-
formation sources. For example the Geodesic Active
Region (GAR) model proposed by Paragios (Paragios
and Deriche, 2002) is one of the first to propose an
edge/region-based energy.
A large number of methods apply these energies
to segment cardiac images. Ultrasound scans (US)
uncovered specific needs for these noisy and poorly
contrasted images. Some authors developed frame-
works to replace gradient-based methods. For in-
stance, some methods use the monogenic signal and
the asymmetry measure (Felsberg and Sommer, 2001;
Kovesi, 1997) to segment cardiac ultrasound images
with an edge-based energy (Rajpoot et al., 2008). In
a region-based context Barbosa et al. (Barbosa et al.,
2013) used Lankton approach with Rayleigh statis-
tics in order to segment US images. The prior knowl-
edge of the heart geometry can be used in a shape-
based energy. This kind of anatomical constraint was
proposed by Paragios (Paragios, 2002) in MRI left
ventricle segmentation by imposing a minimal thick-
ness of the myocardium wall. Finally, Belaid (Belaid
et al., 2011) proposes to merge edge (monogenic sig-
nal) and region-based (Rayleigh distribution) energy
to perform US left ventricle segmentation.
In section 2, we present a general framework to
segment LV in MRI (3D+T)(the framework overall
is presented on figure 1). In this framework a fully
automatic initialization step is performed. Our inten-
tion is to apply the ultrasound approach to MRI by
carving data terms coherent with the underlying im-
age physics. For that, our functional couples region-
based (Weibull model), edge-based (Kovesi asym-
metry measure) and shape-based (myocardium wall
thickness) energy terms. Its particularity is that we
do not use strong prior (spatio-temporal) as in atlas
methods. In section 3, we present some results and
we compare our method with one of the best methods
of the MICCAI 09 challenge.
2 FULLY AUTOMATIC
SEGMENTATION
2.1 Deformable Model Framework
The variational formulation of the segmentation prob-
lem by means of a deformable model is stated as:
S = arg min
S
F
S
E(S
)
δE(S)
δS
= 0 (1)
In our case S corresponds to the final shape of our
model. This shape is taken from a family of solutions
F
S
, by minimizing the energetic functional E. This
optimization problem is solved by means of descent
method on an artificial temporal parameter t. The
model is put into motion, it is a deformable model:
S
t
=
δE
δS
= V n (2)
This problem is equivalent to a front propagation
where the variation is homogeneous to a speed V on
the normal n. The calculus of variations on E can be
computed using shape derivative tools (Aubert et al.,
2003). In the level set framework, Sethian (Sethian,
1999) showed that this problem can be stated as:
∂φ
t
= V |∇φ|, (3)
VISAPP2015-InternationalConferenceonComputerVisionTheoryandApplications
518
Figure 2: (a) Motion between ED and ES (b) Motion bound-
ing box (c) Model Initialisation.
where φ, the level representation, is a higher order
function and φ
1
(0) = S .
2.2 Automatic Initialisation
To initialize our deformable model, we exploit both
geometric and motion information. These data are
more meaningful when you examine the heart: (i) on
a basal SAX slice (ii) between the end of diastole (ED
) and the end of systole (ES).
To detect the LV we use a circular Hough trans-
form on ED basal slice but we restrict voting points to
a region of interest encompassing strong motion be-
tween ED and ES. We select a basal slice in order
to be close to our circular shape hypothesis for the
LV. The diameter associated to the circular detection
is based on myocardium mean size on a basal SAX
slice, which is about 50 mm for a healthy adult heart.
For that, we restrict the detection to a range of 40 to
60 mm. Our deformable model is initialized using
the fast marching method (Sethian, 1999), on a circle
with a diameter equal to 95% of the detected diam-
eter. The whole segmentation is carried out on a 80
x 80 mm region of interest centered on the detection.
This process is illustrated on Figure 2.
For other slices we propagate the basal result, on
which we apply a scale factor following a conical hy-
pothesis (based on the LV shape). Finally, we propa-
gate the segmentation result at time t to initialize the
t + 1 model.
2.3 Functional Definition: Energy
Coupling
To segment endocardium and epicardium contours,
we use two deformable models based on the coupling
three kinds of energy region, edge and shape energies:
(
E
endo
= α
1
E
R
endo
+ α
2
E
E
E
epi
= β
1
E
R
epi
+ β
2
E
E
+ β
3
E
A
(4)
E
endo
and E
epi
correspond respectively to ener-
gies used to segment endocardium and epicardium
contours. E
R
endo
and E
R
epi
correspond to region-based
terms, E
E
is an edge-based term and E
A
is a shape
term. Parameters α
i
and β
i
are used to weigh the
influence of each term.
Edge-based energy: Our edge-based energy data
term is the same for both segmentations. This data
term is based on a geodesic model, however, we use a
step edge detector based on phase information instead
of a classic gradient-based detector:
E
E
=
Z
S
g(FA(I
NL
))dx (5)
where x is a given point, g(a) =
1
1+a
2
and I
NL
is
the locally normalized input image. The edge de-
tector FA, is the feature asymmetry introduced by
Kovesi (Kovesi, 1997). This measure gives reliable,
well localized but incomplete edges even in noisy im-
ages. We also used the monogenic signal (Felsberg
and Sommer, 2001), the multidimensional extension
of the 1D analytic signal, to replace steerable filters
used by Kovesi as in (Belaid et al., 2011).
We apply a local normalization filter on the MRI
raw image in order to uniformize the mean and
variance of an image around local neighbourhoods.
This is especially useful for correcting non-uniform
illumination or shading artifacts.
Region-based Energy. Our region energy terms
are based on a Weibull model (Ayed et al., 2006). The
probability density function of a Weibull random vari-
able is:
P
W
(u) =
β
η
u
η
β1
e
u
η
β
× I1
[0,+[
(u), (6)
where η is a scale parameter and β is a shape parame-
ter. This model tends to maximize the log-likelihood
of a Weibull distribution over a region R:
E
GW
(R) =
Z
R
log(P
W
(I|R))dx (7)
A local version (Lankton framework) of this
global energy might be stated, using a ball B, as:
E
LW
(R
S
) =
Z
S
Z
R
S
B(x;y)log(P
W
(I|R
S
))dydx (8)
Global methods provide more energy but are
meaningful if the distribution over the considered ob-
ject follows a stationary process. Nevertheless, our
images present some non-stationarity over the my-
ocardium and over its neighbourhood. Experimen-
tally, we show that the distributions over the different
regions (using expert segmentations) might be repre-
sented as in Figure 3. This figure, shows that my-
ocardium and tissues distributions can be approxi-
mated using a normal law and an exponential law.
FullyAutomaticDeformableModelIntegratingEdge,TextureandShape-ApplicationtoCardiacImagesSegmentation
519
Cavity distribution can be approximated using a nor-
mal law under the assumption that papillary muscles
are excluded from the cavity. Indeed, in Figure 3 the
intensity step between 20 and 60 comes from the in-
clusion of these muscles by the expert. We build our
region-based energy terms using these observations
and assumptions. The final model we used is stated
as:
E
R
endo
= E
GW
(R
S
) + E
LW
(R
S
) (9)
E
R
epi
= E
LW
(R
S
) + E
LW
(R
S
) (10)
In these models, we capitalize on the relative station-
arity over the cavity to lead the contour during the first
segmentation step and then we work more finely over
the remaining tissues (myocardium and its environ-
ment). The Weibull scale parameters for the different
regions R are computed (depending if they are global
G or local L) as:
η
GR
=
R
R
I
β
R
dx
R
R
dx
!
1
β
R
η
LR
(x) =
R
R
B(x;y)I
β
R
dy
R
R
B(x;y)dy
!
1
β
R
(11)
0 20 40
60
80 100 120
0
2 · 10
2
4 · 10
2
6 · 10
2
8 · 10
2
0.1
Intensity
Probability
Cavity
Myocardium
Tissues
Figure 3: Regions histograms based on an expert segmen-
tation.
Shape-based Energy. Our anatomical constraint is
similar to the one introduced by Paragios (Paragios,
2002). This constraint corresponds to a coupling
force added back to the global energy to ensure the en-
docardium segmentation when region and edge-based
terms failed. This happens, for example, where the
contrast is poor between organs surrounding the heart
and the myocardium wall. This force allows to have a
non-zero wall thickness for the LV.
This term can be considered as a balloon force
with an intensity C depending on the location x and
the endocardium contour position φ
endo
:
E
A
=
Z
R
S
C
A
(x, φ
endo
)dx (12)
We define C
A
, the coupling function as
C
A
(x, φ
endo
) = 1 if d(x, φ
endo
) < m, 1 if
d(x, φ
endo
) > M and 0 elsewhere. This repul-
sive force halts the contour propagation if into does
not fall in an acceptable distance range. This range
[m, M] is defined using standard anatomical wall
thickness properties and is determined relatively to
the endocardium position.
3 RESULTS & PERSPECTIVES
We assessed our method on the MICCAI 2009 chal-
lenge database, which contains 15 patients for the
training set and 15 for the validation set. For each pa-
tient 6 to 12 SAX slices from the atrioventricular ring
to the apex (thickness= 8 mm, gap=8 mm) are given.
The spatial resolution is 1.25 mm. For each patient
endocardial and epicardial contours were segmented
by an experienced cardiologist in all slices at ED and
ES phases. Our algorithm was implemented using the
ITK Library. Outcome results were obtained using
the same parameter set for all patients. The process,
for each patient, takes less than a minute to complete.
Comparing our method to the state of the art
was a challenging task as the MICCAI ground truth
is done by experts who have done smooth manual
segmentations. Nevertheless, we here provide seg-
mentation results visually validated by our experts.
Secondly, we use the Dice similarity coefficient
(DSC) to compare our method with one of the best
fully automatic methods of the challenge proposed
by (Jolly et al., 2009) and belonging to the category
(i). As the DSC measure the amount of overlap
between the segmentation and the ground truth, this
evaluation is not in favour of our non regularized
method. Hence, these results are only here to show
that we provide equivalent results.
Visual Assessment. Figure 4 shows segmentation re-
sults for patient SC-HF-NI-07 at ED. As can be seen,
contours are weakly regularized as expected and the
method also works for apical slices.
Figures 5 and 6 present some results from a full
sequence segmentation for patient SC-HF-I-05. We
can see that the method is quite reliable and provides
good LV contours at all time points even for apical
slices.
Quantitative Assessment. Table 1 sums up some re-
sults for each kind of patient (Heart Failure (HF)
Ischemic/Non-Ischemic, Hypokinesis and Normal)
and for Training and Validation set. As can be seen,
VISAPP2015-InternationalConferenceonComputerVisionTheoryandApplications
520
Figure 4: ED for patient SC-HF-NI-07 (basal to apical
slices).
Figure 5: Full cardiac cycle segmentation from ED to ES
and from ES to ED for patient SC-HF-I-05 basal slices.
Figure 6: Full cardiac cycle segmentation from ED to ES
and from ES to ED for patient SC-HF-I-05 apical slices.
our method gave comparable results to this method.
On 30 patients our automatic initialization fails only
for patient SC-N-40 which leads to a bad segmenta-
tion result.
As we expected, our results differ slightly from
those of Jolly. These differences flow from our need
of non regularized segmentation results and from
the use of the DSC to compare the methods. Nev-
ertheless, our results are coherent with our clinical
context, in which papillary muscles must be included
inside the myocardium for dense strain computation.
Multi-modal Perspective. Finally, to demonstrate
Table 1: Dice on Training and Validation set.
Patient
Jolly Proposed method
Endo Epi Endo Epi
SC-HF-I-02 0.89 0.94 0.9 0.91
SC-HF-NI-04 0.91 0.95 0.88 0.9
SC-HYP-03 0.90 0.94 0.85 0.89
SC-HF-I-08 0.87 0.93 0.87 0.88
SC-HF-NI-11 0.91 0.94 0.9 0.92
SC-HYP-08 0.86 0.91 0.88 0.88
SC-N-06 0.87 0.95 0.87 0.86
the versatility of our energy and its multi-modal per-
spective, Figure 7 presents successful segmentation of
endocardium and epicardium regions on a two-cavity
US slice by simply adapting the β parameters to the
US image histogram. For this test we manually ini-
tialize the model.
Figure 7: US LV segmentation (initialisation and final con-
tour).
4 CONCLUSION
We have presented a segmentation framework to
segment the LV (endorcadium/epicardium) in MRI
(3D+T). Our fully automatic deformable model cou-
ples an edge-based energy (Kovesy asymmetry mea-
sure), a region-based energy (Local/Global Weibull
model) and a shape-based energy (myocardium wall
thickness). Our results, compared to a state of the
art method, demonstrate the potential and reliability
of our approach regarding our choices: not to include
papillary muscle in the endocardium cavity and not to
use strong priors in the model. Finally, we have illus-
trated the versatility of our method, by successfully
segmenting ultrasound LV images, for multi-modal
cardiac segmentation perspective.
REFERENCES
Aubert, G., Barlaud, M., Faugeras, O., and Jehan-Besson,
S. (2003). Image segmentation using active contours:
Calculus of variations or shape gradients? SIAM Jour-
nal on Applied Mathematics, 63(6):2128–2154.
Ayed, I. B., Hennane, N., and Mitiche, A. (2006). Unsu-
pervised variational image segmentation/classification
FullyAutomaticDeformableModelIntegratingEdge,TextureandShape-ApplicationtoCardiacImagesSegmentation
521
using a weibull observation model. Image Processing,
IEEE Transactions on, 15(11):3431–3439.
Barbosa, D. et al. (2013). Fast and fully automatic 3-
d echocardiographic segmentation using b-spline ex-
plicit active surfaces: Feasibility study and validation
in a clinical setting. Ultrasound in Medicine & Biol-
ogy.
Belaid, A., Boukerroui, D., Maingourd, Y., and Lerallut,
J.-F. (2011). Implicit active contours for ultrasound
images segmentation driven by phase information and
local maximum likelihood. In Biomedical Imaging:
From Nano to Macro, 2011 IEEE International Sym-
posium on, pages 630–635.
Caselles, V., Kimmel, R., and Sapiro, G. (1997). Geodesic
active contours. International journal of computer vi-
sion, 22(1):61–79.
Chan, T. F. and Vese, L. A. (2001). Active contours with-
out edges. Image processing, IEEE transactions on,
10(2):266–277.
Felsberg, M. and Sommer, G. (2001). The monogenic
signal. Signal Processing, IEEE Transactions on,
49(12):3136–3144.
Foulonneau, A., Charbonnier, P., and Heitz, F. (2003). Ge-
ometric shape priors for region-based active contours.
In Image Processing, 2003. ICIP 2003. Proceedings.
2003 International Conference on, volume 3, pages
III–413.
Jolly, M.-P., Xue, H., Grady, L., and Guehring, J. (2009).
Combining registration and minimum surfaces for the
segmentation of the left ventricle in cardiac cine mr
images. In Medical Image Computing and Computer-
Assisted Intervention–MICCAI 2009, pages 910–918.
Kovesi, P. (1997). Symmetry and asymmetry from local
phase. pages 2–4.
Lankton, S. and Tannenbaum, A. (2008). Localizing region-
based active contours. Image Processing, IEEE Trans-
actions on, 17(11):2029–2039.
Lu, Y., Radau, P., Connelly, K., Dick, A., and Wright, G. A.
(2009). Segmentation of left ventricle in cardiac cine
mri: An automatic image-driven method. In Func-
tional Imaging and Modeling of the Heart, pages 339–
347.
Paragios, N. (2002). A variational approach for the segmen-
tation of the left ventricle in cardiac image analysis.
International Journal of Computer Vision, 50(3):345–
362.
Paragios, N. and Deriche, R. (2002). Geodesic active re-
gions and level set methods for supervised texture seg-
mentation. International Journal of Computer Vision,
46(3):223–247.
Pednekar, A., Kurkure, U., Muthupillai, R., Flamm, S., and
Kakadiaris, I. A. (2006). Automated left ventricular
segmentation in cardiac mri. Biomedical Engineering,
IEEE Transactions on, 53(7):1425–1428.
Petitjean, C. and Dacher, J.-N. (2011). A review of seg-
mentation methods in short axis cardiac mr images.
Medical image analysis, 15(2):169–184.
Rajpoot, K., Noble, A., Grau, V., and Rajpoot, N. (2008).
Feature detection from echocardiography images us-
ing local phase information. Medical Image Under-
standing and Analysis.
Sethian, J. A. (1999). Level set methods and fast marching
methods: evolving interfaces in computational geom-
etry, fluid mechanics, computer vision, and materials
science, volume 3. Cambridge university press.
VISAPP2015-InternationalConferenceonComputerVisionTheoryandApplications
522