Fully Automatic Deformable Model Integrating Edge, Texture and Shape

Application to Cardiac Images Segmentation

ement Beitone

1,3

, Christophe Tilmant

1,3

and Fr

eric Chausse

2,3

Clermont Universit

e, Univ. Blaise Pascal, Institut Pascal, BP 10448, F-63000 Clermont-Fd, France

Clermont Universit

e, Univ. d’Auvergne, Institut Pascal, BP 10448, F-63000 Clermont-Fd, France

CNRS, UMR 6602, Institut Pascal, F-63171 Aubi

ere, France

Keywords:

Fully Automatic Segmentation, Deformable Model, MRI, Weibull Model, Monogenic Signal.

Abstract:

This article presents a fully automatic left ventricle (LV) segmentation method on MR images by means of

an implicit deformable model (Level Set) in a variational context. For these parametrizations, the degrees of

freedom are: initialization and functional energy. The ﬁrst is often delegated to the practician. To avoid this

human intervention, we present an automatic initialisation method based on the Hough transform exploiting

spatio-temporal information. Generally, energetic functionals integrate edges, regions and shape terms. We

propose to bundle an edge-based energy computed by feature asymmetry on the monogenic signal, a region-

based energy capitalizing on image statistics (Weibull model) and a shape-based energy constrained by the

myocardium thickness. The presence of multiple tissues implies data non-stationarity. To best estimate dis-

tribution parameters over the regions and regarding anatomy, we propose a deformable model maximizing

locally and globally the log-likelihood. Finally, we evaluate our method on MICCAI 09 Challenge data.

1 INTRODUCTION - STATE OF

THE ART

Cardiovascular diseases are the main cause of death

on earth. According to the World Health Organiza-

tion in 2008, they are responsible for 30% of the to-

tal number of deaths. Systolic function impairment

and especially the left ventricle (LV) is one of the

main characteristics reﬂecting that the heart is dam-

aged. Quantitative analysis provides important car-

diac functional parameters for heart disease diagnosis,

for example the strain showed is reliable prognostic

value.

The evaluation of these parameters requires,

among other things, to have an accurate segmentation

result. This step has been the subject of a large num-

ber of studies: a review and an evaluation of segmen-

tation methods applied to MR images can be found in

(Petitjean and Dacher, 2011).

Authors propose the following segmentation

methods classiﬁcation regarding the level of exter-

nal information they require (strong/weak) and their

methodological approach:

Weak Priors Strong Priors

(i) Image-based (a) AAM

(ii) Pixel classiﬁcation (b) Atlas-based

(iii) Deformable models (c) Shape Prior

We propose a variational method, belonging to the

deformable model category with weak priors (iii), in

order to segment LV. There are three degrees of free-

dom: parametrization, initialization and energy func-

tional.

We decided to represent our deformable model us-

ing the level set framework because it allows topolog-

ical changes. These issues arise, for example, when

there are papillary muscles inside the endocardium

cavity.

Deformable model evolution is linked to an op-

timisation problem. The functional minimization is

Figure 1: Overall scheme of the proposed method.

517

Beitone C., Tilmant C. and Chausse F..

Fully Automatic Deformable Model Integrating Edge, Texture and Shape - Application to Cardiac Images Segmentation.

DOI: 10.5220/0005304005170522

In Proceedings of the 10th International Conference on Computer Vision Theory and Applications (VISAPP-2015), pages 517-522

ISBN: 978-989-758-089-5

 2015 SCITEPRESS (Science and Technology Publications, Lda.)

achieved using a descent method, and leads to a lo-

cal minimum dependent on the initialization. Usu-

ally, this step is delegated to the practician (Lu et al.,

2009). Nevertheless, this tedious intervention can

be automated using the geometrical properties of the

left cavity (quasi-conic shape); which can be approx-

imated by a circular shape in a basal short axis (SAX)

slice. Heart motion is also discriminant to initialize

the model (Pednekar et al., 2006).

Using a deformable model requires to design an

energy functional which depends on internal informa-

tion (regularization) and external information (image,

prior). For example, the geodesic model (Caselles

et al., 1997) uses a functional dependent on the image

gradient intensity. The key idea behind edge-based

energy is to force the model to converge on areas

where the gradient intensity is high, i.e where edges

are. The authors also proposed to work on image tex-

ture by integrating region-based energy. The seminal

Chan and Vese method (Chan and Vese, 2001) mini-

mizes inter-class variance on pixels inside/outside the

model. Some generalizations exploit statistics linked

to the physical process of the image formation. For

example, the Weibull model (Ayed et al., 2006) al-

lows to model different distributions which can be

used to ﬁnely characterise regions. To solve the non-

stationarity issue induced by region response, Lank-

ton et al. (Lankton and Tannenbaum, 2008) propose a

local region energy. In this method, statistics are eval-

uated along the deformable model. However in real

images, noise and missing data require to incorpo-

rate geometrical constraints in the method, known as

shape-based energy, to solve these issues. This prior

information can be integrated in different ways into

the variational method. For example, Foulounneau

(Foulonneau et al., 2003) uses a mean square process

over Legendre moments between the current model

shape and a reference shape. One of the main interests

of the variational context is its ability to glue different

constraints (edge, region and shape-based). This cou-

pling allows to strengthen the method by merging in-

formation sources. For example the Geodesic Active

Region (GAR) model proposed by Paragios (Paragios

and Deriche, 2002) is one of the ﬁrst to propose an

edge/region-based energy.

A large number of methods apply these energies

to segment cardiac images. Ultrasound scans (US)

uncovered speciﬁc needs for these noisy and poorly

contrasted images. Some authors developed frame-

works to replace gradient-based methods. For in-

stance, some methods use the monogenic signal and

the asymmetry measure (Felsberg and Sommer, 2001;

Kovesi, 1997) to segment cardiac ultrasound images

with an edge-based energy (Rajpoot et al., 2008). In

a region-based context Barbosa et al. (Barbosa et al.,

2013) used Lankton approach with Rayleigh statis-

tics in order to segment US images. The prior knowl-

edge of the heart geometry can be used in a shape-

based energy. This kind of anatomical constraint was

proposed by Paragios (Paragios, 2002) in MRI left

ventricle segmentation by imposing a minimal thick-

ness of the myocardium wall. Finally, Belaid (Belaid

et al., 2011) proposes to merge edge (monogenic sig-

nal) and region-based (Rayleigh distribution) energy

to perform US left ventricle segmentation.

In section 2, we present a general framework to

segment LV in MRI (3D+T)(the framework overall

is presented on ﬁgure 1). In this framework a fully

automatic initialization step is performed. Our inten-

tion is to apply the ultrasound approach to MRI by

carving data terms coherent with the underlying im-

age physics. For that, our functional couples region-

based (Weibull model), edge-based (Kovesi asym-

metry measure) and shape-based (myocardium wall

thickness) energy terms. Its particularity is that we

do not use strong prior (spatio-temporal) as in atlas

methods. In section 3, we present some results and

we compare our method with one of the best methods

of the MICCAI 09 challenge.

2 FULLY AUTOMATIC

SEGMENTATION

2.1 Deformable Model Framework

The variational formulation of the segmentation prob-

lem by means of a deformable model is stated as:

S = arg min

∗

∈F

E(S

∗

) ⇔

δE(S)

δS

= 0 (1)

In our case S corresponds to the ﬁnal shape of our

model. This shape is taken from a family of solutions

, by minimizing the energetic functional E. This

optimization problem is solved by means of descent

method on an artiﬁcial temporal parameter t. The

model is put into motion, it is a deformable model:

∂S

∂t

= −

δE

δS

= V n (2)

This problem is equivalent to a front propagation

where the variation is homogeneous to a speed V on

the normal n. The calculus of variations on E can be

computed using shape derivative tools (Aubert et al.,

2003). In the level set framework, Sethian (Sethian,

1999) showed that this problem can be stated as:

∂φ

∂t

= V |∇φ|, (3)

VISAPP2015-InternationalConferenceonComputerVisionTheoryandApplications

518

Figure 2: (a) Motion between ED and ES (b) Motion bound-

ing box (c) Model Initialisation.

where φ, the level representation, is a higher order

function and φ

−1

(0) = S .

2.2 Automatic Initialisation

To initialize our deformable model, we exploit both

geometric and motion information. These data are

more meaningful when you examine the heart: (i) on

a basal SAX slice (ii) between the end of diastole (ED

) and the end of systole (ES).

To detect the LV we use a circular Hough trans-

form on ED basal slice but we restrict voting points to

a region of interest encompassing strong motion be-

tween ED and ES. We select a basal slice in order

to be close to our circular shape hypothesis for the

LV. The diameter associated to the circular detection

is based on myocardium mean size on a basal SAX

slice, which is about 50 mm for a healthy adult heart.

For that, we restrict the detection to a range of 40 to

60 mm. Our deformable model is initialized using

the fast marching method (Sethian, 1999), on a circle

with a diameter equal to 95% of the detected diam-

eter. The whole segmentation is carried out on a 80

x 80 mm region of interest centered on the detection.

This process is illustrated on Figure 2.

For other slices we propagate the basal result, on

which we apply a scale factor following a conical hy-

pothesis (based on the LV shape). Finally, we propa-

gate the segmentation result at time t to initialize the

t + 1 model.

2.3 Functional Deﬁnition: Energy

Coupling

To segment endocardium and epicardium contours,

we use two deformable models based on the coupling

three kinds of energy region, edge and shape energies:

(

endo

= α

endo

+ α

epi

= β

epi

+ β

(4)

endo

and E

epi

correspond respectively to ener-

gies used to segment endocardium and epicardium

contours. E

endo

and E

epi

correspond to region-based

terms, E

is an edge-based term and E

is a shape

term. Parameters α

and β

are used to weigh the

inﬂuence of each term.

Edge-based energy: Our edge-based energy data

term is the same for both segmentations. This data

term is based on a geodesic model, however, we use a

step edge detector based on phase information instead

of a classic gradient-based detector:

g(FA(I

))dx (5)

where x is a given point, g(a) =

1+a

and I

the locally normalized input image. The edge de-

tector FA, is the feature asymmetry introduced by

Kovesi (Kovesi, 1997). This measure gives reliable,

well localized but incomplete edges even in noisy im-

ages. We also used the monogenic signal (Felsberg

and Sommer, 2001), the multidimensional extension

of the 1D analytic signal, to replace steerable ﬁlters

used by Kovesi as in (Belaid et al., 2011).

We apply a local normalization ﬁlter on the MRI

raw image in order to uniformize the mean and

variance of an image around local neighbourhoods.

This is especially useful for correcting non-uniform

illumination or shading artifacts.

Region-based Energy. Our region energy terms

are based on a Weibull model (Ayed et al., 2006). The

probability density function of a Weibull random vari-

able is:

(u) =





β−1

−





× I1

[0,+∞[

(u), (6)

where η is a scale parameter and β is a shape parame-

ter. This model tends to maximize the log-likelihood

of a Weibull distribution over a region R:

(R) =

log(P

(I|R))dx (7)

A local version (Lankton framework) of this

global energy might be stated, using a ball B, as:

) =

B(x;y)log(P

(I|R

))dydx (8)

Global methods provide more energy but are

meaningful if the distribution over the considered ob-

ject follows a stationary process. Nevertheless, our

images present some non-stationarity over the my-

ocardium and over its neighbourhood. Experimen-

tally, we show that the distributions over the different

regions (using expert segmentations) might be repre-

sented as in Figure 3. This ﬁgure, shows that my-

ocardium and tissues distributions can be approxi-

mated using a normal law and an exponential law.

FullyAutomaticDeformableModelIntegratingEdge,TextureandShape-ApplicationtoCardiacImagesSegmentation

519

Cavity distribution can be approximated using a nor-

mal law under the assumption that papillary muscles

are excluded from the cavity. Indeed, in Figure 3 the

intensity step between 20 and 60 comes from the in-

clusion of these muscles by the expert. We build our

region-based energy terms using these observations

and assumptions. The ﬁnal model we used is stated

as:

endo

= E

) + E

) (9)

epi

= E

) + E

) (10)

In these models, we capitalize on the relative station-

arity over the cavity to lead the contour during the ﬁrst

segmentation step and then we work more ﬁnely over

the remaining tissues (myocardium and its environ-

ment). The Weibull scale parameters for the different

regions R are computed (depending if they are global

G or local L) as:

(x) =

B(x;y)I

B(x;y)dy

(11)

0 20 40

80 100 120

2 · 10

−2

4 · 10

−2

6 · 10

−2

8 · 10

−2

0.1

Intensity

Probability

Cavity

Myocardium

Tissues

Figure 3: Regions histograms based on an expert segmen-

tation.

Shape-based Energy. Our anatomical constraint is

similar to the one introduced by Paragios (Paragios,

2002). This constraint corresponds to a coupling

force added back to the global energy to ensure the en-

docardium segmentation when region and edge-based

terms failed. This happens, for example, where the

contrast is poor between organs surrounding the heart

and the myocardium wall. This force allows to have a

non-zero wall thickness for the LV.

This term can be considered as a balloon force

with an intensity C depending on the location x and

the endocardium contour position φ

endo

(x, φ

endo

)dx (12)

We deﬁne C

, the coupling function as

(x, φ

endo

) = −1 if d(x, φ

endo

) < m, 1 if

d(x, φ

endo

) > M and 0 elsewhere. This repul-

sive force halts the contour propagation if into does

not fall in an acceptable distance range. This range

[m, M] is deﬁned using standard anatomical wall

thickness properties and is determined relatively to

the endocardium position.

3 RESULTS & PERSPECTIVES

We assessed our method on the MICCAI 2009 chal-

lenge database, which contains 15 patients for the

training set and 15 for the validation set. For each pa-

tient 6 to 12 SAX slices from the atrioventricular ring

to the apex (thickness= 8 mm, gap=8 mm) are given.

The spatial resolution is 1.25 mm. For each patient

endocardial and epicardial contours were segmented

by an experienced cardiologist in all slices at ED and

ES phases. Our algorithm was implemented using the

ITK Library. Outcome results were obtained using

the same parameter set for all patients. The process,

for each patient, takes less than a minute to complete.

Comparing our method to the state of the art

was a challenging task as the MICCAI ground truth

is done by experts who have done smooth manual

segmentations. Nevertheless, we here provide seg-

mentation results visually validated by our experts.

Secondly, we use the Dice similarity coefﬁcient

(DSC) to compare our method with one of the best

fully automatic methods of the challenge proposed

by (Jolly et al., 2009) and belonging to the category

(i). As the DSC measure the amount of overlap

between the segmentation and the ground truth, this

evaluation is not in favour of our non regularized

method. Hence, these results are only here to show

that we provide equivalent results.

Visual Assessment. Figure 4 shows segmentation re-

sults for patient SC-HF-NI-07 at ED. As can be seen,

contours are weakly regularized as expected and the

method also works for apical slices.

Figures 5 and 6 present some results from a full

sequence segmentation for patient SC-HF-I-05. We

can see that the method is quite reliable and provides

good LV contours at all time points even for apical

slices.

Quantitative Assessment. Table 1 sums up some re-

sults for each kind of patient (Heart Failure (HF)

Ischemic/Non-Ischemic, Hypokinesis and Normal)

and for Training and Validation set. As can be seen,

VISAPP2015-InternationalConferenceonComputerVisionTheoryandApplications

520

Figure 4: ED for patient SC-HF-NI-07 (basal to apical

slices).

Figure 5: Full cardiac cycle segmentation from ED to ES

and from ES to ED for patient SC-HF-I-05 basal slices.

Figure 6: Full cardiac cycle segmentation from ED to ES

and from ES to ED for patient SC-HF-I-05 apical slices.

our method gave comparable results to this method.

On 30 patients our automatic initialization fails only

for patient SC-N-40 which leads to a bad segmenta-

tion result.

As we expected, our results differ slightly from

those of Jolly. These differences ﬂow from our need

of non regularized segmentation results and from

the use of the DSC to compare the methods. Nev-

ertheless, our results are coherent with our clinical

context, in which papillary muscles must be included

inside the myocardium for dense strain computation.

Multi-modal Perspective. Finally, to demonstrate

Table 1: Dice on Training and Validation set.

Patient

Jolly Proposed method

Endo Epi Endo Epi

SC-HF-I-02 0.89 0.94 0.9 0.91

SC-HF-NI-04 0.91 0.95 0.88 0.9

SC-HYP-03 0.90 0.94 0.85 0.89

SC-HF-I-08 0.87 0.93 0.87 0.88

SC-HF-NI-11 0.91 0.94 0.9 0.92

SC-HYP-08 0.86 0.91 0.88 0.88

SC-N-06 0.87 0.95 0.87 0.86

the versatility of our energy and its multi-modal per-

spective, Figure 7 presents successful segmentation of

endocardium and epicardium regions on a two-cavity

US slice by simply adapting the β parameters to the

US image histogram. For this test we manually ini-

tialize the model.

Figure 7: US LV segmentation (initialisation and ﬁnal con-

tour).

4 CONCLUSION

We have presented a segmentation framework to

segment the LV (endorcadium/epicardium) in MRI

(3D+T). Our fully automatic deformable model cou-

ples an edge-based energy (Kovesy asymmetry mea-

sure), a region-based energy (Local/Global Weibull

model) and a shape-based energy (myocardium wall

thickness). Our results, compared to a state of the

art method, demonstrate the potential and reliability

of our approach regarding our choices: not to include

papillary muscle in the endocardium cavity and not to

use strong priors in the model. Finally, we have illus-

trated the versatility of our method, by successfully

segmenting ultrasound LV images, for multi-modal

cardiac segmentation perspective.

REFERENCES

Aubert, G., Barlaud, M., Faugeras, O., and Jehan-Besson,

S. (2003). Image segmentation using active contours:

Calculus of variations or shape gradients? SIAM Jour-

nal on Applied Mathematics, 63(6):2128–2154.

Ayed, I. B., Hennane, N., and Mitiche, A. (2006). Unsu-

pervised variational image segmentation/classiﬁcation

FullyAutomaticDeformableModelIntegratingEdge,TextureandShape-ApplicationtoCardiacImagesSegmentation

521

using a weibull observation model. Image Processing,

IEEE Transactions on, 15(11):3431–3439.

Barbosa, D. et al. (2013). Fast and fully automatic 3-

d echocardiographic segmentation using b-spline ex-

plicit active surfaces: Feasibility study and validation

in a clinical setting. Ultrasound in Medicine & Biol-

ogy.

Belaid, A., Boukerroui, D., Maingourd, Y., and Lerallut,

J.-F. (2011). Implicit active contours for ultrasound

images segmentation driven by phase information and

local maximum likelihood. In Biomedical Imaging:

From Nano to Macro, 2011 IEEE International Sym-

posium on, pages 630–635.

Caselles, V., Kimmel, R., and Sapiro, G. (1997). Geodesic

active contours. International journal of computer vi-

sion, 22(1):61–79.

Chan, T. F. and Vese, L. A. (2001). Active contours with-

out edges. Image processing, IEEE transactions on,

10(2):266–277.

Felsberg, M. and Sommer, G. (2001). The monogenic

signal. Signal Processing, IEEE Transactions on,

49(12):3136–3144.

Foulonneau, A., Charbonnier, P., and Heitz, F. (2003). Ge-

ometric shape priors for region-based active contours.

In Image Processing, 2003. ICIP 2003. Proceedings.

2003 International Conference on, volume 3, pages

III–413.

Jolly, M.-P., Xue, H., Grady, L., and Guehring, J. (2009).

Combining registration and minimum surfaces for the

segmentation of the left ventricle in cardiac cine mr

images. In Medical Image Computing and Computer-

Assisted Intervention–MICCAI 2009, pages 910–918.

Kovesi, P. (1997). Symmetry and asymmetry from local

phase. pages 2–4.

Lankton, S. and Tannenbaum, A. (2008). Localizing region-

based active contours. Image Processing, IEEE Trans-

actions on, 17(11):2029–2039.

Lu, Y., Radau, P., Connelly, K., Dick, A., and Wright, G. A.

(2009). Segmentation of left ventricle in cardiac cine

mri: An automatic image-driven method. In Func-

tional Imaging and Modeling of the Heart, pages 339–

347.

Paragios, N. (2002). A variational approach for the segmen-

tation of the left ventricle in cardiac image analysis.

International Journal of Computer Vision, 50(3):345–

362.

Paragios, N. and Deriche, R. (2002). Geodesic active re-

gions and level set methods for supervised texture seg-

mentation. International Journal of Computer Vision,

46(3):223–247.

Pednekar, A., Kurkure, U., Muthupillai, R., Flamm, S., and

Kakadiaris, I. A. (2006). Automated left ventricular

segmentation in cardiac mri. Biomedical Engineering,

IEEE Transactions on, 53(7):1425–1428.

Petitjean, C. and Dacher, J.-N. (2011). A review of seg-

mentation methods in short axis cardiac mr images.

Medical image analysis, 15(2):169–184.

Rajpoot, K., Noble, A., Grau, V., and Rajpoot, N. (2008).

Feature detection from echocardiography images us-

ing local phase information. Medical Image Under-

standing and Analysis.

Sethian, J. A. (1999). Level set methods and fast marching

methods: evolving interfaces in computational geom-

etry, ﬂuid mechanics, computer vision, and materials

science, volume 3. Cambridge university press.

VISAPP2015-InternationalConferenceonComputerVisionTheoryandApplications

522