A Non-Local Diffusion Saliency Model for Magnetic Resonance Imaging

I. Ram

ırez

, G. Galiano

, N. Malpica

and E. Schiavi

Dpt. of Applied Mathematics, Material Sciences and Electronic Technology, King Rey Juan Carlos University,

C/Tulip

an s/n, 28933, M

ostoles, Madrid, Spain

Dpt. of Mathematics, Oviedo University. c/ Calvo Sotelo, 33007-Oviedo, Spain

Keywords:

Saliency Detection, Segmentation, Non-local Diffusion, Non-convex Optimization, Bilateral Filtering,

P-laplacian Operator, MRI, FLAIR, Tumor, Edema.

Abstract:

Based on previous work on image classiﬁcation and recent applications of non-local non-linear diffusion equa-

tions, we propose a non-local p-laplacian variational model for saliency detection in digital images. Focusing

on the range 0 < p < 1 we also consider the regularized non-convex ﬂuxes generated by the related hyper-

laplacian diffusion operators. With the aim of exploring the properties and potential applications of such

non-local, non-convex operators the model is applied to Magnetic Resonace Imaging (MRI) for Fluid Attenu-

ated Inversion Recovery image (FLAIR) modality showing promising numerical results. In this work Saliency

shall be understood as the relevant, outstanding region in a FLAIR image, which is commonly the brightest

part. It corresponds to a tumor and neighborhood edema. Our preliminary experiments show that the proposed

model can achieve very accurate results in this modality in terms of all the considered metrics.

1 INTRODUCTION

There is a general, growing interest in digital im-

age processing and computer vision applications for

(Visual) Saliency based models able to focus on per-

ceptually relevant information in digital images. Se-

mantic segmentation, object detection, object propos-

als, image clustering, retrieval and cognitive saliency

applications such as image captioning and high-

level image understanding are just few examples of

saliency based models. Saliency is also of interest

as a means to improve computationally efﬁciency and

increases robustness to clustering and threshold. De-

spite the lack of a general consensus on a proper math-

ematical deﬁnition of Saliency, it has a biologically

perceptive meaning whereas it models the mechanism

of human attention and consists of the task of ﬁnding

interesting objects in an image with or without prior

knowledge. It turns out that the right role of saliency

in models is then application dependent and this limit

the scope of general purpose methods. Recently there

is been a burst of research on Saliency due to its wide

application in leading medical disciplines such as

Neuroscience and Cardiology. In fact, when consid-

ering medical images such as those acquired in Mag-

netic Resonance Imaging (MRI) or Positron Emis-

sion Tomography (PET), the automatic obtainment

of saliency maps, can be useful for pathology detec-

tion, disease classiﬁcation (Rueda et al., 2013), loca-

tion and segmentation of brain strokes, gliomas, my-

ocardium detection for PET images, tumors quantiﬁ-

cation in FLAIR MRI (Thota et al., 2016) etc. Several

different techniques and approaches have been ap-

plied to construct saliency maps. They vary from low

dimensional manifold features minimization (Zhan,

2011), non-local sparse minimization (Wang et al.,

2014), graphs techniques (Harel et al., 2006), Partial

Differential Equations (PDE) (Li et al., 2013), super-

pixel (Liu et al., 2013), learning (Liu et al., 2014) and

Neural Networks based approaches (Bylinskii et al., )

(MIT-Benchmark).

With the aim to explore the applications and al-

gorithms of non-smooth, non-local, non-convex opti-

mization to PDE saliency based models, we present

in these notes, a new non-Local diffusion model for

PDE applied to Fluid Attenuated Inversion Recovery

(FLAIR) MR images for accurate location of tumor

and edema. The basis for the modeling exercise re-

lies in a recent model proposed by Li (Li et al., 2013)

for natural images saliency detection. Recent results

which justiﬁes the interest for this applications can be

found in (Thota et al., 2016) and multichannel pseudo

RGB (Banerjee et al., 2016). PDE based variational

methods have provided to be effective to model gen-

eral low level computer vision processing tasks (de-

noising, restoration, deblurring, segmentation, super-

100

RamÃ rez I., Galiano G., Malpica N. and Schiavi E.

A Non-Local Diffusion Saliency Model for Magnetic Resonance Imaging.

DOI: 10.5220/0006172101000107

In Proceedings of the 10th International Joint Conference on Biomedical Engineering Systems and Technologies (BIOSTEC 2017), pages 100-107

ISBN: 978-989-758-215-8

resolution) allowing speciﬁc solutions to be obtained

based on Bayesian modelling. When ﬁrst order nec-

essary optimality conditions are imposed on the non-

linear energy functional modeling data and a priori

knowledge, a PDE arises which has to be numerically

solved. Primal gradient-descent is a ﬁrst step for a

problem understanding. Alternatively non-linear dif-

fusion processing has been shown to be a valid ﬁl-

tering data processing method to design speciﬁc pre-

processing steps for further analysis. In any case, we

are leaded to solve non-linear (local) parabolic diffu-

sion equations. Anisotropy is an important character-

istic which can also be modeled in such a local ﬂux

based diffusive processes, (Weickert, 1998). Nev-

ertheless, when non-local interactions are important

for pathology detection, non-local diffusion equations

have to be considered. Non-local integro-differential

equations have a long history, since the Fredholm in-

tegral equations of the ﬁrst kind, but a great deal of

interest has been recently triggered due to its applica-

tions in image processing (Gilboa and Osher, 2008).

Based on such arguing we propose to extend the

local p−laplacian classiﬁcation model of (Samson

et al., 1998) to a non-local framework for saliency

detection, and the non-local model proposed by (Li

et al., 2013) for p > 2 to include non-convex ﬂuxes

related to the range 0 < p < 1. This paper is orga-

nized as follows. In section (2), based on the ap-

proach of (Samson et al., 1998) for image classiﬁca-

tion, we deﬁne a local variational model which in-

cludes a new term for saliency detection. It generates

an absorption-reaction balance which drives the dif-

fusion process promoting a two classes binary image

labeling. Remarkably, such a term has a concave en-

ergy leading to a fast saliency detection. We also de-

ﬁne an approximating family of edge-preserving po-

tential functions which allows to describe non-convex

diffusive ﬂuxes. In section (3) we introduce the non-

local framework for PDE (Andreu-Vaillo et al., 2010).

Based on the approach of (Li et al., 2013), we deﬁne a

new variational model with the non-local p-laplacian

operator that provides non-local diffusion, combined

with a ﬁdelity term coming from the Restoration

Bayesian framework, and a saliency term, inspired

by the Ginzburg-Landau (Aubert et al., 2005) super-

conductor theory. Non-local diffusion has multiple

applications in the literature among which stands out

the detection and preserving patterns capacity. How-

ever, when non-local non-convex ﬂux is acting, the

non-convexity imposed (as a priori knowledge) with

0 < p < 1, forces the non-local gradient to be sparse,

in so far it minimizes the number of jumps (big non-

local gradient modulus) in the image domain. Actu-

ally, only sharp jumps are preserved, and the result-

ing image looks like a cartoon piece-wise constant

image. In section (4) we describe the numerical dis-

cretization using a semi-implicit Euler scheme com-

bined with a quantization step to speed up the com-

putations (Yang et al., 2009). The ﬁnal saliency mask

is ﬁnally computed by a simple thresholding step. A

mask is generated to compare with the ground-truth.

Finally, in section 5 we present some preliminary re-

sults which show the promising performance of the

model in terms of accuracy and speed when applied

to FLAIR sequences of MR images.

2 LOCAL DIFFUSION

EQUATIONS

We start with a Bayesian Restoration model for image

classiﬁcation which produces a Maximum a Posteri-

ori (MAP) estimation of the solution through the min-

imization of a nonlinear energy functional in Ω ⊂R

Let λ and α be ﬁxed positive parameters. We consider

the functional (Samson et al., 1998)

ε,p

(u) = αJ

ε,p

(u) +

H(u) + λF(u) = (1)

= α

Ω

ε,p

(|∇u|)dx +

Ω

h(u)dx+

Ω

|Ru − f |

where h(u) is typically a quadratic function known as

a double-well potential function which we shall pre-

cise later on. The functions

ε,p

(s) =



+ ε



p/2

−

(2)

deﬁne an approximating family that allows to over-

come the no differentiability when 0 < p ≤ 1. When

ε → 0 we have φ

ε,p

(s) → φ

0,p

(s) = (2/p)|s|

. When

ε > 0 the φ

ε,p

family is differentiable for any p and

non-convex for 0 < p < 1. In fact, it is concave except

at a thin layer of thickness ε near the origin (see Fig-

ure 1). Despite of the non convexity, the φ

ε,p

-family is

edge-preserving (Charbonnier et al., 1997). This fam-

ily satisﬁes φ

ε,p

(0) = 0 for any p,ε. We shall assume

0 < p ≤ 1 and ε > 0. We have

ε,p

(s) = 2s



+ ε



(p/2)−1

and then φ

ε,p

(0) = 0 for any p, ε and φ

ε,p

(s) > 0 for

s > 0 and ∀p,ε. When h(u) ≡ 0 the following re-

sults are well known. For parametric values p > 1 the

energy E

ε,p

(u) deﬁned in (1) is convex and differen-

tiable presenting a unique global minimum. The dif-

fusive local ﬂux over-smooths the solutions and blurs

A Non-Local Diffusion Saliency Model for Magnetic Resonance Imaging

101

the edges. For p = 1 and R ≡ I

we recover the cele-

brated ROF model (Rudin et al., 1992). The energy is

convex but no differentiable. We have again a unique

global minimum (Chambolle and Lions, 1997) in a

weak sense, say L

∞

(Ω)∩BV (Ω), where BV (Ω) is the

space of Bounded Variation Functions. This space in-

cludes discontinuous functions and its gradient is a

Bounded Radon measure. Details on the geometric

measure theory underlying this notions can be found

in the book of (Ambrosio et al., 2000).

Notice that when functionals like (1) are used

for image classiﬁcation, partitioning and labeling, the

set of functions on which we minimize (for exam-

ple, piece-wise constant binary functions taking val-

ues {0, 1}) is not convex and the overall minimiza-

tion problem is not convex despite of the convexity of

the energy functional. Moreover, as shown in (Sam-

son et al., 1998), the solutions need not to be piece-

wise constant for α > 0, and this suggests to look for

solutions in the closed, convex interval [0,1]. When

α → 0 the Γ-convergence theory (so far proved only

for p > 1) assures the existence of piece-wise constant

binary solutions.

For 0 < p < 1 the energy is non-convex (in fact

concave) and non-smooth. No much is known the-

oretically in this range because the functional is not

differentiable and we are working with the so called

p−quasi-norms of the L

(Ω) spaces and the direct

method of variational calculus cannot be applied any-

more. The problem lacks of a sound mathematical

theory. A differential approximation is in order to

get insight into the numerical behavior of its solu-

tions. Let |∇u|

= |∇u|

+ ε

. The associated Euler-

Lagrange equation is

−αdiv

ε,p

(|∇u|)

|∇u|

∇u

(u)+λR

∗

(Ru− f ) = 0 (3)

for p > 0, λ > 0 ﬁxed and f ∈ L

∞

(Ω) given. If

p > 1 we can set ε = 0 and no regularization of

the diffusion operator takes place. For p = 1 the

1−Laplacian operator is considered and duality ar-

guments (Chambolle, 2004) have to be invoked to

properly solve the exact (not ε−approximating) TV

minimization problem. For 0 < p < 1 the problem is

open even with h(u) = 0. Remarkably, Hinterm

uller

has recently shown the existence of a viscosity solu-

tion (Hinterm

uller and Wu, 2014). The term h

(u) we

propose for saliency detection is a simple linear func-

tional given by

(u) = δ(1 −δu) (4)

which is the Frechet differential of the function

0 0.2 0.4

0.6

0.8 1

Saddle Point ˆs =

√

2ε

ε,p

(s)

ε,p

(s)

Figure 1: A plot of a member of the φ

ε,p

family. The saddle

point deﬁnes the transition from the convex initial layer to

a concave proﬁle.

h(u) = −

(1 −δu)

(5)

where δ is a real positive parameter which can be con-

veniently ﬁxed to deal with the variability of natu-

ral images. Contrary to the common usage of convex

quadratic terms to model double-well for image clas-

siﬁcation (Aubert and Kornprobst, 2006), we propose

in (5) a concave quadratic energy term

H(u) =

Ω

h(u)dx = −

Ω

(1 −δu)

In fact, the minimum of h(u) when u ∈ [0,1] is

min

u∈[0,1]

h(u) = min

{

h(0),h(1)

}

= m

where m = h(0) for 0 < δ < 2, m = h(0) = h(1) for

δ = 2 and m = h(1) for δ > 2. In any case, the min-

imum m is attained at the boundary of the interval

[0,1], which are the labels we impose for saliency

detection (foreground) u = 1, and u = 0 for back-

ground. The term H(u) promotes the detection of

salient regions of interest (foreground) separated by

regions with no relevant information (background)

while keeping the edges and details of the given data

image f ∈L

∞

(Ω) thanks to the edge-preserving prop-

erties of the φ

ε,p

-family.

Finally, the ﬁdelity term F(u) models image

restoration and can be used to perform joint recon-

struction, deconvolution, super-resolution, denosing

of the given data. In the application we are concerned

with we simply set R ≡ I

The Euler-Lagrange equation (3) is com-

plemented, as usual in image processing, with

homogeneous Neumann boundary conditions on ∂Ω.

Using a gradient descent we are leaded to

BIOIMAGING 2017 - 4th International Conference on Bioimaging

102

= αdiv

ε,p

(|∇u|)

|∇u|

∇u

(δu −1)−λ(u − f )

(6)

which is for p > 1 the classical p−laplacian diffusion

operator

= α∆

u +

(δu −1) −λ(u − f ) (7)

The lower order r.h.s term models a linear

absorption-reaction balance which can produce dif-

ferent asymptotic behaviors depending on the chosen

parametric values. In fact we have

(δu −1) −λ(u − f ) =



−λ



u +



λ f −



= au + b

Typical values used in the simulation, λ ≈ 0.1, α ≈

0.5, δ ≈ 2, ρ ≈ 20, give a ≈ 7.9 and b ≈ −3.9. As a

result, the linear term au +b models absorption when

u is positive and small, say u ≈ 0 (background), and

it acts as a reaction term in the region where u ≈ 1

(foreground). This mechanism pushes the solution

toward the labels of the two classes allowing a sim-

ple threshold for the ﬁnal binary mask targeting the

ground-truth image.

3 NON-LOCAL DIFFUSION

EQUATIONS

We ﬁrst brieﬂy introduce the mathematical frame-

work of Andreu et al. which deﬁnes, for 1 ≤ p ≤ +∞

the non-local p-laplacian diffusion operators. Then, in

the framework of bilateral ﬁltering, we extend the ﬁl-

ter to include diffusion parameters 0 < p < 1. We start

with considering ﬂux functions φ

(s) = (2/p)|s|

for

non-local diffusion. For p > 1 they are strictly con-

vex and differentiable, for p = 1 we have a con-

vex, sub-differentiable function and for 0 < p < 1 we

are not aware of any mathematical result about exis-

tence of variational solutions. Nevertheless many pa-

pers report on the improved numerical performance

of non-convex functions in terms of quality and nu-

merical efﬁciency. This has been observed in the lo-

cal framework where these functions are considered,

when 0 < p < 1, as edge-preserving functions. In the

non-local case this is, so far, unexplored.

3.1 Non-local Strictly Convex Flows

This corresponds to φ

ε=0,p

(s) = φ

(s) = (2/p)|s|

with p > 1. The non-local analogous of the p-

laplacian operator ∆

u is given by

(u) =

Ω

w(x −y)φ

(u(y) −u(x))dy = (8)

Ω

w(x −y)|u(y) −u(x)|

p−2

(u(y) −u(x))dy

which is the Frechet derivative (for p > 1) of

(u) =

Ω×Ω

w(x −y)φ

(u(y) −u(x))dydx =

(9)

Ω×Ω

w(x −y)|u(y) −u(x)|

dydx

As typical in bilateral ﬁltering theory (Tomasi and

Manduchi, 1998) we deﬁne the w(x −y) function in

form

w(x −y) =





−

(|x−y|)

, (x,y) ∈ Ω ×Ω

where C is a normalizing constant.

While the corresponding local case is not to much

relevant to the image processing community because

of its regularizing effect on the solutions which pro-

vokes artiﬁcial smoothing due to elliptic regularity

theory, there is an important result of (Andreu-Vaillo

et al., 2010) which assures that, for the related non-

local diffusion case, the data and the solution space

are the same and no regularization takes efect. This

intriguing effect is due to non-local diffusion and can

be related to recent bilateral ﬁlter theory and applica-

tion.

3.2 Approximating Non-local

Non-convex Flows

When 0 < p ≤ 1 we use the already presented

ε,p

−family (2). Denote x ∈ Ω as a pixel in the im-

age and y ∈ Ω as a pixel in the neighborhood N(x) of

x, u(x) and u(y) as the corresponding range values of

pixel x and y. We consider the functional

ε,p

(u) =

Ω×Ω

w(x −y)φ

ε,p

(u(y) −u(x))dxdy =

(10)

Ω×Ω

w(x −y)



|u(y) −u(x)|

+ ε



p/2

dxdy+

−

Ω×Ω

w(x −y)dxdy

When u ≡ 0 we have u(y) = u(x) for any x,y ∈ Ω and

ε,p

(0) = 0. In the limit

A Non-Local Diffusion Saliency Model for Magnetic Resonance Imaging

103

lim

ε→0

ε,p

(u) = J

0,p

(u) =

Ω×Ω

w(x −y)|u(y) −u(x)|

dxdy

which is the non-local analogous of to the energy

functional associated to the local p-laplacian operator

(u) =

Ω

|∇u|

as before the derivative of the approximated non-local

functional (10) can be formally written as

ε,p

(u) =

Ω

w(x−y)φ

ε,p

(u(y)−u(x))dy = (11)

Ω

w(x −y)(|u(y)−u(x)|

+ε

)

p−2

(u(y)−u(x))dy

Using the gradient descent method to minimize the

functional J

ε,p

(u) we arrive at the evolution equation

(x,t) =

Ω

w(x −y)φ

ε,p

(u(y,t) −u(t, x))dy

where

ε,p

(u(y,t) −u(x,t)) =



|u(y,t) −u(x,t)|

+ ε



p−2

(u(y,t) −u(x,t))

We deﬁne the kernel

ε,p

(s) =

ε,p

(s) = s



+ ε



p−2

and ﬁnally write

(x,t) = K

ε,p

(u)(x,t) = (12)

Ω

w(x −y)k

ε,p

(u(y,t) −u(x,t))dy

which is complemented with initial data u(x,0) =

(x),x ∈ Ω. As remarked in (Andreu-Vaillo et al.,

2010), the homogeneous Neumann boundary condi-

tions are implicitly imposed by the non-local diffu-

sion operator K

ε,p

. The equation above models a non-

linear non-local non-convex p−laplacian bilateral ﬁl-

ter. This will be used in the next section to present

ﬁnal joint model for saliency detection and restora-

tion.

3.3 Proposed Model

We present the general model valid for non-local

strictly convex ﬂows (ε = 0, p > 1) and the approxi-

mated non-local non-convex ﬂows (ε > 0, 0 < p < 1).

We start with the analogous non-local energy func-

tional

ε,p

(u) = αJ

ε,p

(u) +

H(u) + λF(u) (13)

Using the deﬁnitions introduced in section (2) regard-

ing the saliency and the ﬁdelity terms, we consider the

energy functional J

ε,p

(u) deﬁned in (10). The non-

local Euler Lagrange equation reads

−

Ω

w(x −y)φ

ε,p

(u(y) −u(x))dy+ (14)

(u) + λ(u − f ) = 0

and the corresponding gradient ﬂow evolution equa-

tion is

Ω

w(x −y)φ

ε,p

(u(y,t) −u(x,t))dy+ (15)

−

(u) −λ(u − f )

which is in term of kernel k

ε,p

(12)

Ω

w(x −y)k

ε,p

(u(y,t) −u(x,t))dy+ (16)

−

(u) −λ(u − f )

4 NUMERICAL RESOLUTION

We solve the non-local p-laplacian diffusion equa-

tion (16) using the approach proposed in (Yang et al.,

2009). This is based on the quantization process

which allows to reduce diffusion term to a convolu-

tion operation over each quantized value. In fact, this

technique produces solutions in a quantized range of

values which is suitable for our proposal which is to

separate and detecting saliency (u ≈ 1) from back-

ground (u ≈ 0). Such technique allows to speed up

the bilateral non-local ﬁltering and labeling. We then

address the main saliency detection problem with a

primal gradient descent numerical scheme.

4.1 Iterative Scheme

We brieﬂy describe the quantization step which lin-

earizes the model Let q = (q

) be a quantized vector,

q ∈ R

, where Q is the number of quantization lev-

els and q

∈{0, ...,1} are the quantized values. More-

over we denote u

(x) = u(x,t) for t

∈[(k −1)∆t,k∆t)

and then K

ε,p

(u)(x,t) = K

ε,p

)(x). For k = 0 we de-

ﬁne a quantized version of the given data u(x,0) =

(x) ∈ {0,...,1}. Notice that the original FLAIR im-

ages we consider have a 16 depth-bit resolution which

gives 65536 possible gray levels. For computational

purposes we reduce the date image to a 8 depth-bit

resolution corresponding to 256 values. This ﬁx the

number of quantization levels Q = 256 and allows the

BIOIMAGING 2017 - 4th International Conference on Bioimaging

104

exact computation of the reconstructed quantized so-

lution. For each q

, i = 1..Q we introduce and com-

pute the functions

ε,p

(x)) = (k

ε,p

−q

) ∗w)(x) =

Ω

w(x −y)k

ε,p

(y) −q

)dy =

Ω

w(x −y)(|u

(y) −q

+ ε

)

p−2

(y) −q

)dy

where K

ε,p

) = k

ε,p

−q

)∗w is a convolution op-

erator. The contribution of each q

is then summed up

recover the effect of diffusion at any x ∈ Ω. For any x

such that u

(x) = q

we deﬁne F(u(x)) as follows

(x)) =

(

ε,p

(x)), if u

(x) = q

0, otherwise

so that

F(u

(x)) =

∑

(x))

The above formalism leads to a semi-implicit scheme

(for the sake of readability we write u

= u

(x)) using

a ﬁrst-order forward Euler time discretization in (16).

We have

k+1

= u

+ ∆tαF(u

) −∆t

(1 −δu

k+1

)+ (17)

−∆tλ(u

k+1

− f )

Rearranging terms we have the following explicit

scheme. Given a data image f and parameters α, λ,

δ we ﬁx ∆t and set k = 0, u

= f . For any k = 1..T

compute:



1 −

+ λ∆t



k+1

= u

+∆t



αF(u

) + λ f −



Notice that, as typical in bilateral ﬁltering theory, the

diffusion time T is small because of the non-local dif-

fusion.

4.2 Results

Signal in MR images is high or low (bright or dark),

depending on the pulse sequence used, and the type of

tissue in the image region of interest. In this work we

present our preliminary results obtained solving the

model equation (16) when applied to a FLAIR set of

images from the BRATS2015 dataset (Menze et al.,

2014). For all tests we have ﬁxed the parameters:

λ = 0.1, Q = 256, ρ = 20, α = 0.5, ε = 1e −6, and

for each image 1,2, 3,4, we choose δ = 2,1.5,2.5,1.5

respectively. The parameters of the numerical dis-

cretization are summarized in table (1), where ∆t is

the time discretization step, T = k∆t corresponds to

Table 1: Numerical parameters of the simulation.

p = 0.1 p = 0.5 p = 1 p = 2 p=3

∆t 0.0025 0.005 0.01 0.01 0.01

k 80 40 30 20 20

T 0.2 0.2 0.3 0.4 0.4

Table 2: Statistics for different values of p.

p = 0.1 Image Precision Recall DICE

1 0,959 0,916 0,937

2 0,964 0,909 0,936

3 0,984 0,876 0,927

4 0,973 0,887 0,928

Average 0,970 0,897 0,932

p = 0.5 Image Precision Recall DICE

1 0,895 0,955 0,924

2 0,935 0,915 0,925

3 0,974 0,878 0,923

4 0,938 0,920 0,929

Average 0,935 0,917 0,925

p = 1 Image Precision Recall DICE

1 0,862 0,958 0,908

2 0,911 0,919 0,915

3 0,956 0,901 0,928

4 0,932 0,932 0,932

Average 0,915 0,928 0,921

p = 2 Image Precision Recall DICE

1 0,854 0,959 0,903

2 0,868 0,924 0,895

3 0,932 0,911 0,921

4 0,898 0,935 0,916

Average 0,888 0,932 0,909

p = 3 Image Precision Recall DICE

1 0,853 0,959 0,903

2 0,835 0,930 0,880

3 0,926 0,912 0,919

4 0,857 0,939 0,896

Average 0,868 0,935 0,900

the total diffusion time and k is the number of itera-

tions. The results are summarized in table (2) where

we used the typical statistics and performance metrics

for saliency and segmentation: Accuracy, Sensitivity,

Speciﬁcity, G-mean, Precision, Recall and DICE. In

all cases the results are very promising and show the

successful application of the method, as indicated by

the DICE coefﬁcient, is associated to small values of

p. In the non-convex case, the Precision is higher than

the Recall, and in the convex case it is exactly the op-

posite. The accuracy is in all cases extremely high.

A Non-Local Diffusion Saliency Model for Magnetic Resonance Imaging

105

Figure 2: Four FLAIR candidates from BRATS2015 data-set (Menze et al., 2014) (rows). From left to right (columns):

original image, output image, perimeter of the segmentation of the output image, output binary mask, perimeter of the ground

truth, ground truth segmentation, ground truth and output binary mask overlap (TP-green, TN-black, FP-blue, FN-yellow).

5 CONCLUSIONS

In this work we have presented a new non-local non-

convex diffusion model for saliency detection and la-

beling which has shown to be able to perform a fast

foreground detection when it is applied to a FLAIR

given image. The results reveal that this method

can achieve very high accurate statistics metrics over

the ground-truth BRATS2015 data-set (Menze et al.,

2014). Also, as a by-product of the model, the so-

lution has, after few iterations, a reduced number

of quantized values making simpler the ﬁnal thresh-

olding step. Such a technique could be improved

computationally by observing that the diffusion pro-

cess combined with the saliency term evolves produc-

ing more cartoon like piece-wise constant solutions

which can be coded with less number of quantization

values while converging to a binary mask. This is re-

lated to the absorption-reaction balance in the PDE

where absorption is active where the solution is small

and the reaction is active where u ≈1. The non-local

diffusion properties of the model also allow to de-

tect salient objects which are not spatially close and

connected regions (disjoint areas). This can be use-

ful in many other medical images modalities, spe-

cially in functional MRI (fMRI). Non-convex prop-

erties, meanwhile, promote sparse non-local gradi-

ent, pushing the solution to a cartoon piece-wise con-

stant image. Both characteristics combined with our

proposed concave energy term results in a promising

accurate and fast technique suitable to be applied to

FLAIR images and others MRI modalities.

ACKNOWLEDGMENTS

This research has been partially supported by

the Spanish Government research funding ref.

MINECO/FEDER TIN2015-69542-C2-1 and the

Banco de Santander and Universidad Rey Juan Car-

los Funding Program for Excellence Research Groups

ref. Computer Vision and Image Processing (CVIP).

REFERENCES

Ambrosio, L., Fusco, N., and Pallara, D. (2000). Functions

of bounded variation and free discontinuity problems.

Oxford university press.

Andreu-Vaillo, F., Maz

on, J. M., Rossi, J. D., and Toledo-

Melero, J. J. (2010). Nonlocal diffusion problems, vol-

ume 165. American Mathematical Society.

Aubert, G., Aujol, J.-F., and Blanc-F

eraud, L. (2005).

Detecting codimensiontwo objects in an image with

BIOIMAGING 2017 - 4th International Conference on Bioimaging

106

ginzburg-landau models. International Journal of

Computer Vision, 65(1-2):29–42.

Aubert, G. and Kornprobst, P. (2006). Mathematical prob-

lems in image processing: partial differential equa-

tions and the calculus of variations, volume 147.

Springer Science & Business Media.

Banerjee, S., Mitra, S., Shankar, B. U., and Hayashi, Y.

(2016). A novel gbm saliency detection model using

multi-channel mri. PloS one, 11(1):e0146388.

Bylinskii, Z., Judd, T., Borji, A., Itti, L., Durand, F., Oliva,

A., and Torralba, A. Mit saliency benchmark.

Chambolle, A. (2004). An algorithm for total variation min-

imization and applications. Journal of Mathematical

imaging and vision, 20(1-2):89–97.

Chambolle, A. and Lions, P.-L. (1997). Image recovery via

total variation minimization and related problems. Nu-

merische Mathematik, 76(2):167–188.

Charbonnier, P., Blanc-F

eraud, L., Aubert, G., and Barlaud,

M. (1997). Deterministic edge-preserving regulariza-

tion in computed imaging. IEEE Transactions on im-

age processing, 6(2):298–311.

Gilboa, G. and Osher, S. (2008). Nonlocal operators with

applications to image processing. Multiscale Model-

ing & Simulation, 7(3):1005–1028.

Harel, J., Koch, C., and Perona, P. (2006). Graph-based vi-

sual saliency. In Advances in neural information pro-

cessing systems, pages 545–552.

Hinterm

uller, M. and Wu, T. (2014). A smoothing descent

method for nonconvex tvˆ q-models. In Efﬁcient Algo-

rithms for Global Optimization Methods in Computer

Vision, pages 119–133. Springer.

Li, M., Zhan, Y., and Zhang, L. (2013). Nonlocal variational

model for saliency detection. Mathematical Problems

in Engineering, 2013.

Liu, R., Cao, J., Lin, Z., and Shan, S. (2014). Adaptive par-

tial differential equation learning for visual saliency

detection. In Proceedings of the IEEE Conference

on Computer Vision and Pattern Recognition, pages

3866–3873.

Liu, Z., Meur, L., and Luo, S. (2013). Superpixel-based

saliency detection. In 2013 14th International Work-

shop on Image Analysis for Multimedia Interactive

Services (WIAMIS), pages 1–4. IEEE.

Menze, B., Jakab, A., Bauer, S., Kalpathy-Cramer, J., Fara-

hani, K., Kirby, J., Burren, Y., Porz, N., Slotboom,

J., Wiest, R., Lanczi, L., Gerstner, E., Weber, M.-

A., Arbel, T., Avants, B., Ayache, N., Buendia, P.,

Collins, L., Cordier, N., Corso, J., Criminisi, A., Das,

T., Delingette, H., Demiralp, C., Durst, C., Dojat, M.,

Doyle, S., Festa, J., Forbes, F., Geremia, E., Glocker,

B., Golland, P., Guo, X., Hamamci, A., Iftekharud-

din, K., Jena, R., John, N., Konukoglu, E., Lashkari,

D., Antonio Mariz, J., Meier, R., Pereira, S., Pre-

cup, D., Price, S. J., Riklin-Raviv, T., Reza, S., Ryan,

M., Schwartz, L., Shin, H.-C., Shotton, J., Silva, C.,

Sousa, N., Subbanna, N., Szekely, G., Taylor, T.,

Thomas, O., Tustison, N., Unal, G., Vasseur, F., Win-

termark, M., Hye Ye, D., Zhao, L., Zhao, B., Zikic,

D., Prastawa, M., Reyes, M., and Van Leemput, K.

(2014). The Multimodal Brain Tumor Image Segmen-

tation Benchmark (BRATS). IEEE Transactions on

Medical Imaging, page 33.

Rudin, L. I., Osher, S., and Fatemi, E. (1992). Nonlinear to-

tal variation based noise removal algorithms. Physica

D: Nonlinear Phenomena, 60(1):259–268.

Rueda, A., Gonz

alez, F., and Romero, E. (2013).

Saliency-based characterization of group differences

for magnetic resonance disease classiﬁcation. Dyna,

80(178):21–28.

Samson, C., Blanc-F

eraud, L., Aubert, G., and Zerubia, J.

(1998). Image classiﬁcation using a variational ap-

proach. PhD thesis, INRIA.

Thota, R., Vaswani, S., Kale, A., and Vydyanathan, N.

(2016). Fast 3d salient region detection in medical im-

ages using gpus. In Machine Intelligence and Signal

Processing, pages 11–26. Springer.

Tomasi, C. and Manduchi, R. (1998). Bilateral ﬁltering for

gray and color images. In Computer Vision, 1998.

Sixth International Conference on, pages 839–846.

IEEE.

Wang, Y., Liu, R., Song, X., and Su, Z. (2014). Saliency

detection via nonlocal l {0} minimization. In Asian

Conference on Computer Vision, pages 521–535.

Springer.

Weickert, J. (1998). Anisotropic diffusion in image process-

ing, volume 1. Teubner Stuttgart.

Yang, Q., Tan, K.-H., and Ahuja, N. (2009). Real-time o

(1) bilateral ﬁltering. In Computer Vision and Pattern

Recognition, 2009. CVPR 2009. IEEE Conference on,

pages 557–564. IEEE.

Zhan, Y. (2011). The nonlocal-laplacian evolution for im-

age interpolation. Mathematical Problems in Engi-

neering, 2011.

A Non-Local Diffusion Saliency Model for Magnetic Resonance Imaging

107