Brain Tumor Segmentation of Lower-Grade Glioma Across MRI Images

Using Hybrid Convolutional Neural Networks

Amal Jlassi

, Khaoula ElBedoui

1,2

and Walid Barhoumi

1,2

Universit

e de Tunis El Manar, Institut Sup

erieur d’Informatique, Research Team on Intelligent Systems in Imaging and

Artiﬁcial Vision (SIIVA), LR16ES06 Laboratoire de Recherche en Informatique, Mod

elisation et Traitement de l’Information

et de la Connaissance (LIMTIC), 2 Rue Abou Rayhane Bayrouni, 2080 Ariana, Tunisia

Universit

e de Carthage, Ecole Nationale d’Ing

enieurs de Carthage,

45 Rue des Entrepreneurs, 2035 Tunis-Carthage, Tunisia

Keywords:

Deep Learning, Brain Segmentation, MRI, LGG, Hybrid Convolutional Neural Networks.

Abstract:

Low-Grade Gliomas (LGG) are the most common malignant brain tumors that greatly deﬁne the rate of sur-

vival of patients. LGG segmentation across Magnetic Resonance Imaging (MRI) is common and necessary

for diagnosis and treatment planning. To achieve this challenging clinical need, a deep learning approach that

combines Convolutional Neural Networks (CNN) based on the hybridization of U-Net and SegNet is devel-

oped in this study. In fact, an adopted SegNet model was established in order to compare it with the most

used model U-Net. The segmentation uses FLuid Attenuated Inversion Recovery (FLAIR) of 110 patients of

LGG for training and evaluations. The highest mean and median Dice Coefﬁcient (DC) achieved by the hybrid

model is 83% and 85.7%, respectively. The obtained results of this work lead to the potential of using deep

learning in MRI images in order to provide a non-invasive tool for automated LGG segmentation for many

relevant clinical applications.

1 INTRODUCTION

According to the World Health Organisation (WHO),

Low-Grade Gliomas (LGG) are a class of grade I

and grade II brain tumors. Contrary to LGG grade

I, which is frequently curable by surgical resection,

LGG grades II and III are inﬁltrative and reach

to reproduce the higher-grade lesion (Louis et al.,

2016). Furthermore, and as reported by WHO also,

an increasing number of LGG grade II has been

incidentally found throw cervical MRI (Magnetic

Resonance Imaging), however 3.8% to 10.4 % of pa-

tients do not have obvious tumor-related symptoms.

Furthermore, in its ﬁfth edition of 2021 relating to the

classiﬁcation of tumors of the central nervous system,

the WHO afﬁrms that LGG and glioneuronal tumors

account more than 30% of pediatric neoplasms of

the central nervous system. Thus, LGG is one of the

most commonly encountered brain tumors among

children, and the number of affected children may

dramatically rise. Indeed, as per the data published on

the site cancer.net, it is estimated that approximately

5, 900 brains will be diagnosed with brain tumors

this year (02/2022) in children ages 0 to 19 years

in the United States. In terms of diagnosis, MRI is

usually used throughout the neuro-oncology patient

treatment since routine structural imaging provides

particular anatomical and pathological information.

However, predicting patient outcomes based only on

MRI data for these tumors are imprecise and suffers

from the clinicians’ inter-variability (Network,

2015). To deal with this issue, subtypes of LGG

were deﬁned across the clustering of patients based

on DNA methylation, gene expression, DNA copy

number, and microRNA expression (Mazurowski,

2015). Radiogenomics, as a new research direction

in this ﬁeld, aims to explore the relationship between

tumor genomic characteristics and medical imaging

such as MRI (Mazurowski, 2015). Currently, the

ﬁrst step when extracting tumor features was the

manual segmentation of MRI by neuroradiologists or

clinicians. However, manual segmentation is costly,

and time-consuming, and results often lead to inter-

observer variability, which can signiﬁcantly sway the

diagnosis. In an effort to overcome these limitations,

automatic LGG segmentation seems to be one of

the effective solutions. Recently, progress in Deep

Learning (DL) for automatic brain segmentation has

carried out a level that achieves the performance of a

skilled radiologist. However, most of the existing DL

works have been focused on glioblastoma, compar-

atively to LGG (Booth et al., 2020). Several studies

454

Jlassi, A., ElBedoui, K. and Barhoumi, W.

Brain Tumor Segmentation of Lower-Grade Glioma Across MRI Images Using Hybrid Convolutional Neural Networks.

DOI: 10.5220/0011895900003393

In Proceedings of the 15th International Conference on Agents and Artiﬁcial Intelligence (ICAART 2023) - Volume 2, pages 454-465

ISBN: 978-989-758-623-1; ISSN: 2184-433X

 2023 by SCITEPRESS – Science and Technology Publications, Lda. Under CC license (CC BY-NC-ND 4.0)

suggest that LGG can be associated with different

genomic subtypes, which are signiﬁcant factors in

determining the course of treatment. Based on the

recent literature, there is no noninvasive approach

identifying genomic subtypes. Does previous litera-

ture demonstrate a correlation between LGG shape

characteristics and subtypes(Buda et al., 2019). In

fact, it leads to conducting radiogenic analysis and

enhances inferences about these correlations.

In this work, we propose a fully automated seg-

mentation method that identify whether the assessed

shape features are prognostic of tumor molecular

subtypes or not. To do so, the proposed method is

based on an integrated deep learning architecture

combining SegNet and U-Net architectures. In fact,

to the best of our knowledge, none of the state-of-

the-art methods have tested the performance of the

well-known CNN architecture SegNet on delineating

LGGs. Most of the literature methods are based

on U-Net variants which have shown promising

performances. Thus, in order to take advantage of the

beneﬁts of both U-Net and SegNet algorithms, we

have conducted a comparative study which allowed

us to propose an effective method that combines

both architectures in order to further enhance the

diagnosis accuracy. Literally, this work aims to

investigate the correlation between selected shape

features and genomic subtypes in order to provide the

information to clinicians sooner via a non-invasive

method. Further, in some cases, it could perform

better delineation of tumors where the resection is

not provided. Indeed, the obtained results show that

the proposed automated tool based on deep learning

could be helpful for the diagnosis and the treatment

planning of LGG.

The remainder of this paper is organized as fol-

lows. Section 2. describes the state of the art whereas

section 3. presents the proposed hybrid CNN ar-

chitectures for segmenting LGG from MRI images.

Then, in section 4. we show results for the segmen-

tation model. In section 5. we produce a conclusion

with some directions.

2 RELATED WORK

Various segmentation approaches have been devel-

oped to delineate LGG on MRI scans. The vast ma-

jority of these approaches are based on machine learn-

ing. For instance, generative and discriminative mod-

els have been widely used. On the one hand, Genera-

tive Models (GMs) have the capacity to handle small-

sized datasets. On the other hand, Discriminant Mod-

els (DMs) are more efﬁcient when using ”wide data”.

However, GMs are generally less accurate than DMs.

2.1 Generative Models

GMs such as atlas-based models need prior knowl-

edge of anatomy and take on posterior probabilities

for voxels’ classiﬁcation. For instance, Parisot et al.

have explored ﬁrstly prior knowledge in order to clas-

sify the tumor then they used another graph to identify

the class of each voxel (Parisot et al., 2012). How-

ever, Huang et al. have used the sparseness of sam-

ples to construct a particular dictionary and develop

a softmax model in order to optimize the error re-

construction coefﬁcients for different classes (Huang

et al., 2014). Furthermore, the Random Forest (RF)

approach, notably in the cases of high number of fea-

tures, has succeeded to be good to accomplish accu-

rate brain tumor segmentation (Zikic et al., 2012). In

this context, Meier et al. have used a set of dedicated

features-based decision RF to discriminate patholog-

ical regions within brain MRI volumes (Meier et al.,

2015). Likewise, Meier et al. have investigated the

CRF method to improve the voxel-wise classiﬁcation

accuracy on the summit of the RF classiﬁer. Dif-

ferently, Markov Random Field (MRF) and Condi-

tional Random Field (CRF) are also frequently used

for brain tumor segmentation. For instance, Zhao

et al. have proposed a semi-segmentation approach

based on the MRF, in which one slice was labeled

and the other slices were sequentially labeled using

the MRF label (Zhao et al., 2013). Nevertheless, GMs

usually focus on the distribution of a dataset in order

to return a probability for a given example.

2.2 Discriminative Models

DMs, such as the Support Vector Machine (SVM),

do not require prior knowledge of anatomy and

use imaging features extracted from MRI instead

of the original MRI data for the classiﬁcation task.

Thus, dimensionality reduction or imaging feature

selection is mostly developed before the model

training task. Deep Learning (DL) based on CNN is

a promising approach that is different from classical

DMs since it is based on end-to-end classiﬁers. In

fact, unlike classical DM, imaging feature extraction

and selection is automated during model training, and

this approach has shown relevant results in automatic

tumor segmentation. Furthermore, in recent years,

CNN models have shown promising performances

in medical image processing, not only in terms of

accuracy but also in terms of efﬁciency. Pereira

Brain Tumor Segmentation of Lower-Grade Glioma Across MRI Images Using Hybrid Convolutional Neural Networks

455

et al. have developed two different structures with

dissimilar depths to deal with the LGG (Thaha et al.,

2019). Similarly, Dvorak et al. have evaluated the

effectiveness of different patch selection techniques

based on the segmentation results of CNNs (Zhang

et al., 2020). Havaei et al. have proposed a multiscale

CNN structure in order to enhance the use of local

and global information (Havaei et al., 2015a). A

combination of RF with the ﬁnal output of CNNs is

used to make better classiﬁcation results. Zhao et al.

have introduced a method that combines FCNN and

CRF (Havaei et al., 2015b). The main advantage of

this method is that it treats the subproblem of unbal-

anced data. Overall, the patches are often randomly

extracted with controlling their number per class.

However, the size or quality of the patches can affect

easily the LGG segmentation. For example, a patch

of a small size cannot have all the spatial information

whereas a patch of considerable size will need more

computational resources. To address these problems,

recent studies used CNN-based encoder-decoder

networks. For instance, Buda et al. have recently

proposed a fully automatic way to quantify LGG

characteristics using U-Net architecture and test

whether these characteristics are predictive of tumor

genomic subtypes (Buda et al., 2019). Due to the

excellent performance of U-Net, other segmentation

networks based on the U structure of U-Net are

produced such as UNet++. Xu et al. have proposed

an LGG segmentation tool based on the UNet++

model (Xu et al., 2020) which uses nested dense

skip connections to reduce the semantic gap between

encoder and decoder caused by the U-Net model.

Moreover, Naser et al. have combined CNN based

on the U-Net for LGG segmentation and transfer

learning based on a pre-trained convolution-base of

Vgg16 and a fully connected classiﬁer (Naser and

Deen, 2020). The latter U-Net architecture uses

skip connections to the corresponding layers in the

decoding part. Thus, it leads to a shortcut for gradient

ﬂow in shallow layers during the training task.

More recently, two models, which are U-Net with

a ResNeXt-50, have been investigated in (Paradkar

and Paradkar, 2022). This work includes analyzing

LGGs through deep learning-based segmentation,

shape feature extraction, and statistical analysis to

identify correlations between selected shape features

and genomic subtypes.

As best as we know, no CNN architecture based

on SegNet is used for LGG segmentation. The most

used one is the U-Net model which requires higher

computational time compared to SegNet. However,

the skip connection saddles the set of captured fea-

tures to the corresponding upsampling convolution

blocks in the SegNet decoder module. This paper fo-

cuses on the hybridization of the CNN architecture,

the hybrid U-SegNet. The idea comes after a com-

parative study between U-Net and SegNet models.

Thus, the proposed architecture is a U-shape model

with properties mimicked from the SegNet.

3 MATERIALS AND METHODS

In this section, we ﬁrstly present the dataset that we

investigated in this work. Then, the proposed method

for the LGG segmentation is described comparatively

to used SegNet and U-Net and evaluated within the

used dataset.

3.1 Materials

The dataset used in this study contains brain MR im-

ages together with manual FLAIR abnormality seg-

mentation masks. The images were obtained from

The Cancer Imaging Archive (TCIA). In fact, these

scans correspond to 110 patients included in The Can-

cer Genome Atlas (TCGA) LGG collection with fully

FLAIR sequence and genomic cluster data available.

The collection of patients comes from ﬁve different

institutions (Thomas Jefferson University – 16 pa-

tients; Henry Ford Hospital – 45 patients; UNC – 1

patient; Case Western – 14 patients; and Case West-

ern St. Joseph’s – 34 patients). The patients are

distributed as 50 patients with Grade II, and 58 pa-

tients with Grade III. Figure 1 summarises the char-

acteristics of the patient’s data such as tumor grades,

tumor sub-types, genders, and ages. Each MRI per

patient contains from 20 to 88 slices with the size

of 256 pixels and shows cross-sectional areas of the

brain as shown in Figure 2. Tumor shape assessment

was based only on the FLAIR abnormality since tu-

mor enhancement in LGG is infrequent. The Ground

Truth (GT) generated by tumor masks was performed

by Buda et al. (Buda et al., 2019) using the FLAIR

MRI images and they made it publicly available for

download from (https://www.kaggle.com/).

3.2 Methods

An overview of the proposed approach used for LGG

segmentation is shown in Figure 3. In fact, the pro-

posed fully automatic method of LGG segmentation

based on a hybrid CNN is composed of three main

procedures: image preprocessing, data augmentation,

and segmentation.

ICAART 2023 - 15th International Conference on Agents and Artiﬁcial Intelligence

456

Figure 1: The patients’ data includes tumor grades, tumor sub-types, genders, and ages.

Figure 2: A sample of MRI scans from he TCGA dataset:

(a) T1 modality, (b) T2 modality, and (c) FLAIR modality).

3.2.1 Preprocessing

The Skull Stripping (SS) process is used in order to

extract brain tissue from the non-brain tissue. The

output of the SS is a new image with only a brain pixel

(without non-brain tissue) as presented in Figure 4 or

a binary value assigning value 1 for brain pixels and

value 0 for the rest of the tissue. More precisely, the

preprocessing of the MRI sequences consists of the

following steps:

1. Scaling images to the joint frame of reference.

2. Stripping of the skull to concentrate the analysis

of the brain region.

3. Normalizing the tissue intensity.

3.2.2 Data Augmentation

The number of images containing tumors was signiﬁ-

cantly lower than the number of those with only back-

ground class present. To deal with this issue, data

augmentation seems to be as a good solution. How-

ever, in our context, we cannot apply all transforma-

tions because the segmentation results could consid-

erably change. Consequently, we opted to work on

three possible transformations in order to not degrade

the training performance (Buda et al., 2018). Indeed,

for each oversampled slice, we applied random rota-

tion, ﬂip and for the other slice, we applied random

scale, as shown in Figure 5. Finally, in order to reduce

the unbalance between tumor and non-tumor classes,

we isolated empty slices that did not contain any brain

or other tissue after applying the Skull Stripping pro-

cess.

3.2.3 Segmentation

Recently, deep neural networks are payoff popularity

among researchers and have shown outstanding per-

formance with appreciated accuracy in medical im-

age segmentation. CNN is a type of deep neural net-

work, which can learn and extract features from im-

ages. In fact, many researchers have used CNN for

automatic brain tumor segmentation in MRI images,

especially for LGG segmentation. The objective of

this paper is to generally explore the CNN architec-

tures for brain tumor segmentation and speciﬁcally

those of SegNets and U-Net. So, it is important to

ﬁnd the relevant advantages of each model in order

Brain Tumor Segmentation of Lower-Grade Glioma Across MRI Images Using Hybrid Convolutional Neural Networks

457

Figure 3: An overview of the proposed hybrid CNN.

Figure 4: Preprocessing Example: (a) Original MRI, (b)

Skull Stripped MRI, and (c) Preprocessed MRI.

to develop a hybrid architecture by inheriting the ad-

vantages of these models. It is noticeably expected

that the hybrid architecture will give a more devoted

result. Particularly, U-Net has achieved good results

in medical image segmentation. Hence, it is the most

commonly used in the LGG segmentation task. It has

performed outstanding results in this challenge and

it has overcome the problems of fewer data capac-

ity, fuzzy boundaries, and high gray scales in med-

ical image analysis. In fact, the U-Net method in-

cludes an encoder for processing input MRI images

and a decoder for generating outputs (Drozdzal et al.,

2016). Firstly, the encoder decomposes the image

into different levels of feature maps. Then, it extracts

the coarse-grained features of the main feature maps.

Next, the decoder restores the feature maps of each

layer by an up-sampling process. The concatenation

cascades the features of each layer of the encoder with

Figure 5: Example of corresponding data augmentation re-

sults. (a) Original MRI. (b) Flip. (c) Scale by 4% –8%. (d)

rotation by 5°–15°.

the features obtained by the transpose convolution op-

eration in the decoder. Thus, it reduces the loss of

accuracy in the feature extraction process. Regarding

the SegNet, it can be classiﬁed based on the number

of convolution blocks (Li et al., 2021). The SegNet

basically, has two convolutional layers with 3 × 3 ﬁl-

ters. In each convolution block, the feature extraction

and the convolution operation are performed from the

input by sliding the ﬁlter kernel. Moreover, batch nor-

malization layers are developed after each convolu-

tional layer in order to normalize the channels of the

extracted features. Moreover, ReLU layers are used

in order to convert the negative value to zero with-

out changing its dimensions. It seems that U-Net is

able to capture ﬁne and soar pieces of information

from the encoder to the decoder using skip linking,

but it requires a higher computational time compared

to SegNet. Since none of the state-of-the-art works

ICAART 2023 - 15th International Conference on Agents and Artiﬁcial Intelligence

458

have tested the performance of the well-known CNN

architecture SegNet on delineating LGGs. A compar-

ative study is established between the U-Net used by

Buda et.al (Buda et al., 2019) and the SegNet. The

latter is composed of an encoder network and a corre-

sponding decoder network, followed by a ﬁnal classi-

ﬁcation layer in pixels. This architecture is illustrated

in Figure 6. In our case, the encoder network con-

sists of 10 layers followed by encoders, of the same

number of blocks set-top boxes.

In order to keep the higher-resolution feature maps

at the deepest encoder output, fully connected layers

were removed. The ﬁnal decoder output is fed to a

Sigmoid classiﬁer to produce class probabilities for

each pixel independently. For our dataset, the SegNet

architecture is trained with various parameters and

then we chose the relevant ones that gave a promis-

ing result for our task. The developed SegNet has the

following encoder layers:

• Input: MRI scans.

• Conv-1: The convolutional layer consists of 16

3times3 ﬁlters applied with a stride of 1 and a

padding of 1.

• Conv-2: The convolutional layer consists of 16