TA Hybrid Approach using Set Theory (HAST) for

Magnetic Resonance (MR) Image Segmentation

Liu Jiang

, Chee Kin BanP

, Tan Boon PinP

Shuter Borys

and Wang Shih-ChangP

Department of Computer Science, School of Computing

National University of Singapore (NUS)

Diagnostic Radiology, Yong Loo Lin School of Medicine,

National University of Singapore (NUS)

Abstract. This paper describes a new Hybrid Approach using Set Theory

(HAST) for Magnetic Resonance (MR) Image segmentation based on two ex-

isting techniques, region-based and level set methods. In our approach, instead

of using the typical pipeline methodology to integrate the two techniques, a hy-

brid set-based methodology will be proposed. To evaluate the effectiveness of

HAST, MR images taken from a national hospital that reflects the quality of

real world medical images are used. A comparison between the two individual

techniques and HAST will also be made to demonstrate the effectiveness of the

latter.

1 Introduction

Medical Resonance Imaging (MRI) [1,2] is a non-invasive technique used to capture

internal body structures using magnetism and radio waves. MRI is used to obtain a

two-dimensional view of the internal organs and the images that are produced are

very crucial for early diagnosis of tumors and analysis of fluid flow in organs like

kidney. Due to the importance of this imaging technique, it forms a motivation to

work on new methods that can effectively perform image analysis tasks like image

segmentation. Despite being a safer imaging technique compared to x-ray, images

produced using MRI often suffer from problems of noise and geometrical complexity

of inner body tissues. These two problems bring about two common errors during the

segmentation process, under-segmentation (leaking) and over-segmentation. Under-

segmentation [3] is defined as the inclusion of the unwanted region in the segmented

image. Over-segmentation [3] is when the segmented image does not contain the

complete desired region of interest. See Fig. 1 .

Jiang L., Kin Ban C., Boon Pin T., Borys S. and Shih-Chang W. (2006).

TA Hybrid Approach using Set Theory (HAST) for Magnetic Resonance (MR) Image Segmentation.

In 6th International Workshop on Pattern Recognition in Information Systems, pages 159-168

DOI: 10.5220/0002473801590168

 SciTePress

Fig. 1. (a) Original kidney, (b) Under-segmented kidney, (c) & (d) Over-segmented kidney, (e)

Desired segmented kidney.

2 MRI Segmentation Techniques

Much works had been carried out in the past to propose new segmentation tech-

niques. Such works typically involve complicated mathematical formulas which form

the basis of the techniques. Some of the most common techniques are the region-

growing techniques [3,4], level set technique [3,4] and segmentation techniques based

on watershed [4]. As we will be using the first two classes of techniques mentioned

above in our approach, in this section we will provide a brief overview of the two

techniques.

2.1 Region Growing

Region-based class of algorithm [3,4] typically makes use of information present in

the input image to perform the segmentation task. One of the most commonly used

information is the image intensity. In the implementation of a simple region-based

technique, a seed index, x together with a lower intensity threshold t1 and upper in-

tensity threshold t2 are initialized by the user. The algorithm first analyzes x to check

if the intensity, I(x) falls between the intensity intervals. If so, x will be added to a set,

S which forms the result of the algorithm and all the neighbors, N(x) of s will be

added into a queue. The process of analyzing the intensity of each pixel is carried out

repeatedly until the queue becomes empty. To prevent the same pixels from being

repeatedly analyzed, the algorithm also keeps track of which pixels had already been

analyzed. The equation below describes the relationship between the pixels in the

result set, S,

∀x, x ∈ S → (I(x) ≤ t2) ∧ (I(x) ≥ t1) ∧ (∃y, y ∈ N(x) ∧ y ∈ S)

(1)

Although region-based techniques tend to work ideally in images which have distinct

intensity difference between adjacent tissues, due to its emphasis on image-based

information, this class of algorithm is typically susceptible to noisy images, which is

the common shortcoming in many imaging techniques.

160

2.2 Level Set

Level set technique [2,4], unlike region based which works on image intensities,

works on the geometry of isosurfaces. It is a powerful approach which is able to han-

dle geometrical issues like convolution, splitting and merging. This approach tracks

the evolution of contours and surfaces in the image and uses a differential equation to

control the evolution of contours. Mean intensities and gradient are typically used in

the differential equation to perform segmentation. For this cause, level set techniques

usually produce outputs that are based on the shape of the tissue of interest. However,

this does not necessary determine that level set algorithms typically work better. In a

human body which contains many organs and tissues, the shape of the tissue of inter-

est may sometimes be too complicated to be tracked by any level set algorithms. This

makes the technique unsuitable to segment very complex tissues.

3 Hybrid Approaches

As we had given a brief overview of the two segmentation techniques used in this

hybrid approach, it is not difficult to realize that each segmentation technique has its

own strength and weakness, and they do not always work well on all types of images.

This gives rise to the idea of hybrid approaches to integrate different segmentation

techniques to amplify the benefits and minimize the weaknesses present in each of the

techniques. Much works had been carried out so far, and some of them provide prom-

ising results. One such integration [11] involves the integration of Fuzzy-

Connectedness, Voronoi Diagram-based [5] and the deformable model [6,7,8,9]. In

this approach, the fuzzy-connectedness [10] method is first used to segment a region

that consists of the tissue of interest from the input image. In this method, the seg-

mentation task is achieved using the affinity between two elements in the image. The

affinity between any two elements is defined by their degree of adjacency as well as

the variation of their intensities. The segmentation result of this step is subsequently

used to perform statistical analysis. In this analysis, statistical data is generated based

on the average and variance of the strongest three channels in two color spaces, Red,

Green and Blue (RGB) and Hue, Value and Chroma axis (HVC). This analysis is

done to derive a homogeneity operator which will be used as a multi channel operator

to perform classification based on the Voronoi Diagram-based algorithm [5]. The

final result is then generated using the deformable model which will determine the

smooth boundary of the segmented region. [3, 11, 12, 13, 14]

It can be seen that the pipeline methodology is used in the approach described

above. This is not the only hybrid approach that adopts the pipeline methodology,

many other approaches adopts the methodology in different ways. Although these

hybrid methods provide segmentation results that are better than the individual tech-

niques, one of the common problems is that all of them require user intervention in

the form of intensive parameter initialization. In this paper, a hybrid approach that

uses set operations to produce better segmentation results and reduce the hassle of

parameter initialization will be discussed.

161

3.1 HAST

In HAST, instead of integrating the segmentation techniques using the common pipe-

line methodology, set theory will be used in the integrating process. Past works like

SHA[15] had also adopted a similar approach. In SHA, the union and intersection

operations are used for integration to reduce leaking and over-segmentation. In

HAST, instead of using the same operations, we will focus on the union and differ-

ence operations to achieve the same aim. Furthermore, the two techniques, region-

based and level set technique works on different basis. This makes the two techniques

ideal for combination as the regions that are detected by both techniques are different.

Notice that it will be less interesting to combine two techniques that have the same

characteristic, as it would consequently mean that they have the same benefits and

flaws. Therefore such combination will still encounter the same flaw. The region-

based and level set combination will ideally be able to maximize the benefits of both

techniques, and hopefully restrict each other’s flaws. The next paragraph and Fig.2

will give a detailed description of the steps involved in this approach.

Step 1: Generate using the region-based technique and the level set technique two

segmentation images consisting of the tissue of interest. We call the two images gen-

erated RG

B and LSB

B respectively

Step 2: Combine RG

B and LSB

B using the union operation to form the image, I.

Step 3: If I is acceptable, GOTO step 5, Else GOTO step 4

Step 4: Identify an adjacent tissue where the leaking occurs, and generate the im-

ages RG

B and LSB

B using the two techniques. Combine RGB

B and LSB

B using the union

operation and subtract the result from I. GOTO step 3

Step 5: Apply the region growing technique using a seed point within the tissue of

interest to segment the image, I. The intensity boundaries should be set to 1 and 300

respectively. The result of this step, I

B will alienate any region that is disjoint with the

tissue of interest.

Fig. 2. Overview of HAST.

162

3.1.1 Step 1 of HAST

In this step, the main task is to generate two segmentation images that consist of the

tissue of interest using the two techniques. The two images, RG

B and LSB

B generated

should ideally consist of the entire tissue of interest. This is to say that over-

segmentation is manifested as holes in the images. This may simply mean a bolder set

of parameters may be used to produce such a segmentation image. At this point, we

will first ignore the amount of leaking that may arise by such a bold set of parameters.

We will handle the issue of leaking at a later stage.

3.1.2 Step 2 of HAST

The main purpose of this step is to reduce over-segmentation in the resulting image I.

Such reduction is done by combining RG

B and LSB

B using the union operation. As we

had mentioned in the earlier sections that the two segmentation techniques used in

HAST works on different basis, therefore the chance that RG

B and LSB

B covers exactly

the same pixels is extremely low. As we had mentioned earlier, this step serves to

reduce over-segmentation, we will ignore leaking in the explanation of this step and

leave the explanation to later.

Let F be the set of pixels, denoted by (x, y) contained in the region defined by the

tissue of interest, and U, O be the set of pixels contained in the under-segmented

region and the over-segmented region respectively.

O ⊆ F

(2)

U ∩ F = ∅

(3)

O ∩ F = O

(4)

Let R

B and EB

B be the set of pixels, denoted by (x, y) contained in the region ob-

tained RG

B and, LSB

B respectively, and OB

B and HB

B be the set of pixels, denoted by (x, y)

contained in the over-segmented region of RG

B and LSB

B respectively.

Since we know that all the pixels of the over-segmented regions must belong to

the tissue of interest, we know O

B and HB

B must be a subset of F.

B ∩ RB

B = ∅

(5)

B ∩ EB

B = ∅

(6)

3.1.2.1 O

B and HB

B are disjoint (Case 1) Fig 2a

In this case, O

B and HB

B are disjoint. This is the best case for over-segmentation reduc-

tion of the union operation. Since over-segmentation is defined as the set of pixels

that are included in the set F but not in the segmentation output, RG

B or LSB

B there-

fore from this we know that the over-segmented region of RG

B, OB

B is included in the

region of LS

B and the over-segmented region of LSB

B, HB

B must be included in the re-

163

gion of RGB

B. The union operation in this case will cause the over-segmentation rate of

the union result to be 0.

B ⊆ EB

(7)

B ⊆ RB

(8)

3.1.2.2 O

B and HB

B intersect (Case 2) Fig 2b

In this case, like the explanation in case 1, the regions which O

B and HB

B do not inter-

sect will be included in the union region after the operation. Therefore in this case the

over-segmentation of the union output will be the intersection between O

B and HB

Since the intersection between O

B and HB

B is a proper subset of both OB

B and HB

B, there-

fore the over-segmentation of the union result in this case is better than both RG

B and

B – OB

B)⊆ RB

(9)

B – HB

B) ⊆ EB

(10)

B ∩ OB

B) ⊈ (RB

B ∪ EB

(11)

3.1.2.3 O

B is a subset of HB

B or vice versa. (Case 3) Fig 2c

This is the worst case for over-segmentation. The final over-segmentation after the

union operation in this case is the region that the smaller set covers. In this case the

over-segmentation rate of the larger set is reduced but remains the same for the

smaller set.

B ∩ OB

B) = OB

(12)

B⊈ (RB

B ∪ EB

(13)

Fig. 3. Venn diagram describing the three cases of the union operation.

3.1.3 Step 3 of HAST

Step 3 of HAST is a decision step which requires the user to decide whether the im-

age is acceptable. If the output from step 2 is acceptable, the output is taken as the

result and the process ends here. However such a situation may not be true all the

time. Very frequently, we may find that leaking exists in the output from step 2. This

164

is because a union operation will generate an image with more leaking than the two

source images. Note that if both source images, RG

B and LSB

B do not contain any leak-

ing; the union output will contain no leaking. In a case where one of the source im-

ages contain leaking, or in the worst case when both contain leaking, the union output

will contain more leaking. In this step, when such a case occurs, users will carry out

the steps in step 4 to reduce leaking.

3.1.4 Step 4 of HAST

This step is designed to reduce leaking in the union output when the user has identi-

fied leaking in the previous step. This step will begin with the user generating two

segmentation images using the two techniques respectively. The segmentation will be

generated based on the tissue adjacent to the tissue of interest where the leaking oc-

curs. Upon generating the images, the two images are combined using the union op-

eration before the combined image is subtracted from the union output in step 2. The

leaking-reducing step lies in the subtraction operation. The two images generated in

this step need not be a good segmentation of the adjacent tissue. In actual fact, the

reduction in leaking will take place as long as the union of the two images of the

adjacent tissue does not contain leaking. Unlike the first step where bold parameter

values are used to generate the segmentation of the tissue of interest, in this step,

more conservative values are chosen. As it is probably apparent that we have reduced

over-segmentation in step 2, therefore we would want to minimize the increase in

over-segmentation in this step of leaking-reduction. After this step, the control will be

passed back to the user (step 3) to decide whether there is any more leaking to be

reduced. Note that after the subtraction operation, the region where leaking reduction

is directed to will consist of small portions of the adjacent tissue that are not yet re-

moved. These portions will be removed only at the end when the bulk of all the leak-

ing regions have been subtracted.

3.1.5 Step 5 of HAST

This is the last step of HAST. In the previous steps, the bulk of all the leaked regions

have already been subtracted from the main union output, leaving behind small por-

tions of the leaked regions on the union output, I. The purpose of this step is to touch

up I by removing all these small portions that are disjoint from the tissue of interest.

This is done using the region-based technique by specifying a seed point within the

tissue of interest and setting the lower and upper intensity threshold to be 1 and 300

respectively. By doing this, all connected pixels to the seed point will be included in

the final result, I

B, omitting all the unconnected pixels from step 4.

4 Experimental Results

In order to investigate the effectiveness of HAST, experiments were conducted on

real MR images taken from a national hospital. Since HAST is part of the effort taken

to reduce the hassle in manual segmentation, which is a process carried out in many

hospitals, therefore tests and experiments should be conducted using images that can

165

reflect the quality of MR images in the real world. Before proceeding to view the

experimental results, in order to measure the over-segmentation and leaking present

in segmentation images, the calculation of the over-segmentation rate (OR), under-

segmentation rate (UR) and the overall error rate (ER) are given as below. Three

variables that are used in the formulas, namely the number of pixels missing from the

segmentation image, denoted by O

B, the number of pixels outside the desired of the

segmentation image, denoted by U

B, and the number of pixels inside the desired re-

gion of the template that we had generated for experimental purpose, denoted by D

The formulas below describe the three calculations:

OR = O

B / (UB

B + DB

(14)

UR = U

B / (UB

B+ DB

(15)

ER = (O

B + UB

B) / DB

(16)

A total of ten experiments were carried out in the initial phase using ten different MR

images taken from different patients. These MR images reflect the real-world quality

and contain sufficient noises that make the images susceptible to the problems of

segmentation. The following tables show the summary of the experimental results

compared to the results obtained using each of the two techniques used in HAST.

Table 1. Relative Over-Segmentation rate and Relative Under-Segmentation Rate of 10 ex-

periements.

Over-Segmentation Under-Segmentation

Experiment

RGB

LSB

I IB

RGB

LSB

I IB

1 0.64 0.11 0.08 0.09 0.04 0.33 0.26 0.21

2 0.69 0.25 0.09 0.09 0 0.42 0.29 0.29

3 0.36 0.77 0.21 0.21 0.08 0 0.08 0.08

4 0.25 0.56 0.19 0.24 0.51 0 0.19 0.01

5 0.21 0.56 0.12 0.16 0.60 0 0.35 0.18

6 0.44 0.61 0.37 0.37 0.10 0.05 0.11 0.10

7 0.57 0.47 0.34 0.45 0.14 0.20 0.24 0.03

8 0.71 0.49 0.54 0.55 0 0.09 0.01 0

9 0.38 0.25 0.23 0.24 0.29 0.13 0.15 0.13

10 0.37 0.51 0.46 0.53 0.39 0.11 0.13 0

Table 2. Relative Overall Error rate of the 10 experiments.

Experiment

RGB

LSB

I IB

1 0.71 0.65 0.47 0.38

2 0.69 1.17 0.54 0.54

3 0.48 0.77 0.32 0.32

4 1.57 0.56 0.47 0.25

5 2.03 0.56 0.72 0.41

6 0.61 0.69 0.54 0.54

7 0.83 0.85 0.77 0.50

8 0.72 0.65 0.56 0.56

9 0.93 0.44 0.45 0.42

166

10 1.24 0.69 0.68 0.53

From the experimental results, we can see clearly that this method reduces the

problem of both leaking and over-segmentation. Before we proceed further to elabo-

rate on the results, note that in all experiments except for experiment 3, the subtrac-

tion step for HAST (Step 4) is performed. For over-segmentation, in all ten experi-

ments, the over-segmentation of the final result is reduced when compared to the

initial two images, RG

B and LSB

B. However, one necessary point to notice is that the

over-segmentation in the union of RG

B and LSB

B, in most cases is lesser than the final

result. As we had mentioned in the theoretical part above, the ideal result of step 4 of

HAST is to generate a segmentation of the adjacent tissue where the leaking occurs

which does not leaked into the tissue of interest. However, such an assumption is not

reasonable. In all these experiments except for experiment 3 which stops before the

subtraction step, the segmentation of the adjacent tissues leak into the desired region,

thus causing the over-segmentation in the final result to increase. One encouraging

observation of this behavior is that the leaking in the segmentation of the adjacent

tissues are usually very minimal. This will result in the increase in over-segmentation

unnoticeable when compared to the amount of leaking that is reduced. In terms of

leaking as a whole, we can easily notice that the union of RG

B and LSB

B will usually

cause a steep increase in leaking. As we had mentioned in the theoretical explanation,

the increase in leaking at this point is expected. In the later steps where the segmenta-

tion of the adjacent tissue which leaks is subtracted from the union image, in all these

cases except for experiment 3, the leakages are reduced. This is once again due to the

fact that the subtraction step is not performed in experiment 3. In order to summarize

the experiments conducted, the overall error rates of all the 10 experiments are also

provided. Finally the most crucial explanation in terms of how HAST serves to im-

prove both over-segmentation and leaking is that even though in the initial steps, we

reduce over-segmentation at the expense of leaking and in the later steps we reduce

leaking at the expense of over-segmentation, the overall improvements that we get are

still much more that the price paid. This is clearly reflected in the improvement in the

overall error rate in all the experiments.

5 Conclusion and Future Work

In this paper we discuss a new hybrid segmentation approach; the Hybrid Approach

using Set Theory (HAST), which is a procedure incorporating set operations into a

pipeline. In HAST, union is performed on the segmented images to reduce over-

segmentation at the expense of leaking. However, the leaking from the initial images

and those that are caused by the union step are then reduced altogether in the subtrac-

tion stage. Experiments were carried out using HAST, out of the ten experiments

carried out, all the ten experiments provide better results when compared to the seg-

mentation generated using any of the two individual images.

Although this approach is still somewhat new and the process is still subjected to

the hassle of parameter initialization, new works have already been done to reduce the

problem of initialization. More experiments are likely to be carried out in the future.

167

Acknowledgements

This research is supported by the Research Grant No R-252-000-210-112 from the

Academic Research Fund at the National University of Singapore. The authors would

like to express their gratitude to Prof. Leong Tze Yun for her guidance and sugges-

tions.

References

1. A. Carrington and A.D. McLachlan, Introduction To Magnetic Resonance, Chapman and

Hall, London 1967.

2. D.D. Stark and W.G. Bradley, Magnetic Resonance Imaging, C.V. Mosby Co., St. Louis,

MO 1988

3. T.S. Yoo and the Insight Software Consortium, Insight into images, A K Peters, Wellesley,

Massachusetts, 2004

4. L. Ibáñez, W. Schroeder, L. Ng, J. Cates, Insight Software Consortium. ITK Software

Guide

5. Imielinska, C. Downes, M. Yuan, W., Semi-Automated Color Segmentation of Anatomical

Tissue, accepted J. of CMIG ., 24(2000), 173-180, April, 2000.

6. Terzopoulous. D. Regularization of Inverse Visual Problems Involving Disambiguities,

IEEE Transaction on Pattern Analysis and Machine Intelligence 8(4)(1986): 413-424

7. Staib, L. and J.Duncan. Deformable Fourier Models for Surface Finding in 3D images. In

Visualization In Biomedical Computing 1992 (Proceedings of VBC92, Chapel Hill,

NC1992), R. A. Robb, ed. SPIE 1808(1992): 90-104

8. McInerney, T. and D, Terzopolous. A Dynamic Finite Element Surface Model for Segmen-

tation and Tracking in Multidimensional Medical Images with Application to Cardiac 4D

Image Analysis. Computerized Medical Imaging and Graphics 19(1)(1995

): 69-83

9. Whitaker, R. Volumetric Deformable Models: Active blobs. Visualization in Biomedical

Computing 1994 (R. A. Robb, ed.), (Mayo Clinic, Rochester, Minnesota) SPIE (1994):

122-134

10. J. Udupa and S. Samarasekera. Fuzzy connectedness and object definition: Theory, algo-

rithms and applications in image segmentation, Graphical Models and Image Processing

58(1996): 246-261

11. C. Imielinska, D. Metaxas, J. Udupa, Y. Jin, and T. Chen. Hybrid Segmentation of the

Visible Human Data. In Proceedings of the Third Visible Human Project Conference, Be-

thesda, MD, 5 October 2000.

12. C. Imielinska, D. Metaxas, J.K. Udupa, Y. Jin, and T. Chen. Hybrid Segmentation of Ana-

tomical Data. In Proceedings MICCAI, pages 1048–1057, Utrecht, The Netherlands, 2001.

13. Y. Jin, C. Imielinska, and A. Laine. A Homogeneity-Based Speed Term for Level-set

Shape Detection. In SPIE Medical Imaging, San Diego, 2002.

14. J. Udupa, V.R. Leblanc, H. Schmidt, C. Imielinska, K.P. Saha, G.J. Grevera, Y. Zhuge, P.

Molholt, L. Currie, and Y. Jin. A Methodology for Evaluating Image Segmentation Algo-

rithms. In SPIE Medical Imaging, San Diego, 2002.

15. J. Liu, T.Y. Leong, B.P. Tan, K.B. Chee, S. Borys, S.C. Wang. A Set-based Hybrid Ap-

proach (SHA) for Magnetic Resonance Image (MRI) Segmentation. [Submitted to 2006

Workshop on Pattern Recognition in Bioinformatics (PRIB- 2006)]

168