Color Image Segmentation upon a New Unsupervised Approach

using Amended Competitive Hebbian Learning

Meriem Timouyas

, Souad Eddarouich

and Ahmed Hammouch

ENSET, LRGE, Mohammed 5 University, Rabat, Morocco

Regional Educational Center, Rabat, Morocco

ywords: Probability Density Function, Competitive Neural Networks, Mahalanobis Distance,

Competitive Hebbian Learning, Topology Preserving Feature, K-means, Segmentation,

Competitive Concept, Thresholding.

Abstract: This paper proposes a new unsupervised color image segmentation procedure based on the competitive

concept, divided into three processing stages. It begins by the estimation of the probability density function,

followed by a training competitive neural network with Mahalanobis distance as an activation function. This

stage allows detecting the local maxima of the pdf. After that, we use the Competitive Hebbian Learning to

analyze the connectivity between the detected maxima of the pdf upon Mahalanobis distance. The so

detected groups of Maxima are then used for the segmentation. Compared to the K-means clustering or to

the clustering approaches based on the different competitive learning schemes, the proposed approach has

proven, under a real and synthetic test images, that does not pass by any thresholding and does not require

any prior information on the number of classes nor on the structure of their distributions in the dataset.

1 INTRODUCTION

The aim of automatic classification is to partition a

set of observations into groups or classes such as

observations belonging to the same class are more

similar than those belonging to different classes

(Eddarouich and Sbihi, 2007).

For the multidimensional data classification

methods (Muthanna et al., 2010; Hammouche et al.,

2005; Verikas et al., 1996), cluster analysis

techniques attempt to separate a set of

multidimensional observations into groups or

clusters which share some properties of similarity.

The objects are generally represented by N-

dimensional vectors of observed features. The

statistical approach in cluster analysis postulates that

the input patterns are drawn from an underlying

probability density function (pdf), which describes

the distribution of the data points through the data

space. Regions of high local density, which might

correspond to significant classes in the population,

can be found from the peaks or the modes of the

density function estimated from the available

patterns (Devijver and Kittler, 1982). Then, the key

problem is to partition the data space with a

multimodal pdf into subspaces over which the pdf is

unimodal (Mizoguchi and Shimura, 1976).

Among the most common applications of

automatic classification is image segmentation.

Color image segmentation is one of the most

important pre-processing step towards image

understanding, image compression and coding. It is

a process that consists of partition the image into

disjoint region as sets of connected pixels that are

homogenous with respect one or more color

characteristics (Uchiyama and Arbib, 1994).

Generally, most algorithms of segmentation

treated are based on threshold selection or on

parameters adjustment which may change the its

results.

Considering the analogy between clustering and

segmentation, the color image segmentation is

achieved by pixel classification according to color

features. It is generally assumed that homogeneous

regions in the image correspond to clusters of color

points in the color space. The sample of observations

is composed of image pixels represented as data

points scattered in a color space; it determines a

partition of these points into subgroups in a way that

makes points within a group more similar than

points in different groups. In an unsupervised

context, the number of these subgroups is not a

priori known and has also to be determined.

Timouyas, M., Eddarouich, S. and Hammouch, A.

Color Image Segmentation upon a New Unsupervised Approach using Amended Competitive Hebbian Learning.

In Proceedings of the 18th International Conference on Enterprise Information Systems (ICEIS 2016) - Volume 2, pages 205-210

ISBN: 978-989-758-187-8

205

Amongst the non-classical methods, the

application of artificial neural networks (ANN) is

prominent. In recent years, motivated by the

remarkable characteristics of the human visual

system (HVS), research has applied ANNs to

various problems in classification (Yeo et al., 2005).

ANNs have several advantages over many

conventional computational algorithms, among

which the most important are parallelism,

adaptability to different data sets and optimal

performance.

Many clustering procedures based on modes

detection concepts, have been proposed. In some of

them, modes are considered as local maxima of the

estimated probability density function and are

detected by hill climbing procedures using some

gradient search technique (Fukunagal and Hostler,

1975). As these procedures are based on differential

operators, they face some difficulties where noise is

present in the data set. In practical situations, they

are known to generate a greater number of modes

than the true pdf. Another approach is based on the

analysis of the convexity properties of the

underlying pdf. Modes are then considered as

concave regions of this function and are detected by

means of a test which determines, locally, the

convexity of the multivariate pdf (Vasseur and

Postaire, 1980; Moussa, Sbihi and Postaire, 2008).

Although this approach yields more robust results

than the previous one, it remains sensitive to local

irregularities in the pattern distribution, especially

for small data sets.

In this context, we are going to present a new

unsupervised approach for the segmentation of color

images based on the detection of the modes of

multivariate probability density function (pdf)

without using either differential operators or any

procedures of filtering

The remainder of the paper is organized as

follows. Section 2 presents the different steps of the

proposed approach. Section 3 is consecrated to other

results and discussion. The paper ends with a

conclusion and perspectives.

2 THE MODES DETECTION

PROCEDURE

The new algorithm of the modes detection is carried

out in three processing stages. The first one consists

in estimating the underlying pdf using a non-

parametric estimator. In the second stage, we use an

artificial neural network with competitive training

(CNN) to extract the local maxima of the pdf. In the

third stage, we develop a new technique for the

detection of the interneural connection.

2.1 The Estimation of Underlying

Probability Density Function

Let

{ , ,..., }

XXΓ=

, be the set of Q N-

dimensional observations of a random variable X

with a probability density function P(X).To estimate

this underlying density function when what is

available is only a set

,1 ,2 , ,

{ , ,..., ,..., }

qqq qnqN

Xxx x x=

, q=1,2,...,Q of Q

observations, the analyst may use non-parametric

techniques.

The Parzen (1962) window method proves well

adapted to the proposed procedure in this paper.

However, this estimation procedure needs

prohibitive calculus when the dimension of the space

is very important. So, we have opted for the fast

estimation algorithm which is proposed by Postaire

and Vasseur (1982).

First, the range of variation of each component

of these observations is normalized to the interval [0,

R], where R is an integer such as

2R ≥

, by means

of the transformation defined as:



,



,

−min





,



max





,

−min





,



∗

(1)

Each axis of the so normalized data space is then

partitioned into R exclusive intervals of unit width.

This discretization defines a set of

ℜ

hypercube of

unit side length. Each hypercube noted

()HX

, is a

site defined by its N coordinates

, ,..., ,...,

xxx

which are the integer parts of

the coordinates of its center

To be more specific, let

1, 2, , ,

[ , ,..., ,..., ]

qqqnqNq

Yyy y y=

, q=1, 2, Q be the Q

observations in the normalized space. Each

observation Yq is found inside a non-empty

hypercube with the coordinates

int( )

nnq

, n=1,

2, N, where

int( )

designates the integer parts of

,nq

. If several observations fall in the same

hypercube, this one appears many times on the list

of non–empty hypercubes. Furthermore, the number

of times the hypercube H(X) appears in that list

ICEIS 2016 - 18th International Conference on Enterprise Information Systems

206

indicates the number of data points q[H(X)] which

falls in this hypercube. Subsequently, the value of

the local density estimated is:



















(2)

Since the volume of H(X) is equal to unity.

So, this fast procedure allows only the estimation

of the underlying probability density function at the

centers of the non-empty hypercubes whose number

never exceeds the number Q of available

observations. At the centers of the hypercube cells,

which are not on that list, the density estimates are

known to be null. At the end of this fast algorithm,

all the available information for clustering is in the

discrete set

of estimated values of the underlying

probability density function

()pX

2.2 The Extraction of Local

Maxima by Neural Network

Assimilating the modes to the local maxima of the

pdf, the proposed approach uses the Neural

Networks with Competitive Training (NNCT)

(Eddarouich and Sbihi, 2007).

In the training algorithm, we work only on the

pdf by presenting, sequentially, the centers of the

non-empty hypercubes of the set X to the network,

instead of the Q observations.

The neural network is composed of two layers:

the input layer and the output layer (Cf. Fig.1). The

first one is made of N units I

, n=1,2,...,N, such that

unit I

is solicited by the attribute X

of the non-

empty hypercube H(X) when this one is presented to

the network. However, each output neuron

materializes an hypercube which represents the site

of one local maximum of the pdf in the set X, and

presents its weight by the mean vector µ

(X),

k=1,2,…,K. The number of the output neuron is

first initialized arbitrary.

Figure 1: Competitive neural network.

During the training phase, The output neurons

enter into competition with each other by comparing

the distance D[µ

(X), H(X)], k=1,2,…,K, between

the input hypercube H(X) and each output neuron

(X), the winner is the closest one to the hypercube,

then we compare the values of the pdf associated to

the winner neuron µ

(X) and to H(X). The distance

measure used in this learning algorithm is

Mahalanobis distance that gives best results for the

non Gaussian distribution (Timouyas et al., 2012)

instead of Euclidian distance as in the (NNCT).

The following algorithm describes the different

steps of the learning phase:

Training Algorithm:

Initialization Phase:

 Initialize the mean vectors

()

; k=1, 2, K, of

the K output neural, with an arbitrary choice of K

non-empty hypercubes from the set X,

 Initialize the coefficients of the training function

and

. The choice of these parameters is not

a real problem. We should only give a very

important value to

for the algorithm to search

the sites of the local maxima before its

convergence (Eddarouich and Sbihi, 2007).

Processing Phase:

1) Present to the network, with an arbitrary pulling,

a non-empty hypercube H(X),

2) Search for the winner neuron g defined by

calculating the distance that separate µ

(X) and

H(X) and seeking for the minimal distance:



















X−HX



X−HX



(3)

∑



Inverse of covariance matrix.

3) Compare the fdp(µ

(X)) and fdp(H(X)):

If fdp(µ

(X)) < fdp(H(X)) update the parameters

of the winner neuron as follow:





















− 







−



















=











(4)

Where

t is the number of iterations and

()t

is a one of

“search then converge” learning functions defined as:

αt=





1







(5)

Then go to step1. Else, go directly to step1,

4) Stopping criteria: after the processing of all

hypercubes, compare

()

−

for k∈K.

( ( ) ( )) 1, 2,...,

XXk K

μμ

−

≠∀=

, pass to

the next iteration and go to step 1.

Else, end of the processing.

…

(X)

Input Layer

Output Layer

Color Image Segmentation upon a New Unsupervised Approach using Amended Competitive Hebbian Learning

207

2.3 Detection of Significant Modes

of Pdf

As the modes that are detected during the learning

phase perfectly mark modal regions and are divided

into a number equal to the number of classes present in

the sample, we thought about connecting each groups

of the closest modes in such a way that we get a map

which preserves the shape and structure of the classes.

One of the perfect methods which forms

topology preserving maps is Competitive Hebbian

Learning (CHL) proved by Martinetz (1993). The

basic principle which governs the change of

interneural connection strength been formulated by

Hebb (1949). According to Hebb’s postulate, a

presynaptic unit i increases the strength of synaptic

link to a postsynaptic unit j if both units are

concurrently active (Martinetz, 1993).

CHL is usually not used on its own but in

conjunction with other method derived as Neural

Gas plus Competitive Hebbian Learning (Martinetz,

1993)

and Growing Neural Gas (Fritzke, 1995).

Let

{}

)X(H),...,X(H),X(HF

, be the set of

K hypercubes which represent the sites in X of the K

detected local maxima; the output of CNN. In this

phase, we are going to seek the interneural

connection of these modes using a new method of

Competitive Hebbian Learning adapting

Mahalanobis distance as measure of resemblance.

The clustering in CHL is based on three

concepts. Firstly, the Vector Quantization (VQ),

which is searching for centroids as density points of

nearby lying samples, it can be also directly used as

prototype-based clustering method: each centroid is

then associated to one prototype. By aiming to

minimize the expected squared quantization error

(Gray, 1984).

The second and third concepts are Voronoi

diagram and the Delaunay triangulation illustrated

below:

(a) 14 point from data set in R

(b) The Voronoi diagram

Figure 2: The detection of the induced Dt by masking the

Delaunay triangulation with the data set.

The Voronoi diagram V

of a set

{}

)X(H),...,X(H),X(HF

of hypercubes H(X

)

ϵ R

is given by K N-dimensional polyhedra, the

Voronoi polyhedra V

, which is defined as follows:

The Voronoi polyhedron V

of a hypercube H(X

) ϵ F

is given by the set of hypercubes v ϵ R

which are

close to H(X

) than any other H(X

) ϵ F for i≠j

(Martinetz and schulten, 1994) :





=∈





‖

−





‖

−



∀



(6)

The dual graph of Voronoi diagram is Delaunay

triangulation Dt (Delaunay, 1934), it’s the

connection of all pairs H(X

), H(X

) ϵ F, where the

circumcircles of the triangles consisting of each of 3

hypercubes of the set, such that no hypercube in F is

inside.

The Dt is also the graph where hypercubes with a

common Voronoi edge V

and V

are connected by an

edge, that is (Martinetz & schulten, 1994):









=,



=1,…,



∩



∅ (7)

To generate the induced Delaunay triangulation

(Fig. 2(a)), competitive Hebbian learning, given the

K modes detected by CNN as prototypes in R

successively adds connections among them. The

method does not change the weight of prototypes,

but only generates topology according to these

prototypes. For each mode H(X

), its two closest

prototypes are connected by an edge using

Mahalanobis distance (3) as measure of resemblance

instead of Euclidian distance, it works as an

activation function for competition between

neurons. This leads to the induced Delaunay

triangulation, which is limited to those regions of the

input space R

3 RESULTS AND DISCUSSION

To illustrate the behavior of the procedure, we

present two examples. The first one is a synthetic

image constituted of 4 regions of colors in different

shapes. The three RGB components, coded on 256

levels, constitute the axes of the coordinates of the

representation space of the image pixels (fig. 3(a)).

The proposed procedure of the pdf modes

detection of the observations sample in figure 3.b

permits to extract the six related regions with the

resolution parameter R=30.

ICEIS 2016 - 18th International Conference on Enterprise Information Systems

208

(a) (f)

(b) (e)

Figure 3: (a) Original color image; (b) Pixels in the RGB

color space; (c) Estimation of the underlying pdf; (d)

Prototypes in the RGB color space; (e) Prototypes

connected by edges; (f) Segmented color image.

This figure provides the clear idea about the

different steps of the proposed method. Below we

illustrate separately the 4 classes of the image:

Figure 4: the four classes of the image.

As shown, the proposed technique success to

detect the four groups of connected neurons

presenting the four classes in the image (fig. 3(e))

and have been used to get the segmented image (fig.

3(f)).

Now, we will apply the approach on a real color

image, with a significant overlapping degree,

constituted of five homogeneous regions with

different shapes:

(a) (b)

Figure 5: (a) Original pepper image; (b) Segmented pepper

image; (c) Pixels in the RGB color space; (d) Prototypes

connected by edges defining 8 classes in 8 iterations.

Despite the difficulty of the treatment of this

image where the various clusters present a

significant overlapping degree in the RGB color

space ( fig.5(c)), the use of the Mahalanobis Metric

as criterion of resemblance, allows the proposed

approach to give more powerful results compared to

the Neuromemetic approach.

In order to prove the approach’s efficiency more,

we evaluate the homogeneity in the image compared

to K-means result applying on the same image by

the experimental parameters exposed below:

Table 1: Statistical parameters of the two methods.

Parameters K-Means Proposed method

avg voxel intensity

45.764

47.0573

Std Dev

21.6

19.3433

Coeff Var

47.21%

41.106%

Figure 6: Comparison of statistical parameters.

The image segmented by the suggested

procedure is most consistent because of its lower

standard deviation and higher average voxel

avg voxel

intensity

Std Dev Coeff Var

K-Mean

Propose

method

Color Image Segmentation upon a New Unsupervised Approach using Amended Competitive Hebbian Learning

209

intensity, which is clearly proved by its lower

coefficient of variance compared to the K-means

result. Hence, the proposed procedure demonstrates

its accuracy in color image segmentation, knowing

that this method has all advantages of artificial

neural network mentioned before.

In spite of that, The Mahalanobis Distance has a

higher execution time than Euclidian Distance

because of its processing complexity but the nearest-

neighbor search can be performed in only O(LogN)

instead of O(N) time by exploiting the Delaunay

triangulation (Knuth, 1973). Also, with this so

reduced number N of neurons, the proposed

detection of modes procedure stays faster than this

phase in both Neuromimetic and

Neuromorphological procedures (Timouyas et al.,

2014). Although, our aim to further minimize the

execution time of the new approach in overall.

4 CONCLUSIONS

In this paper, a new approach of unsupervised color

image segmentation has been introduced, based

essentially on neural network concepts.

In order to conceive an unsupervised

classification procedure, we have searched to

connect the detected local maxima, by CNN, in such

away, every connected set of neurons represents a

class, using the Competitive Hebbian Learning.

The proposed procedure permits good

unsupervised image color segmentation without

resorting to any thresholding and does not require

any priori information about the number of classes

nor about the structure of their distributions in the

sample.

REFERENCES

Delaunay, B. (1934). Sur la Sphère vide. Bulletin of the

Academy of Sciences USSR,VII, pp.793-800.

Devijver, P.A., & Kittler, K. (1982) ‘Pattern recognition:

A statistical approach’, Englewood Cliff, NJ, Prentice-

Hall international.

Eddarouich, S., & Sbihi, A. (2007) ‘Neural Network for

Modes Detection in Pattern Classification’. ICTIS’07,

Morocco, Fez, 3-5 pp. 300-303.

Fritzke, B. (1995) ‘ A growing neural gas network learns

topologies’, In G. Tesauro, D. S. Touretzky, & T. K.

Leen (Eds.), Advances in neural information

processing systems: 7. Cambridge, MA: MIT Press,

pp. 625_632.

Fukunagal, K., & Hostler, D. (1975). The estimation of the

gradient of a density function with applications in

pattern recognition. IEEE Trans. Inf. Theory, vol. IT-

21, n°1, p 32-40.

Gray, R.M. (1984) ‘Vector Quantization’, ASSP Magazine,

IEEE (Vol 1, Issue 2 ), ISSN :0740-7467, 1984, pp. 4–29.

Hammouche, K. Diaf, M. and Postaire, J.-G. (2005) ‘A

clustering method based on multidimensional texture

analysis’, Pattern Recognition, pp. 1-13.

Hebb, D. (1949) ‘Organisation of Behavior’, Wiley, New

York.

Knuth, D. E. (1973) ‘The art of computer programming’,

Volume III: Sorting and searching. Reading, MA:

Addison-Wesley.

Martinetz, T.M. (1993) ‘Competitive Hebbian learning

rule forms perfectly topology preserving maps’,

(ICANN), Gielen S. and Kappen B. (eds), Springer,

Heidelberg, pp. 427_434.

Martinetz, T., & schulten K. (1994). Topology

Representing Networks. Neural Networks. Vol. 7, No

3, pp. 507-522.

Mizoguchi, R., & Shimura, S. (1976). Nonparametric

learning without a teacher based on mode estimation.

IEEE Trans. Comput., C-25(11), pp.1109-1117.

Moussa. A, Sbihi. A and Postaire. J-G, (2008) ‘A Markov

random ﬁeld model for mode detection in cluster

analysis’. Patt. Recog. Letters 29, pp. 1197–1207.

Muthanna, A. H.,Touahni, R., Sbihi, A., & Messoussi, R.

and Eddarouich, S. (2010) ‘Détection des modes d’une

distribution de données multidimensionnelles par

réseau de neurones et morphologie mathématique’,

Journées d'optique et du traitement de l'information.

Parzen, E. (1962) ‘An Estimation of a Probability Density

Function and Mode’, Ann. Math. Stat., vol. 33, pp.

1065-1076.

Postaire, J.-G., & Vasseur, C. P. A. (1982) ‘A fast

Algorithm for non Parametric Probability Density

Estimation’, IEEE, Trans. on Pattern Anal. and

Machine Intel. PAMI-4, n°6, pp. 663-666.

Timouyas, M., Eddarouich, S., & Hammouch, A. (2012).

A new approach of classification for non-Gaussian

distribution upon competitive training, (ICCS’12),

Agadir, Morocco, pp.1-6.

Timouyas, M., Eddarouich, S., Hammouch, A, Touahni,

R. & Sbihi, A. (2014)‘Unsupervised Neural-

Morphological Colour Image Segmentation Using the

Mahalanobis as Criteria of Resemblance’, ICMCS’14,

Marrakech, Morocco, pp.314–320.

Uchiyama, T., & Arbib, M. A. (1994) ‘Color image

segmentation using competitive learning’, IEEE

Transactions on Pattern Analysis and Machine

Intelligence, Vol. 16, n 12, pp. 1197-1206.

Vasseur, C. P. A. & Postaire, J-G. (1980) ‘A connexity

testing method for cluster analysis’ I.EEE Trans. Syst.

Man. Cyber, vol SMC-10, n°3, p 145-179.

Verikas, A., Malmqvist, K., & Bergman, L. (1997). Colour

image segmentation by modular neural network. Pattern

Recognition Letters, Vol 18, Issue 2, pp. 173–185.

Yeo, N.C., Lee, K.H., Venkatesh, Y.V. and Ong, S.H.

(2005) ‘Colour image segmentation using the self-

organizing map and adaptive resonance theory’, Image

and Vision Computing 23, pp. 1006-1079.

ICEIS 2016 - 18th International Conference on Enterprise Information Systems

210