A COMPARATIVE STUDY TO DESIGN A CODE BOOK FOR

VECTOR QUANTIZATION

Yoshitaka Takeda, Eiki Noro

Department of Information and Electronic Engineering, Muroran Institute of Technology, Muroran, Japan

Junji Maeda, Yukinori Suzuki

Department of Information and Electronic Engineering, Muroran Institute of Technology, Muroran, Japan

Keywords:

Image compression, Vector quantization, Code book optimization, GA, Afﬁnity propagation, Fuzzy.

Abstract:

In this paper, we examined six algorithms to construct an optimal code book (CB) for vector quantization

(VQ) experimentally. Four algorithms are GLA (generalized Lloyd algorithm), FCM (fuzzy c meams), GA

(genetic algorithm), and AP (afﬁnity propagation). The other two algorithms are hybrid methods: AP+GLA

and GA+FCM. Performance of the algorithms was evaluated by both PSNR (peak-signal-to-noise-ratio) and

NPIQM (normalized perceptual image quality measure) of decoded images. Computational experiments

showed that the performance of each algorithm could be categorized as higher performance and lower per-

formance. GLA, AP and AP+GLA belong to the higher performance group, while FCM, GA and GA+FCM

belong to the lower performance group. AP+GLA shows the best performance of algorithms in the higher

performance group. Thus, AP+GLA is an optimal algorithm for constructing a CB for VQ.

1 INTRODUCTION

Communication technologies to improve transmis-

sion band width are rapidly improving and these tech-

nologies have enabled high-speed data transmission.

However, the demand for transmission capacity con-

tinues to outstrip the transmission band width real-

ized by current technologies. Data compression tech-

nologies are therefore being developed for effective

use of communication channels. A huge amount of

data, including data for characters, voices, music, im-

ages and videos is being transferred via communica-

tion channels. In these multimedia data, since images

and videos need a wide communication band width,

effective image and video compression technologies

are developing. For still image compression, JPEG

(Joint Photographic Experts Group) is used as a de

facto standard. It is lossy baseline coding. In JPEG,

an image is segmented into 8 × 8 subimages. The

subimages are transformed by DCT (Discrete Fourier

Transform). The energy of an image is concentrated

into lower frequency components in DCT. This en-

ergy compaction provides a good effect to compress

images (Gonzalez(2008); Sayood(2000)).

We have been studying vector quantization

(VQ) for image compression (Miyamoto(2005);

Sasazaki(2008)). In VQ, encoding and decoding

an image involves only looking up a code book

(CB). Therefore, once the CB is completed, com-

putational cost for image compression is negligi-

ble. This is a very attractive point for communica-

tion terminals whose computational ability is small.

PSNR (peak-signal-to-noise-ratio) sharply decreases

as compression rate increases in the case of image

compression with JPEG. However, in the case of

VQ, PSNR slowly decreases as compression rate in-

creases (Fujibayashi(2003)). Furthermore, Laha et

al. (Laha(2004)) showed that images compressed by

VQ provide better PSNR than do those compressed

by JPEG under the condition of the same bits rate.

These are advantageous points of VQ. In principle,

since performance of image compression with VQ is

determined by a CB, design of a CB is essential for

VQ.

A CB is indispensable to carry out VQ, and also

quality of a decoded image depends on the CB. In

this sense, a CB is essential for VQ. To design a CB,

we ﬁrst prepare learning images. The learning images

are segmented into blocks. These blocks constitute

learning vectors. To generate CVs, which constitute

79

Takeda Y., Noro E., Maeda J. and Suzuki Y..

A COMPARATIVE STUDY TO DESIGN A CODE BOOK FOR VECTOR QUANTIZATION.

DOI: 10.5220/0003064200790084

In Proceedings of the International Conference on Fuzzy Computation and 2nd International Conference on Neural Computation (ICFC-2010), pages

79-84

ISBN: 978-989-8425-32-4

Copyright

c

2010 SCITEPRESS (Science and Technology Publications, Lda.)

a CB, the learning vectors are classiﬁed into clusters

using a clustering algorithm. The prototype of each

cluster is a CV. In this paper, we comparatively study

clustering algorithm to construct an optimal CB. Al-

gorithms studied in this paper are GLA (generalized

Lloyd algorithm), FCM (fuzzy c means), AP (afﬁn-

ity propagation), and GA (genetic algorithm). Two

hybrid algorithms, AP+GLA and GA+FCM are also

examined. Performance of the algorithms is evaluated

by PSNR and NPIQM (normalized perceptual image

quality measure).

The paper is organized as follows. Algorithms to

construct a CB are described in section 2. Computa-

tional experiments to determine an optimal algorithm

for constructing a CB are shown in section 3. Finally,

the paper is concluded in section 4.

2 ALGORITHMS TO

CONSTRUCT A CB

2.1 GLA and FCM

One the most widely used clustering algorithm is

GLA (generalized Lloyd algorithm) (Bezdek(1981)).

GLA is a so-called hard clustering algorithm, in

which a learning vector is assigned to only one clus-

ter. For clustering, we ﬁrst determine the number

of clusters, k. To formulate the GLA, there are k

CVs, Y =

{

y

1

,y

2

,...,y

k

}

and M learning vectors,

M =

{

x

1

,x

2

,...,x

M

}

. The learning vector x

i

is as-

signed to the jth cluster when the following equation

is satisﬁed.

d(x

i

,y

j

) = min

y

j

∈Y

d(x

i

,y

j

) =

x

i

− y

j

2

,(i = 1, 2,...,M).

(1)

Membership function is degree of belonging to a clus-

ter. Since a learning vector belongs to only one cluster

in the GLA, membership function is deﬁned as

u

j

(x

i

) =

(

1(ifd(x

i

,y

j

) = min

y

j

∈Y

d(x

i

,y

j

))

0(otherwise)

. (2)

When clustering is completed, CVs are computed as

y

j

=

M

∑

i=1

u

j

(x

i

)x

i

M

∑

i=1

u

j

(x

i

)

(∀ j = 1,2,...,k). (3)

Computation from (1) to (3) is repeated to minimize

distortion.

J =

k

∑

j=1

M

∑

i=1

u

j

(x

i

)

x

i

− y

j

2

. (4)

The computation ends when J becomes smaller than

the predetermined value ε. While the GLA is hard

clustering, fuzzy clustering assigns the learning vec-

tor to multiple clusters. Formulations are carried

out in the same manner of GLA (Bezdek(1981);

Baraldi(1999a); Baraldi(1999b); H

¨

oppner(1999)).

2.2 Real-coded GA

We designed a CB using a genetic algorithm (GA).

GA have been extensively studied to ﬁnd the global

optimal solution in multi-dimensional space for com-

plex problems (Iba(1999)). It is expected that

an optimal CB for VQ can be designed using a

GA (Hall(1999)). A GA is a stochastic search method

and its idea is based on the mechanism of natural se-

lection and genetics. The principal procedures of a

GA consists of selection, crossover, and mutation. In

a GA, genes are usually coded by binary values: zero

or one. This is a bit-string GA. It shows good per-

formance in searching for a solution in a global area.

However, the solution is necessarily precise. For this

reason, a bit-string GA is used in combination with

a local search method. Furthermore, phase structure

of a genotype space is much different from that of a

phenotype space in a bit-string GA. We select two in-

dividuals from parents that are close each other in a

phenotype space, and we carry out crossover to pro-

duce their children. The children are not necessarily

produced in the neighborhood of their parents. Even

though a GA ﬁnds a promising search area by selec-

tion, the crossover may drag the GA away from the

area. Thus, a GA does not work well at the middle

and last search stages (Kita(1998); Ono(1999)).

To overcome this disadvantage, we employed a

real-coded GA in which genes are coded by real val-

ues instead of binary values. In the real-coded GA,

variable space is continuous, while it is not continu-

ous for the binary-coded GA. This continuity of vari-

able space may produce good results (Kita(1998);

Ono(1999)). We also employed the minimal gener-

ation gap (MGG) algorithm for selection and simu-

lated binary crossover (SBX) is used to generate a

new population (Sato(1997); Deb(1999)). The MGG

algorithm could avoid evolutionary stagnation in the

last stage of the search, which may be involved by

simple GA.

In SBX, we ﬁrst randomly select two individuals,

P

1

and P

2

, from N individuals (parents) such as

P

1

: x

1

1

x

1

2

···x

1

k

x

1

k+1

···x

1

n

P

2

: x

2

1

x

2

2

···x

2

k

x

2

k+1

···x

2

n

.

A crossover point is also set randomly. We suppose

that genes at the crossover point are x

1

k

and x

2

k

. Mean

ICFC 2010 - International Conference on Fuzzy Computation

80

and variance of alleles of these genes are computed

as µ

k

and σ

k

. Then random numbers with a normal

distribution are generated by µ

k

and σ

k

. New genes

whose alleles are ¯x

1

k

and ¯x

2

k

at the crossover point are

produced by this normal distribution such as

P

1

: x

1

1

x

1

2

··· ¯x

1

k

x

1

k+1

···x

1

n

P

2

: x

2

1

x

2

2

··· ¯x

2

k

x

2

k+1

···x

2

n

.

Then we carry out crossover to generate two children

as

P

1

: x

2

1

x

2

2

··· ¯x

2

k

x

1

k+1

···x

1

n

P

2

: x

1

1

x

1

2

··· ¯x

1

k

x

2

k+1

···x

2

n

.

These procedures are repeated until we generate N

children. In the GA with SBX, the distance between

two individuals may be large in the early stage of a

search. Therefore, the GA can search for a solution

in a global area. On the other hand, since the distance

between individuals may be small in the last stage of

the search, the GA search can be carried out in a local

area. In this manner, the shift from a global area to

a local area enables an effective search to be carried

out.

2.3 Afﬁnity Propagation

We designed a CB using afﬁnity propagation

(AP) (Frey(2007)). Computation was carried out

based on the programs provided at the website

(http://www.psi.toronto.edu/afﬁnitypropagation/).

AP is an effective clustering algorithm that can not

only avoid the initial value problem but also realize

fast clustering for a large amount of data. Frey

and Dueck proposed AP and demonstrated its good

performance for clustering tasks such as clustering

images of faces, putative exons to ﬁnd genes, and

the problem of identifying a restricted number of

Canadian and American cities (accessibility from

large subsets of other cities). Most clustering al-

gorithms proposed so far compute cluster centers

from the data points forming respective clusters. It

is usually a mean of data points in a cluster. The AP

algorithm ﬁnd data points as the cluster centers for

respective clusters. This is an essentially different

point from other clustering algorithms. There two

kinds of data points: exemplar and just data point.

An exemplar corresponds to the cluster center of a

previous clustering algorithm. Similarity s(i, k) is

deﬁned to indicate how well the data point with index

k is appropriate to be the exemplar for data point i. It

is formulated for points x

i

and x

j

as

s(i,k) = −

k

x

i

− x

k

k

2

, (5)

where indexes i and k indicate a data point and a po-

tential exemplar.

There are two messages, responsibility r(i,k) and

availability a(i, k), that are exchanged between data

points in the process of clustering. The responsibility

is sent from data point i to candidate exemplar k. It

is accumulated evidence of how well data point k is

appropriate for data point i as an exemplar. The re-

sponsibility is sent to all potential exemplars to ﬁnd

the optimal exemplar for point i. r(i,k) is computed

during the message exchange as follows:

r(i, k) = s(i,k) − max

k

0

6=k

a(i,k

0

) + s(i,k

0

)

, (6)

where initial a(i,k

0

) is set to zero. The other message

is the availability which is sent from exemplar k to

data point i. It is a message of appropriateness from

exemplars to data point i to choose k as the exemplar

for i. a(i,k

0

) is computed as

a(i,k) = min

(

0,r(k,k) +

∑

i

0

/∈(i,k)

max(0,r(i

0

,k)

)

.

(7)

Self-availability is also computed as

a(k, k) =

∑

i

0

6=k

max

0,r(i

0

,k)

. (8)

3 COMPUTATIONAL

EXPERIMENTS

We constructed four CBs using GLA, FCM, GA, and

AP. The image consists of four popular images used

in image processing: Mandrill, Milk drop, Parrots,

and Peppers. The training images are segmented into

4 × 4 blocks in size to make training vectors. The

number of CVs is 256. For both the GLA and FCM,

the number of iterations to update CVs is 100. In

the FCM, m is set to 1.2. In the GA, N is 30 and

T is 600. We selected ﬁve test images (Lenna, Earth,

Airplane, Sailboat, and Aerial) and encoded these im-

ages using CBs constructed by the GLA, FCM, GA,

and AP (Takeda(2010)). Performance of the individ-

ual clustering algorithms are examined by quality of

decoded test images. Image quality is evaluated by

both PSNR and NPIQM. PSNR is computed as

PSNR = 10 log

10

PS

2

MSE

(dB). (9)

NPIQM is introduced by Al-Otum (Al-Otum(2003)).

The measure is proposed to evaluate perceptual image

quality. There is a ﬁve-step image quality scale: 1-

unacceptable, 2- poor quality, 3- acceptable, 4- good,

and 5- pleasant and excellent quality. We also exam-

ined hybrid methods. In one method, initial CVs are

A COMPARATIVE STUDY TO DESIGN A CODE BOOK FOR VECTOR QUANTIZATION

81

generated using afﬁnity propagation and then CVs are

computed by GLA. In the other method, initial CVs

are generated using GA and then CVs are computed

by FCM.

Performance evaluation by PSNR for each algo-

rithm is shown in Table 1. The performance of

each algorithm is categorized as higher or lower per-

formance. The higher performance group consists

of GLA, AP and AP+GLA, and lower performance

group consists of FCM, GA and GA+FCM. Table 2

shows performance evaluation by NPIQM for each

algorithm. The performance of each algorithm is also

categorized as higher or lower performance. In the

same manner as PSNR, GLA, AP and AP+GLA be-

long to the higher performance group, while FCM,

GA and GA+FCM belong to the lower performance

group. From the two performance evaluations, GLA,

AP, and AP+GLA are able to produce a CB with

higher quality. AP+GLA shows the best performance.

In AP+GLA, initial CVs are generated by AP and

clustering of learning vectors is carried out by GLA

using those initial CVs. The higher performance of

GLA, AP, and AP+GLA is supported by an average

distortion. It is computed as

D

ave

=

1

M

M

∑

i=1

min

y

j

∈Y

d(x

i

,y

j

), (10)

where d(x

i

,y

j

) =

x

i

− y

j

2

. x

i

is a vector to be en-

coded and y

j

is a CV. M is the number of vectors to

be encoded. Table 3 shows values of D

ave

. GLA,

AP and AP+GLA show smaller values of D

ave

than

those of FCM, GA and GA+FCM. AP+GLA shows

the smallest D

ave

. Figure 1 shows examples of the de-

coded image “Lenna ”. Corresponding to the results

described above, images decoded by the CBs con-

structed with GLA, AP and AP+GLA show higher

quality than those cecoded by the CBs constructed

with FCM, GA and GA+FCM. In conclusion, CBs

constructed by GLA, AP, and AP+GLA are superior

to those constructed by FCM, GA and GA+FCM. The

hybrid method AP+GLA is the best method for con-

struting a CB for VQ.

In the computational experiments, AP is an effec-

tive method for designing a CB. AP+GLA shows the

best performance. AP is a clustering algorithm and

it ﬁnds a data point as a cluster center. The other

clustering algorithms determine the cluster center as

an average of data belonging to the cluster. The AP

algorithm recursively sends messages to obtain data

points that become cluster centers. As stated above,

AP determines data points as cluster centers. These

data points considered to be good initial cluster cen-

ter. In our experiments, GLA showed better perfor-

mance than that of FCM. Both GLA and FCM have

an initial value problem, so that clustering depends on

initial values. However, since AP gives good initial

values for GLA and FCM, AP+GLA shows the best

performance. In the GA, we could not obtain good re-

sults. The reason is thought to be smaller the number

of individuals, 30, in the experiments. N = 30 was

determined by the basis of computational cost. The

GA requires huge computational cost to ﬁnd good so-

lutions. This relatively small N may not ﬁnd good

solutions. Further study is needed for conﬁrming this

speculation.

4 CONCLUSIONS

We constructed four kinds of CB for VQ. GLA,

FCM, AP and GA algorithms were used to con-

struct the CBs. Two hybrid algorithms, AP+GLA and

GA+FCM, were also employed to construct CBs. The

six algorithms were comparatively studied to ﬁnd the

best algorithm. PSNR and NPIQM were used to eval-

uate CBs constructed by those algorithms. Compu-

tational experiments show that AP+GLA is the best

algorithm for constructing a CB.

Table 1: PSNRs of decoded images.

test images

algorithms Lenna Earth Airplane Sailboat Aerial

GLA 27.45 28.12 25.91 26.53 24.84

FCM 26.36 27.82 24.83 24.89 24.50

GA 26.24 27.47 24.74 24.95 24.47

AP 27.39 28.47 26.16 26.47 25.00

AP+GLA 27.52 28.47 26.31 26.71 25.05

GA+FCM 26.32 27.79 24.82 24.89 24.51

Table 2: NPIQMs of decoded images.

test images

algorithms Lenna Earth Airplane Sailboat Aerial

GLA 4.29 4.31 4.06 4.20 4.15

FCM 4.11 4.18 3.97 3.97 4.02

GA 4.12 4.21 4.01 3.98 4.06

AP 4.28 4.31 4.19 4.19 4.15

AP+GLA 4.30 4.35 4.15 4.22 4.16

GA+FCM 4.10 4.18 3.93 3.94 4.02

ACKNOWLEDGEMENTS

This research project is partially supported by grant-

in-aid for scientiﬁc research of Japan Society for the

Promotion of Science (21500211).

ICFC 2010 - International Conference on Fuzzy Computation

82

Table 3: Average distortion.

algorithms GLA FCM GA AP AP+GLA GA + FCM

average distortiuon 33.25 37.31 38.24 33.28 32.64 37.10

Figure 1: Decoded images of Lenna. The top left image is decoded by GLA. The top middle image is decoded by FCM. The

top right image is decoded by GA. The bottom left image is decoded by AP. The top middle image is decoded by AP+GLA.

The bottom right image is decoded by GA+FCM.

REFERENCES

R. C. Gonzalez, and R. E. Woods, Digital image processing,

Person Prentice Hall/New Jersy, 2008.

K. Sayood, Introduction to data compression, Morgan

Kaufmann Publisher: Boston, 2000.

T. Miyamoto, Y. Suzuki, S. Saga, and J. Maeda, Vector

quantization of images using fractal dimensions, 2005

IEEE Mid-Summer Workshop on Soft Computing in

Industrial Application, pp. 214-217, 2005.

K. Sasazaki, S. Saga, J. Maeda, and Y. Suzuki, Vector quan-

tization of images with variable block size, Applied

Soft Computing, vol. 8, pp. 634-645, 2008.

M. Fujibayashi, T. Nozawa, T. Nakayama, K. Mochizuki,

M. Konda, K. Kotani, S. Sugawara, and T. Ohmi, A

still-image encoder based on adaptive resolution vec-

tor quantization featuring needless calculation elimi-

nation architecture, IEEE Journal of Solid-State Cir-

cuit, vol. 38, no. 5, pp. 726-733, 2003.

A. Laha, N. R. Pal, and B. Chanda, Design of vector quan-

tization for image compression using self-organizing

feature map and surface ﬁtting, IEEE Tans Image Pro-

cessing, vol. 13, no. 10, pp. 1291-1303, 2004.

J.C. Bezdek, Pattern recognition with fuzzy objective func-

tion algorithms, Plenum: New York, 1981.

A. Baraldi and P. Blonda, Survey of fuzzy clustering algo-

rithms for pattern recognition-Part I, IEEE Trans Sys-

tems, Man and Cybernetics-Part B: Cybernetics, vol.

29, no. 6, pp. 778-785, 1999(a).

A. Baraldi and P. Blonda, Survey of fuzzy clustering algo-

rithms for pattern recognition-Part II, IEEE Trans Sys-

tems, Man, and Cybernetics-Part B: Cybernetics, vol.

29, no. 6, pp. 786-801, 1999(b).

F. H

¨

oppner, F. Klawonn, R. Kruse, and T. Runkler, Fuzzy

Cluster Analysis, John Wiley & Sons, LTD: New

York, 1999.

H. Iba, To solve the mystery of genetic algorithms (in

Japanese), Ohmusha: Tokyo, 1999.

L. O. Hall, I. B. Ozyurt and J. Bezdek, Clustering with a

A COMPARATIVE STUDY TO DESIGN A CODE BOOK FOR VECTOR QUANTIZATION

83

generally optimized approach, IEEE Trans Evolution-

ary Computation, vol. 3, no. 2, pp. 103-112, 1999.

H. Kita, I. Ono, and S. Kobayashi, Theoretical analysis of

the unimodal distribution crossover for real-coded ge-

netic algorithms, Evolutionary Computation Proceed-

ings, 1998. IEEE World Congress on Computational

Intelligence, volume, issue, 4-9, pp. 529-534, 1998.

I. Ono, H. Sato, and S. Kobayashi, Real-coded genetic al-

gorithm for function optimization using the unimodal

distribution crossover, Journal of Japanese Society for

Artiﬁcial Intelligence, vo. 14, no. 6, pp. 1146-1155,

1999.

H. Sato, I. Ono, and S. Kobayashi, A new generation alter-

nation model of genetic algorithms and its assessment,

J. Japanese Society of Artiﬁcial Intelligence, vo. 12,

no. 5, pp. 1-10, 1997.

K. Deb, H-G., Beyer, Self-adaptive genetic algorithms with

simulated binary crossover. Technical Report No. CI-

61/99, University Dortmund, 1999.

B. J. Frey and D. Dueck, Clustering by Passing Messages

Between Data Points, Science, vol. 315, pp. 972-976,

2007.

Y. Takeda, Code book optimization with afﬁnity propaga-

tion for image compression (in Japanese), Master The-

sis of Muroran Institute of Technology, 2010.

H. M. Al-Otum, Qualitative and quantitative image quality

assessment of vector quantization, JPEG, and JPEG

2000 compressed image, Journal of Electronic Imag-

ing, vol. 12. No. 3, pp. 511-521, 2003.

ICFC 2010 - International Conference on Fuzzy Computation

84