EVALUATING THE POTENTIAL OF TEXTURE AND COLOR

DESCRIPTORS FOR REMOTE SENSING IMAGE RETRIEVAL AND

CLASSIFICATION

Jefersson A. dos Santos, Ot´avio A. B. Penatti and Ricardo da S. Torres

RECOD Lab / LIS, Institute of Computing, University of Campinas – Unicamp, 13084–970, Campinas, SP, Brazil

Keywords:

Image descriptors, Remote sensing image, Image classiﬁcation, Image retrieval.

Abstract:

Classifying Remote Sensing Images (RSI) is a hard task. There are automatic approaches whose results nor-

mally need to be revised. The identiﬁcation and polygon extraction tasks usually rely on applying classiﬁcation

strategies that exploit visual aspects related to spectral and texture patterns identiﬁed in RSI regions. There

are a lot of image descriptors proposed in the literature for content-based image retrieval purposes that may be

useful for RSI classiﬁcation. This paper presents a comparative study to evaluate the potential of using suc-

cessful color and texture image descriptors for remote sensing retrieval and classiﬁcation. Seven descriptors

that encode texture information and twelve color descriptors that can be used to encode spectral information

were selected. We perform experiments to evaluate the effectiveness of these descriptors, considering image

retrieval and classiﬁcation tasks. To evaluate descriptors in classiﬁcation tasks, we also propose a methodology

based on KNN classiﬁer. Experiments demonstrate that Joint Auto-Correlogram (JAC), Color Bitmap, Invari-

ant Steerable Pyramid Decomposition (SID) and Quantized Compound Change Histogram (QCCH) yield the

best results.

1 INTRODUCTION

Agriculture has an important role in the economy of

several countries. The results of agricultural activi-

ties are directly linked to the productivity. Therefore,

many researches have been investigating new ways to

improve agricultural practices and, consequently, to

increase the quantity and quality of what is produced.

In this scenario, crop monitoring is a fundamental

activity and using Geographic Information Systems

(GIS) has made it easier.

Some of the main issues related to crop monitor-

ing are: How is the land occupation? What is cul-

tivated in a given region? Where are some cultures

cultivated?

Remote Sensing Images (RSIs) provide the ba-

sis for the creation of information systems that sup-

port the decision-making process based on land cover

changes. Using RSI in crop monitoring requires the

recognition of the regions of interest and the extrac-

tion of the polygons around these regions.

The identiﬁcation and polygon extraction tasks

usually rely on applying classiﬁcation strategies that

exploit visual aspects related to spectral and texture

patterns identiﬁed in RSI regions. These tasks can

be performed automatically or manually. The “man-

ual” approach is based on image editors by witch

users can deﬁne or draw polygons that represent re-

gions of interest using the raster image as back-

ground. In general, automatic approaches use clas-

siﬁcation strategies based on pixel information. How-

ever, the most used pixel classiﬁcation algorithm,

MaxVer (Showengerdt, 1983) is not always effective.

The main drawback of automatic approaches is

concerned with its sensitivity to image noises (e.g.,

for example, distortions that can be found in moun-

tainous regions). Another important problem in the

automatic approaches is concerned with the fact that

they usually fail to correctly identify borders between

distinct regions within the same image. Thus, in prac-

tical situations, the results obtained need to be revised.

As these revisions take a lot of time, it is sometimes

more convenient to the user to perform recognition

manually.

Content-based Image Retrieval (CBIR) systems

are developed to provide efﬁcient and effective means

to retrieve images. In these systems, the searching

process consists of, for a given image, computing the

most similar images stored in the database, consider-

ing only image properties, like color and texture, for

203

A. dos Santos J., A. B. Penatti O. and da S. Torres R. (2010).

EVALUATING THE POTENTIAL OF TEXTURE AND COLOR DESCRIPTORS FOR REMOTE SENSING IMAGE RETRIEVAL AND CLASSIFICATION.

In Proceedings of the International Conference on Computer Vision Theory and Applications, pages 203-208

DOI: 10.5220/0002843402030208

 SciTePress

instance. The searching process relies on the use of

image descriptors. A descriptor can be characterized

by two functions: feature vector extraction and sim-

ilarity computation. The similarity between two im-

ages is computed as a function of their feature vectors

distance.

This paper presents an evaluation of image de-

scriptors for RSI retrieval and classiﬁcation. Seven

descriptors that encode texture information and

twelve color descriptors that can be used to encode

spectral information were selected. We perform ex-

periments to evaluate the effectiveness of these de-

scriptors in retrieval sessions and classiﬁcation tasks.

2 RELATED WORK

Several methods have been proposed to improve the

performance of RSI classiﬁcation techniques. In (Mo

et al., 2007), a new method considering image seg-

mentation, GIS, and data mining algorithms was pre-

sented. Compared with pixel-based classiﬁcation, the

results showed best agreement with visual interpre-

tation. The work proposed in (Yildirim et al., 2005)

applied a morphological ﬁlter in an image which was

classiﬁed by MaxVer algorithm. The results were

compared with the other classiﬁcation algorithms

(Fisher linear likelihood, minimum Euclidean dis-

tance and ECHO). In (hyung Kim et al., 2007), three

Land Cover Classiﬁcation Algorithms are compared

for monitoring North Korea using multi-temporal

data.

Recently, some descriptors for RSI purposes have

been proposed. Tusk et. al. (Tusk et al., 2003) pre-

sented algorithms that allow automatic selection of

features for region and tile similarity searches apply-

ing relevance feedback. Samal et. al. (Samal et al.,

2009) proposed a RSI descriptor, called SIMR (Satel-

lite Image Matching and Retrieval). SIMR computes

spectral and spatial attributes of the images using a

hierarchical representation. A unique aspect of this

descriptor are the couples of second-level spatial au-

tocorrelation with quad tree structure.

There is a large number of image descriptors pro-

posed in the literature for CBIR that can be useful

to classify and recognize RSI regions. Using de-

scriptors, systems can compute how similar regions

of an image are when compared to a spectral or tex-

ture pattern in which users are interested. This infor-

mation can, therefore, be used to classify the whole

image. Santos et. al. (dos Santos et al., 2009) pre-

sented a semi-automatic method to vectorize regions

from remote sensing images using relevance feedback

based on genetic programming (GP). The solution

consists of using image descriptors to encode texture

and spectral features from the images, applying rel-

evance feedback based on GP to combine these fea-

tures with information obtained from the users inter-

actions and, ﬁnally, segment the image. At the end,

segmented image (raster) is converted into a vector

representation.

Descriptors effectiveness can vary from one ap-

plication to another. This fact shows the importance

of evaluating descriptors considering speciﬁc appli-

cations. A comparative study of color descriptors for

Web image retrieval is presented in (Penatti and Tor-

res, 2008). However, to the best of our knowledge no

study has been conducted to evaluate the performance

and effectiveness of image descriptors in RSI retrieval

and classiﬁcation tasks.

3 IMAGE DESCRIPTORS

The descriptors chosen for the evaluation are impor-

tant descriptors from the literature and recently pro-

posed descriptors.

The color descriptors evaluated in this work are:

GCH (Swain and Ballard, 1991), CGCH (Stricker

and Orengo, 1995), LCH (Swain and Ballard, 1991),

CCV (Pass et al., 1996), ACC (Huang et al., 1997),

JAC (Williams and Yoon, 2007), BIC (de O. Stehling

et al., 2002), CBC (de O. Stehling et al., 2001),

Color Bitmap (Lu and Chang, 2007), CSD (Man-

junath et al., 2001), CW-HSV (Utenpattanant et al.,

2006) and CM (Paschos et al., 2003).

The texture descriptors evaluated in this work are:

LBP (Ojala et al., 2002), HTD (Wu et al., 2000),

SID (Zegarra et al., 2007), CCOM (Kovalev and

Volmer, 1998), Unser (Unser, 1986), QCCH (Huang

and Liu, 2007), and LAS (Tao and Dickinson, 2000).

4 EXPERIMENTS

This section presents the databases used in the exper-

iments and the measures used to evaluate the descrip-

tors.

4.1 Image Databases

Two image databases were created to evaluate image

descriptors based on distinct RSIs. One of them can

be classiﬁed as “easy recognition” (pasture image)

while the other as “hard recognition” (coffee image).

Information about the used RSIs is showed in Table 1.

In the experiments, one image is represented by a

tile from the original RSI. The size of the tile was

VISAPP 2010 - International Conference on Computer Vision Theory and Applications

204

Table 1: Remote Sensing Images used in the experiments.

Image1 Image2

Region of interest pasture coffee

Terrain plain mountainous

Satelite CBERS SPOT

Spatial resolution 20 meters 2,5 meters

Bands composition R-IR-G (342) IR-NIR-R (342)

Acquisition date 08–20–2005 08–29–2005

Location Laranja Azeda Monte Santo

Basin, MS County, MG

Dimensions (px) 1310× 1842 2400× 2400

ﬁxed according to the common extension value of

a region of interest. Coffee crops are normally in

small parcels on the same farm. We deﬁned that

75 × 75 meters is a good value to the size of the

partition. For pasture parcels, that are larger, the

chosen value was 400 × 400 meters. The dimen-

sion of partitions are ﬁxed in the experiments. We

used 30× 30 pixels to partition the coffee image and

20 × 20 pixels for the pasture image. The number of

partitions for the pasture and coffee images was 5980

and 6400, respectively.

A “mask” containing all regions of interest from

the RSIs used in the experiments was used to know

the class of each tile. A “mask” is a binary image

where value 1 represents pixels of regions of interest.

The “masks” used in our experiments were classiﬁed

manually by agricultural specialists.

4.2 Evaluation Measures

The main objective of the experiments was to evaluate

and compare the descriptors considering effectiveness

issues. For this purpose, we conﬁgured two experi-

ments: retrieval effectiveness evaluation and overall

accuracy classiﬁcation.

To evaluate retrieval effectiveness, Precision ×

Recall curves were used. Precision quantiﬁes the per-

centage of relevant images present in the retrieved

results. Recall is a measure that represents the per-

centage of the relevant images that are retrieved. A

Precision × Recall curve indicates the variation in

Precision values as the rate of relevant images from

the database (Recall) changes. Intuitively, the higher

the curve, the better the effectiveness. The Precision

and Recall curves were computed based on the av-

erage values obtained for each query image in each

database. We used 340 and 100 queries in the Pasture

and Coffee image sets respectively for all the color

and texture descriptors presented in Section 3.

To compute the overall accuracy of each descrip-

tor we implemented a variation of K-Nearest Neigh-

boor (KNN) classiﬁer. First of all, a set of tiles

from the database was randomly selected to be used

as training set. The set, corresponding to 10% of

the database size, is composed by relevants and non-

relevants samples in the same proportion found in the

full database. To classify one tile, each descriptor

evaluated was used to compute the distance between

the given tile and all the training set tiles. Based on

the descriptor distances, the training set is ranked and

the ﬁrst K tiles are weighted inversely proportional

to their position in the rank. Finally, the sum of the

tiles’ weights for each class (relevant or non-relevant)

is computed. The greater sum indicates the class of

the input tile. To test the classiﬁcation effectiveness

of the descriptors 100 tiles were used for each RSI.

4.3 Results

Figures 1, 2, 3, and 4 show the Precision × Re-

call curves for color and texture descriptors in the

databases used.

From Figure 1 we can see that good descrip-

tors considering retrieval effectiveness are JAC, Color

Bitmap, and ACC.

0.2

0.4

0.6

0.8

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

Precision

Recall

ACC

BIC

CBC

CCV

CGCH

CSD

CW-HSV

GCH

JAC

LCH

Color Bitmap

Figure 1: Precision × Recall curves for color descriptors in

the Pasture Image Set.

From Figure 2, it is possible to see that JAC

presents the highest Precision values even for small

values of Recall and for Recall equal to 1.

Analyzing Figure 3 it is possible to notice that SID

has the highest Precision values for all values of Re-

call.

Considering curves for the Coffee database in Fig-

ure 4, it is possible to see that the descriptors present

very similar Precision valuesand these values are near

32% when Recall reaches 10%.

After analyzing the curves for color and texture

descriptors it is possible to say that color descrip-

tors are slightly better than texture descriptors for the

databases used. For example, in the Pasture database,

for Recall equal to 10%, the highest Precision value

for color descriptors is around 62% (JAC) and for

EVALUATING THE POTENTIAL OF TEXTURE AND COLOR DESCRIPTORS FOR REMOTE SENSING IMAGE

RETRIEVAL AND CLASSIFICATION

205

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

Precision

Recall

ACC

BIC

CBC

CCV

CGCH

CSD

CW-HSV

GCH

JAC

LCH

Color Bitmap

Figure 2: Precision × Recall curves for color descriptors in

the Coffee Image Set.

0.2

0.4

0.6

0.8

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

Precision

Recall

CCOM

HTD

LAS

LBP

QCCH

SID

Unser

Figure 3: Precision × Recall curves for texture descriptors

in the Pasture Image Set.

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

Precision

Recall

CCOM

HTD

LAS

LBP

QCCH

SID

Unser

Figure 4: Precision × Recall curves for texture descriptors

in the Coffee Image Set.

texture descriptors is near 47%. For Recall equal to

1, color descriptors achieve Precision of 25% (Color

Bitmap) and texture descriptors achieve almost 23% .

For the Coffee database, it is possible to notice that,

for Recall equal to 10%, the highest curve of a color

descriptor reaches 61% (JAC) while the highest curve

of a texture descriptor reaches almost 40% (SID). For

Recall equal to 1, there is almost no difference in the

Precision values.

According to the results for the coffee database

presented (Figure 5), is observed that some descrip-

tors achieved high overall accuracy values. The color

descriptors BIC, ACC, CBC, Color Bitmap, and JAC

were the best ones reaching more than 60% of overall

accuracy for any k. JAC produced the highest accu-

racy values, being the only one with values over 70%

(72% for k=1, 79% for k=3, and 73% for k=7 and

k=10). With regard to the texture descriptors, QCCH,

SID, and LAS yielded the highest accuracy values,

52% for k=3. For k values different than 3, the tex-

ture descriptors presented accuracy below 48%. The

CCOM descriptor did not reach 25% of accuracy in

any of the experiments in the coffee database.

According to the results for the pasture database

presented (Figure 6) we can see that some descrip-

tors yielded good accuracy values. The color descrip-

tors JAC, Color Bitmap, and CBC reached near or

more than 60% of overall accuracy. JAC descriptor

was again the descriptor with highest accuracy value,

reaching 78% for k=3 and being over 65% for all k

values. The texture descriptors yielded lower accu-

racy values when compared with most of color de-

scriptors. QCCH, SID and Unser were the only tex-

ture descriptors to reach accuracy above 50%. For

k=3, QCCH reached 58% of accuracy, SID 55% and

Unser 53%. CCOM descriptor reached the lowest ac-

curacy values, being below 25% for all k values.

Considering the accuracy values in both image

databases, we can point JAC as the best color de-

scriptor. However, JAC generates big feature vec-

tors and so, it is slower to compare them. If storage

and time requirements are not critical, JAC is the best

choice. Other descriptors with near effectiveness are

CBC and Color Bitmap. CBC has complex extraction

and distance function. Color Bitmap is the best choice

among the color descriptors, which balance simple al-

gorithms and good effectiveness. Amongst the texture

descriptors, QCCH and SID reached the highest ac-

curacy values, being SID computationally more com-

plex than QCCH for features extraction.

5 CONCLUSIONS

This paper presents a comparative study of image de-

scriptors for the classiﬁcation and recognition of RSI

regions. Twelve color descriptors and seven texture

descriptors were compared considering effectiveness

issues. The effectiveness was measured by precision-

recall curves and overal accuracy. JAC and Color

Bitmap presented the best results among the color de-

scriptors evaluated, while SID was the best texture de-

scriptor. We also proposed methodology to evaluate

VISAPP 2010 - International Conference on Computer Vision Theory and Applications

206

Figure 5: Overall accuracy classiﬁcation of each descriptor for Coffee using KNN 1, 3, 7 and 10.

Figure 6: Overall accuracy classiﬁcation of each descriptor for Pasture KNN 1, 3, 7 and 10.

image descriptors in classiﬁcation problems by using

KNN classiﬁer.

The next stage of this work is to combine the best

descriptors and to evaluate their use in RSI classiﬁca-

tion tasks.

ACKNOWLEDGEMENTS

Authors thank to FAPESP, CNPq (BioCORE project)

and Fapesp-Microsoft Research Virtual Institute

(eFarms project) for ﬁnancial support.

REFERENCES

de O. Stehling, R., Nascimento, M. A., and Falcao, A. X.

(2001). An adaptive and efﬁcient clustering-based

approach for content-based image retrieval in im-

age databases. In Proceedings of the International

Database Engineering & Applications Symposium,

pages 356–365, Washington, DC, USA.

de O. Stehling, R., Nascimento, M. A., and Falc˜ao, A. X.

(2002). A compact and efﬁcient image retrieval ap-

proach based on border/interior pixel classiﬁcation. In

Proceedings of the eleventh international conference

on Information and knowledge management, pages

102–109, New York, NY, USA.

dos Santos, J. A., Lamparelli, R. A., and da Silva Torres,

R. (2009). Using relevance feedback for classifying

EVALUATING THE POTENTIAL OF TEXTURE AND COLOR DESCRIPTORS FOR REMOTE SENSING IMAGE

RETRIEVAL AND CLASSIFICATION

207

remote sensing images. In Proceedings of Brazilian

Remote Sensing Symposium, pages 7909–7916, Natal,

RN, Brazil.

Huang, C. and Liu, Q. (2007). An orientation indepen-

dent texture descriptor for image retrieval. Interna-

tional Conference on Communications, Circuits and

Systems, pages 772–776.

Huang, J., Kumar, S. R., Mitra, M., Zhu, W., and Zabih, R.

(1997). Image indexing using color correlograms. In

Proceedings of the 1997 Conference on Computer Vi-

sion and Pattern Recognition, page 762, Washington,

DC, USA.

hyung Kim, D., gyu Jeong, S., and hwa Park, C. (2007).

Comparison of three land cover classiﬁcation algo-

rithms - isodata, sma, and som - for the monitoring of

north korea with modis multi-temporal data. Korean

Journal of Remote Sensing.

Kovalev, V. and Volmer, S. (1998). Color co-occurence de-

scriptors for querying-by-example. MultiMedia Mod-

eling, 0:32–38.

Lu, T. and Chang, C. (2007). Color image retrieval tech-

nique based on color features and image bitmap. In-

formation Processing and Management, 43(2):461–

472.

Manjunath, B. S., Ohm, J.-R., Vasudevan, V. V., and Ya-

mada, A. (June 2001). Color and texture descriptors.

IEEE Transactions on Circuits and Systems for Video

Technology, 11(6):703–715.

Mo, D.-K., Lin, H., Li, J., Sun, H., and Xiong, Y.-J. (2007).

Design and implementation of a high spatial resolu-

tion remote sensing image intelligent interpretation

system. Data Science Journal, 6:S445–S452.

Ojala, T., Pietik¨ainen, M., and M¨aenp¨a¨a, T. (2002). Mul-

tiresolution gray-scale and rotation invariant texture

classiﬁcation with local binary patterns. IEEE Trans-

actions on Pattern Analysis and Machine Intelligence,

24(7):971–987.

Paschos, G., Radev, I., and Prabakar, N. (2003). Image

content-based retrieval using chromaticity moments.

IEEE Transactions on Knowledge and Data Engineer-

ing, 15(5):1069–1072.

Pass, G., Zabih, R., and Miller, J. (1996). Comparing im-

ages using color coherence vectors. In Proceedings of

the fourth ACM international conference on Multime-

dia, pages 65–73, New York, NY, USA.

Penatti, O. A. B. and Torres, R. d. S. (2008). Color descrip-

tors for web image retrieval: A comparative study.

XXI Brazilian Symposium on Computer Graphics and

Image Processing, pages 163–170.

Samal, A., Bhatia, S., Vadlamani, P., and Marx, D. (2009).

Searching satellite imagery with integrated measures.

Pattern Recogn., 42(11):2502–2513.

Showengerdt, R. (1983). Techniques for Image Process-

ing and Classiﬁcation in Remote Sensing. Academic

Press, New York.

Stricker, M. A. and Orengo, M. (1995). Similarity of

color images. In Niblack, W. and Jain, R. C., edi-

tors, Proc. SPIE Storage and Retrieval for Image and

Video Databases III, volume 2420, pages 381–392.

Swain, M. J. and Ballard, D. H. (1991). Color indexing. In-

ternational Journal of Computer Vision, 7(1):11–32.

Tao, B. and Dickinson, B. W. (2000). Texture recognition

and image retrieval using gradient indexing. Journal

of Visual Communication and Image Representation,

11(3):327 – 342.

Tusk, C., Koperski, K., Aksoy, S., and Marchisio, G.

(2003). Automated feature selection through rele-

vance feedback. In Geoscience and Remote Sens-

ing Symposium, 2003. IGARSS ’03. Proceedings.

2003 IEEE International, volume 6, pages 3691–3693

vol.6.

Unser, M. (1986). Sum and difference histograms for

texture classiﬁcation. IEEE Transactions on Pattern

Analysis and Machine Intelligence, 8(1):118–125.

Utenpattanant, A., Chitsobhuk, O., and Khawne, A. (20-22

February 2006). Color descriptor for image retrieval

in wavelet domain. Eighth International Conference

on Advanced Communication Technology, 1:818–821.

Williams, A. and Yoon, P. (2007). Content-based image

retrieval using joint correlograms. Multimedia Tools

and Applications, 34(2):239–248.

Wu, P., Manjunath, B. S., Newsam, S., and Shin, H. D.

(2000). A texture descriptor for browsing and simi-

larity retrieval. Signal Processing: Image Communi-

cation, 16(1-2):33 – 43.

Yildirim, I., Ersoy, O. K., and Yazgan, B. (2005). Improve-

ment of classiﬁcation accuracy in remote sensing us-

ing morphological ﬁlter. Advances in Space Research.

Zegarra, J. A. M., Leite, N. J., and Torres, R. d. S. (2007).

Rotation-invariant and scale-invariant steerable pyra-

mid decomposition for texture image retrieval. In

XX Brazilian Symposium on Computer Graphics and

Image Processing, pages 121–128, Washington, DC,

USA. IEEE Computer Society.

VISAPP 2010 - International Conference on Computer Vision Theory and Applications

208