A Multi-features Fusion of Multi-temporal Hyperspectral Images
using a Cooperative GDD/SVM Method
Selim Hemissi
1
and Imed Riadh Farah
2
1
ITI, Telecom Bretagne, Brest, France
2
RIADII, ENSI, Manouba, Tunisia
Keywords:
Hyperspectral Data, Feature Fusion, Hyperion, Remote Sensing, SVM, Generalized Dirichlet Distribution,
Generative/Discriminative Model.
Abstract:
Considering the emergence of hyperspectral sensors, feature fusion has been more and more important for
images classification, indexing and retrieval. In this paper, a cooperative fusion method GDD/SVM (General-
ized Dirichlet Distribution/Support Vector Machines), which involves heterogeneous features, is proposed for
multi-temporal hyperspectral images classification. It differentiates, from most of the previous approaches, by
incorporating the potentials of generative models into a discriminative classifier. Therefore, the multi-features,
including the 3D spectral features and textural features, can be integrated with an efficient way into a unified
robust framework. The experimental results on a series of Hyperion images confirm the improved performance
and show that this cooperative fusion approach has consistence over different testing datasets.
1 INTRODUCTION
Presently, the considerable archive produced by satel-
lite remote-sensing sensors is becoming an increas-
ingly valuable source of information. This leads to a
better interpretation of land-cover and land-use evolu-
tion by analyzing the spectral response of the different
earth’s surface elements. Hyperspectral signatures af-
ford a compact recording of reflectance values over a
large domain of the electromagnetic spectrum. They
allow practitioners to map, quantify and qualify effec-
tively the spatio-temporal variations of land surface
(Heinz et al., 2010). The goal of fusion techniques is
to extract complementary information from different
sources to allow for a more informed decision than
one could gain from any of the sources alone. In the-
ory, data fusion provides significant advantages over
single source of features. In addition to the statisti-
cal advantage, the use of multiple kinds of features
may increase the possibility of a target of interest be-
ing observed and characterized resulting in a reduced
error rate. In contrast, fusion may not always result in
an improved decision over simply selecting the most
appropriate source for the task because accurate data
may be fused with very inaccurate data (Nakariyakul
and Casasent, 2004).
2 PROBLEM STATEMENT
The semantic categorization of remote-sensing im-
ages requires analysis of many features of the images
such as texture, spectral profiles, etc. Current feature
fusion approaches commonly concatenate different
features. It gives, generally good results and several
approaches have been proposed using this schema.
However, most of them have various conditional con-
straints, such as noise and imperfection, which might
retain the use of such systems under degraded perfor-
mance. However, how to fuse heterogeneous features
in a flexible way is still an open research question.
Similarly, in the area of Supervised Machine
Learning (SML), diversity with respect to the er-
rors committed by component classifiers has received
much attention (Bishop, 2006). Generative and dis-
criminative approaches are two distinct schools of
probabilistic machine learning. It has shown that
discriminative approaches such as SVM (Cristianini
and Shawe-Taylor, 2000) outperform model based ap-
proaches due to their flexibility in decision bound-
aries estimation. Conversely, since that discrimina-
tive methods are concerned with boundaries, all the
classes need to be estimated conjointly (Ulusoy and
Bishop, 2006). Complementary, one of the interest-
ing characteristics, that generative models have over
discriminative ones, is that they are learnt indepen-
681
Hemissi S. and Riadh Farah I..
A Multi-features Fusion of Multi-temporal Hyperspectral Images using a Cooperative GDD/SVM Method.
DOI: 10.5220/0004377406810685
In Proceedings of the 2nd International Conference on Pattern Recognition Applications and Methods (PRG-2013), pages 681-685
ISBN: 978-989-8565-41-9
Copyright
c
 2013 SCITEPRESS (Science and Technology Publications, Lda.)

dently for each class. Moreover, following their mod-
eling power, generative models are able to deal with
missing data. An ideal fusion method should combine
these two approaches in order to improve the classifi-
cation accuracy (LeBlanc and Saffiotti, 2007).
3 COOPERATIVE SVM/GDD
METHOD FOR
HYPERSPECTRAL IMAGES
CLASSIFICATION
3.1 Overview of the proposed Fusion
Schema
In this paper, we propose a new technique in remote-
sensing images classification by fusing heterogeneous
representations. The proposed approach involve sev-
eral steps including preprocessing; features extrac-
tion; features fusion; matching and classification
stages. The block diagram of the proposed technique
is shown in Fig. 1. In our previous work (Farah et al.,
2010), we proposed a novel 3D model which design
the spectral signature as a three dimensional function
which are the time, reflectance, and wavelength band
(equation 1). For each pixel, we generated a surface
(3D Mesh) which generalizes the usual signature by
adding a time dimension. We call this new representa-
tion the multi-temporal spectral signature. Interested
readers can refer to (Farah et al., 2010).
Figure 1: General workflow of the proposed approach.
3.2 Images Pre-processing and Features
Extraction
In this study multi-temporal hyperspectral images
constitutes the source data. Spectral and textural fea-
tures are the foundational data for this kind of images.
The 3D spectral features are extracted from the rel-
ative mesh of a given pixel (multi-temporal spectral
signature) while the textural ones are derived directly
from images. Mainly, two features vectors are gener-
ated for each pixel as follows:
Heat Kernel Signature (HKS). The HKS is a sig-
nature computed only from the intrinsic geometry of
an object. Suppose (m,g) is a compelte Rieman-
nian manifold, g is the Riemannian metric. δ is the
Laplace-Beltrami operator. The eigenvalues {λ
n
} and
eigenfunctions {φ
n
} of δ are δφ
n
= λ
n
φ
n
, where φ
n
is
normalized to be orthonormal in L
2
(M). The Laplace
spectrum is given by 0 = λ
0
< λ
1
≤ λ
2
≤ . . . ,λ
n
→ ∞.
4 is the Laplace-Beltrami operator. As a local shape
descriptor, Sun et al. (Sun et al., 2009) defined the
heat kernel signature (HKS) by :
h(x,t) = K
x,t
(x,x) =
∞
∑
i=0
e
−λ
t
φ
2
i
(x) (1)
where λ
0
,λ
1
,··· ≥ 0 are eigenvalues and φ
0
,φ
1
,...
are the corresponding eigenfuctions on the Laplace-
Beltrami operator, satisfying δ
X
φ
i
= λ
i
φ
i
. Let’s de-
note this vector by Y .
Spatio-temporal Gabor Filters. Texture is one of
the important characteristics used in identifying ob-
jects or regions of interest. It contains important infor-
mation about the structural arrangement of surfaces.
Fusing texture with 3D spectral information is con-
ducive to the interpretation of remote seeing image
(Wang and Chua, 2005). We use a method for dy-
namic texture modeling based on spatio-temporal Ga-
bor filters. Briefly, the sequence of images is con-
volved with a bank of spatiotemporal Gabor filters
and a feature vector is constructed with the energy of
the responses as components. Let’s denote this vector
by Y
0
.
3.2.1 Multi-features Fusion based on a
Cooperative GDD/SVM Classifier
In this section, we present an approach that combines
an SVM classifier (Burges, 1998) with a generatively
trained GDD model and profits, accordingly, from the
advantages of both techniques. The key idea here is to
concatenate the extracted features into one vector and
to project it in a new space. First, a straightforward
feature combination approach is used to concatenate
feature vectors (Y and Y
0
) to a single feature vector
X = (X
i1
,...,X
idim
). The dim size may differ from
one pixel to another making the fusion and classifi-
cation a challenging tasks. To overcome this limit,
we use the Generalized Dirichelet Distribution (GDD)
model (Bouguila and Ziou, 2010) to map each fea-
ture vector into its Fisher score. Therefore, the Fisher
kernel function from the GDD is used to replace the
Gaussian kernel in the classical SVM.
Let (X
1
,...,X
N
) denote a collection of N multi-
temporal hyperspectral pixels. Each data X
i
is as-
sumed to have dim size, X = (X
i1
,...,X
idim
). Each
ICPRAM2013-InternationalConferenceonPatternRecognitionApplicationsandMethods
682

data X
i
is assumed to be drawn from the following
finite mixture model :
p(X
i
/θ) =
M
∑
j=1
p(X
i
/ j,θ
j
)P( j) (2)
where M is the number of components, the P( j),
(0 < P( j) < 1 and
∑
dim
j=1
P( j) = 1) are the mixing
proportions and p(X / j,θ
j
) is the Probability Density
Function PDF. θ is the set of parameters to be esti-
mated : θ = (α
1
,...,α
M
,P(1),...,P(M)).
If the random vector X = (X
i1
,...,X
idim
) follows
a Dirichelet distribution, the joint density function is
given by :
X = (X
i1
,...,X
idim
) =
τ(
|
α
|
)
∏
dim+1
i=1
τ(α
i
)
dim+1
∏
i=1
X
α
i
−1
i
(3)
Since that each feature vector X may has an arbitrary
dimension, the proposed method defines the fusion
as a projection from one feature vector space (spec-
tral bands) to another with a fixed dimentionnality.
Accordingly, the feature-level fusion is done by pro-
jecting the vector X combining into one vector in the
Fisher space. Thus, the generative model will have
its impact on the final classification result through the
projection of the extracted features in this new space.
SVM classifier is used to classify the fused fea-
tures and the multi-temporal dataset of images. Given
the generative model obtained by GDD with parame-
ters θ, we compute for each sample X the Fisher score
U
d
= 5
θ
logP(x|θ) (the gradient of the log likehood
of x for model θ). The Fisher kernel operates in the
gradient space of the generative mode and provides a
natural similarity measure between data samples. For
each sample, this score is a vector of fixed dimention-
ality. Using this score, the Fisher Information matrix
is defined as I = E
X
i

U
X
i
T
U
X
i

. After Fisher score
normalization, we compute the Fisher kernel function
on the basis of the Euclidean distance between the
scores of the new sample and the training samples :
K(X,X
0
) = U
X
i
I
−1
U
X
0
i
T
(4)
In the second stage, suppose our training set S
consists of labels input vectors (X
i
,z
j
),i = 1, . . . ,m
where X
i
∈ R
n
and z
i
∈ {±1}. Given a kernel ma-
trix and a set of labels z
i
for each sample, the SVM
proceeds to learn a classifier of the form,
z(x) = sign(
∑
i
α
i
z
i
)K(X
i
,X)) (5)
where the coefficients α
i
are determined by solv-
ing a constrained quadratic program which aims to
maximize the margin between classes. In our exper-
iments we used the LIBSVM package. Our research
deals with multi-class problem. The One-Vs-One ap-
proach is adopted to extend the proposed approach to
multi-temporal hyperspectral classification.
4 EXPERIMENTAL RESULTS
The images set used in this experiment were jointly
collected from the Tunisian Institute of Remote-
Sensing (CNT) and the USGS library through the
Glovis Viewer (Clark et al., 2007). Some earlier re-
sults and ground truth maps produced by the CNT
were also used to perform the analysis of the selected
test sites and for validation purposes.
The studied area is being within the line between
the northwest tip of Djerba island on the southeast and
Ras Yonga on the northwest. The centroid for the
study area is at 33
◦
50
0
16
00
N 10
◦
07
0
41
00
E. It is char-
acterized by typical Mediterranean climate with max-
imum temperatures reaching, in the period between
June and August (48
◦
), whereas the coldest tempera-
tures are measured between December and February.
Due to the sea proximity, the climate of the study area
slightly differs from the typical arid or semi-arid ar-
eas. The rainfall is very irregular and ranges between
150 − 240mm with an average of 30 rainy days per
year (September/October). The region has been cho-
sen not only due to the great interest from govern-
mental and non-governmental organizations, but also
because of the coexistence of several oasis such as
Mareth and Teboulbou including various types of veg-
etations that change over time. The vegetation has a
cover of 40% to 60%, comprising predominantly an-
nual plants which develop from the autumn rains and
persist until the end of the following spring. The veg-
etation cover is marked by the predominant species,
Palm, Lythracea (Henn
´
e) and Carex. In this set of
experiments, two time series are available, and thus,
the season spectral variability can be well mapped
through this set of images. An external digital eleva-
tion model and a reference land-cover map provided
by the Tunisian Institute of Remote-Sensing (CNT)
were also available for results assessment. Consider-
ing the differences in multi-temporal images acqui-
sition, we first perform a pre-processing step. Im-
ages were geometrically corrected and geo-coded to
the Universal Transverse Mercator (UTM) coordinate
system based on a topographic map of the study area.
45 regularly distributed ground control points (GCPs)
were used for this purpose. Then, Hyperion images
were converted to reflectance and co-registered and
re-sampled to 30 × 30 m with the nearest neighbor
algorithm. The registration was performed at a sub-
pixel level, obtaining a rootmean-squared error of
about 0.65 pixels. After co-registration, all images
were radiometrically corrected to surface reflectance
by Atmosphereic CORrection Now (ACRON) soft-
ware, which is based on the MODTRAN-4 radiative
transfer code. In the following experiments, we se-
AMulti-featuresFusionofMulti-temporalHyperspectralImagesusingaCooperativeGDD/SVMMethod
683

Table 1: Sequence of real images (red/green/blue (rgb) composition, bands [6,19,33]) and their corresponding true classifica-
tion maps (first time serie: 2009/2010).
03/06/2009 23/09/2009 27/12/2009 09/01/2010 30/04/2010
Real Hyperionimages
Thematic map True Map
Table 2: Confusion matrix of the proposed approach for real experience set.
Percentages Classification Data
Reference Data Carex Lythracea Bare soil Palm Water Producer’s Accuracy Errors of Omission
Carex 243 17 6 5 0 88.47% 11.52%
Lythracea(henn
´
e) 29 514 18 7 11 87.35% 12.64%
Bare Soil 7 33 722 12 25 89.65% 10.66%
Water 2 5 13 279 8 89.96% 10.03%
Palm 0 9 12 19 308 87.01% 12.98%
User’s accuracy 84.63% 87.56% 93.65% 84.58% 85.11% OA=88.48%
Errors of commission 15.63% 12.45% 06.78% 15.41% 14.28%
Table 3: Evaluation of the Proposed Approach Aiganest Several Conventionnal Approaches.
Classifier Overall Accuracy Kappa
Proposed Approach 88.48 0.73
Maximum-Likelihood Classifier 81.46 0.69
Support Vector Machines (SVMs) 87.84 0.71
lected subset images from the whole temporal data
set of images containing 3300 pixels-per-image in ar-
eas with substantial changes. Pixels belonging to un-
known classes were not considered. Once the features
were extracted from the reconstructed images, their
potential use for image classification is investigated
in the following steps. Tables 2 and 3 the obtained
results.
5 CONCLUSIONS AND FUTURE
WORKS
We have presented a novel fusion method in the con-
text of multi-temporal hyperspectral images, mixtures
of dirichlet and SVM classifiers. Accordingly, the
generalization capacity of generative models can con-
ICPRAM2013-InternationalConferenceonPatternRecognitionApplicationsandMethods
684

siderably be enhanced by training them discrimina-
tively. Our experiments show that the cooperative
generative-discriminative model can lead to a the-
matic maps with superior quality. There are two ob-
vious extensions to the work that has been covered in
this project. The first is to improve the estimation ac-
curacy, and the second is to examine the possibility of
using other 3D feature vectors.
REFERENCES
Bishop, C. M. (2006). Pattern Recognition and Ma-
chine Learning (Information Science and Statistics).
Springer-Verlag New York, Inc., Secaucus, NJ, USA.
Bouguila, N. and Ziou, D. (2010). A dirichlet process mix-
ture of generalized dirichlet distributions for propor-
tional data modeling. IEEE Transactions on Neural
Networks, 21(1):107–122.
Burges, C. J. C. (1998). A tutorial on support vector ma-
chines for pattern recognition. Data Min. Knowl. Dis-
cov., 2(2):121–167.
Clark, R., Swayze, G., Wise, R., Livo, E., Hoefen, T.,
Kokaly, R., and Sutley, S. (2007). Usgs digital spec-
tral library splib06a: U.s. geological survey. In Digital
Data Series 2312007. NASA, USGS.
Cristianini, N. and Shawe-Taylor, J. (2000). An Introduction
to Support Vector Machines. Cambridge University
Press, Cambridge, UK.
Farah, I. R., Hemissi, S., Ettabaa, K. S., and Souleiman, B.
(2010). Multi-temporal Hyperspectral Images Unmix-
ing and Classification Based on 3D Signature Model
and Matching. Piers Online, 6:480–484.
Heinz, D., Davidson, C., and Ben-David, A. (2010).
Temporal-spectral detection in long-wave ir hyper-
spectral imagery. IEEE Sensors Journal, 10(3):509
–517.
LeBlanc, K. and Saffiotti, A. (2007). Cooperative infor-
mation fusion in a network robot system. In Proceed-
ings of the 1st international conference on Robot com-
munication and coordination, RoboComm ’07, pages
42:1–42:4, Piscataway, NJ, USA. IEEE Press.
Nakariyakul, S. and Casasent, D. (2004). Hyperspectral fea-
ture selection and fusion for detection of chicken skin
tumors. In Proc. SPIE, pages 128–139.
Sun, J., Ovsjanikov, M., and Guibas, L. (2009). A concise
and provably informative multi-scale signature based
on heat diffusion. In Proceedings of the Symposium
on Geometry Processing, SGP ’09, pages 1383–1392.
Eurographics Association.
Ulusoy, I. and Bishop, C. M. (2006). Comparison of gen-
erative and discriminative techniques for object detec-
tion and classification. Toward CategoryLevel Object
Recognition, pages 173–195.
Wang, Y. and Chua, C. (2005). Face recognition from 2d
and 3d images using 3d gabor filters. Image and Vision
Computing, 23(11):1018–1028.
AMulti-featuresFusionofMulti-temporalHyperspectralImagesusingaCooperativeGDD/SVMMethod
685