BOOSTING STEERABLE FEATURES

FOR 2D FACE RECOGNITION ON IV² DATABASE

Nefissa Khiari Hili

1,2

, Sylvie Lelandais

, Christophe Montagne

and Kamel Hamrouni

IBISC Laboratory,University of Evry Val d’Essonne, 40 Rue du Pelvoux, 91020, Evry Cedex, France

TSIRF Laboratory, ENIT, University of Tunis El Manar, BP-37 Le Belvédère, 1002, Tunis, Tunisia

Keywords: 2D Face Recognition, Feature Extraction, Feature Selection, Steerable Pyramid, Adaboost.

Abstract: In this paper, a novel approach for 2D face recognition is proposed, based on local feature extraction

through a multi-resolution multi-orientation linear method, Steerable Pyramid (SP) and on a feature

selection and classification by means of a non-linear method, Adaboost. Many strategies have been

elaborated and tested on IV² database including challenging variability such as pose, expression,

illumination and quality. To show the robustness of the method, it was compared to five algorithms

submitted to the first evaluation campaign on 2D face recognition using IV² database. Proposed algorithm is

almost among the two best ones.

1 INTRODUCTION

In the last two decades, security concerns have

deeply increased because of the incessant fraud

attempts. Being considered as the ultimate solution,

biometrics took part to research works to a great

extent, especially, 2D-Face recognition since it is

non-invasive and requires less user cooperation.

Despite the considerable efforts in 2D-face

recognition, it is still difficult to achieve high

accuracy under non-constrained situations. Since

most of the existing databases don’t offer either

enough variabilies or sufficient number of subjects,

relevant databases, like the IV² one, were recently

developed to permit efficient evaluation of the

proposed methods. Classical algorithms such as

Eigenfaces, Fisherfaces, Gabor (Li and Jain, 2005 )

perform well in controlled environments; however,

their performances drastically drop when variability

like quality, pose and illumination occur. Therefore,

new solutions are being suggested to overcome these

challenges. Many of them were based on combining

conventional algorithms and brought quite good

results. As instance, Mellakh et al. (2009) proposed

a method based on LDA and Gabor; Zhang and Jia

(2005) operated an identification by means of SP

and LDA and Su et al. (2009) allied local and global

features. With the persuasion of the enhancement a

merging strategy could provide, the 2D face

recognition system presented in this paper, was

developed by combining a space-scale feature-

extraction method: the Steerable Pyramid (SP) to a

non-linear feature-selection and classification

method: Adaboost. The idea of applying Steerable

Pyramids to characterize Face images, emanates

from a previous work on iris recognition by Khiari et

al. (Khiari, 2008) where good results were reached.

The Steerable Pyramid (SP) transform introduced by

Simoncelli and Freeman (1995) associates multi-

scale decompositions with differential

measurements, thus able to capture both frequency

and orientation information, which perfectly suits

this application. On the other hand, AdaBoost

method, proposed by Freund and Schapire (1995),

provides a simple yet effective stage-wise learning

approach for feature selection and classification.

Therefore, we adopt AdaBoost to select the most

discriminant SP features and build a strong

classifier.

Through this paper, a promising 2D-face

recognition method combining SP and Adaboost is

introduced. The following is divided into four

sections. Section 2 explains SP and Adaboost

formulations. Section 3 illustrates the collection of

data and the evaluation protocol elaborated in the

IV² project. Section 4 describes the proposed method

and reports experimental results in comparison to

five other algorithms submitted in the first IV²

evaluation campaign (Mellakh, 2009). Finally,

488

Khiari Hili N., Lelandais S., Montagne C. and Hamrouni K..

BOOSTING STEERABLE FEATURES FOR 2D FACE RECOGNITION ON IVÂš DATABASE.

DOI: 10.5220/0003888104880493

In Proceedings of the International Conference on Bio-inspired Systems and Signal Processing (MPBS-2012), pages 488-493

ISBN: 978-989-8425-89-8

 2012 SCITEPRESS (Science and Technology Publications, Lda.)

section 5 ends up with conclusions and perspectives

for future work.

2 BACKGROUND CONCEPTS

This section introduces the basical concepts of

Adaboost and Steerable Pyramids (SP) as being the

association that gave birth to proposed approach.

2.1 Adaboost

Boosting is a method to combine a collection of

weak classification functions (weak learner) to form

a stronger classifier. AdaBoost is an adaptive

algorithm to boost a sequence of classifiers, in that

the weights are updated dynamically according to

the errors in previous learning (Viola and Jones,

2001).

AdaBoost Algorithm:

Input: n training of examples (x

, y

)…(x

, y

)

with y

in {+1, -1} is the class label for the positive

or negative sample x

; where i = 1,…,n. In our case

(face recognition), x

= (I

1,i

2,i

) is a pair of images.

Initialize: weights D

1,i

=1/n

Do for t = 1,…,T:

1. For each filter Φ

, compute the best weak

classifier h

, that uses Φ

. This amounts to finding the

optimum threshold t

, minimizing the error e

for

each possible filter.

2. Choose the classifier h

with the lowest error

, according to weighted examples and their labels.

3. Choose α and β that define the feature f

, based

on e

and the estimated labels y

i,t

4. Normalize the weights D

t,i

so that they are a

distribution.

Output: The final Strong classifier :

























(1)

More details about Adaboost algorithm and

equations are available at (Viola and Jones, 2001).

2.2 Steerable Pyramid (SP)

The steerable pyramid, introduced by Simoncelli and

Freeman (1995), is a linear multi-scale multi-

orientation decomposition that provides a front-end

to many image-processing applications particularly

in texture analysis. The basis functions of a steerable

pyramid are directional derivative operators that

come in different sizes and orientations. The

pyramid can be designed to produce any number of

orientation bands. The representation is translation

invariant (it is aliasing free) and rotation invariant

(the sub-bands are steerable). More importantly, the

transform is a tight-frame, specifically; the same

filters used in the decomposition are used for the

reconstruction.

Figure 1: First level of the diagram system of a steerable

pyramid.

The block diagram of a steerable pyramid

(Krasaridis and Simoncelli, 1996) is given in figure

1 for both analysis and synthesis. In the analysis

part, the image is decomposed into highpass and

lowpass subbands using H

and L

filters. The

lowpass band continues to break down into a

collection of oriented n+1 bandpass subbands B

,…, B

and a lower lowpass subband L

. The

lower lowpass subband is subsampled by a factor of

2 in the x and y directions. This process constitutes

the first level of decomposition of a steerable

pyramid. Repeating the enclosed area on the output

of subsampling provides the recursive (pyramid)

structure, hence the next levels. In the synthesis part,

the reconstructed image is obtained by upsampling

the lower lowpass subband by a factor of 2 and

adding up the collection of bandpass subbands and

the highpass subband.

Figure 2: Face image decomposition using a 3-level

steerable pyramid with 4 orientations.

Figure 2 illustrates a three level steerable

pyramid decomposition of a face image, with 4

orientations (n=3). Shown are the four orientated

bandpasss images at three scales and the final

lowpass image. The initial highpass image is not

shown.

BOOSTING STEERABLE FEATURES FOR 2D FACE RECOGNITION ON IV² DATABASE

It is important to point out that only the analysis

part of the steerable pyramid diagram system is

applied while extracting features from the face

texture.

3 THE IV² DATABASE AND THE

EVALUATION PROTOCOL

In biometric studies, it is very crucial to have a big

set of data on which the efficiency of proposed

algorithms can be evaluated. Some databases are

available (Petrovska, 2009) but they don’t offer

enough data either in number or in variability. The

IV² database was designed with the aim of proposing

multiple test situations to allow evaluation with

regard to variability well known to be critical for the

biometric systems performance, such as pose,

expression, illumination and quality (Figure 3). The

IV² database has been realized during the Techno

Vision program and has been supported by the

French Research Ministry in collaboration with the

French Ministry of Defence.

Expression variability

(1.a)

(1.b)

(1.c)

Illumination variability

(2.a)

(2.b)

(2.c)

Quality variability

(3.a) High quality DVCAM

(3.b) low quality WEBCAM

Figure 3: Examples of variability related to (1.a-c)

expression, (2.a-c) illumination and (3.a-b) quality.

3.1 Database Description

The publicly available IV² database allows

monomodal and multimodal experiments using face

data. It contains 315 subjects with one session data

where 77 of them also participated to a second

session. From this database, a subset of 52 subjects,

distributed as a development set, constitutes also the

training set.

The face and sub-face data that are present in the

IV² database are: 2D audio-video talking face

sequences, 2D stereoscopic data acquired with two

pairs of synchronized cameras, 3D facial data

acquired with a laser scanner, and iris images

acquired with an portable infrared camera. This

database has been collected in several locations, by

many operators. From the totality of the acquired

data, are available two disjoint sets for development

and evaluation purposes, and also an evaluation

package.

3.2 The 2D Face Evaluation Protocol

As a closing stage of the IV² project, an evaluation

campaign was performed involving iris recognition,

2D and 3D-face recognition and also multimodal

recognition. In the 2D-Face evaluation (Petrovska,

2008), the strategy of having “one variability” at a

time was adopted in order to evaluate how

challenging variability - related to illumination,

expression, quality or multi-session images - can be

for the biometric systems.

In this evaluation campaign a set of more than

15000 images were divided into four subsets. Table

1 gives a description of the test images according to

the corresponding experiment. The protocol was

constructed so as to have almost the same number of

client and imposters tests. This strategy allows

having equivalent FAR (False Acceptance Rate) and

FRR (False Rejection Rate).

Table 1: Description for the images used for the 2D-Face

experiments (V. = variation and N. = number)

Experiment

Exp1

Exp2

Exp3

Exp4

Sessions

Mono

Multi

Quality

High

Low

High

Expression V.

Small

Illumination V.

Yes

N. Intraclass

2595

2503

1774

1796

N. Interclass

2454

2364

1713

1796

Each team who proposed an algorithm conducted

a list of tests with no indication of their type

(intraclass or interclass). Only one still image per

subject was used for enrolment and one still image

for the test. A set of 156 images from 52 different

subjects acquired at the development stage has been

as well distributed to each team for the training

phase. Five appearance based methods were

evaluated on the IV² database. Details about the

algorithms are given in (Mellakh, 2009) and

comparative results are shown in table 2.

BIOSIGNALS 2012 - International Conference on Bio-inspired Systems and Signal Processing

4 PROPOSED METHOD AND

EXPERIMENTAL RESULTS

Beforehand the recognition, normalization process is

performed on gray-scale images in order to extract

efficiently the region of interest. Normalization is

operated so as to deal with pose variations (rotation

and scale) taking place at the acquisition step, and

get all the images at the same scale and size (64x64).

4.1 Characterization through Adaboost

First experiments have been conducted only with

Adaboost. It has been noticed that the performance

increased with the number of iterations of adaboost

model. Limited by the computational time, a number

of 400 iterations has been kept. It can be seen from

table 2 (Test 1) that Adaboost reach very good EER:

3.7%, in Experiment 1. However, performance

drastically drops in Experiments 2 to 4 that involve

challenges related to illumination, quality and

multisession variabilities. This is due to the fact that

Adaboost is very dependent on the training set;

witch has been acquired, almost under the same

conditions as in Experiment 1.

4.2 Characterization through SP

Many experiments have been run on how the SP has

to be performed. A first experiment constructed the

feature vector from the whole information at all

orientations and scales provided by the entire filtered

image. Another experiment was carried out by

composing the feature vector of the 49 (8x8) filtered

blocs issued from the initial image. A third try was

run to build the feature vector with the 49 energy

values computed from the (8x8) filtered blocs. Other

tests have been completed to investigate on how the

recognition performance is further enhanced with

the increase of the orientations and scales numbers.

Specifically an exhaustive set of experimentations

has been fulfilled. Optimum parameters were

obtained by extracting features from the entire

(64x64) filtered image by utilizing a four-level fifth

derivative steerable pyramid (6 orientations). The

feature vector was made of more than 10

intensity

values. Details about EER are given in table 2 (Test

2) where it can be seen that the results obtained by

the SP-based algorithm are worse than the IV² ones.

As a matter of fact, SP features are over-

complete and stand for a high dimensional

representation of face images. Straightforward,

implementation exhibits a lack of efficiency.

4.3 Characterization through Adaboost

applied to SP

To make up for the SP and Adaboost shortcomings

when operated separately, AdaBoost was adopted as

a feature selector and classifier that reduces the SP

feature space size in order to keep only relevant

characteristics.

In the next, the influence of a non exhaustive list

of parameters, related to the application strategy of

Adaboost on SP-filtered images, is firstly presented.

Then, a comparison with the submitted algorithms at

the IV² evaluation campaign is brought. The results

are reported with the Equal Error Rate percentage.

4.3.1 Applying Adaboost on the Whole SP

A first set of tests was carried out by applying

Adaboost on the whole steerable features so as to

select the more discriminant ones. Increasing the

number of iterations related to the Adaboost model

(i.e. the number of selected features) improves the

recognition rate as shown in figure 4. But,

unfortunately, running tests was so time consuming

that reaching more than 400 features was not

practical.

Figure 4: Equal Error Rate versus number of iterations in

Adaboost model.

It is obvious from EERs in table 2 (Test3.a) that

allying adaboost to SP, is much better than applying

Adaboost or SP separately. The results are almost

acceptable, when compared to the IV² other tests;

but they are still not good enough. The aim to

ameliorate the method, and at the same time,

encounter the computation time restriction, led to the

idea of applying Adaboost per band.

4.3.2 Applying Adaboost on Every

Sub-band of SP

Many strategies of application have been tested:

Using only one sub-band: A set of experiments

focused on which oriented sub-band of the SP

should be took into account as an input to Adaboost.

BOOSTING STEERABLE FEATURES FOR 2D FACE RECOGNITION ON IV² DATABASE

Previous works (Khiari, 2011) on SP showed that

there is no favourite orientation for all tests. Each

one has its own contribution. That’s why; the

strategy of fusing all sub-bands has been adopted, so

as to take advantage of complementarities between

different orientations.

Operating sub-band fusion: By adopting the

alternative of fusion, several parameters had to be

fixed. Among them, is the choice of the score fusion

rule. Usual operators such as maximum score, scores

product and scores sum were tried. Best results were

achieved with Sum rule, with an augmentation

reaching 4.4% compared to the best band used

separately. Another question was about the number

of features to keep on every sub-band. two options

were tested:

Same feature number for all sub-bands :

Referring to results of Test 3.b in table 2, this kind

of fusion is better than applying Adaboost on the

whole SP. Moreover, it is much less time

consuming, making possible to increase the number

of considered features.

Weighting feature number per sub-band : While

conducting the test of Adaboost on the entirety of

the SP, it has been noticed that the number of

selected features was not the same for all sub-bands.

Based on this observation, the idea of weighting the

number of features for every sub-band (oriented at

all scales, high-pass and low-pass) was suggested.

Once the total number of features is fixed, the

weights were attributed based on feature distribution

found in Test 3.a as follows: 2%; 12.75%; 13.5%;

10.25%; 18%; 15.25%; 18.5%; 9.75% respectively

for High-pass, oriented band-pass 1 to 6, and low-

pass sub-bands. As instance, assuming a total feature

number of 800, Adaboost selects respectively: 16,

102, 108, 82, 144, 122, 148 and 78 features from the

pre-cited sub-bands. Almost experiments were

improved attaining 0.8% of enhancement (table 2,

Test 3.c) when compared to taking the same feature

number for all sub-bands.

Another possibility was to weight the scores of

the different classifiers before the sum fusion:

Weighting scores of classifiers with same number

of features : This is equivalent to a weighted sum-

rule at the score level while keeping the same

number of features for all sub-bands. Assuming that

is the EER of Sub-band sb, then, the weights W

associated to the scores of sub-band sb are

calculated from Equation 2 (Su, 2009).

An improvement reaching 0.7% was denoted

(table 2, Test 3.d) when compared to taking the same

feature number for all sub-bands.















 













with

















(2)

and and nb_sb the total number of sub-bands.

Weighting scores of classifiers with weighted

feature numbers : A final try, was to proceed by

weighting at the characterization level (feature

numbers) as well as at the score level (weighted sum

rule). This strategy of fusion gave almost best results

for the four experiments (table 2, Test 3.e).

To summarize, the method having the optimum

configuration was then to filter the entire (64x64)

image by a 6-orientation and 3-scale Steerable

Pyramid. Then, apply Adaboost on each sub-band

(oriented at all scales, high-pass and low-pass) with

weighted numbers of features. Afterward, score fusion

was operated on classifiers by weighted sum rule.

Table 2 illustrates also the evaluation of the

proposed method put side by side with the other

ones. It can be seen that combining SP to Adaboost

improves considerably the performance of SP and

Adaboost applied separately. On another hand, in a

comparison to PCA1 (Chaari, 2009) enhancements

are obvious in all experiments.

Regarding PCA2, it has to be underlined that the

training set on which the face space has been

constructed isn’t the same as indicated by the

protocol. In fact, it is built using 300 images from

BANCA database (30 subjects, 10 images per

subject) (Petrovska, 2009) with 3 different quality

images. While proposed method strictly followed the

protocol using only 156 images of 52 individuals (3

images/person) acquired under quite good

conditions, which is not the case of the test subsets

where many variations are present. The small

number of trained images besides the different

acquisition conditions between training and test

subsets constitutes an additional challenge, which

explains the results obtained in experiment 4 that are

better than ours. Despites, proposed method

outperforms PCA2 in the first three experiments.

Compared to LDA, except for the first controlled

scenario, proposed method achieves higher performance

in the other more challenging ones. But it still remains

less robust than LDA/Gabor which is a combining

approach of a projection-based method (LDA) and a

space-scale feature-extraction method (Gabor).

5 CONCLUSIONS

Through this work, a combining approach based on

BIOSIGNALS 2012 - International Conference on Bio-inspired Systems and Signal Processing

Steerable Pyramid and Adaboost has been

introduced for 2D-face recognition. It has been

proved that joining a non-linear classifier to the SP

brought significant enhancements, especially when

weighting both sub-bands feature numbers and

classifiers scores. Future works are intended to

consider Adaboost only as a feature selector, rather

than a classifier, on SP outputs, and study the

effectiveness of conventional projective classifiers

such as PCA and LDA on the Ada-SP-features.

REFERENCES

A. Chaari, S. Lelandais and M. Ben Ahmed, 2009. A

pruning approach improving face identification

system. Advanced Video and Signal Based

Surveillance (AVSS), Italy.

Y. Freund and R. E. Schapire, 1995. A decision-theoretic

generalization of on-line learning and an application to

boosting. In Computational Learning Theory:

Eurocolt ’95, pages 23–37. Springer-Verlag.

N. Khiari, H. Mahersia, K. Hamrouni, 2008. Iris

Recognition Using Steerable Pyramids. In 1st IEEE

Inter. Conf. on Image Processing, Theory, Tools and

Applications (IPTA), 368-374, Sousse, Tunisia.

N. Khiari, S. Lelandais, C. Montagne and K. Hamrouni,

2011. 2D Face recognition on IV² database by

Steerable pyramid and LDA. In Conf on Traitement et

Analyse de l’information, Méthodes et Applications

(TAIMA), Tunisia.

A. Krasaridis and E. P. Simoncelli, 1996. A Filter Design

Technique for Steerable Pyramid Image Transforms.In

IEEE, Inter. Conf. on Acoustics, Speech and Signal

Processing (ICASSP), 4:2387-2390, Atlanta, USA

S. Z. Li, A. K. Jain, 2005. Handbook of Face Recognition.

Springer.

A. Mellakh, A. Chaari., S. Guerfi & all., 2009. 2D Face

Recognition in the IV² Evaluation Campaign. In Conf.

on Advanced Concepts for Intelligent Vision Systems

(ACIVS), Bordeaux, France.

D. Petrovska-Delacrètaz, G. Chollet and B. Dorizzi, 2009.

Guide to Biometric Reference Systems and

Performance Evaluation. Springer.

D. Petrovska-Delacretaz, S. Lelandais, J.Colineau & all.,

2008. The IV

Multimodal Biometric Database

(Including Iris, 2D, 3D, Stereoscopic, and Talking

Face Data), and the IV2- 2007 Evaluation Campaign.

In 2nd IEEE Inter. Conf. on Biometrics: Theory,

Applications and Systems (BTAS), USA.

E. P. Simoncelli and W. T. Freeman, 1995. The Steerable

Pyramid: A Flexible Architecture For Multi-scale

Derivative Computation. In IEEE Inter. Conf. on

Image Processing (ICIP), 3(2):444-447, USA.

Y. Su, S. G. Shan, X. L. Chen, and W. Gao, 2009.

Hierarchical ensemble of global and local classifers

for face recognition. In Journal of IEEE Trans. on

Image Processing, IEEE, 18(8): 1885-1896.

P. Viola and M. Jones, 2001. Rapid Object Detection

using a Boosted Cascade of Simple Features. In Proc

of IEEE Comp. Soc. Conf on Computer Vision and

Pattern Recognition, 2001. (CVPR). I(1):511-518.

X. Zhang and Y. Jia, 2005. Face Recognition Based on

Steerable Feature and Random Subspace LDA. In Inter.

workshop on Analysis and Modelling of Faces and

Gestures (AMFG), 3723(2):170-183, Beijing, China.

APPENDIX

Table 2: Comparative results between proposed algorithms (blue and green) and IV² first evaluation campaign ones (black).

3.a: Adaboost on total SP. 3.a to 3.e: Adaboost on SP bands. Description: feature number; rate = weighting number of

features per band; s = sum rule; ws = weighted sum rule.

Numéro test

Description

Participants

Exp1

Exp2

Exp3

Exp4

Previous Tests IV²

PCA1

6.7 (±0.8)

20.7 (±1.3)

20.1 (±1.6)

22.2 (±1.6)

PCA2

7.3 (±0.8)

21.6 (±1.4)

13.6 (±1.4)

16.3 (±1.4)

mod PCA

5.3 (±0.7)

20.7 (±1.4)

19.5 (±1.6)

20.5 (±1.5)

LDA

3.7 (±0.6)

22.5 (±1.4)

21.7 (±1.7)

19.7 (±1.5)

LDA/Gabor

4.2 (±0.6)

12.0 (±1.1)

8.3 (±1.1)

11.3 (±1.2)

Adaboost

3.7(±0.6)

22.7(±1.4)

44.3(±2.0)

28.6(±1.8)

11.0 (±1.0)

23.0 (±1.4)

20.1 (±1.6)

23.9 (±1.7)

3.a

400(total_SP)

Ada/SP

5.0(±0.7)

19.5(±1.3)

14.1(±1.4)

22.9(±1.6)

3.b

400(8*50)s

Ada/SP

4.4(±0.7)

18.8(±1.3)

13.7(±1.4)

22.4(±1.6)

800(8*100)s

Ada/SP

4.6(±0.7)

18.6(±1.3)

13.7(±1.4)

21.3(±1.6)

3.c

400(rate)s

Ada/SP

4.5(±0.7)

18.6(±1.3)

13.0(±1.3)

21.8(±1.6)

800(rate)s

Ada/SP

4.7(±0.7)

18.2(±1.3)

12.9(±1.3)

20.9(±1.6)

3.d

400(8*50) ws

Ada/SP

3.7(±0.6)

18.7(±1.3)

13.5(±1.3)

22.4(±1.6)

800(8*100)ws

Ada/SP

3.9(±0.6)

18.5(±1.3)

12.9(±1.3)

21 .0(±1.6)

3 .e

400(rate) ws

Ada/SP

4.1(±0.6)

18.6(±1.3)

13.0(±1.3)

21.7(±1.6)

800(rate) ws

Ada/SP

4.1(±0.7)

18.1(±1.3)

12.9(±1.3)

20.4(±1.6)

BOOSTING STEERABLE FEATURES FOR 2D FACE RECOGNITION ON IV² DATABASE