MODIFIED LOCAL BINARY PATTERN (MLBP) FOR ROBUST

FACE RECOGNITION

Mohammad Moinul Islam

, Vijayan K. Asari

, Mohammed Nazrul Islam

and Mohammad A. Karim

Department of Electrical and Computer Engineering, Old Dominion University, Norfolk, VA 23529, U.S.A.

Department of Electrical and Computer Engineering, University of Dayton, Dayton, OH 45469, U.S.A.

Department of Security Systems, Farmingdale State University of New York, Farmingdale, New York, 11735, U.S.A.

Keywords: Local binary patterns, Object recognition, Illumination invariance, Modified local binary pattern.

Abstract: This paper presents an improvement of Local Binary Pattern (LBP) for robust face representation under

varying lighting conditions. Original LBP operator compares pixels in a local neighbourhood with the

centre pixel and converts the resultant binary string to 8-bit integer value. So, it is less effective under

difficult lighting conditions where variation between pixels is negligible. Our proposed MLBP uses two

stage encoding procedure which is more robust in detecting this variation in a local patch. The performance

of the proposed method is compared with the baseline LBP under different illumination conditions.

1 INTRODUCTION

Face recognition, although not a new area of

research, still attracting a lot of researchers for its

wide range of applications and the challenges that

occur in real world environments. In practice, face

images are obtained from different sources (ex.

Facebook, Flickr etc) and at different times causing

pose, appearance and illumination variations. So, a

key challenge in face recognition problems is to find

an efficient descriptor which is robust to these

unconstrained conditions. Many face representation

techniques have been proposed so far including

Gabor feature (Liu and Wechsler, 2002); (Let et. al.,

2011); (Yang and Zhang, 2010), principal

component analysis (PCA) (Turk and Pentland,

1991), modified PCA (Gottumukkal and Asari,

2004), 2D PCA (Yang et. al., 2004), Fisher’s linear

discriminant analysis (FLDA) (Belhumeur et. al.,

1997), independent component analysis (ICA) (Liu

and Wechsler, 2003; Comon, 1994) etc. All these

methods have been widely investigated and found to

perform well under controlled settings.

Recently, local texture descriptor using LBP has

been shown to be effective in face recognition (Tan

and Triggs, 2010); (Ahonen et. al., 2006). It has

been used in combination with other descriptors

such as Gabor, histogram etc. (Zhang et. al., 2005);

(Xie et. al., 2010) in order to improve recognition

accuracy but a little attention is given to the

improvement of original LBP operator. Zhao and

Pietikäinen proposed volume local binary patterns

(VLBP) for dynamic texture recognition which

extracts textures in spatiotemporal domain by

applying LBP in three orthogonal directions (Zhao

and Pietikäinen, 2007). Another extension (Lei et.

al., 2011) was conducted on Gabor face volume to

explore the neighbouring relationship in spatial,

frequency and orientation domains. Wolf et al.

(Woolf et. al., 2010) proposed three-patch and four-

patch LBP codes where the centre pixel in a 3×3

neighbourhood is encoded using 8 (for three-patch

and 16 for four-patch) additional 3×3 patch and the

distance between two patches is thresholded to

estimate the corresponding bit value. Tan and Triggs

(Tan and Triggs, 2010) quantized LBP to three

levels namely local ternary patterns (LTP) in order

to reduce noise effects in near uniform regions. All

these methods perform well under small perturbation

of lighting conditions.

The contribution of this paper can be

summarised as follows: We propose a novel

technique to improve conventional LBP coding

which can better handle lighting variations. Any

small change or uniform texture pattern (which is

the case in difficult lighting conditions) can easily be

detected using the proposed MLBP coding scheme

and it is computationally efficient. Unlike other

methods, we do not require any pre-processing to

adjust illumination effects.

147

Moinul Islam M., K. Asari V., Nazrul Islam M. and A. Karim M..

MODIFIED LOCAL BINARY PATTERN (MLBP) FOR ROBUST FACE RECOGNITION.

DOI: 10.5220/0003678001470152

In Proceedings of the International Conference on Neural Computation Theory and Applications (NCTA-2011), pages 147-152

ISBN: 978-989-8425-84-3

 2011 SCITEPRESS (Science and Technology Publications, Lda.)

2 LOCAL BINARY PATTERN

The LBP operator, introduced by Ojala et al. (Ojala

et. al., 1996) is a powerful tool for texture

description. It has been widely used in various

recognition algorithms for its discriminative nature

in texture classification. The LBP operator was

originally defined for a 3×3 neighbourhood and 8-bit

binary pattern which gives 2

= 256 possible texture

units. It takes a neighbourhood around each pixel

and compares every pixel in the neighbourhood with

its centre pixel. The result of the comparison is then

thresholded to give a binary number which is given a

particular weight based on its position in the

neighbourhood. This gives an integer number of

8-bit LBP code around the centre pixel and is a local

descriptor of that pixel.

The LBP coding of a 3×3 example patch with the

centre pixel as threshold is shown in Figure 1.



Threshold

= 180



10110100

Figure 1: LBP encoding is shown for a 3×3

neighbourhood with the centre pixel as the threshold.

Mathematically, LBP operator can be described

as (Ahonen et al., 2004).













otherwise,0

ppL

(1)









2LBP

ppL

(2)

Where p

where k = 0, 1, 2, ..., 7) represents

neighborhood pixels and

is the center pixel of

that neighborhood. The LBP pattern of the pixel is

calculated by assigning a binomial factor

for

each L(p

– p

). The LBP operator was later

extended to different sizes of neighborhood in order

to deal with textures at different scales (Ojala et. al.,

2002). Another extension is the so-called uniform

patterns (Zhao and Pietikäinen, 2007). A pattern is

called uniform if it contains at most two bitwise

transitions from ‘0’ to ‘1’ or vice versa in a circular

fashion.

3 MODIFIED LOCAL BINARY

PATTERN (MLBP)

LBP operator has been successfully applied in many

recognition applications for discriminative feature

extraction. It has also proven its robustness in small

change in lighting conditions. Since texture features

are usually described by the relative change in pixel

itensity with respct to its neighborhood and since

LBP operator compares and thresholds the

neighborhood pixels at exactly the center pixel, it

can extract the texture feature when there are

significant variations in image intensity. But in case

of difficult lighting conditions (extreme dark or

bright), there are small variations in a local

neighborhood; i.e, the variance of a local patch is

negligibly small and pixel intensity repeats itself. So,

there is a tendency that a ‘0’ is encoded as ‘1’ or

vice versa when compared with an image with

neutral light settings. As a result, the bit error rate

increases and decoded value differs significantly.

In order to overcome the problem of LBP

operator, we propose a modified local binary pattern

(MLBP). Our method uses two steps to encode the

final pattern. First, we assign a status bit to each

pixel based on its local neighbours. Then, these

status bits are used to encode the LBP of the centre

pixel. Figure 2 illustrates the proposed MLBP

coding scheme. An input image is first converted to

a binary status image. For each pixel we select a 3×3

(for MLBP3 and 5×5 for MLBP5) local

neighbourhood and calculate absolute intensity

differences of all the pixels within the patch with

respect to the centre pixel. The sum of all deviations

is denoted as total deviation (TD) for the centre

pixel, P

. Then we compare all the pixels in the

patch with P

and select those pixels which are equal

or above the centre pixel. Now, deviation for these

pixels are estimated and denoted as positive

deviation (PD). The status bit of P

is calculated as:















otherwise,0

PDif,1

(3)

This gives us a binary status image where each bit is

estimated based on its local neighbourhood. Now,

this status image is taken as the input and a 3×3

neighbourhood is taken to calculate MLBP code.

The MLBP code of the centre pixel is denoted using

the status bits of all the neighbouring pixels as

shown in Figure 2. Thus the value of the centre pixel

is calculated by assigning a binomial factor as:









2MLBP

(4)

NCTA 2011 - International Conference on Neural Computation Theory and Applications

148







(TD)deviation Total









PPts

(PD)deviationPositive















otherwise,0

PDif,1



12345678

SSSSSSSS

Figure 2: MLBP encoding; from left to right: a 3 ͯ 3

neighbourhood for calculating the status bit of the centre

pixel

P , status bit calculation, MLBP code of centre pixel

is calculated from the status bits of its neighbourhood.

In order to investigate robustness of the proposed

method, we plot histograms of three segments

extracted from three images of the same person with

different lighting. We consider three white rectangle

regions shown in Figures 3a, 3b and 3c to illustrate

this and obtain their LBP and MLBP generated

histograms and are shown in Figure 4. In Figure 3,

image ‘a’ represents the most neutral lighting

condition. Figure 4(a) shows original intensity plot

of the images and they are at three different regions

of the dynamic range of gray level. Figure 4(b) and

Figure 4(c) plot histograms of their LBP and MLBP

encoded images respectively. From the figures, we

can see that LBP generated histograms are separated

from each other significantly whereas their MLBP

generated histograms resembles each other. We also

measure this deviation quantitatively.

The L2-norm distances of image ‘b’ and image

‘c’ from image ‘a’ are calculated as 14.63 and 21.17

for LBP while it is more uniform in the case of

MLBP, which are obtained as 12.65 and 12.73

respectively. Figures 3a to 3c show images of a

person with three different lighting conditions and

their LBP and MLBP images are depicted in Figure

3(d) and Figure 3(e) respectively.

(a) (b) (c)

(d)

()

(e)

Figure 3: (a-c) An example image with three different

illuminations and their corresponding (d) LBP image and

(e) MLBP image.

4 EXPERIMENTAL ANALYSIS

In this section we illustrate the effectiveness of our

proposed method on Extended Yale B database. The

database contains 38 subjects under 64 illumination

conditions. It has little variability in pose and

expressions but its extreme lighting variations make

it a difficult problem in face recognition. From the

database we select 2413 images of 38 individuals.

The images are cropped and resized to 36×30 pixels.

We divide the database into two non-overlapping

groups (group A and group B). Group A contains the

subjects with odd numbered ID in total of 20

subjects and group B contains the remaining 18

subjects. For training purpose, we select two images

from each subject which have the most neutral

lighting conditions. We applied the proposed MLBP

to each of the cropped face image and perform

nearest neighbour (NN) classification in Euclidean

space.

MODIFIED LOCAL BINARY PATTERN (MLBP) FOR ROBUST FACE RECOGNITION

149

(a)

0 2 4 6 8 10 12 14 16

(Gray Level)*16

Number of Pixels

Image 'a'

Image 'b'

Image 'c'

(b)

0 2 4 6 8 10 12 14 16

(Gray Level)*16

Number of Pixels

LBP of Image 'a'

LBP of Image 'b'

LBP of Image 'c'

(c)

0 2 4 6 8 10 12 14 16

(Gray Level)*16

Number of Pixels

MLBP of Image 'a'

MLBP of Image 'b'

MLBP of Image 'c'

Figure 4: Histogram plot for three different illuminations

(a) Original intensity histograms (b) histograms of LBP

images and (c) histograms of MLBP images.

The experimental results are listed in Table 1. It

shows that LBP has higher error rate than MLBP

and increasing the neighbourhood size further

improves recognition accuracy.

Table 1: Comparison of the Performance on Error Rate

Using Yale B Database.

Method Error Rate

LBP-NN 0.14

MLBP3-NN 0.08

MLBP5-NN 0.05

In this experiment we compare our MLBP

approach with LBP operator for determining

recognition accuracy at various illumination

conditions. For this purpose, we divide the database

into six subsets according to their azimuth angles

(5°, 20°, 35°, 50°, 85° and 120°). The performance

of LBP and MLBP algorithms are compared for

each test image and the results are plotted in Figure

5. From the figure, we see that at small lighting

variations both LBP and MLBP perform almost

equally, but at strong lighting variations the

performance of LBP degrades significantly.

0 20 40 60 80 100 120

0.2

0.4

0.6

0.8

Azimuth angle (in degrees)

Recognition rate

LBP-NN

MLBP3-NN

MLBP5-NN

Figure 5: Performance comparison of LBP and MLBP

with respect to variations of different illumination

conditions.

At 85° and 120° our proposed method

outperforms LBP by as much as 20%. In all these

experiments MLBP5 shows better performance than

both LBP and MLBP3. We also compare the

performance of LBP and MLBP at various

dimensionalities. Figure 6 is a plot of recognition

accuracy with dimensionality (row × column). This

indicates that the performance of MLBP is much

better than LBP at reduced dimensions.

NCTA 2011 - International Conference on Neural Computation Theory and Applications

150

5 CONCLUSIONS

In this paper we present a new technique for image

representation and feature extraction named

modified local binary pattern (MLBP) which shows

many advantages over original LBP approach. First,

it is less sensitive to variations in lighting conditions.

We conducted several experiments by changing

lighting conditions and almost in all cases MLBP

performed better than LBP in terms of recognition

accuracy. Although in some experiments LBP

showed better results but the difference is not

significant and MLBP is more consistent in all cases.

This is because LBP only compares with the centre

pixel whereas MLBP uses two layer comparisons. It

is noted that the recognition accuracy is improved in

difficult lighting conditions based on the magnitude

difference of each pixel from the centre pixel.

MLBP considers this in every neighbourhood of a

given pixel in a given patch. This was evident when

we used MLBP5. It performed better than MLBP3.

We only used two different neighbourhood size but

it can also be used for different neighbourhood size

although there will be maximum limit on recognition

accuracy. In addition, MLBP has better recognition

accuracy than LBP at reduced dimensions.

The objective of this paper is to improve the

existing LBP method so that it is more robust in

difficult lighting conditions. So, we use simple

nearest neighbour classifier. The proposed MLBP

method can also be combined with other feature

extraction techniques to improve recognition

accuracy.

0.6

0.65

0.7

0.75

0.8

0.85

0.9

0.95

Dimension in row*column (log scale)

Recognition rate

LBP

MLBP3

MLBP5

Figure 6: Performance comparison of LBP and MLBP

with respect to dimensions in row*column vector.

REFERENCES

Liu, C., Wechsler, H., 2002. Gabor feature based

classification using the enhanced Fisher linear

discriminant model for face recognition. In IEEE

Transactions on Image Processing, 11, 467-476.

Lei, Z., Liao, S., Pietikäinen,M., Li, S., 2011. Face

recognition by exploring information jointly in space,

scale and orientation. In IEEE Transactions on Image

Processing, 20, 247-256.

Yang, M., Zhang, L., 2010. Gabor feature based sparse

representation for face recognition with Gabor

occlusion dictionary. In Proceedings of the European

Conference on Computer Vision, ECCV 2010, 448-

461.

Turk, M., Pentland, A., 1991. Face recognition using

eigenfaces. In Proceedings of IEEE Computer Society

Conference on Computer Vision and Pattern

Recognition, CVPR 1991, 586-591.

Gottumukkal, R., Asari, K. V., 2004. An improved face

recognition technique based on modular PCA

approach. Pattern Recognition Letters, 25, 4, 429-436.

Yang, J., Zhang, D., Frangi , A., Yang, J., 2004. Two-

dimensional PCA: A new approach to appearance-

based face representation and recognition. In IEEE

Transactions on Pattern Analysis and Machine

Intelligence, 26, 131-137.

Belhumeur, P., Hespanha, J., Kriegman, D., 1997.

Eigenfaces vs. fisher faces: Recognition using class

specific linear projection. In IEEE Transactions on

Pattern Analysis and Machine Intelligence, 19, 711-

720.

Liu, C., Wechsler, H., 2003. Independent component

analysis of Gabor features for face recognition. In

IEEE Transactions on Neural Networks, 14, 919-928.

Comon, P., 1994. Independent component analysis-a new

concept? In Signal Processing, 36, 287-314.

Tan , X., Triggs, B., 2010. Enhanced local texture feature

sets for face recognition under difficult lighting

conditions. In IEEE Transactions on Image

Processing, 19, 1635-1650.

Ahonen, T., Hadid , A., Pietikäinen, M., 2006. Face

description with local binary patterns: Application to

face recognition. In IEEE Transactions on Pattern

Analysis and Machine Intelligence, 28, 2037-2041.

Zhang, W., Shan, S., Gao, W., Chen, X., Zhang, H., 2005.

Local Gabor binary pattern histogram sequence

(LGBPHS): A novel non-statistical model for face

representation and recognition. In Proceedings of

IEEE International Conference on Computer Vision

ICCV 2005, 786-791.

Xie, S., Shan, S., Chen, X., Chen, J., 2010. Fusing local

patterns of Gabor magnitude and phase for face

recognition. In IEEE Transactions on Image

Processing, 19, 1349-1361.

Zhao, G., Pietikäinen, M., 2007. Dynamic texture

recognition using local binary patterns with an

application to facial expressions. In IEEE

Transactions on Pattern Analysis and Machine

Intelligence, 29, 915-928.

MODIFIED LOCAL BINARY PATTERN (MLBP) FOR ROBUST FACE RECOGNITION

151

Wolf, L., Hassner, T., Taigman, Y., 2010. Effective

unconstrained face recognition by combining multiple

descriptors and learned background statistics. In IEEE

Transactions on Pattern Analysis and Machine

Intelligence, 1-16.

Ojala, T., Pietikäinen, M., Harwood, D., 1996. A

comparative study of texture measures with

classification based on feature distribution. In Pattern

Recognition, 29, 51-59.

Ahonen, T., Hadid, A., Pietikäinen, M., 2004. Face

recognition with local binary patterns. In Proceedings

of European Conference on Computer Vision, ECCV

2004, 469-481.

Ojala, T., Pietikäinen, M., Mäenpää, T., 2002.

Multiresolution gray-scale and rotation invariant

texture classification. with local binary patterns. In

IEEE Transactions on Pattern Analysis and Machine

Intelligence, 24, 971-987.

NCTA 2011 - International Conference on Neural Computation Theory and Applications

152