MODIFIED LOCAL BINARY PATTERN (MLBP) FOR ROBUST
FACE RECOGNITION
Mohammad Moinul Islam
1
, Vijayan K. Asari
2
, Mohammed Nazrul Islam
3
and Mohammad A. Karim
1
1
Department of Electrical and Computer Engineering, Old Dominion University, Norfolk, VA 23529, U.S.A.
2
Department of Electrical and Computer Engineering, University of Dayton, Dayton, OH 45469, U.S.A.
3
Department of Security Systems, Farmingdale State University of New York, Farmingdale, New York, 11735, U.S.A.
Keywords: Local binary patterns, Object recognition, Illumination invariance, Modified local binary pattern.
Abstract: This paper presents an improvement of Local Binary Pattern (LBP) for robust face representation under
varying lighting conditions. Original LBP operator compares pixels in a local neighbourhood with the
centre pixel and converts the resultant binary string to 8-bit integer value. So, it is less effective under
difficult lighting conditions where variation between pixels is negligible. Our proposed MLBP uses two
stage encoding procedure which is more robust in detecting this variation in a local patch. The performance
of the proposed method is compared with the baseline LBP under different illumination conditions.
1 INTRODUCTION
Face recognition, although not a new area of
research, still attracting a lot of researchers for its
wide range of applications and the challenges that
occur in real world environments. In practice, face
images are obtained from different sources (ex.
Facebook, Flickr etc) and at different times causing
pose, appearance and illumination variations. So, a
key challenge in face recognition problems is to find
an efficient descriptor which is robust to these
unconstrained conditions. Many face representation
techniques have been proposed so far including
Gabor feature (Liu and Wechsler, 2002); (Let et. al.,
2011); (Yang and Zhang, 2010), principal
component analysis (PCA) (Turk and Pentland,
1991), modified PCA (Gottumukkal and Asari,
2004), 2D PCA (Yang et. al., 2004), Fisher’s linear
discriminant analysis (FLDA) (Belhumeur et. al.,
1997), independent component analysis (ICA) (Liu
and Wechsler, 2003; Comon, 1994) etc. All these
methods have been widely investigated and found to
perform well under controlled settings.
Recently, local texture descriptor using LBP has
been shown to be effective in face recognition (Tan
and Triggs, 2010); (Ahonen et. al., 2006). It has
been used in combination with other descriptors
such as Gabor, histogram etc. (Zhang et. al., 2005);
(Xie et. al., 2010) in order to improve recognition
accuracy but a little attention is given to the
improvement of original LBP operator. Zhao and
Pietikäinen proposed volume local binary patterns
(VLBP) for dynamic texture recognition which
extracts textures in spatiotemporal domain by
applying LBP in three orthogonal directions (Zhao
and Pietikäinen, 2007). Another extension (Lei et.
al., 2011) was conducted on Gabor face volume to
explore the neighbouring relationship in spatial,
frequency and orientation domains. Wolf et al.
(Woolf et. al., 2010) proposed three-patch and four-
patch LBP codes where the centre pixel in a 3×3
neighbourhood is encoded using 8 (for three-patch
and 16 for four-patch) additional 3×3 patch and the
distance between two patches is thresholded to
estimate the corresponding bit value. Tan and Triggs
(Tan and Triggs, 2010) quantized LBP to three
levels namely local ternary patterns (LTP) in order
to reduce noise effects in near uniform regions. All
these methods perform well under small perturbation
of lighting conditions.
The contribution of this paper can be
summarised as follows: We propose a novel
technique to improve conventional LBP coding
which can better handle lighting variations. Any
small change or uniform texture pattern (which is
the case in difficult lighting conditions) can easily be
detected using the proposed MLBP coding scheme
and it is computationally efficient. Unlike other
methods, we do not require any pre-processing to
adjust illumination effects.
147
Moinul Islam M., K. Asari V., Nazrul Islam M. and A. Karim M..
MODIFIED LOCAL BINARY PATTERN (MLBP) FOR ROBUST FACE RECOGNITION.
DOI: 10.5220/0003678001470152
In Proceedings of the International Conference on Neural Computation Theory and Applications (NCTA-2011), pages 147-152
ISBN: 978-989-8425-84-3
Copyright
c
2011 SCITEPRESS (Science and Technology Publications, Lda.)
2 LOCAL BINARY PATTERN
The LBP operator, introduced by Ojala et al. (Ojala
et. al., 1996) is a powerful tool for texture
description. It has been widely used in various
recognition algorithms for its discriminative nature
in texture classification. The LBP operator was
originally defined for a 3×3 neighbourhood and 8-bit
binary pattern which gives 2
8
= 256 possible texture
units. It takes a neighbourhood around each pixel
and compares every pixel in the neighbourhood with
its centre pixel. The result of the comparison is then
thresholded to give a binary number which is given a
particular weight based on its position in the
neighbourhood. This gives an integer number of
8-bit LBP code around the centre pixel and is a local
descriptor of that pixel.
The LBP coding of a 3×3 example patch with the
centre pixel as threshold is shown in Figure 1.
Threshold
= 180

2
10110100
Figure 1: LBP encoding is shown for a 3×3
neighbourhood with the centre pixel as the threshold.
Mathematically, LBP operator can be described
as (Ahonen et al., 2004).

otherwise,0
,1
ck
ck
pp
ppL
(1)

7
0
2LBP
k
ck
k
ppL
(2)
Where p
k
where k = 0, 1, 2, ..., 7) represents
neighborhood pixels and
c
p
is the center pixel of
that neighborhood. The LBP pattern of the pixel is
calculated by assigning a binomial factor
k
2
for
each L(p
k
– p
c
). The LBP operator was later
extended to different sizes of neighborhood in order
to deal with textures at different scales (Ojala et. al.,
2002). Another extension is the so-called uniform
patterns (Zhao and Pietikäinen, 2007). A pattern is
called uniform if it contains at most two bitwise
transitions from ‘0’ to ‘1’ or vice versa in a circular
fashion.
3 MODIFIED LOCAL BINARY
PATTERN (MLBP)
LBP operator has been successfully applied in many
recognition applications for discriminative feature
extraction. It has also proven its robustness in small
change in lighting conditions. Since texture features
are usually described by the relative change in pixel
itensity with respct to its neighborhood and since
LBP operator compares and thresholds the
neighborhood pixels at exactly the center pixel, it
can extract the texture feature when there are
significant variations in image intensity. But in case
of difficult lighting conditions (extreme dark or
bright), there are small variations in a local
neighborhood; i.e, the variance of a local patch is
negligibly small and pixel intensity repeats itself. So,
there is a tendency that a ‘0’ is encoded as ‘1’ or
vice versa when compared with an image with
neutral light settings. As a result, the bit error rate
increases and decoded value differs significantly.
In order to overcome the problem of LBP
operator, we propose a modified local binary pattern
(MLBP). Our method uses two steps to encode the
final pattern. First, we assign a status bit to each
pixel based on its local neighbours. Then, these
status bits are used to encode the LBP of the centre
pixel. Figure 2 illustrates the proposed MLBP
coding scheme. An input image is first converted to
a binary status image. For each pixel we select a 3×3
(for MLBP3 and 5×5 for MLBP5) local
neighbourhood and calculate absolute intensity
differences of all the pixels within the patch with
respect to the centre pixel. The sum of all deviations
is denoted as total deviation (TD) for the centre
pixel, P
c
. Then we compare all the pixels in the
patch with P
c
and select those pixels which are equal
or above the centre pixel. Now, deviation for these
pixels are estimated and denoted as positive
deviation (PD). The status bit of P
c
is calculated as:
otherwise,0
TD
2
1
PDif,1
c
S
(3)
This gives us a binary status image where each bit is
estimated based on its local neighbourhood. Now,
this status image is taken as the input and a 3×3
neighbourhood is taken to calculate MLBP code.
The MLBP code of the centre pixel is denoted using
the status bits of all the neighbouring pixels as
shown in Figure 2. Thus the value of the centre pixel
is calculated by assigning a binomial factor as:
7
0
1
2MLBP
k
k
k
S
(4)
NCTA 2011 - International Conference on Neural Computation Theory and Applications
148
8
1
(TD)deviation Total
i
ci
PP
ck
PPts
ik
ck
PP
.
(PD)deviationPositive
otherwise,0
TD
2
1
PDif,1
c
S

2
12345678
SSSSSSSS
Figure 2: MLBP encoding; from left to right: a 3 ͯ 3
neighbourhood for calculating the status bit of the centre
pixel
c
P , status bit calculation, MLBP code of centre pixel
is calculated from the status bits of its neighbourhood.
In order to investigate robustness of the proposed
method, we plot histograms of three segments
extracted from three images of the same person with
different lighting. We consider three white rectangle
regions shown in Figures 3a, 3b and 3c to illustrate
this and obtain their LBP and MLBP generated
histograms and are shown in Figure 4. In Figure 3,
image ‘a’ represents the most neutral lighting
condition. Figure 4(a) shows original intensity plot
of the images and they are at three different regions
of the dynamic range of gray level. Figure 4(b) and
Figure 4(c) plot histograms of their LBP and MLBP
encoded images respectively. From the figures, we
can see that LBP generated histograms are separated
from each other significantly whereas their MLBP
generated histograms resembles each other. We also
measure this deviation quantitatively.
The L2-norm distances of image ‘b’ and image
‘c’ from image ‘a’ are calculated as 14.63 and 21.17
for LBP while it is more uniform in the case of
MLBP, which are obtained as 12.65 and 12.73
respectively. Figures 3a to 3c show images of a
person with three different lighting conditions and
their LBP and MLBP images are depicted in Figure
3(d) and Figure 3(e) respectively.
(a) (b) (c)
(d)
()
(e)
Figure 3: (a-c) An example image with three different
illuminations and their corresponding (d) LBP image and
(e) MLBP image.
4 EXPERIMENTAL ANALYSIS
In this section we illustrate the effectiveness of our
proposed method on Extended Yale B database. The
database contains 38 subjects under 64 illumination
conditions. It has little variability in pose and
expressions but its extreme lighting variations make
it a difficult problem in face recognition. From the
database we select 2413 images of 38 individuals.
The images are cropped and resized to 36×30 pixels.
We divide the database into two non-overlapping
groups (group A and group B). Group A contains the
subjects with odd numbered ID in total of 20
subjects and group B contains the remaining 18
subjects. For training purpose, we select two images
from each subject which have the most neutral
lighting conditions. We applied the proposed MLBP
to each of the cropped face image and perform
nearest neighbour (NN) classification in Euclidean
space.
MODIFIED LOCAL BINARY PATTERN (MLBP) FOR ROBUST FACE RECOGNITION
149
(a)
0 2 4 6 8 10 12 14 16
0
5
10
15
20
25
30
35
(Gray Level)*16
Number of Pixels
Image 'a'
Image 'b'
Image 'c'
(b)
0 2 4 6 8 10 12 14 16
0
5
10
15
20
25
30
35
(Gray Level)*16
Number of Pixels
LBP of Image 'a'
LBP of Image 'b'
LBP of Image 'c'
(c)
0 2 4 6 8 10 12 14 16
0
5
10
15
20
25
30
35
(Gray Level)*16
Number of Pixels
MLBP of Image 'a'
MLBP of Image 'b'
MLBP of Image 'c'
Figure 4: Histogram plot for three different illuminations
(a) Original intensity histograms (b) histograms of LBP
images and (c) histograms of MLBP images.
The experimental results are listed in Table 1. It
shows that LBP has higher error rate than MLBP
and increasing the neighbourhood size further
improves recognition accuracy.
Table 1: Comparison of the Performance on Error Rate
Using Yale B Database.
Method Error Rate
LBP-NN 0.14
MLBP3-NN 0.08
MLBP5-NN 0.05
In this experiment we compare our MLBP
approach with LBP operator for determining
recognition accuracy at various illumination
conditions. For this purpose, we divide the database
into six subsets according to their azimuth angles
(5°, 20°, 35°, 50°, 85° and 120°). The performance
of LBP and MLBP algorithms are compared for
each test image and the results are plotted in Figure
5. From the figure, we see that at small lighting
variations both LBP and MLBP perform almost
equally, but at strong lighting variations the
performance of LBP degrades significantly.
0 20 40 60 80 100 120
0
0.2
0.4
0.6
0.8
1
Azimuth angle (in degrees)
Recognition rate
LBP-NN
MLBP3-NN
MLBP5-NN
Figure 5: Performance comparison of LBP and MLBP
with respect to variations of different illumination
conditions.
At 85° and 120° our proposed method
outperforms LBP by as much as 20%. In all these
experiments MLBP5 shows better performance than
both LBP and MLBP3. We also compare the
performance of LBP and MLBP at various
dimensionalities. Figure 6 is a plot of recognition
accuracy with dimensionality (row × column). This
indicates that the performance of MLBP is much
better than LBP at reduced dimensions.
NCTA 2011 - International Conference on Neural Computation Theory and Applications
150
5 CONCLUSIONS
In this paper we present a new technique for image
representation and feature extraction named
modified local binary pattern (MLBP) which shows
many advantages over original LBP approach. First,
it is less sensitive to variations in lighting conditions.
We conducted several experiments by changing
lighting conditions and almost in all cases MLBP
performed better than LBP in terms of recognition
accuracy. Although in some experiments LBP
showed better results but the difference is not
significant and MLBP is more consistent in all cases.
This is because LBP only compares with the centre
pixel whereas MLBP uses two layer comparisons. It
is noted that the recognition accuracy is improved in
difficult lighting conditions based on the magnitude
difference of each pixel from the centre pixel.
MLBP considers this in every neighbourhood of a
given pixel in a given patch. This was evident when
we used MLBP5. It performed better than MLBP3.
We only used two different neighbourhood size but
it can also be used for different neighbourhood size
although there will be maximum limit on recognition
accuracy. In addition, MLBP has better recognition
accuracy than LBP at reduced dimensions.
The objective of this paper is to improve the
existing LBP method so that it is more robust in
difficult lighting conditions. So, we use simple
nearest neighbour classifier. The proposed MLBP
method can also be combined with other feature
extraction techniques to improve recognition
accuracy.
10
2
10
3
10
4
0.6
0.65
0.7
0.75
0.8
0.85
0.9
0.95
1
Dimension in row*column (log scale)
Recognition rate
LBP
MLBP3
MLBP5
Figure 6: Performance comparison of LBP and MLBP
with respect to dimensions in row*column vector.
REFERENCES
Liu, C., Wechsler, H., 2002. Gabor feature based
classification using the enhanced Fisher linear
discriminant model for face recognition. In IEEE
Transactions on Image Processing, 11, 467-476.
Lei, Z., Liao, S., Pietikäinen,M., Li, S., 2011. Face
recognition by exploring information jointly in space,
scale and orientation. In IEEE Transactions on Image
Processing, 20, 247-256.
Yang, M., Zhang, L., 2010. Gabor feature based sparse
representation for face recognition with Gabor
occlusion dictionary. In Proceedings of the European
Conference on Computer Vision, ECCV 2010, 448-
461.
Turk, M., Pentland, A., 1991. Face recognition using
eigenfaces. In Proceedings of IEEE Computer Society
Conference on Computer Vision and Pattern
Recognition, CVPR 1991, 586-591.
Gottumukkal, R., Asari, K. V., 2004. An improved face
recognition technique based on modular PCA
approach. Pattern Recognition Letters, 25, 4, 429-436.
Yang, J., Zhang, D., Frangi , A., Yang, J., 2004. Two-
dimensional PCA: A new approach to appearance-
based face representation and recognition. In IEEE
Transactions on Pattern Analysis and Machine
Intelligence, 26, 131-137.
Belhumeur, P., Hespanha, J., Kriegman, D., 1997.
Eigenfaces vs. fisher faces: Recognition using class
specific linear projection. In IEEE Transactions on
Pattern Analysis and Machine Intelligence, 19, 711-
720.
Liu, C., Wechsler, H., 2003. Independent component
analysis of Gabor features for face recognition. In
IEEE Transactions on Neural Networks, 14, 919-928.
Comon, P., 1994. Independent component analysis-a new
concept? In Signal Processing, 36, 287-314.
Tan , X., Triggs, B., 2010. Enhanced local texture feature
sets for face recognition under difficult lighting
conditions. In IEEE Transactions on Image
Processing, 19, 1635-1650.
Ahonen, T., Hadid , A., Pietikäinen, M., 2006. Face
description with local binary patterns: Application to
face recognition. In IEEE Transactions on Pattern
Analysis and Machine Intelligence, 28, 2037-2041.
Zhang, W., Shan, S., Gao, W., Chen, X., Zhang, H., 2005.
Local Gabor binary pattern histogram sequence
(LGBPHS): A novel non-statistical model for face
representation and recognition. In Proceedings of
IEEE International Conference on Computer Vision
ICCV 2005, 786-791.
Xie, S., Shan, S., Chen, X., Chen, J., 2010. Fusing local
patterns of Gabor magnitude and phase for face
recognition. In IEEE Transactions on Image
Processing, 19, 1349-1361.
Zhao, G., Pietikäinen, M., 2007. Dynamic texture
recognition using local binary patterns with an
application to facial expressions. In IEEE
Transactions on Pattern Analysis and Machine
Intelligence, 29, 915-928.
MODIFIED LOCAL BINARY PATTERN (MLBP) FOR ROBUST FACE RECOGNITION
151
Wolf, L., Hassner, T., Taigman, Y., 2010. Effective
unconstrained face recognition by combining multiple
descriptors and learned background statistics. In IEEE
Transactions on Pattern Analysis and Machine
Intelligence, 1-16.
Ojala, T., Pietikäinen, M., Harwood, D., 1996. A
comparative study of texture measures with
classification based on feature distribution. In Pattern
Recognition, 29, 51-59.
Ahonen, T., Hadid, A., Pietikäinen, M., 2004. Face
recognition with local binary patterns. In Proceedings
of European Conference on Computer Vision, ECCV
2004, 469-481.
Ojala, T., Pietikäinen, M., Mäenpää, T., 2002.
Multiresolution gray-scale and rotation invariant
texture classification. with local binary patterns. In
IEEE Transactions on Pattern Analysis and Machine
Intelligence, 24, 971-987.
NCTA 2011 - International Conference on Neural Computation Theory and Applications
152