AN EFFECTIVE METHOD FOR IMAGE MATCHING BASED ON

MODIFIED LBP AND SIFT

Yinan Wang, Nuo Zhang, Toshinori Watanabe and Hisashi Koga

Graduate School of Information Systems, the University of Electro-Communications,

1-5-1 Chofugaoka, Chofu-shi, Tokyo, 182-8585 Japan

Keywords:

Feature Matching, Local Binary Pattern (LBP), Scale Invariant Feature Transform (SIFT).

Abstract:

Scale Invariant Feature Transform (SIFT) is a very powerful and popular descriptor for image registration,

which is commonly used in feature matching. However, there is still a need for improvement with respect to

the matching accuracy of SIFT. In this paper, we present a combination of modiﬁed LBP and SIFT method

for more reliable feature matching. The main idea of the proposed method is to extract spatially enhanced

image features with modiﬁed Local Binary Pattern (LBP) from the images before implementation Difference-

of-Gaussian (DoG) in SIFT. The proposed method is also robust to illumination changes, rotation and scaling

of images. Experimental results show signiﬁcant improvement over original SIFT.

1 INTRODUCTION

The image matching, an important component in im-

age processing, is widely used in numerous appli-

cations that range from object recognition, building

panoramas guidance, automatic surveillance, navi-

gation, robot vision and mapping sciences, and so

forth. For image matching, the technology based on

Lowe’s descriptor (Lowe, 1993; Lowe, 1991; Lowe,

2004) has merits of selecting stable features in a scale

space, which is named Scale Invariant Feature Trans-

form (SIFT). Recently, improvement techniques de-

veloped for SIFT are mostly foucsed on minimization

of the reduction of computational time (H. Bay, 2006;

G. Michael, 2006; Y. Ke, 2004).

Local Binary Pattern (LBP) is a powerful descrip-

tor for texture feature analysis (T. Ojala and Maen-

paa, 2002). In this paper, we propose a combination

of modiﬁed LBP and SIFT method for more reliable

feature matching. The proposedmethod helps empha-

size some features, such as edges, in original images

and then improves performance of image matching.

The proposed method also carries on the good prop-

erties of LBP and SIFT, so that it is robust to gray

scale, illumination, rotation and scaling variation.

By combining SIFT operator and Local Binary

Pattern, it is shown to be a robust and effective ap-

proach for feature matching. In some cases, we got

results of 100% correct matching . The advantages of

our method is illustrated in the following part.

The rest of the paper is organized as follows.

In Section 2, we ﬁrst brieﬂy describe the original

SIFT and LBP. Sections 3 gives details for the pro-

posed approach. Experiment results with the pro-

posed method, and comparisons between our method

and the SITF are shown in Section 4. The conclusion

is given in the last section.

2 SIFT AND LBP METHODS

Before presenting in detail about our method, we give

a brief review of the SIFT and LBP methods that form

the basis of our work.

2.1 Scale Invariant Feature Transform

The scale invariant feature transform algorithm

(Lowe, 2004) is an algorithm for image features gen-

eration which is invariant to image translation, rota-

tion, scaling, illumination changes and partially afﬁne

projection. SIFT ﬁrst searches over all scales and

locations of the original image. It is implemented

efﬁciently by using a Difference-of-Gaussian (DoG)

function to identify potential interesting points.

Then for each candidate interesting location, a de-

tailed model is ﬁt to determine location and scale.

Key-points are selected based on measures of their

stability. After the key-point location, one or more

orientations are assigned to each key-point location

410

Wang Y., Zhang N., Watanabe T. and Koga H..

AN EFFECTIVE METHOD FOR IMAGE MATCHING BASED ON MODIFIED LBP AND SIFT.

DOI: 10.5220/0003827004100413

In Proceedings of the International Conference on Computer Vision Theory and Applications (VISAPP-2012), pages 410-413

ISBN: 978-989-8565-03-7

 2012 SCITEPRESS (Science and Technology Publications, Lda.)

Figure 1: LBP features for a neighborhood of 8 pixels.

Figure 2: The circular model (8,1), (8,2), and (16,2) neigh-

borhoods.

based on local image gradient directions. All further

operations are performed on the features which are

assigned orientation, scale, and location, thereby pro-

viding invariance to these transformations.

2.2 Local Binary Pattern

The LBP operator was originally designed for tex-

ture description. The operator assigns a label to every

pixel of an image by thresholdingthe neighborhoodof

each pixel in comparison with the center pixel value

and transforms the pixel to a binary number (Fig. 1).

To be able to deal with textures in different scales,

the LBP operator was later extended to use neighbor-

hoods with differentsizes (T. Ahonen and Pietikainen,

2004). Deﬁning the local neighborhood as a set of

sampling points evenly spaced on a circle centered at

a pixel with radius in any length and sampling points

in any number, bilinear interpolation is used when a

sampling point does not fall in the center of a pixel.

The notation (P,R) stands for pixel neighborhoods

with P sampling points on a circle with radius of R

(Fig. 2). If the graylevel of a pixel on the circle is

equal to or greater than that of the central pixel, its

value is set to be one, otherwise zero. The descriptor

shows the results over the neighborhood

as a binary number (binary pattern):

LBP

(x,y) =

P−1

∑

i=0

s(p

− p

,s(x) =



0,x<0

1,x≥ 0

(1)

where p

corresponds to the graylevel of the center

pixel of a local neighborhood, and p

to the graylevels

of P equally spaced pixels on the circle of radius R.

Figure 3: A sample of modiﬁed LBP model.

Since correlation between pixels decreases with

distance, most texture information can be obtained

from local neighborhoods, thus R is usually kept

small. In practice, Eq. (1) means that the signs of

the differences in a neighborhood are interpreted as

a P-bit binary number, resulting in 2

distinct values

for the binary pattern.

3 THE COMBINATION OF

MODIFIED LBP AND SIFT

The main objective of SIFT is to identify locations

and key-points in an image. The SIFT generates key-

points through ﬁnding the extrema of (DoG) function

in an image. After a key-point candidate is found

in comparison with its neighbors, detailed ﬁt to the

nearby data for location, scale, and ratio of princi-

pal curvatures is performed. These information al-

lows pixels that have low contrast or are poorly local-

ized along an edge to be rejected. The correspond-

ing edges are discarded because they are sensitive to

noise. However, this process is overly simpliﬁed, be-

cause it completely ignores the geometric informa-

tion among descriptors. Some useful key-points are

also discarded with eliminating edge responses. In

order to improve the performance of describing an

enhanced texture features image for SIFT-matching,

we extracted spatially enhanced texture feature with

LBP from images before the implementation of DoG

in SIFT. Local Binary Pattern descriptor is used to

help achieve ﬁnding more correct features. Unlike the

original LBP, we used the points (the red ones in Fig.

3) distributed uniformly on the frame of the window,

sized D

= 2d, where D

is a chessboard distance,

and d is a user-settable parameter. The points used

in LBP instead of interpolation value, also helps re-

duce the computational cost. Through experiments,

this modiﬁed method has outstanding performance to

emphasize features such as edges in the image, and

based on which SIFT can ﬁnd more correct features

(Fig. 4).

In case of processing rotated images (Fig. 11),

the LBP value changes after an image is rotated.

We record the sequence of the binary pattern which

forms the original LBP. For example, the pattern is

AN EFFECTIVE METHOD FOR IMAGE MATCHING BASED ON MODIFIED LBP AND SIFT

411

Figure 4: Results of feature extraction in comparison with

SIFT method and our method.

Figure 5: Comparison of LBP operator between the original

image and the rotated image.

11101001 and the LBP value is 151 for the original

image. The pattern changes to be 01111010 and the

LBP value becomes 94 (Fig. 5) after the rotation of

the image. Finally, we get two completely different

LBP. During this transformation, SIFT cannot ﬁnd

correct features and match them.

In order to solve this problem, we also modify the

LBP descriptor as:

LBP

(x,y) =

P−1

∑

i=0

s(p

− p

)256/P, s(x) =



0,x<0

1,x≥ 0

(2)

We used average value 256 /P instead of 2

shown in

Eq. (2).

Based on this simple modiﬁcation, the LBP value

keeps the same after rotation (Fig. 6), and SIFT can

ﬁnd the corresponding features correctly.

4 EXPERIMENTAL EVALUATION

In this section, a performance evaluation of the pro-

posed method is presented. Our proposed method is

assessed on the feature matching problem. The imple-

mentation of our proposal is based on the open source

work from Lowe’s SIFT detector (link-a). To ensure

the reproducibility of the tests, the publicly available

image data set (link-b) was utilized to test the per-

formance of the proposed method. The test data con-

tains images with different geometric and photometric

transformations. Different transformations evaluated

in this study are scale change, image rotation, image

blur and illumination change. In the test we chose two

different sets (illumination changed images are shown

in Fig. 9, and blurred images are shown in Fig. 10) of

the test data. And in order to study in more detail for

Figure 6: Comparison of the modiﬁed LBP operator be-

tween the original image and the rotated image.

Figure 7: Random face matching results of original SIFT

and the proposed method.

Figure 8: Enlarged random face matching results of original

SIFT and the proposed method.

the tolerance of our descriptor, we took another 4 sets

of photos (shown in Fig. 7, Fig. 8, Fig. 11 and Fig.

12). We also present the advantages of our proposed

method compared to original SIFT (shown in Table.

1) through all the experiments.

First, we introduce two experiments which has

very high accuracy of matching: we took photos of

faces, and randomly cut one face from them. Then,

original SIFT and our proposed method were used to

ﬁnd corresponding features between the two images

(Fig. 7). As the results, we got 100% of correct pos-

itive match, 174 correct matches and 0 false match

(Table. 1). On the other hand, we enlarged the size

of the cut face, to compare our proposal with original

SIFT method. We also got excellent results: 100% of

correct positive match, 54 correct matches and 0 false

matches (Table. 1).

Furthermore, the proposed method outperformed

original SIFT on both illumination changed (Fig. 9)

and blured images (Fig. 10). The advantages of

our proposed method compared to original SIFT were

shown in Table. 1.

To test matching of image rotation, we took pho-

tos of the same type projectorsin different rooms. The

level of illumination is not the same in the two rooms,

VISAPP 2012 - International Conference on Computer Vision Theory and Applications

412

Figure 9: Illumination changed image matching results of

original SIFT and the proposed method.

Figure 10: Image blur matching results of original SIFT and

the proposed method.

Figure 11: Image rotation (90

) matching results of original

SIFT and the proposed method.

Table 1: Comparison of the accuracy between the proposed

method and original SIFT.

that one photo is in normal direction, the other one is

rotated 90

(Fig. 11). As the result, we got 100% of

correct match, 55 correct matches and 0 false match

(Table. 1). Through all the experiments, our method

showed outstanding performance comparing to origi-

nal SIFT.

5 CONCLUSIONS

In this paper we presented a combination of modi-

ﬁed LBP and SIFT method with more reliable feature

matching performance. We adopted the idea that the

appearance of image edges can be well characterized

and emphasized by local features. To combine the

strengths of SIFT and LBP, we modiﬁed LBP as a fea-

ture enhance descriptor for SIFT. The performance of

the proposed method was compared to SIFT on fea-

ture matching in several different cases. For all of

the test images, our proposed method gave outstand-

ing performance compared to original SIFT. The pro-

posed method can get more correct matches with less

error. In the future work, an adaptation of our pro-

posed method to more challenging image transforma-

tion is going to be carried out.

REFERENCES

G. Michael, G. Helmut, B. H. (2006). Fast approximated

sift. In Conference on Computer Vision, Hyderabad,

India , Springer. Volume 3851/2006, 918-927.

H. Bay, T. Tuytelaars, L. V. G. (May 2006). Surf, speeded

up robust features. In Proceedings of the ninth

European Conference on Computer Vision. Volume

3951/2006, 404-417

link-a. http://blogs.oregonstate.edu/hess/code/sift/. In SIFT

Library.

link-b. http://www.robots.ox.ac.uk/∼vgg/research/afﬁne/.

In Afﬁne Covariant Features.

Lowe, D. G. (2004). Distinctive image features from scale-

invariant key-points. In International Journal of Com-

puter Vision. pp. 91-110.

Lowe, D. G. (June 1991). Local feature view clustering for

3d object recognition. In IEEE Conference on Com-

puter Vision and Pattern Recognition, Kauai, Hawaii.

12(2):291-301.

Lowe, D. G. (November 1993). Object recognition from lo-

cal scale-invariant features. In International Confer-

ence on Computer Vision, Corfu, Greece. 15(5):795-

825.

Mikolajczyk, K. and Schmid, C. (October 2005). A perfor-

mance evaluation of local descriptors. In IEEE Trans-

actions on Pattern Analysis and Maching Inelligence.

VOL. 27, NO. 10.

T. Ahonen, A. H. and Pietikainen, M. (2004). Face recog-

nition with local binary patterns. In Eighth European

Conf. Computer Vision. pp. 469-481.

T. Ojala, M. P. and Maenpaa, T. (July 2002). Multiresolu-

tion gray-scale and rotation invariant texture classiﬁ-

cation with local binary patterns. In IEEE Trans. Pat-

tern Analysis and Machine Intelligence. vol. 24, no. 7,

pp. 971-987.

Y. Ke, R. S. (2004). Pca-sift: A more distinctive represen-

tation for local image descriptors. In Proceedings of

the IEEE Computer Society Conference. Proc. CVPR.

Volume 2, pp. 506-513.

AN EFFECTIVE METHOD FOR IMAGE MATCHING BASED ON MODIFIED LBP AND SIFT

413