Reliable Image Matching using Binarized Gradient Features Obtained

with Multi-ﬂash Camera

Yasunori Sakuramoto

, Yuichi Kanematsu

, Shuichi Akizuki

, Manabu Hashimoto

Kiyotaka Watanabe

and Makito Seki

Graduate School of Computer and Cognitive Sciences, Chukyo University,

101-2, Yagoto-Honmachi, Showa-ku, Nagoya, Aichi, Japan

School of Information Science and Technology, Chukyo University,

101-2, Yagoto-Honmachi, Showa-ku, Nagoya, Aichi, Japan

Advanced Technology R&D Center, Mitsubishi Electric Corporation,

8-1-1, Tsukaguchi-Honmachi, Amagasaki, Hyogo, Japan

Keywords:

Object Detection, Binarized Gradient Features, Image Matching, Texture-less, Binary Code.

Abstract:

In this paper, we propose an object detection method using features describing information about a concavo-

convex shape of an object that are obtained by using a small camera that controls the illumination direction. A

feature image containing information about the shape of the object is generated by integrating images obtained

by turning on, one by one, light emitting diodes (LEDs) annularly arranged around the camera. Our method

can reliably detect a texture-less object by using this feature image in the matching process. Experiments

using 200 actual images conﬁrmed that the method achieves a 97.5% recognition success rate and a 4.62 sec

processing time.

1 INTRODUCTION

Object detection using an image sensor is a useful

technique in factory production systems. Most ob-

jects handled in a production system are intermediate

workpieces having a slightly concavo-convex shape

and few surface patterns such as characters or de-

signs. Therefore, a practical object detection method

that can be utilized for low-texturedobjects is desired.

Some object detection methods use edge pixels

(Barrow et al., 1977) or distinctive pixels (Hashimoto

et al., 2010) in the template image for detecting ob-

jects in general. These methods achieve reliable de-

tection by analyzing the distinctiveness of the tem-

plate image. They are also able to detect objects at

high speed by using a small number of pixels in the

matching process. For low-textured images, however,

their detection performance is low since it is difﬁcult

for them to obtain effective pixels for matching.

To address this issue, in recent years keypoint-

based matching methods such as SIFT (Lowe, 2004),

SURF (Bay et al., 2006), and ORB (Rublee et al.,

2011) have been proposed. However, these methods

describe features on the basis of gradient distribution

in the neighborhood of the key points, so it is difﬁcult

for them to describe effective features for matching

with low-textured images. Therefore, there is con-

cern that their detection performance will be lowered

as well.

Three robust approaches for low-textured images

that have been reported are the DOT method (Hinter-

stoisser et al., 2010) which uses the main gradient di-

rection in local regions, the BOLD method (Tombari

et al., 2013) which uses line segments of edge, and

an object detection method (Akizuki and Hashimoto,

2013) using pixel pairs that are selected optimally in

accordance with the intensity of the edge in the tem-

plate image. However, if the surface of object is only

slightly concavo-convex in shape, it is extremely dif-

ﬁcult to obtain shading information about the surface.

Since this information is an important clue in image

matching, it is difﬁcult for these methods to detect ob-

jects with sufﬁcient accuracy.

The purpose of this research is to achieve a

method that detects objects reliably even if the sur-

face of the object has few or no patterns, which are

important clues in image matching. We believe that if

a concavo-convexshape of the object can be captured,

it will be possible to detect an object even if its sur-

face has few or no patterns. To achieve our purpose

we propose an object detection method using features

260

Sakuramoto Y., Kanematsu Y., Akizuki S., Hashimoto M., Watanabe K. and Seki M..

Reliable Image Matching using Binarized Gradient Features Obtained with Multi-ﬂash Camera.

DOI: 10.5220/0005267902600264

In Proceedings of the 10th International Conference on Computer Vision Theory and Applications (VISAPP-2015), pages 260-264

ISBN: 978-989-758-090-1

 2015 SCITEPRESS (Science and Technology Publications, Lda.)

that describe the information about a concavo-convex

shape of the object. The features are obtained by us-

ing a small camera that can control the illumination

direction.

Using a multi-ﬂash camera (MFC) (Raskar et al.,

2004) with light emitting diodes (LEDs) annularly ar-

ranged around the camera, we obtain images by turn-

ing on the LEDs one by one. We consider that with

these images, input images and template images are

very similar at correct matching positions because

they reﬂect the concavo-convex shape of the object

for each illumination direction. Therefore, a reliable

object detection method is achieved by using the im-

ages obtained with the MFC in the matching process.

Moreover, the processing cost for matching can be re-

duced by integrating these images into a feature im-

age that describes the information about the concavo-

convex shape of the object.

Our method can also reliably detect texture-less

target objects by using this feature image in the

matching process.

In addition to the above mentioned methods, there

are other methods (Hinterstoisser et al., 2012; Drost

and Ilic, 2012) that use a 3-D sensor to obtain the

concavo-convex shape of the object. However, while

the sensor is quite a large device, the MFC we use is a

compact device and thus is considered to be practical.

In the rest of this paper, section 2 describes the

proposed object detection method and its use of the

binary gradient features based on gradient informa-

tion, section 3 presents experimental results to con-

ﬁrm the effectiveness of our method, and section 4

concludes the paper with a summary of key points.

2 THE PROPOSED METHOD

2.1 Basic Idea

The basic idea of our work is to use images obtained

by turning on, one by one, N LEDs (in this research,

N=8) of the MFC that Rasker et al. proposed (Figure

1).

Figure 1: Example images obtained from the MFC.

These images contain details about the concavo-

convex shape (e.g., horizontal and vertical edges)

that are emphasized in each illumination direction.

In other words, they contain pseudo-3-D informa-

tion. Therefore, an object that has few or no sur-

face patterns can be detected by using these images,

for which there is very high similarity between in-

put images and template images at correct matching

positions. However, the problem is that the process-

ing cost is increased when the eight images obtained

from the MFC are used in the matching process. In

this research, we were able to reduce the process-

ing cost in matching by integrating the eight images

into a feature image that has the information about the

concavo-convex shape of the object. This is because

the integration makes it possible to achieve match-

ing based on the pseudo-3-D information with a small

amount of calculation.

2.2 Binarized Gradient Features based

on Gradient Information

This section explains how we integrate the images ob-

tained from the MFC into a feature image. Figure

2 shows the process for extracting binarized gradient

features.

Figure 2: Method for extracting binarized gradient features.

Images T

(k = 1,2,...,8) are obtained by turning

on the LEDs of the MFC one by one, and images

(k = 1,2,...,8) of the gradient magnitude are gen-

erated from these images. The feature B

of each

pixel is described as an 8-bit code using a gradient

image G

. The method of generating an image of the

binarized gradient feature is to compare the gradient

ReliableImageMatchingusingBinarizedGradientFeaturesObtainedwithMulti-flashCamera

261

magnitudes of eight images at the same pixel. Next,

the 8-bit binary code is described by assigning 1 bit

for the four highest gradient magnitudes and 0 bits

for the four lowest. The image F

is generated by

the binarized feature B

of all pixels in the image.

The processing cost of matching can be reduced by

integrating the eight images into a feature image and

designing a feature as the binary code.

2.3 Image Matching using the Binarized

Gradient Features

This subsection explains a matching method we pro-

pose that uses the above mentioned binarized gradient

features. Figure 3 shows a schematic diagram of the

proposed algorithm.

Figure 3: Schematic diagram of the proposed algorithm.

The proposed algorithm consists of template gen-

eration and image matching steps.

In template generation, the binarized gradient fea-

tures are generated by the above described method,

i.e., using template images obtained by turning on the

LEDs of the MFC one by one. The image of the gen-

erated binarized gradient features is the template.

In the image matching process, the image F

(i, j)

of the binarized gradient features is generated in the

same way as the template. Next, the template scans

the input image in the same way as in conventional

template matching, and the position of the maximum

similarity in the input image is detected as the ﬁnal

output. The high-speed matching is achieved by us-

ing the Hamming distance between the input image

features and the template in the similarity calculation.

In this way, the proposed method is able to detect

a texture-less object by using the binary features that

reﬂect the concavo-convex shape of the object.

3 EXPERIMENTS AND

DISCUSSION

3.1 Performance Comparison of

Proposed Method and Other

Methods

This section explains the results we obtained in com-

paring the performance of our method with that of

other methods. We used the following four methods

as comparative methods.

(1) ZNCC: The Zero-mean Normalized Cross-

Correlation method, which uses all pixels.

(2) Chamfer Matching (Barrow et al., 1977): A

method in which edge pixels in the template im-

age are used.

(3) SURF (Bay et al., 2006): A method in which the

SIFT keypoints are used.

(4) OCPTM (Akizuki and Hashimoto, 2013): A

method using pixel pairs that are selected opti-

mally in accordance with the intensity of the edge

in the template image.

We used 50 images taken of four objects (Figure 4)

whose surfaces have different concavo-convex shapes

in the experiment.

Figure 4: The four types of objects used in the experiment.

Figure 5 shows example feature images generated

by the proposed method. While it is difﬁcult to distin-

guish the objects in the grayscale images, it is easy to

distinguish those in the feature images.

Table 1 shows recognition success rate P

[%] and

processing time for matching T[sec] of each method.

VISAPP2015-InternationalConferenceonComputerVisionTheoryandApplications

262

Figure 5: Image of the binarized gradient features generated

by the proposed method.

The recognition success rate is deﬁned by the value P

(the number of successfully matched images within

accuracy of ±2 pixels) / the total number of images

× 100. It should be noted that the comparative meth-

ods used images taken under environment light con-

ditions.

Table 1: Recognition success rate and processing time for

each method.

Method M [pixels] P

[%] T [sec]

(1) ZNCC 10,000 22 2.13

(2) Chamfer Matching 230 18 0.07

(3) SURF 9 [points] 0 0.18

(4) OCPTM 500 20 0.21

(5) Proposed method 10,000 94 4.62

(CPU: Intel

CORE

i5-2.50GHz, RAM: 4GB)

The proposed method achieved a 94% recognition

success rate, considerably higher than the rates the

comparative methods achieved. The comparative

methods achieved much lower rates because it is difﬁ-

cult for them to distinguish a target object from other

objects in grayscale images. On the other hand, the

proposed method is able to detect target objects that

do not have a surface pattern by using the features that

reﬂect the concavo-convex shape of the object.

3.2 Evaluation of recognition

Performance for Various Images

To test the general applicability of the proposed

method, we performed matching experiments with 50

input images of each of three different types. The in-

put image size was 640 pixels by 480 pixels. Figure

6shows the feature images of each object and their

feature images, and Table 2 shows the recognition

success rate achieved for each object. With regard

to the experiment conditions, the ZNCC method and

the proposed method used all pixels in the template

image and the OCPTM method used pixels selected

by the parameters described in reference (Akizuki and

Hashimoto, 2013).

Figure 6: Example input grayscale images [(a), (b), and (c)]

and images of binarized gradient features [(a’), (b’), and

(c’)]. Images enclosed in red frames are template images.

Table 2: Recognition success rate achieved for various ob-

jects.

Method

Plastic parts Metal plates Printed board

[%] P

[%]

ZNCC 58 58 100

OCPTM 84 98 100

Proposed method 100 98 98

The proposed method achieved a high (more than

98%) recognition success rate for all objects. On av-

erage this was 5% higher than that of the previous

OCPTM method. These results conﬁrm the proposed

method has high general applicability.

ReliableImageMatchingusingBinarizedGradientFeaturesObtainedwithMulti-flashCamera

263

3.3 Similarity Map Analysis

We conﬁrmed the effectiveness of the binarized gra-

dient features by analyzing the similarity map ob-

tained for the proposed method. The experimental

images used were images of the printed circuit board

and plastic parts shown in Figure 6. The compari-

son method used was the OCPTM method. Figure 7

shows the similarity maps obtained for the proposed

method and the OCPTM method.

Figure 7: Similarity maps obtained for the proposed method

and the OCPTM method.

In a high-textured object (the printed circuit

board), the similarity maps obtained by the two meth-

ods showed a sharp peak at the position of the target

object. However, in low-textured objects (the plas-

tic parts), the similarity map of the OCPTM method

showed a high degree of similarity in positions other

than the position of target object, while the score map

of the proposed method showed a high degree of sim-

ilarity only in the position of target object. These re-

sults conﬁrmed the effectiveness of the binarized gra-

dient features.

4 CONCLUSION

We have proposed binarized gradient features that re-

ﬂect the concavo-convexshape of an object and an ob-

ject detection method using these features. By using

the features in the matching process, we conﬁrmed

that our method is able to achieve reliable object de-

tection even if a target object is low-textured. Ex-

periments using 200 actual images conﬁrmed that our

method achievesa 97.5% recognition success rate and

a 4.62 sec processing time. In future work, we will at-

tempt to even further speed up the processing time.

REFERENCES

Akizuki, S. and Hashimoto, M. (2013). Robust match-

ing for low-texture images based on co-occurrence of

geometry-optimized pixel patterns. In Proc. QCAV,

pages 113–116.

Barrow, H., Tenenbaum, J., Bolles, R., and Wolf, H. (1977).

Parametric correspondence and chamfer matching:

Two new techniques for image matching. In Proc. of

IJCAI, pages 659–663.

Bay, H., Tuytelaars, T., and Gool, L. (2006). Surffspeeded

up robust features. In Proc. of ECCV, pages 404–417.

Drost, B. and Ilic, S. (2012). 3d object detection and local-

ization using multimodal point pair features. In Proc.

3DIMPVT, pages 9–16.

Hashimoto, M., Fujiwara, T., Koshimizu, H., Okuda, H.,

and Sumi, K. (2010). Extraction of unique pix-

els based on co-occurrence probability for high-speed

template matching. In Proc. of ISOT, pages 1–6.

Hinterstoisser, S., Cagniart, C., Ilic, S., Sturm, P., Navab,

N., Fua, P., and Lepetit, V. (2012). Gradient response

maps for real-time detection of texture-less objects. In

IEEE Trans. on PAMI, pages 876–888.

Hinterstoisser, S., Lepetit, V., Ilic, S., Fua, P., and Navab,

N. (2010). Dominant orientation templates for real-

time detection of texture-less objects. In Proc. CVPR,

pages 2257–2264.

Lowe, D. (2004). Distinctive image features from scale-

invariant keypoints. In IJCV, volume 60, pages 91–

110.

Raskar, R., Tan, K., Feris, R., Yu, J., and M.Turk (2004).

Non-photorealistic camera: Depth edge detection and

stylized rendering using multi-ﬂash imaging. In ACM

Trans. on Graphics, volume 23, pages 679–688.

Rublee, E., Rabaud, V., Konolige, K., and Bradski, G.

(2011). ORB : An efﬁcient alternative to SIFT or

SURF. In Proc. of ICCV, pages 2564–2571.

Tombari, F., Franchi, A., and Stefano, L. D. (2013). Bold

features to detect texture-less objects. In Proc. of

ICCV, pages 1265–1272.

VISAPP2015-InternationalConferenceonComputerVisionTheoryandApplications

264