The Improved Harris Operator based on Steerable Filter

Lin Zhang, Jinfeng Lu, Jing Wang, Yangyu Wu, Zijuan Jiang

College of Bioresources Chemical and Material Engineering, Shaanxi University of Science and Technology, Xi' an,

P R China

zhanglin@sust.edu.cn

Keywords: Corner detection, Harris operator, Steerable filter, Gray image.

Abstract: The conventional Harris corner detection operator is improved to enhance the missing rate and the detection

capability of false corners in this paper. With the materials recognition on automated logistics and

packaging line as an example, the acquired images were firstly pretreated to achieve the grayscale images.

The rotations of four different angles were performed by the steerable filter based on the grayscale and the

corner points were detected. Finally the authenticity corner points were determined through the integrated

logic operations. The image data pre-processed was detected using the improved Harris corner operator and

compared with the data by the traditional corner detection operator. The false detection rate was decreased

to 1.3 % and the missing rate reduced to 2.9% in the experiment. The results show that the improved

operator has a strong capability of discerning authenticity angular point and this method can effectively

improve the recognition accuracy of corner detection operator.

1 INTRODUCTION

Angular point is generally defined as the area in

which the grey value of image changes the most. By

detecting the angle point, the image can be

abstracted and replaced only through the typical

points. The image data volume is greatly reduced

and the speed and efficiency of image processing

were improved considerably (Chen, Lyu and Ding,

2005). Corner detection is a method of image

features extraction for computer vision system and is

used widely in the motion detection, image

matching, video tracking, 3D modelling and target

recognition, etc.

At present the corner detection algorithm can be

classified into two categories: the corner detection

based on image edge (Lee, Sun and Chen, 1995;

Beus, Steven and Tiu, 1987) and based on gray

image (Barbara, Jaroslav and Gabriele, 1999). The

corner detection based on image edge to a large

extent depends on image edge detection and image

segmentation. But the two operations have a great

deal of difficulty to achieve the good results. For

example, it is difficult to detect the complete edges

when it subjects to the local shelter or overlapping

situations, so that the scope of application is

restricted. The corner detection based on gray image

avoids this problem tactfully, and the corner is

detected by checking the gray variation of

neighborhood, and mainly by computing the gradient

and curvature. A successful corner detection

operator should be able to detect all the real angular

point. Harris operator is a recognized corner

detection algorithm for gray image, but the choice of

its threshold has a great deal of uncertainty due to

without clear theory regulation, just according to the

experience or the experiment (Harris and Stephens,

1988). Harris corner detection algorithm is widely

used in various fields. Yang et al. (Yang, Ning and

He, 2011) used Harris operator to study grains with

characteristics of tips such as corn, squash and

pumpkin for identifying tips. Wang et al. reflected

precisely the geometry quality changes of the remote

sensing image before and after compression through

Harris corner detection algorithm (Wang, Yang and

Wu, 2011). Various cases proved that Harris corner

detection operator had good angular point

recognition, but it still exist some shortage and need

to be improved.

Liu et al. put forward an improved Harris corner

detection algorithm based on image edge for fault

detection and residual angular point. The experiment

results proved that the corner detection algorithm

305

Jiang Z., Wu Y., Wang J., Lu J. and Zhang L.

The Improved Harris Operator based on Steerable Filter.

DOI: 10.5220/0006449503050311

In ISME 2016 - Information Science and Management Engineering IV (ISME 2016), pages 305-311

ISBN: 978-989-758-208-0

305

had higher accuracy (Liu, Zhao and Sun, 2013).

Fang et al. used double mask to set the local scope of

the non-maximum inhibition combined with k-

means clustering method for the non-maximum

inhibition. This method overcame the lost or

redundancy of angular point caused by uncertainty

of the traditional Harris corner detection algorithm

which selected the threshold value based on the

experience and improved the precision of corner

detection (Fang, Wang and Niu, 2011). Wang et al.

put forward a similar angular point algorithm for the

T shape and inclined T shape whose positions were

not accurate with low arithmetic speed. The

experimental results showed that the improved

algorithm was more suitable for real-time

demanding (Wang, Tang, Ren and et al., 2008).

Zhang et al. built a new Harris multi-scale corner

detection algorithm based on the wavelet transform

for defect of Harris operator without scales change

(Zhang, Li and Yang, 2007).

The paper proposed a corner detection algorithm

with improved Harris operator based on steerable

filters with material identification on the automatic

logistics packaging line as example. The improved

Harris operator, traditional Harris operator and

classical Susan operator were compared on error

detection rate and miss rate. The experiments proved

that the improved Harris operator was a good way to

reduce the error detection rate and the miss rate of

angular point.

2 HARRIS OPERATOR

The logic behind the Harris corner detection

algorithm (Harris and Stephens, 1988; Beis and

Lowe, 1997) is that, the image intensity will change

significantly in multiple directions at a corner, while

the image intensity will change greatly in a certain

direction at an edge. This phenomenon can be

formulated by examining the changes in intensity

resulting from shifts in a local window. Around a

corner point, the image intensity will change greatly

when the window is shifted in an arbitrary direction.

While around an edge point, the image intensity will

change greatly when the window is shifted in the

perpendicular direction. Following this intuition, the

Harris detector uses a second order moment matrix

as the basis of its corner decisions. Unless otherwise

specified, all corner points and edge points detected

by the Harris corner detector are hereafter referred to

Harris corner interest points.

For a given image I, its autocorrelation matrix M

at point

(, )

y can be calculated as:

I(, ) II(, )

M(,)= (,)

II(, ) I(, )

xxy

xy y

yxy

xy xy

yxy

⎡

⎤

⎢

⎥

⎢

⎥

⎣

⎦

∑

(1)

Where I

and I

is the respective derivative of pixel

intensity in the x and y direction at point (x, y). That

is,

[

]

II 1,0,1 I/

⊗− ≈∂∂

(2)

[

]

II 1,0,1 I/

⊗− ≈∂∂

(3)

Where the operator

⊗

denotes convolution.

The off-diagonal entries are the product of I

and

, while the diagonal entries are squares of the

respective derivatives. The weighting function

(, )

can be uniform, but it is more typically an

isotropic, circular Gaussian:

( , ) ( , , ) exp( )

xy gxy

ωσ

πσ σ

== −

(4)

It assigns more weight to those values near the

center of a local region. As it turns out, the matrix

M(x, y) describes the shape of the autocorrelation

measure as a result of the shifts in window location

(Harris and Stephens, 1988; Beis and Lowe, 1997).

Let α and β be the eigenvalues of M(x, y), and then

these values can provide a quantitative description of

how the autocorrelation measure changes in space,

its principal curvatures. According to the

eigenvalues of the autocorrelation matrix, the image

regions can be divided into three categories as

follows (Harris and Stephens, 1988; Beis and Lowe,

1997):

1. Plain regions: α≈0 and β≈0. Both eigenvalues

are small, thus the intensity variation is negligible in

any direction. In this case, the region can be

considered almost plain;

2. Edges: α》β. The difference between the two

eigenvalues is large, which implies that the intensity

variation is noticeable only in one direction, i.e., the

edge gradient direction;

3. Corners: α≈β, with αβ》0. Intensity variation

is strong along all directions, which is usually

achieved by ‘small spot’ regions.

The (α, β) space can be divided into three regions

by the heavy lines (Harris and Stephens, 1988; Beis

and Lowe, 1997). However, aside from the corner

and edge classification regions, a measure of their

quality or response is also needed. Note that the

product αβ is sensitive to corners, while the sum

ISME 2016 - Information Science and Management Engineering IV

306

ISME 2016 - International Conference on Information System and Management Engineering

306

α+β is sensitive to both edges and corners. Besides,

the determinant and the trace of a general

diagonalizable matrix agree with the product and the

sum of its eigenvalues, respectively, i.e.,

()

tr M , I ( , ) I ( , )

yxyxy

αβ

=+= + (5)

()

22 2

detM , I(, )I(, ) (II(, ))

xy xy

yxyxyxy

αβ

== ⋅ −

(6)

Therefore, it is attractive to use

tr(M( , ))

and

det(M( , ))

to measure the corner response since it

avoids the explicit eigenvalue decomposition of

M(x, y). The corner response can be measured using

Eq. (7) (Harris and Stephens, 1988; Beis and Lowe,

1997).

( , ) det(M( , ))- tr (M( , )) ( )Rxy xy k xy k

βαβ

=⋅=−⋅+

(7)

Where k is a scalar value empirically chosen from

the range [0.04,0.16]. Corner points have large

positive eigenvalues and hence a large Harris

measure response. Thus, corner points that are

greater than a specified threshold are identified as

local maxima of the Harris measure response, i.e.,

{

}

{

}

(, ) (, ) (, ) (,),(,) (, ),(, )

cc cc cc ii ii cc cc th

y xy Rxy Rxy xy Wxy Rxy t=>∀∈ >

(8)

Where {

(, )

} is the set of all corner points,

(, )

Rx y

is the Harris measure response calculated at

point (x, y);

(, )

Wx y

is an 8-neighbor set centered

around the point

(, )

, and

is a specified

threshold. Obviously, the number of detected Harris

corner points depends on the threshold

t .

Above all, Harris operator is a kind of efficient

point feature extracting operator with simple

algorithm and low time-consuming(Harris and

Stephens, 1988; Beis and Lowe, 1997). The operator

only needs to calculate the first-order difference and

filtering on the basis of gray level and the operation

is simple. The point feature information is relatively

uniform, and the Harris operator calculates the

interested values for all points in the image, and then

performs the optimal selection in the neighborhood.

The algorithm process is relatively stable, and

involves only a derivative operator calculation, not

affected by the image rotation, noise and perspective

transformation easily (Harris and Stephens, 1988;

Beis and Lowe, 1997). But it has also some

irremovable drawbacks. It is very sensitive to the

changes of scale, and does not have scale invariance.

Extraction of angular point level is pixel level. The

positioning accuracy is poor and is easy to miss

some practical angular point. It is necessary for the

maximum inhibition, and the extraction accuracy of

angular point depends on the set threshold. If the

threshold setting is too big, it will cause the loss of

angular point information. If the threshold setting is

too small, it will cause a lot of false corners

extracted (Harris and Stephens, 1988; Beis and

Lowe, 1997).

3 STEERABLE FILTER

Steerable Filter was put forward as a filter design

method of Fourier series expansion in 1991 by

Freeman (Stephen and Smith, 1997) based on polar

coordinates (r, θ). It is adopted as a linear

calculation of convolution. The basic idea is that the

filter along arbitrary direction can be expressed as a

linear combination of a set of base filter functions.

Image after responded by filter in one direction can

be showed by a set of linear combination of the base

filter response. This can avoid the repeated

convolution operation, thus reducing the

computational complexity.

Steerable filter has the function of arbitrary

rotation and can be represented by a linear

combination of the fixed base filter for any direction.

Therefore it can accurately detect the target features

such as edge, texture and singular point. The base

filter corresponding to the steerable filter is a set of

filters having certain overlap between each other in

the frequency domain. And each filter can be

expressed as the form of the angle rotating to a

certain way for steerable filter. The basic idea is to

produce a rotating steerable filter based on a linear

combination of filter from a fixed set. The structure

of a universal steerable filter was shown in

Figure 1.

It is defined as the following form: if a filter

Figure 1: The basic structure of the filter steerable

Image

after

filter

()

The based

filter

The

Interpolatio

n Function

The Improved Harris Operator based on Steerable Filter

307

The Improved Harris Operator based on Steerable Filter

307

(, )

can rotate, the rotation

(, )

in any

direction

is as Eq. (7).

(, ) () (, )

xy K f xy

∑

(9)

Where

(, )

- based filter, M – the number of

filter,

()

- the interpolation function.

A given function f is determined in the polar

coordinate (

,arg(,)rxy xy

=+ =

). And f

after the rotation can be expressed as the function of

a Fourier series in the polar angle

(, ) ()

rare

=−

∑

(10)

Where j=-1, N-discrete distance.

This method has a certain requirements on the

direction angle of the directional derivative filter

with a rotation angle. The filter function must be

continuous, otherwise this method will not be

applied. The solution process for the algorithm is

complex, and the computation is larger. It cannot be

directly used in industrial automation geometry

image identification and is usually combined with

other operator for application, such as combining

pyramid algorithm for face recognition (William and

Edward, 1991). In the paper, the angle corner in any

rotation direction can be expressed as a linear

combination of the same group of base filter by

using steerable filter.

4 THE IMPROVED HARRIS

OPERATOR

For the uncertainty of image rotation due to selecting

threshold of Harris operator, the paper improved the

Harris corner detection operator using rotation

invariance of steerable filter. The rotation of

different direction by steerable filter was increased

during the detection of Harris operator, and the

Harris test was. Finally the authenticity angular point

of all directions was determined by logic operations.

4.1 The algorithm process

Harris corner detection needs to set the threshold for

the non-maximum inhibition, and the accuracy of

extracting angular point depends on the setting

threshold (Xie, Zhan and Jiang, 2003; Du, Wen and

Chen, 2009). If the threshold setting is too large, the

angular point information will be lost; if the

threshold setting is too small, it will cause a lot of

false corners extracted, such as some image noise

(Xie, Zhan and Jiang, 2003; Du, Wen and Chen,

2009). The traditional Harris operator sets threshold

according to the experience and there is large

uncertainty so that it causes some false detection of

non-angular point and leak detection of actual

angular point.

This paper proposed an improved Harris corner

detection algorithm by steerable filter which can

change the selection algorithm for threshold

artificially to a great degree and improve the

accuracy of the corner detection algorithm. The

algorithm structure was as shown in Figure 2.

By image collecting of CCD camera, the binary

image of experiment was achieved by preprocessing

steps such as enhancement and smoothing. Each

pixel of binary image was regarded as pixels of

discrete grey value. And for each pixel, its four

neighbourhoods which constituted the cross point

group were calculated, as shown in Figure 3.

The brightness of the center point was judged by

detecting the four points

Ⅰ, Ⅱ, Ⅲ and Ⅳ around the

cross point group, and then its gradient was judged

according to the brightness value. If the higher

Image

Steerable

Filter

0°

45°

90°

135°

Harris

Summary

Angular

point

detection

Figure 2: The structure of the improved Harris

algorithm

Figure 3: Cross point model

Ⅰ

Ⅱ

Ⅲ

Ⅳ

ISME 2016 - Information Science and Management Engineering IV

308

ISME 2016 - International Conference on Information System and Management Engineering

308

accuracy needs to be achieved, the gradient value of

the center point can be detected by the brightness

values of 8 points surrounding. For case of the real-

time packing line system, the brightness values of

four points surrounding were selected for the

experiment considering factors such as shape and

material.

1) At first, the image was rotated by the steerable

filter of different direction.

From the figure 4 it can be seen that the position

relations between the four points of cross point

group and the center point were four rotation angles

of 0 °, 45 °, 90 ° and 135 °. According to the

brightness of the cross point, the brightness of the

center can be judged, so the four rotation angles of 0

°, 45 °, 90 ° and 135 ° were chosen. The gray image

processed was rotated according to the four angles

respectively.

2) According to the image clarity (focusing

degree), Laplace energy function was evaluated. The

brightness values of cross point model was

calculated respectively and then the gradient value of

the center point was judged.

3) The corners of each direction were detected

using Harris operator. The angular corner

information of image processed by the steerable

filter for four directions was detected using Harris

operator respectively.

4) All the authenticity corners detected were

performed by the logical "or" operation. The real

corners were selected and the false corners were

removed.

4.2 The examples

The experiment was based on Microsoft Visual C++

6.0 programming environment and Open CV. And

the classic image of corner detection operator and

the material on automatic packaging line were

selected. In this paper, all of the processing images

were gray images after the early pretreatment. The

Susan operator, the classical Harris operator and the

Harris operator improved by steerable filter were

performed by corner detection on the basis of

grayscale. The test results were analyzed and

compared as shown in Table 1.

It can be clearly observed that the extraction of

different operator for corner point information had a

large difference, especially for distinguishing

authenticity corner point. The error detection rate

and the miss rate were analyzed and compared as

shown in Table 2 and Table 3.

The false detection rate is

(11)

Where k –the non-corners false detection point

which is regarded as a corner point, N-the number of

real corner points.

Table 1: The results with different algorithms of Sample 1,

2, 3

Sample 1 Sample 2 Sample 3

Original

image

Susan

operator

Harris

operator

The

improved

Harris

operator

Table 2: The comparison of false detection rate

Algorithm

The number of angular point

for false detection

The false

detection

rate/%

Sample

Susan 8 6 7 20.34

Harris 4 5 4 7.2

The

improved

Harris

4 2 1 1.3

Table 3: The comparison of undetected rate

Algorithm The number of angular point

for false detection

The false

detection

rate/%

Sample

Susan 8 7 2 5.2

Harris 19 4 3 14.4

The

improved

Harris

3 1 1 2.9

The Improved Harris Operator based on Steerable Filter

309

The Improved Harris Operator based on Steerable Filter

309

The average error detection rate is

∑

(12)

Where n -the total numbers of subjects.

It can be seen from table 1 that for the error

detection rate, the evaluation of the Susan operator

was the highest, the traditional Harris operator was

the second place and the improved Harris operator

was the lowest.

The miss rate is

(13)

Where L-the numbers of the missed real corner

points, N-the numbers of the real corner points.

The average miss rate is

∑

(14)

Where n-the total numbers of subjects.

From table 3 it can be seen the miss rate of Susan

operator was the highest, traditional Harris operator

was middle-level, and the improved Harris operator

miss rate was the lowest.

In conclusion it can be seen that the traditional

Harris operator was superior to Susan operator in

detecting the image error detection rate and the miss

rate, and the improved Harris operator was better

than traditional Harris operator.

5 CONCLUSIONS

The paper put forward an improved Harris operator

algorithm based on steerable filter which enhanced

the leak detection and mistakenly identification for

Harris operator during the corner detection. The

gradient of the four different rotation angles for the

suspected corner pixels were further tested by

steerable filter so as to confirm whether it was really

a corner point. The experiment proved that the

method was a good way to improve the detection

accuracy of the real corner point and reduce error

detection rate of the false corner point containing the

noise. But for some high-speed case, the algorithm

program took too much time and need to be further

improved.

ACKNOWLEDGEMENTS

This paper was supported by National "863"

Program (2011AA06A101), Shaanxi Science and

Technology Department for Industrial Research

Program (2015GY120) and Doctor Startup Fund

(BJ13-18).

REFERENCES

Chen, L.，Lyu, W, G.，Ding, S, H., 2005. A survey of

corner detection algorithms. Techniques of Automation

and Applications, 5, 1-4.

Lee, J, S.

，Sun, Y, N.，Chen, C, H., 1995. Multiscale

corner detection by using wavelet transform. In IEEE

Transactions on Image Processing

：

a Publication of

the IEEE Signal Processing Society

，

41, 100-104.

Beus, H, L.

，Steven, S.， Tiu, H., 1987. An improved

corner detection algorithm based on chain-coded plane

curves. Pattern Recognition

，

20, 291-296.

Barbara, Z.

， Jaroslav, K.， Gabriele, P., 1999. Robust

detection of significant points in multiframe images 1

electronic annexes available. Pattern Recognition

Letters

，

202, 199-206.

Harris, C.

，Stephens, M., 1988. A combined corner and

edge detector. In Proceedings of the 4th Alvey Vision

Conference

，147-151.

Yang, S, Q.

，Ning, J, F.，He, D, J., 2011. Identification

of tip cap of agricultural kernel based on harris

algorithm. Transactions of the Chinese Society for

Agricultural Machinery, 3, 166-169.

Wang, M, F., Yang, S, H., Wu, Q, Z ., 2011. A geometric

quality assessment algorithm of remote sensing image

based on corner detection. Acta Geodaetica et Carto-

graphica Sinica, 2, 175-179.

Liu, B, C.

，Zhao, J. ，Sun, Q., 2013. Improved harris

corner detection method based on Edge. Chinese

Journal of Liquid Crystals and Displays

，

6, 939-942.

Fang, C.

，Wang, X, P.，Niu, Y, P.，et al., 2011. Corner

detection based on an improved Harris algorithm.

Computer and Digital Engineering, 5, 142-144.

Wang, W., Tang, Y, P., Ren, J, L.

， et al., 2008. An

improved algorithm for harris corner detection. Optics

and Precision Engineering

，

10, 1995-2001.

Zhang, X, H., Li, B., Yang, D., 2007. A novel harris multi-

scale corner detection algorithm. Journal of

Electronics & Information Technology

，

7, 1735-1738.

Beis, J., Lowe, D., 1997. Shape indexing using

approximate nearest neighbour search in high-

dimensional spaces. In Proceedings of IEEE

International Conference on Computer Vision and

Pattern Recognition, 1000-1006.

Stephen, M.

，Smith, J, M., 1997. A new approach to low

level image processing. International Journal of

Computer Vision, 23, 45-78.

William, T, F.

，Edward, H, A., 1991. The design and use

of steerable filters. In IEEE Trans Pattern Anal Mach

Intell, 13, 891-906.

Xie, D, H.

，Zhan, Z, Q.，Jiang, W, S., 2003. Improving

harris corner detector. Journal of Geomatics, 2, 22-23.

ISME 2016 - Information Science and Management Engineering IV

310

ISME 2016 - International Conference on Information System and Management Engineering

310

Du, F, M., Wen, Y, M., Chen, F., 2009. On constructing

and applying clarity-evaluation-model of the cross-line

gray-scale image. Computing Technology and

Automation, 3, 83-85.

The Improved Harris Operator based on Steerable Filter

311

The Improved Harris Operator based on Steerable Filter

311