SPIKING HIERARCHICAL NEURAL NETWORK FOR CORNER

DETECTION

Dermot Kerr

, Martin McGinnity

, Sonya Coleman

, Qingxiang Wu

and Marine Clogenson

Intelligent Systems Research Centre, University of Ulster, Magee, Derry, BT48 7JL, U.K.

CPE Lyon, Domaine Scientifique de la Doua, BP 82077, 69616, Villeurbanne, France

Keywords: Spiking neural network, Corner detection.

Abstract: To enable fast reliable feature matching or tracking in scenes, features need to be discrete and meaningful,

and hence corner detection is often used for this purpose. We present a new approach to corner detection

inspired by the structure and behaviour of the human visual system, which uses spiking neural networks.

Standard digital images are processed and converted to spikes in a manner similar to the processing that is

performed in the retina. The spiking neural network performs edge and corner detection using receptive

fields that are able to detect edges and corners of various orientations. The locations where neurons emit a

spike indicate the positions of detected features. Results are presented using synthetic and real images.

1 INTRODUCTION

Previous research has illustrated that edges,

contours and corners are very important for visual

perception (Shapley and Tolhurst, 1973). Many

derivative-based feature detection operators have

been proposed in the past 30 years, in particular

many detectors have been proposed to detect edge

junctions and corners. Moravec (1977) developed a

corner detector that shifted a small square window

in vertical, horizontal, and diagonal directions.

Harris and Stephens (1988) expanded the Moravec

operator, removing the limitation of discrete

window shifts, to develop a combined corner and

edge detector. The operator response determines

whether the detected feature is a corner, edge, or a

flat region. Smith and Brady’s SUSAN corner

detector (Smith and Brady, 1997) is based on

brightness comparisons over neighbourhoods and

the detector can distinguish both corner and edge

pixels. Shen and Wang (2001) have expanded a

local edge detector so that corners may also be

detected.

However, when comparisons are drawn

between the performance of such artificial vision

feature detectors and the processing capabilities of

the human visual system (HVS) it becomes

apparent that current approaches suffer serious

weaknesses. Recently researchers have started to

examine the possibility of using biologically

inspired image processing techniques. In the HVS a

visual scene is processed starting in the retina where

light intensity is converted into nerve signals within

the photoreceptors. These signals are then pre-

processed and propagated through the various layers

within the retina with varying delays and lateral

inhibition onto the retinal ganglion cells. The majority

of the resulting spike train output from the retinal

ganglion cells travels along the optic nerve for further

processing in the lateral geniculate nucleus (LGN),

and other areas of the brain. Biological research has

shown that the brain deals with information

processing by using a complicated network of neurons

(Hodgkin and Huxley, 1952). The process of

simulating biological information processing in

engineering is termed neuro-engineering (O’Connor,

Huber and Svoboda, 2008) and such techniques are

typically used for artificial intelligent systems.

Spiking neural networks (SNNs) are a new class of

artificial neural network that mimic biological

information processing in the brain more accurately

than traditional neural networks. However there are

very few attempts to use SNNs to model aspects of the

human visual system. In (Van Rullen and Thorpe,

2002) scene categorisation is performed and this work

is then expanded in (Masquelier and Thorpe, 2010) to

perform object recognition. In (Escobar, Masson,

Vieville and Kornprobst, 2009) a SNN is used to

model two areas of the brain concerned with motion,

with the aim of action recognition. A SNN model is

230

Kerr D., McGinnity M., Coleman S., Wu Q. and Clogenson M..

SPIKING HIERARCHICAL NEURAL NETWORK FOR CORNER DETECTION.

DOI: 10.5220/0003682402300235

In Proceedings of the International Conference on Neural Computation Theory and Applications (NCTA-2011), pages 230-235

ISBN: 978-989-8425-84-3

 2011 SCITEPRESS (Science and Technology Publications, Lda.)

proposed in (Meftah, Lezoray and Benyettou,

2010) that performs segmentation and edge

detection. In (Chevallier et al., 2006) a distributed

SNN is proposed for extracting saliencies in an

image and in (Hugues et al., 2002) contours are

detected in images through the synchronisation of

integrate and fire neurons. SNN approaches have

also recently been applied for the purpose of image

segmentation, in (Wu et al., 2007a) which has

proven to be fast and efficient. However, these

approaches focus on edge or contour detection but

to a lesser extent, corner or interest point detection.

In (Wu et al., 2007b) a SNN was proposed that

detected right angle corners only.

In this paper we present a SNN approach to

corner detection. Our approach is based on a

biologically inspired hierarchical structured SNN

that is capable of detecting various features (edges

and corners). The network uses difference of

Gaussian filters, replicating the retinal ganglion

cells in the retina, for converting images to spikes.

Receptive fields are formed using a hierarchical

structure, with inputs from two types of retinal

ganglion cells that are capable of detecting edges

and corners. The network detects corners at angles

of 45 and 90 degrees.

In Section 2 we present the neuron model used

in the simulations and in Section 3 we present our

spiking neural network structure. Experiments and

results are presented in Section 4 with discussion

and further work presented in Section 5.

2 SPIKING NEURON MODEL

Biological neurons use short and sudden increases

in voltage (commonly known as action potentials,

spikes or pulses) to send information. The first

scientific model of a spiking neuron, proposed by

Hodgkin and Huxley (1952), is based on

experimental recordings from the giant squid axon

using a voltage clamp method. The complexity in

simulating the model is very high due to the

number of differential equations and the large

number of parameters. Thus, most computer

simulations choose to use a simplified neuron

model such as the integrate-and-fire model (I&F).

The I&F model models the state of the neuron by

its membrane potential, which receives excitatory

or inhibitory signals from synaptic inputs from

other neurons. Each input is weighted by its

associated synaptic strength. The leaky I&F model

produces a more biologically realistic neuron

model adding a “leak” term to the membrane

potential, reflecting the diffusion of ions that occurs

through the membrane when some equilibrium is not

reached in the cell. For implementation purposes, the

leaky I&F model has been selected to model the

network neurons in this paper. A full review of the

biological behaviour of single neurons can be found in

(Gerstner and Kistler, 2002) and a comparison of

different neuron models can be found in (Izhikevich,

2004).

3 NETWORK STRUCTURE

In a biological system a receptive field is where a

spiking neuron integrates the spikes from a group of

afferent neurons as illustrated in Figure 1 where

neuron N has a receptive field with a 9 neuron array.

Each neuron in the receptive field connects to neuron

N through both excitatory and inhibitory synapses.

Figure 1: Receptive field of a spiking neuron.

We construct a spiking neural network using

receptive fields with the leaky I&F neuron model

(outlined in Section 2). Within the network structure

proposed we have four processing layers

corresponding to the receptor layer, the edge detection

layer, the corner detection layer, and the output layer.

We define our spiking neural network structure as

illustrated in Figure 2. The first layer in the network

represents the retinal ganglion cells found in the

retina. Here the input image mimics the On-Centre

Off-Surround and Off-Centre On-Surround ganglion

cells found in the retina by convolution with

difference of Gaussian (DoG) filters. This layer

produces two images that are converted into spike

trains in the time domain. In summary the conversion

from input image to spike trains involves converting

the DoG responses to spike trains where high DoG

responses correspond to spike trains with short delays

and low DoG responses correspond to spike trains

SPIKING HIERARCHICAL NEURAL NETWORK FOR CORNER DETECTION

231

Figure 2: Spiking Neural Network Structure.

with long delays. The highest output from the DoG

filtered images are areas where the image intensity

changes rapidly, corresponding to the most rapidly

firing neuron. Zero or negative DoG responses are

areas where the image intensity remains constant

corresponding to the slowest firing neurons

Due to the nature of the On-Centre Off-

Surround and Off-Centre On-Surround images any

neurons that fire slow in one image fire rapidly in

the other, and vice-versa as the images are inverses

of each other. Thus, areas of interest in the images,

such as a rapid change in intensity can be detected

with a change in the firing rate of neurons in a

particular region. To detect this change in firing

rate between images we construct receptive fields

of various orientations that receive input on either

side from the On-Centre and Off-Centre images. If

the receptive field has all the neurons firing rapidly

i.e. both sides of the receptive field, this

corresponds to an area of the image with rapidly

changing intensity, such as an edge.

In this work we define the edge detection layer

with four types of neurons corresponding to four

different receptive fields respectively. There are

four parallel arrays of neurons in the edge

detection layer each of the same dimension as the

Receptor layer with only one neuron (labeled

) in

each array illustrated in Figure 2 for simplicity. Each

of these layers performs the processing for a different

edge direction and is connected to the receptor layer

by differing weight matrices. The receptive field

receives input on either side from both the On- Centre

and Off-Centre receptor level inputs. The arrangement

of the inputs determines the edge orientation that may

be detected. In the experiments presented here we

have four types of edge receptive fields corresponding

to horizontal, vertical, and both diagonal directions.

The synaptic weights for all the edge detection

receptive fields are identical and are chosen

heuristically.

The corner detection layer is composed of eight

types of neurons and performs in a similar manner to

the edge detection layer. The inputs to the corner

detection neurons are the outputs from the edge

detection neurons at different orientations. The

arrangement of the inputs determines the type of

corner that may be detected. Each corner detection

neuron has a receptive field formed by different edge

detection neurons. For example, in Figure 2 we

illustrate that the corner detection neuron

forms a

NCTA 2011 - International Conference on Neural Computation Theory and Applications

232

receptive field with edge detection neurons

and NE

corresponding to horizontal and vertical

edges respectively. Thus actual connectivity of the

synapses within the receptive field defines the type

of corner the neuron can detect. In Figure 2 we

have illustrated three types of corner detection

receptive fields for visual clarity. The synaptic

weights for all the corner detection receptive fields

are identical and are chosen heuristically.

The output layer integrates all the responses

from the corner detection layer and produces a

firing map. The corner neuron firing map indicates

those neurons that have reached each individual

neuron’s firing threshold and thus produced a

spike. Hence, a corner point is detected at a

location where a neuron in the corner detection

layer has fired at least one spike. For visual clarity

detected points are superimposed over the original

image in the presented results. The network model

was implemented with the Brian simulator

(Goodman and Brette, 2009) using a standard

leaky I&F model with parameters

that are

consistent with biological neurons (Gerstner and

Kistler, 2002).

4 EXPERIMENTS AND

RESULTS

In order to test the performance of our proposed

spiking neural network we construct a synthetic

image with two rectangular shapes at different

orientations. The image intensities used to

construct the step edges in the synthetic image are

100, 129, and 158 (where the possible range of

intensities is [0-255]) and the image size is 45



pixels. In the case of the orientated rectangle shape

the intensities are obtained through bilinear

interpolation using the same step edge intensities,

the synthetic image is presented in Figure 3.

Figure 3: Example synthetic input image.

As described in Section 3 we then convolve the

input image with DoG filters, mimicking the On-

Centre Off-Surround and Off-Centre On-Surround

ganglion cells found in the retina and the outputs after

convolution with the two DoG filters are illustrated in

Figure 4.

(a) On-Centre

Off-Surround

(b) Off-Centre

On-Surround

Figure 4: Example retinal ganglion cell filtered image.

The retinal ganglion cell images are converted into

spike trains. Figure 5 illustrates an example raster plot

for the spike activities of the image in Figure 4(a). In

Figure 5, individual neurons are represented on the y-

axis and the x-axis represents the spike activities of

each neuron over the simulation time. This spike

raster plot illustrates that in areas of the retinal

ganglion cell image with negative or zero values no

spikes are produced and in areas of the retinal

ganglion cell image with the strongest responses the

corresponding neuron fires rapidly.

Figure 5: Example spike trains computed from On-Centre

Off-Surround ganglion cell image.

The On-Centre and Off-Centre image spike trains

are presented to the hierarchical network and

processed by the edge detection layer using the

receptive fields. This layer then provides input to the

corner detection layer. To illustrate the performance

of the edge detection layer we show an example

output raster plot for the horizontal edge in Figure 6(a)

and the combined outputs from the edge detection

layer translated to image positions highlighted over

the original input image in Figure 6(b). The spikes

output from the edge detection layer are then input

into the corner detection layer where the various types

of receptive fields process them in order to perform

SPIKING HIERARCHICAL NEURAL NETWORK FOR CORNER DETECTION

233

(a) Firing neurons for horizontal edge (b) Edge neurons firing locations

Figure 6: Example neurons firing in edge detection layer.

(a) Firing neurons for corner type 1 (b) Corner neurons firing locations

Figure 7: Example neurons firing in corner detection layer.

(a) Input image (b) SNN Corner detector (c) Harris Corner detector

Figure 8: Example output from simple real image.

corner detection. The outputs from the corner

detection layer are then integrated in the output layer

where neurons are tuned to fire upon receiving a

spike from any neuron in the corner detection layer.

In Figure 7(a) we illustrate the output raster plot

for all neurons connected to one particular type of

corner detection receptive field (in this case a 90°

corner orientated between 90°-180°). There is only

one spike firing in the raster plot indicating that

there is only one particular type of corner present

corresponding to that type of neuron in the image.

The outputs from all the corner neurons that have

fired in the corner detection layer are illustrated in

Figure 7(b) where the firing neurons have been

transformed into image locations and marked where

the centre of the square is the firing neuron location.

We have also applied the network to a simple real

image to examine its performance in comparison to

the standard corner detection algorithm of Harris and

Stephens (1988), as illustrated in Figure 8. This

visual comparison illustrates the SNN provides

similar results to the Harris corner detector (with a

threshold equal to 120) and in some cases the

corners are more accurately located using the SNN

approach than the Harris corner detector.

NCTA 2011 - International Conference on Neural Computation Theory and Applications

234

5 DISCUSSION AND FUTURE

WORK

The spiking neural network presented in this paper is

constructed using a hierarchical structure that is

composed of spiking neurons with various receptive

fields. The input image is converted to retinal

ganglion cell output spike trains by convolving with

DoG filters. The spike trains are presented to the

network and the various receptive fields process the

image, performing edge detection and corner

detection. The spiking neuron models provide

powerful functionality for integration of inputs and

generation of spikes. Synapses are able to perform

different complicated computations. This paper

demonstrates how a spiking neural network can

detect edge and corners in an image. The

performance illustrates that the proposed detector is

currently only capable of detecting simple edges at

specific orientations and similarly only particular

corner types. However, the current results appear

promising when compared with the standard Harris

approach to corner detection. Future work will

involve the incorporation of biologically plausible

unsupervised learning algorithms (STDP) to set the

synaptic weights, automatic development of

receptive fields to deal with different edge and

corner types.

ACKNOWLEDGEMENTS

This work was supported by the Centre of

Excellence in Intelligent Systems project, funded by

InvestNI and the Integrated Development Fund.

REFERENCES

Chevallier, S., Tarroux, P. and Paugam-Moisy, H., (2006).

Saliency extraction with a distributed spiking neural

network. ESANN'06.

Escobar M. J., Masson G. S., Vieville T. and Kornprobst

P., (2009). Action Recognition Using a Bio-Inspired

Feedforward Spiking Network. International Journal

of Computer Vision. 82, 3, 284-301.

Gerstner, W. and Kistler, W., (2002). Spiking Neuron

Models: Single Neurons, Populations, Plasticity,

Cambridge University Press, 2002.

Goodman D. F. and Brette R., (2009). The Brian

simulator, Front. Neurosci. 3, 2, 192-197.

Harris, C. and Stephens, M., (1988). A Combined Corner

and Edge Detector. Proceedings 4th Alvey Vision

Conf. 147-151.

Hodgkin, A. and Huxley, A. (1952). A quantitative

description of membrane current and its application to

conduction and excitation in nerve. Journal of

Physiology, London, 117, 500-544.

Hugues, E., Guilleux, F. and Rochel, O., (2002). Contour

Detection by synchronization of Integrate and Fire

Neurons. BMVC, 2002.

Izhikevich, E.M., Which model to use for cortical spiking

neurons? IEEE Trans. on Neural Networks. 15, 5.

Masquelier, T. and Thorpe, S. J. (2010). Learning to

recognize objects using waves of spikes and Spike

Timing-Dependent Plasticity. The 2010 International

Joint Conference on Neural Networks (IJCNN). 1-8.

Meftah, B., Lezoray, O. and Benyettou, A. (2010).

Segmentation and Edge Detection based on Spiking

Neural Network Model. Neural Processing Letters,

32, 2, 2010.

Moravec, H. P., (1977) Towards Automatic Visual

Obstacle Avoidance. Proceedings 5th Int. Joint

Conference. Artificial Intelligence. Cambridge, MA,

USA. 584.

O’Connor, D. H., Huber, D., and Svoboda, K. (2008).

Reverse engineering the mouse brain. Nature 461,

923-929.

Shapley, R. M. and Tolhurst, D. J., (1973). Edge detectors

in human vision. Journal of Physiology. 229, 1, 165-

183.

Shen, F. and Wang, H., (2001). A Local Edge Detector

Used for Finding Corners. Proceedings 3rd Int. Conf.

Inf, Comm. And Signal Processing. Singapore.

Smith, S. M. and Brady, J. M., (1997). SUSAN – A New

Approach to Low Level Image Processing. IJCV. 23,

1, 45-78.

van Rullen R. and Thorpe S., (2002). Surfing a spike wave

down the ventral stream. Vision Research. 42, 2593-

2615.

Wu, Q. X., McGinnity, T. M., Maguire, L. P., Glackin B.

and Belatreche, A., (2007a). Information Processing

Functionality of Spiking Neurons for Image Feature

Extraction. 7th Int. Workshop on Information

Processing in Cells & Tissue. Jesus College, Oxford,

UK.

Wu, Q. X., McGinnity, T. M., Maguire, L. P., Belatreche,

A., and Glackin, B., (2007b). Edge Detection Based

on Spiking Neural Network Model. Proc of ICIC.

SPIKING HIERARCHICAL NEURAL NETWORK FOR CORNER DETECTION

235