OBJECT DETECTION AND TRACKING USING KALMAN FILTER

AND FAST MEAN SHIFT ALGORITHM

A. Ali and K. Terada

The University of Tokushima, 2-1 Minami-Josanjima, Tokushima, 770-8506 Japan

Keywords:

Change Detection, Object Tracking, Kalman ﬁlter, Mean shift Algorithm.

Abstract:

Object detection in videos involves verifying the presence of an object in image sequences and possibly locat-

ing it precisely for recognition. Object tracking is to monitor an object’s spatial and temporal changes during

a video sequence, including its presence, position, size, shape, etc. These two processes are closely related be-

cause tracking usually starts with detecting objects, while detecting an object repeatedly in subsequent image

sequence is often necessary to help and verify tracking. In this paper, a novel approach is being presented for

detecting and tracking object. It includes combination of Kalman ﬁlter and fast mean shift algorithm. Kalman

prediction is measurement follower. It may be misled by wrong measurement. In order to cater it, fast mean

shift algorithm is used. It is used to locate densities extrema, which gives clue that whether Kalman prediction

is right or it is misled by wrong measurement. In case of wrong prediction, it is corrected with the help of

densities extrema in the scene. The proposed approach has the robust ability to track the moving object in the

consecutive frames under some kinds of difﬁculties such as rapid appearance changes caused by image noise,

illumination changes, and cluttered background.

1 INTRODUCTION

Object detecting and tracking has a wide variety

of applications in computer vision such as video

compression, video surveillance, vision-based con-

trol, human-computer interfaces, medical imaging,

augmented reality, and robotics. It also plays an

important role in video database such as content-

based indexing and retrieval. Change detection tech-

niques presented in the literature can be divided

in two classes: pixel-based and region-based algo-

rithms(Jain, 1989). Pixel-based algorithms compute

the output by analyzing the values assumed by corre-

spondent pixels in the two analyzed images; region-

based algorithms compare features extracted from

correspondence regions in the two images. Pixel-

based methods, (e.g. Change detection based on bi-

nary difference) present the advantage of the sim-

plicity that makes possible real-time applications,

whereas region-based techniques (e.g. Change de-

tection based on the illumination model(Skifstadt and

Jain, 1989)) provide results more robust to false

alarms introduced by noise. A further class of CD al-

gorithms detects changed regions by means of edge

comparisons(Jain, 1989). The Kalman ﬁlter has

been extensively used in the vision community for

tracking. Broida and Chellappa(Broida and Chel-

lappa, 1986) used the Kalman ﬁlter to track points in

noisy images. In stereo camera-based object tracking,

Beymer and Konolige(Beymer and Konolige, 1999)

use the Kalman ﬁlter for predicting the objects po-

sition and speed in x - z dimensions. Rosales and

Sclaroff(Rosales and Sclaroff, 1999) use the extended

Kalman ﬁlter to estimate 3D trajectory of an object

from 2D motion. A common approach to handle com-

plete occlusion during tracking is to model the ob-

ject motion by linear dynamic models or by nonlin-

ear dynamics and, in the case of occlusion, to keep

on predicting the object location until the object reap-

pears. For example, a linear velocity model is used in

Beymer and Konolige(Beymer and Konolige, 1999)

and a Kalman ﬁlter is used for estimating the loca-

tion and motion of objects. For the image segmen-

tation problem, Mean-Shift Clustering is commonly

used. Comaniciu and Meer(Comaniciu and Meer,

2002) propose the mean-shift approach to ﬁnd clus-

ters in the joint spatial and color space.

In this paper we propose a novel object tracking

scheme showing good tracking performance in the

consecutive frames under some kinds of difﬁculties.

It combine Kalman ﬁlter and fast mean shift algo-

585

Ahmed A. (2009).

OBJECT DETECTION AND TRACKING USING KALMAN FILTER AND FAST MEAN SHIFT ALGORITHM.

In Proceedings of the Fourth International Conference on Computer Vision Theory and Applications, pages 585-589

DOI: 10.5220/0001787705850589

 SciTePress

rithm. The paper is organized as follows: section 2

gives a brief overview on the tracking algorithm and

also gives details on the proposed tracking algorithm ;

The experimental results are demonstrated in Section

3. Finally, the paper is concluded in section 4.

2 TRACKING ALGORITHM

Object tracking is the problem of estimating the po-

sitions and other relevant information of moving ob-

jects in image sequences. Two-frame tracking can

be accomplished using correlation-based matching

methods, optical ﬂow techniques, or change-based

moving object detection methods. In this paper, the

tracking accomplished by using change-based mov-

ing object detection method. The tracking algorithm

is brieﬂy described in next subsections.

2.1 Change-based Object Tracking

Method

Change detection by background subtraction is a

common approach to detect moving foreground. The

resulting difference image is usually thresholded to

obtain objects based on pixel connectedness and re-

sulting blob objects are subsequently tracked(Bovik,

2000). The Change-based tracking algorithm can be

described as follows:

• Image subtraction to detection motion blobs:

– Compute the difference image between two

frames

– Thresholding to ﬁnd the blobs

– Locate the blob center as the position of object

• Track the object by track the blob center

For target with not so much cluttered background or

little illumination changes and there is no motion in

the images being used to generate the stationary back-

ground, results obtained by this technique is of very

high accuracy as shown in ﬁg.1.

(a) Frame 151 (b) Frame 226

Figure 1: Tracking using change-based detection.

2.2 Difﬁculties with the Tracking

Method

In change detection where the reference frame is a

stationary background image that has been generated

in an initialization phase using a simple averaging, it

is of paramount importance that there is no motion

in the images being used to generate the stationary

background. However, in practice, it is very difﬁcult

or almost impossible to ensure that there is no motion

during the initialization phase in real-life applications

(in many situations where you do not have full con-

trol over the environments – for example, in trafﬁc

monitoring and video surveillance applications)(Ong

and Spann, 1999). Cheung and Kamath(Cheung

and Kamath, 2004) studied the methods for generat-

ing a background model, such as frame differencing,

median ﬁlter, linear predictive ﬁlter, non-parametric

model, kalman ﬁlter and Mixture of Gaussians model.

An example of such situation, only change-based

tracking technique didn’t correctly track the object as

shown in ﬁg.2.

(a) Frame 135 (b) Frame 196

Figure 2: The tracker lost the object.

In order to tackle the above-mentioned problem,

Kalman tracker was embedded in the algorithm.

Kalman ﬁlter continuously predicts the next state of

the target(Grewal and Andrews, 2001) based on the

measurement of change-based technique. Kalman

predictor helps in ﬁnding where the object is most

likely to be found in the next frame. A good result ob-

tained after embedding Kalman Tracker as shown in

ﬁg.3. its clear that Kalman tracker successfully track

the object.

2.3 Tracking Using Kalman Filter

A Kalman ﬁlter is used to estimate the state of a

linear system where the state is assumed to be dis-

tributed by a Gaussian. Kalman ﬁltering is composed

of two steps, prediction and correction(Bar-Shalom

and Foreman, 1988). TheKalman ﬁlter a recursivees-

timator. This means that only the estimated state form

the previous time step and the current measurement

are needed to compute the estimate for the current

VISAPP 2009 - International Conference on Computer Vision Theory and Applications

586

state. In contrast to batch estimation techniques, no

history of observations and/or estimates is required.

Fig.3(a) and (b) show this scenario. Red window

shows the location where the target is detected, while

green window shows the Kalman prediction window.

It was decided that when the measurement differ from

the predicted position by more than a certain thresh-

old value, then the measurement will be discarded and

the Kalman prediction will be taken as the next posi-

tion of the target.

(a) Frame 130 (b) Frame 215

Figure 3: The object successfully tracked after embedding

Kalman tracker; the red rectangle show the tracking using

change-based and the green rectangle show the tracking us-

ing Kalman.

2.4 Difﬁculties Using Kalman Filter

The major problem with the tracking was that of the

target detection. Measurements given by the detec-

tor are sometimes very noisy due to rapid appearance

changes caused by image noise, illumination changes,

cluttered background. Kalman prediction is measure-

ment follower. It may be misled by wrong measure-

ment. An example of such situation is shown in ﬁg.4.

(a) (b)

Figure 4: Kalman ﬁlter lost the object by wrong measure-

ments.

In order to treat it, fast mean shift algorithm is used.

It is used to locate densities extrema, which gives clue

that whether Kalman prediction is right or it is misled

by wrong measurement. In case of wrong prediction,

it is corrected with the help of densities extrema in the

scene. Local density maxima in the difference image

– usually representing moving objects – are outlined

by a fast non-parametric mean shift clustering proce-

dure.

2.5 Fast Mean Shift Algorithm

The mean shift algorithm is a nonparametric tech-

nique to locate density extrema or modes of a given

distribution by an iterative procedure(C. Beleznai and

Bischof., 2005). Starting from a location x the local

mean shift vector represents an offset to x

′

, which is a

translation towards the nearest mode along the direc-

tion of maximum increase in the underlying density

function. The local density is estimated within the

local neighborhood of a kernel by kernel density es-

timation where at a data point a kernel weights K(a)

are combined with weights associated with the data.

.For digital images sample weights are deﬁned by the

pixel in intensities at pixel locations a. the new lo-

cation vector x

′

obtained after applying the mean shift

offset. Fast mean shift algorithm was embedded in the

algorithm. If pixel difference between detected mea-

surement obtained by change-based method (the blob

center of the object)and Kalman ﬁlter predicted po-

sition is greater than a certain threshold, then motion

region is detected by triggering fast mean shift algo-

rithm and it guides Kalman ﬁlter, by ﬁnding actual

motion region in the scene, for more accurate predic-

tion of next state of the object. Below the proposed

tracking algorithm scheme can be described by the

following diagram (ﬁg.5).

Figure 5: The proposed Tracking Algorithm scheme.

OBJECT DETECTION AND TRACKING USING KALMAN FILTER AND FAST MEAN SHIFT ALGORITHM

587

3 EXPERIMENTS AND RESULTS

This section show the results obtained by the pro-

posed algorithm. The system was tested using In-

tel(R) Core Duo CPU device with 2.10 GHZ and 3GB

of RAM and Window XP operation system, and Mat-

lab R2007b are used. In addition, four image se-

quences(both indoors and outdoors) are used for test-

ing the system and each image sequence consists of

video frames with 320x240 resolutions per frame. It

is clear from ﬁg.6 (a) that a person walking in a clut-

tered background under the light sources and lighting

changes reﬂected by the white boards put aside.

(a) Image sequence one

(b) Image sequence two

Figure 6: Kalman tracker successfully tracked the object

guided by mean shift algorithm.

Figure 7: Image sequence four; Kalman tracker success-

fully tracked pedestrian in outdoor scenes guided by mean

shift algorithm.

After emerging mean shift algorithm, the object suc-

cessfully detected by mean shift algorithm and pre-

cisely tracked by Kalman predictor by following the

true measurement taken by mean shift detector. An-

other example to test the system using different noisy

images caused by different lighting sources as shown

in ﬁg.6(b). the system was chalenged with a very

noisy image sequence caused by rapid illumination

changes due to different sources of lighting and due to

the walking person shadow as shown in ﬁg.6(c). An

example of outdoor waking person to test the system

by using PETS 2001 Image Sequences. The results

show that the system excellently tracked the object as

shown in ﬁg.7. The satisfactory results are achieved.

By comparing the results taken only using Kalman ﬁl-

ter (see ﬁg. 4) and the result taken by the system after

embdding Mean Shift Algorithm (see ﬁg. 6), its clear

that the system overcomes these difﬁcults by success-

fully detected the motion area. Finally, the perfor-

mance of the system was evaluated using the process-

ing time of the test video sequences as shown in Table

1. The average processing time of every frame is be-

tween 0.025 and 0.031 second. Therefore, it can be

applied to real-time application.

Table 1: The processing time of test video sequences.

Test video seqs No. Frames Track. T.(s)

video seq.1 50 1.56

video seq.2 70 1.95

video seq.3 60 1.71

video seq.4 100 2.53

4 CONCLUSIONS

This paper presents a novel approach for object de-

tection and tracking. It includes combination of

Kalman ﬁlter and fast mean shift algorithm. The sys-

tem start by detecting the object using change detec-

tion technique and track the object by track the blob

center of the detected object. To increase the efﬁ-

ciency of the algorithm, Kalman ﬁlter works parallel

with the tracking algorithm. Kalman ﬁlter continu-

ously predicts the next state of the target based on

the measurement of change-based technique. Under

some kinds of difﬁculties such as rapid appearance

changes caused by image noise, illumination changes,

and cluttered background, Kalman prediction may be

misled by wrong measurement. If pixel difference

between detected measurement obtained by change-

based method and kalman ﬁlter predicted position is

greater than a certain threshold, then fast mean shift

algorithm is used. It is used to locate densities ex-

trema, which gives clue that whether Kalman predic-

VISAPP 2009 - International Conference on Computer Vision Theory and Applications

588

tion is right or it is misled by wrong measurement.

In case of wrong prediction, it is corrected with the

help of densities extrema in the scene. The proposed

method is an efﬁcient video object tracking algorithm.

Furthermore, to consider the situations of tracking

multiple objects, every one of multiple objects can be

set an Kalman ﬁlter to track it. Also since the pro-

cessing time using the proposed method to track the

moving object is short, therefore, the system imple-

mented by the proposed method can afford to track a

moving objects in real time.

REFERENCES

Bar-Shalom, Y. and Foreman, T. (1988). Tracking and Data

Association. Academic Press Inc.

Beymer, D. and Konolige, K. (1999). Real-time tracking of

multiple people using continuous detection. In IEEE

International Conference on Computer Vision (ICCV)

Frame-Rate Workshop.

Bovik, A. (2000). Handbook of Image and Video Processin.

Academic Press.

Broida, T. and Chellappa, R. (1986). Estimation of object

motion parameters from noisy images. In IEEE Trans.

Patt. Analy. Mach. Intell.

C. Beleznai, B. F. and Bischof., H. (2005). Ieee inter-

national workshop on visual surveillance and perfor-

mance evaluation of tracking and surveillance. In

Breckenridge.

Cheung, S. C. S. and Kamath, C. (2004). Robust techniques

for background subtraction in urban trafﬁc video. In

SPIE Electronic Imaging: Video Communications and

Image Processing.

Comaniciu, D. and Meer, P. (2002). Mean shift: A ro-

bust approach toward feature space analysis. In IEEE

Trans. Patt. Analy. Mach. Intell.

Grewal, M. S. and Andrews, A. G. (2001). Kalman Filtering

Theory and Practice using MATLAB, second edition.

John Wiley & Sons, Inc.

Jain, A. K. (1989). Foundamentals of Digital Image Pro-

cessing. Prentice-Hall.

Ong, E. P. and Spann, M. (1999). Robust optical ﬂow com-

putation based on least-median of squares regression.

In International Journal of Computer Vision.

Rosales, R. and Sclaroff, S. (1999). 3d trajectory recov-

ery for tracking multiple objects and trajectory guided

recognition of actions. In IEEE Conference on Com-

puter Vision and Pattern Recognition (CVPR).

Skifstadt, K. and Jain, R. (1989). Illumination independent

change detection for real world sequences. In Graph-

ics, and Image Processing.

OBJECT DETECTION AND TRACKING USING KALMAN FILTER AND FAST MEAN SHIFT ALGORITHM

589