ROADGUARD

Highway Control and Management System

Salma Kammoun Jarraya

MIRACL-FSEG, Sfax University, Rte Aeroport Km 4, 3018 Sfax, Tunisia

Adam Ghorbel, Ahmed Chaouachi, Mohamed Hammami

MIRACL-FS, Sfax University, Rte Soukra Km 3 BP 802, 3018 Sfax, Tunisia

Keywords: Moving object detection, Tracking, Counting, Shadow detection, Shadow removal.

Abstract: In this paper, we propose a new approach, called RoadGuard, for Highway Control and Management

System. RoadGuard is based on counting and tracking moving vehicles robustly. Our system copes with

some challenges related to such application processing steps like shadow, ghost and occlusion. A new

algorithm is proposed to detect and remove cast shadow. The occlusion and ghost problems are resolved by

the adopted tracking technique. A comparative study by quantitative evaluations shows that the proposed

approach can detect vehicles robustly and accurately from highway videos recorded by a static camera

which include several constraints. In fact, our system has the ability to control highway by detecting strange

events that can happen like sudden stopped vehicles in roads, parked vehicles in emergency zones or even

illegal conduct such going out from the road. Moreover, RoadGuard is capable to manage highways by

saving information about date and time of overloaded roads.

1 INTRODUCTION

Highway surveillance is an active research subject in

computer vision. In fact, the rapidly increasing of

car’s numbers makes the roads overloaded and

traffic congestion growing up. The traditional

solution has been to construct more and larger

roadways, but that is no longer seen as an option by

transportation planners, due to the high financial,

social, and environmental costs of such giant

projects. More efficient use of the existing roadways

especially highways network using advanced

technologies seems to be the answer. Therefore, in

perfect harmony with the big international

orientations, considerable investigations aim to keep

the world moving safely, comfortably,

economically, and without harm to our environment

by creating the best transportation system through

proactive excellence, leadership and innovation in

services. The objective is to ensure that roadways

continue to be safer and more technologically up-to-

date. Software solutions like Highway Control and

Management systems (HCMSs) are used to solve

these problems. These systems can lead to semantic

results, such as ‘87 cars are in the right side of the

highway’ or ‘an obstacle is on the road!!’ or ‘car

No.5 is faster than car No.1’ etc.

Semantic results of HCMS are based on counting

and tracking moving vehicles starting by detecting

moving objects (foregrounds). In fact, foregrounds

detection could be considered as one of the

fundamental and critical step in this field. Moving

object detection methods cannot differentiate real

foregrounds from their shadow since they shared

some motional features. This has bad effects on

performance of the upper steps of HCMS. Actually,

researches related to these tasks are still far from

being solved.

In this paper, we present RoadGuard, a new

HCMS that counts and tracks robustly vehicles in

highway. Moreover, our system copes with different

challenges such as shadow, occlusion, ghost and

others outdoor conditions. Since shadow detection

and removal takes a careful consideration in HCMS,

we propose a new algorithm which detect moving

shadows and suppress the cast part. Indeed, we

adopt a tracking technique that resolve occlusion and

ghost problems.

632

Kammoun Jarraya S., Ghorbel A., Chaouachi A. and Hammami M..

ROADGUARD - Highway Control and Management System.

DOI: 10.5220/0003369406320637

In Proceedings of the International Conference on Computer Vision Theory and Applications (VISAPP-2011), pages 632-637

ISBN: 978-989-8425-47-8

 2011 SCITEPRESS (Science and Technology Publications, Lda.)

This paper is organized as follow. In section 2,

we started out with over viewing state of art on

common process steps of HCMSs. Following that,

our proposed approach RoadGuard is reviewed in

section 3. In section 4, an intensive experimental

evaluation and comparison results are discussed. The

proposed approach is summarized and future works

directions are presented in section 5.

2 LITERATURE SURVEY

In this section, we give a state of art on the common

process steps of HCMSs. In general, process of such

systems includes four steps: (1) Moving object

detection, (2) Shadow detection and removal, (3)

Tracking and (4) Counting. The (1), (2) and (3) are

widely treated in literature. The proposed

approaches for each step are presented in the

following subsections.

2.1 Moving Object Detection

Moving object detection is a well known research

area in computer vision. Many methods have already

been proposed in this field. We classify

contributions reported in literature into four main

categories according to the inter-image processing

they adopt: detection based on Inter-Frame

Difference (IFD), detection based on Background

Modeling (BM), detection based on Optical Flow

(OF), and detection based on Hybrid approaches.

BM methods (Tang, 2007) (Grimson, 1998)

(Elhabian, 2008) are the most popular approach,

they detects moving objects by comparing

background model to each input frame. The

efficiency of the BM methods depends on the ‘ideal

background model’ that is not easy to obtain and

easy to be influenced by the environmental

conditions.

2.2 Shadow Detection and Removal

Shadow is occurred when a direct light from a light

source is blocked by a moving object. In fact, there

are two types of moving shadow: cast and self

shadow (Cucchiara, 2001). Generally, the self

shadow appears in the portion of a moving object

which is not illuminated by direct light. Indeed, the

cast shadow presents the area projected by the

moving object in the direction of direct light.

The moving shadows detected erroneously as

part of foregrounds affect the shape of the real

moving objects and/or can cause occlusion between

them. Thus, moving shadow presents the most

serious problem in the performance of the upper

levels of computer vision applications such as

HCMS. The core problem discussed in literature is

the detection and suppression of cast shadow and not

self shadow since it is a part of the moving object.

Indeed, we classify previous works into two

categories of approaches, according to the decision

process, which are: Statistical (Julio, 2005) (Mikic,

2000) and Determinist (Cucchiara, 2001) (Xiao,

2007). In the statistical approach, probabilistic

functions are used to describe the class membership.

In the determinist approach, an on/off decision is

used to classify each pixel to shadow/non-shadow

pixel. Moreover, two subclasses are presented, the

Determinist Model Based (DM) and Determinist

Non-model Based (DNM). DM methods require a

priori knowledge about shape and motion of objects

such as vehicles or human bodies (Kilge, 1992).

Despite its simplicity, this method cannot be robust

to many variations like illumination, objects shape,

shadow edge. Besides, DNM methods classify pixels

to shadow/non-shadow pixels without any prior

knowledge about the scene (Cucchiara, 2001).

2.3 Tracking

Object tracking is used to estimate the trajectory of

moving objects over time in every frame. However,

the complexity of tracking objects is more

complicated by abrupt object motion, occlusion

between moving objects and/or between background

and foreground objects. Several tracking methods

(Kass, 1988) (Masoud, 2001) (Kanhere, 2006) are

proposed in literature. We distinguish three major

tracking approaches which are: Region-based,

Boundary-based and Feature-based approaches. The

Region-based methods rely on information provided

by the entire region such as texture, size, color,

shape and motion based proprieties using motion

estimation techniques. These methods work well for

small numbers of moving objects. However, they

cannot solve the occlusion problems in dense traffic.

The Boundary-based methods rely on information

provided by the moving objects edges. A good

initialization step is required. We notice that

Boundary-based methods do not work well in

presence of occlusions because the model is strongly

dependent on local-based information that can be

inaccurate. The Feature-based methods perform with

tracking the moving object’s sub-features such as

distinguishable points, lines or corners which are

extracted from the blobs between frames

. Tracking

methods based on features are useful in situations of

ROADGUARD - Highway Control and Management System

633

partial occlusions where only a portion of a moving

object is visible.

3 PROPOSED APPROACH

RoadGuard is based on a set of steps: (1) Moving

region detection to generate foreground mask, (2)

Cast shadow detection to obtain cast shadow pixels

mask, (3) Real moving object detection, (4)

Tracking and counting real moving objects.

Each RoadGuard’s step is detailed in next

sections.

3.1 Moving Region Detection

Accurate segmented moving regions are obtained by

generating moving pixels mask and grouping the

connected foreground pixels.

To detect moving pixels, we have adopted BM

approach. BM methods can be recursive or non-

recursive. The non-recursive methods use a sliding

window to estimate the background. They build a

buffer of frames and estimate the background model

with temporal variation of each pixels value in the

buffer. Besides, the recursive methods update

periodically a simple or multiple background models

for each input frame. Since recursive background

models require less storage, we use the recursive

version of Approximate Median (AM) method

(McFarlane, 1995) to detect moving pixels. This

method is able to deal with illumination and scene

changes.

The AM method presents an accurate technique

for background model updating. It updates

periodically a background model for each input

frame.

This process starts by initializing a reference

with the first input frame. Then, if a pixel in the

current frame has a value larger than the

corresponding background pixel, the background

pixel is incremented by 1. Likewise, if the current

pixel is less than the background pixel, this latter is

decremented by 1. In this way, the background

eventually converges to an estimate value where half

of the input pixels are greater than the background

and the other half is less than the background,

approximately the median.

Moreover, each pixel is classified as belonging to

the foreground (assigned 1) or background (assigned

0) after the updating process. A binary moving

pixels mask (M) is generated by a comparative stage

between the input frame (I) and the background

model (B). This comparative stage is based on a pre-

determined threshold (Th) (for highway sequences

equals to 25).

The obtained foreground pixel masks contain lot

of misclassifications which correspond to cast

shadow pixels detected erroneously as foreground

pixels. This misclassification causes occlusion

between the moving objects

Thus, we present our contribution in detecting

cast shadow in the next section.

3.2 Cast Shadow Detection

In our approach, we identify cast shadow pixels

without using the foreground masks. The basic idea

is to detect all moving shadows in a frame sequence

and then substitute from it the self shadow.

Furthermore, we start by classifying each moving

pixel into shadow/non-shadow. Then, we suppress

the self shadow part from the moving shadow mask.

3.2.1 Automatic Classification

of Moving Pixels

We consider shadow as moving pixels that have

lower intensity value. For that, we adopt three

successive differences to detect it. In fact, we

consider four successive grayscale frames (f), (f-1),

(f-2) and (f-3). Firstly, we perform by differences

between ((f),(f-1)), ((f),(f-2)) and ((f), (f-3)) to

obtain grayscale images I

, I

and I

respectively.

Secondly, we compute for I

, I

and I

the respective

binary masks MI

, MI

and MI

Furthermore, we propose an iterative algorithm

to each difference (I

). In our algorithm, we define

two classes: (S) for shadow and (NS) for non-

shadow and two variables last and current which

represent respectively the value of the threshold in

last and current rounds.

We proceed by classifying pixels according to

their intensity value. Then, the current value is

updated by taking the median value between the

medians of the two classes S and NS.

We iterate these instructions until the current

value is equal to last value. We notice that the loop

iteration converges rapidly after three or four

iterations. Besides, the moving shadow mask (MI

)

is obtained by applying a logical AND between the

three masks: MI

, MI

and MI

. Indeed, pixels of

class S are assigned 1 in MI, 0 otherwise.

3.2.2 Self Shadow Detection

Considering the similarity between visual features of

self and cast shadow pixels, this phase constitutes a

considerable work. We propose to build an

VISAPP 2011 - International Conference on Computer Vision Theory and Applications

634

automatic self shadow detection solution based on a

machine learning approach using a set of manually

classified shadow pixels from highway frames, in

order to generate a predict model, which makes it

possible to classify shadow pixels into two classes:

self shadow and non-self shadow. We are based on

KDD process (Fayyad, 1996) to extract useful

knowledge from volumes data. The general principle

of the classification approach is the following:

{

}

()

C :S self shadow, non self shadow

P S C P

→ς= −

∈→ ∈ς

Let S be the population of shadow pixels to be

classified. To each pixel P of S one can associate a

particular attribute namely; its class label C. C takes

its values in the class of labels (0 for self shadow, 1

non-self shadow).

To do it, firstly, we split our corpus into training

data set and test data set and we identify the

effectiveness shadow pixels features in order to

build a two dimensional table from our training

corpus. Each table row represents a shadow pixel

and each column represents a feature. In the last

column, we save the shadow pixel class (0 or 1).

Brightness, color and edge distortions are the most

exploitable features for describing shadow pixels in

literature. Secondly, we employ supervised learning

to produce a significant predict model. In literature,

there are several techniques of supervised learning,

each having its advantages and drawbacks. So,

among the most criterion to compare supervised

learning techniques is the comprehensibility of the

learned model wish leads us to a well-accepted

techniques, that is, the induction of decision trees

(Hammami, 2006). The SIPINA technique (Zighed,

1996) was used in our work to predict model.

Considering this model, self shadow pixels

(assigned 1 in MI) are restored in the foreground

pixels mask (M).

3.3 Real Moving Object Detection

After the post-processing step on the masks of

foreground (M) and cast shadow (MI), a simple

difference between them leads to get the real moving

objects mask (RMO) without their shadows.

3.4 Tracking and Counting

The control phase of our system is based on tracking

vehicles in a defined region of interest (ROI) and the

emergency area.

We adopt a mean shift tracker based on features

selection method (Nedovic, 2008). Several images

features are computed for the Real Moving Objects

(RMO). These features are classified into two main

classes: color based features and texture based

features. The color based features support the RGB,

HSV and normalized RGB color spaces. The RGB

color space reflects any changes in lighting intensity

and color in the ROM. Normalized RGB (rgb) is

considered as invariant feature to the lighting

intensity changes. The HSV color space is based on

human color perception. The texture features include

co-occurrence matrixes, Gabor filters and Wavelet

packets. Co-occurrence matrixes can distinguish

between region pixels with the same color

distribution and different texture. Both Gabor filters

and Wavelet packets identify a texture changes since

the energy signatures of different textures will be

different.

In the mean-shift tracker, the real moving objects

are characterized by the probability density

functions (pdfs) of their color or texture features.

Unlike the most tracking methods wish use fixed

features as an indicator of the RMO location, the

adopted tracker use an adaptive feature selection. In

fact, the selection issue is seen as a RMO class

discrimination problem. Distribution of the two

classes is estimated by computing their features

histograms. Discriminative power of the features

evaluated independently is based on the variance

ratio measure of likelihood distributions.

We obtain a positive value for RMO features and

a negative one to those corresponding to the

background. The distribution values in the likelihood

images are used as an input to the Augmented

Variance Ratio (AVR). We sort features according

to their AVR score and top N features are used for

tracking the object.

The management phase of the RoadGuard is

based on counting vehicles in highways in order to

obtain statistics information. In fact, we have

implemented an algorithm which counts moving

vehicles based on the enhanced provided masks and

save some important information like the date and

time of overloaded highways. Counting process is

done in the ROI.

The counter is incremented for

each vehicle enters the ROI. Also, we have defined a

number of vehicles (here more than 2) to generate an

alert and save an overloaded event.

4 EXPERIMENTAL RESULTS

In order to validate our system, RoadGuard was

ROADGUARD - Highway Control and Management System

635

evaluated by intensive experiments. We carried out

our study on a set of highway sequences very

referred in researches which are: Highway

{i=I,…,V}

. These sequences are recorded in typical

conditions that include shadows. This section

presents some results of these experiments.

Since pixels classifications into foreground or

shadow pixels has important effects on performance

of the upper steps of RoadGuard, the evaluation can

be assessed by segmenting manually pixels of

significant frames

to compute three global

indicators: (1) Global Error Rate (GER), (2) A Priori

Error Rate (PER) and (3) A Posteriori Error Rate

(PSER). GER is the complement of classification

accuracy rate, while PER (respectively, PSER) is the

complement of the classical recall rate (respectively,

precision rate). We identify two classes A and B

which represent respectively “moving pixel” and

“non-moving pixel”.

To illustrate the relevance of our system, we

present the obtained results for the set of significant

frames

. Figure 1 and Figure 2 show respectively

GER and PER(A)-PSER(A) before and after

suppressing cast shadow.

The aim of cast shadow detection and removal is

to decrease GER and PSER (A). The average value

of the obtained GER in our experiment is decreased

from 12% (88% correct classification) to 5,78%

(94,22% correct classification) after suppressing cast

shadow. Figure 2(a) shows that PSER (A) is almost

the time greater than the PER (A), this means that

the algorithm classify pixels erroneously as moving

ones. These pixels correspond to the cast shadow

pixels. The low PER (A) (average value equal to

17.45%) shows the robustness of the whole system

against the environmental changes. After removing

the cast shadow, the average value of PSER(A) is

decreased from 49.78% to 23.74% which means that

our RoadGuard performs highly in detecting cast

shadow. Thus, a better segmentation of moving

vehicles is provided. These results confirm the

important affect of shadow problem.

The high-quality results given above allow us to

compare our contribution with other existing

techniques. The obtained results are compared to the

best results of each shadow detection approach

represented by well-know methods (Table1) given in

a comparative reviewing (Prat, 2003).

Let ε represents the complement of the global

error rate (GER) and η represents complement of the

a priori error rate (PER) of the shadow pixels class.

The Highway video sequences are courtesy of the Computer

Vision and Robotics Research Laboratory at UCSD.

The results show the robustness of our proposed

algorithm against shadow challenge.

The tracker of the RoadGuard is able to draw the

trajectory of a moving object (Figure 3) in spite of

its speed (fast/slow), shape (big/small) or location

(near/far from the camera).

Table 1: Comparative Results.

ε % η %

SNP 81.59 63.76

SP 59.59 84.70

DNM1 69.72 76.93

DNM2 75.49 62.38

Our results 93.51 82.55

Moreover, RoadGuard is able to count vehicles

(Figure 3) and save some important information like

the date and time of overloaded highways.

5 CONCLUSIONS

In this paper, a novel and accurate approach

RoadGuard for Highway Control and Management

System is presented. The system is enough capable

to detect, count and track vehicles in highway while

no scene-specific knowledge is required.

RoadGuard was evaluated with various video files

against different Highway conditions. According to

the presented results, we can conclude that we can

consider RoadGuard as a robust computer vision

application. In fact, our approach combines rapidity

of adaptation in scene changes, a high precision in

moving object detection, performance to detect and

suppress cast shadow and accuracy to count and

track moving vehicles.

REFERENCES

Cucchiara, R., Grana, C., Neri, G., Piccardi, M., Prati, A.,

2001. The Sakbot System for Moving Object

Detection and Tracking. In Video-Based Surveillance

Systems Computer Vision and Distributed Processing.

Elhabian, K., El-Sayed, M., Sumaya, H.A., 2008. Moving

Object Detection in Spatial Domain using Background

Removal Techniques - State-of-Art. In Recent Patents

on Computer Science.

Fayyad, U. M., Piatetsky-Shapiro, G., Smyth, P., 1996.

The kdd process for extracting useful knowledge from

volumes of data. Communications of the ACM

(CACM).

Grimson, Y., Stauffer, C., Romano, R., Lee, L., 1998.

Using adaptive tracking to classify and monitor

activities in a site. In Proc. Conf. Comp. Vision Pat-

VISAPP 2011 - International Conference on Computer Vision Theory and Applications

636

tern Rec.

Hammami, M., Chahir, Y., Chen, L., 2006. WebGuard: A

Web Filtering Engine Combining Textual, Structural,

and Visual Content-Based Analysis. In IEEE

Transaction on Knowledge and Data Engineering.

Julio, C. S. J., Jung, C. R., Musse, S. R., 2005.

Background Subtraction and Shadow Detection in

Grayscale Video Sequences. In Computer Graphics

and Image Processing.

Kanhere, N., Pundlik, S., Birchfield, S., Sarasua, W.,

2006. Vehicle Segmentation and Tracking in the

Presence of Occlusions. In Transportation Research

Board Annual Meeting.

Kass, M., Witkin, A., Terzopoulos, D., 1988. Snakes:

Active contour models. In International J. Computer

Vision.

Kilger, M., 1992. A Shadow Handler in a Video-Based

Real-Time Traffic Monitoring System. In IEEE

Workshop Applications of Computer Vision.

Masoud, O., Papanikolopoulos, N. P., 2001. A novel

method for tracking and counting pedestrians in real-

time using a single camera. In IEEE Trans. Vehicular

Technology.

Mikic, I., Cosman, P., Kogut, G., Trivedi, M.M., 2000.

Moving Shadow and Object Detection in Traffic

Scenes. In Proceeding Int’l Conference Pattern

Recognition.

Nedovic, V., Liem, M., Corzilius, M., Smids, M., 2008.

Kernel-based object tracking using adaptive feature

selection. In technical report, Informatics Institute,

University of Amsterdam.

Prati, A., Mikic, I., Trivedi, M. M., Cucchiara, R., 2003.

Detecting Moving Shadows: Algorithms and

Evaluation. In IEEE Trans. Pattern Analysis and

Machine Intelligence.

Tang, Z., Miao, Z., 2007. Fast Background Subtraction

and Shadow Elimination Using Improved Gaussian

Mixture Model. In IEEE International Workshop on

Haptic, Audio and Visual Environments and Games.

Xiao, M., Han, C. Z., Zhang, L., 2007. Moving Shadow

Detection and Removal for Traffic Sequences. In

International Journal of Automation and Computing.

Zighed, D. A., Rakotomalala, R., 1996. A method for non

arborescent induction graphs. In technical report,

Loboratory ERIC, University of Lyon 2.

Figure 1: GER (a) before and (b) after suppressing the cast shadow.

Figure 2: PER and PSER (a) before and (b) after suppressing the cast shadow.

Figure 3: Results of counting and tracking vehicles in HighwayI.

ROADGUARD - Highway Control and Management System

637