VISION-BASED TRAFFIC SIGN DETECTION FOR ASSISTED

DRIVING OF ROAD VEHICLES

Miguel Ángel García, Miguel Ángel Sotelo, Ernesto Martín Gorostiza

Department of Electronics, University of Alcalá, Alcalá de Henares, 28871

Keywords: Assisted Driving, Intelligent Vehicles, Traffic Sign Detection.

Abstract: A system for real-time traffic sign detection is described in this paper. The vision-based traffic sign

detection module developed in this work is intended for assisted driving of road vehicles by handling color

images in RGB (Red, Green and Blue) format. In a first step a preattentive area of interest is determined

based on the vertical projection of edge pixels. In a second step, a shape analysis is performed. In a third

step, a color analysis is performed, and finally, a template is fitted. Some results obtained on a series of real

road images are presented in order to illustrate the robustness of the detection system.

1 INTRODUCTION

Traffic sign detection and recognition have

experimented increasing importance in the last

times. This is due to the wide range of applications

where this kind of systems can be used, specially as

driver active aid systems.

There are four types of traffic signs in the traffic

code: prohibition, warning, obligation and

informative. Depending on the shape and color, the

warning signs are equilateral triangles with one

vertex at the top. Prohibition signs are circular,

having a specific figure in each case over a white or

blue background, and a red border. To indicate

obligation, signs are circular with a white figure over

a blue background. The most important traffic signs

are prohibition signs; therefore they have priority to

be detected in this work.

One of the greatest inconveniences of using the

RGB color space is that it is very sensitive to

changes in light (A. de la Escalera,2003). This is the

reason why other color spaces are used in computer

vision applications, specially the hue, saturation,

intensity (HSI) one. This system keeps high

immunity to changes in light (R. C. Gonzalez,1993).

The problem with HSI is that transformation

equations (between RGB and HSI) are nonlinear,

making the computational cost prohibitive. Instead,

we propose to use the relation between the RGB

components for traffic sign detection, as this work is

intended for real-time systems and no further

processing is needed after digitalization.

To detect a traffic sign in an image, the algorithm

follows these steps:

• Candidate image regions are obtained by

accumulative vertical and horizontal edge

projections.

• Centre and radius of circular prohibition

and obligation signs are obtained by a

centre determination technique using three

points of the contour.

• Candidate image regions are validated

based on:

a. Red image thresholding, for

prohibition sign.

b. Blue image thresholding, for

obligation sign.

• Blob shape analysis from red or blue image

thresholding.

• Circular ring templates- based correlation method is

used to identify potential traffic signs in images (

M. Gavrila,1999)

2 CANDIDATE IMAGE REGIONS

Candidate regions of interest are computed for

preattentive purposes based on vertical and

horizontal projections of edge pixels.

2.1 Edge image

The appropriate choice of the color features to use in

the process is of crucial importance in order to

García M., Sotelo M. and Gorostiza E. (2004).

VISION-BASED TRAFFIC SIGN DETECTION FOR ASSISTED DRIVING OF ROAD VEHICLES.

In Proceedings of the First International Conference on Informatics in Control, Automation and Robotics, pages 19-24

DOI: 10.5220/0001133300190024

 SciTePress

achieve proper and fast detection. Accordingly, only

the Red component is considered as it provides a

high capacity for color discrimination in visual

analysis of traffic signs and no further processing is

needed after digitalization. In an attempt to carry out

a preattentive strategy, a coarse analysis of vertical

edges is performed in a first stage based on

differential characteristics computed on the Red

component of the image using the method of edge

detection described by john Canny (Canny, 1986).

This method has been extremely influential in many

applications. Numerous implementations of edge

detectors based on Canny’s idea have been

developed. Canny described a method of generating

edge detector using an optimization approach and

showed how to use the technique to generate edges-

robust detector. Canny’s method preserves contours

that are very important for detecting traffic sign

using shape information. Figure 1 shows the result

of applying the Canny edge detector road image.

2.2 Vertical projection of edge pixels

One of the most common techniques for traffic sign

segmentation is to use grey-level images, the red

component in our case, and to project accumulative

edge pixels onto the axes.

Vertical projections of different types of signs

are shown in Figure 1. As it can be observed, a

maximum in the projected signal occurs in the area

of the image where the sign is placed.

As a first step, an adaptive thresholding is

performed aiming at removing the common offset

component in the projection profile. For this

purpose, a threshold u is computed as expressed in

(1).

(1)

Where µ

stands for the average value of the

projection profile, while µ

represents the average

of all points in the projection whose value is greater

than µ. Finally, the coarse detection stage ends by

removing narrow peaks from the projection profile.

This yields a set of candidate image regions that

highly reduces and constraints the portions of the

image where traffic signs are likely to appear.

2.3 Horizontal projection of edge

pixels

In order to restrict the area of interest a bit more, a

similar method is applied to horizontal projections of

edge pixels in the region of interest. In this case the

adaptive threshold is obtained as expressed in

equation (2)

(2)

Where µ

denotes the standard average value of

the horizontal projection. After applying this

segmentation process, regions of interest remain

more restricted. Nevertheless, not all the regions

include a traffic sign inside. Resultant regions of

interest are shown in Figure 2.

source Image

50 100 150 200 250 300 350

100

150

Canny Edge Image

50 100 150 200 250 300 350

100

150

Source Image

50 100 150 200 250 300 350

100

150

Edge Image

50 100 150 200 250 300 350

100

150

Figure 1: Canny edge image. Accumulated vertical and horizontal projection and their corresponding threshold-levels

ICINCO 2004 - ROBOTICS AND AUTOMATION

3 SHAPE ANALYSIS

The shape of traffic sign represents essential

information for road-sign detection and further

classification. The main signs, prohibition and

obligation are circular and their contours are defined

by (3):

()()

yyxxr +++=

(3)

Where ‘r’ denotes the radius of circumference

and (x

) indicates the centre of circumference. The

centre can be determinate using a graphic method of

three points of the circular contour, as it is shown in

Figure 3, following these steps:

1. The three points are grouped in pairs;

), (x

, y

) and (x

, y

2. Centre points of the segments determined

by every pairs of point are obtained by (4).

(

) ()

() ()







+=+=

32123223

21122112

yyyxxx

(4)

3. Orthogonal lines, r

and r

, to segments

obtained in previous step, are traced

through respective central points already

obtained as well by (5).

()











−

−=

−

−=

2323

1212

xxyr

(5)

4. Centre of circumference (x

, y

) from

intersection between orthogonal lines r

and r

is obtained by solving equation (5).

These steps are repeated using three different

points of the contour every time and along the

contour that might include a large enough number of

pixels.

Regions

50 100 150 200 250 300 350

100

150

Horizontal Vertical Projections & Regions

50 100 150 200 250 300 350

100

150

Figure 2: Vertical and horizontal projection to obtaining regions of interest

m12

m23

Figure 3: method of calculus of centre

VISION-BASED TRAFFIC SIGN DETECTION FOR ASSISTED DRIVING OF ROAD VEHICLES

With this method a two-dimensional distribution

of possible points of centre is obtained, as it is

shown in Figure 4. A maximum in the distribution

occurs in the area of the image where the sign is

placed, see Figure 4 right.

4 COLOR ANALYSIS

Color is a very important parameter to be taken into

account for road-sign detection and further

classification. Some authors choose color-spaces

that exhibit high immunity to changes in light, HSI

(Hue, saturation Intensity) (T. Hibi, 1996) (G.

Piccioli, 1996), so that color regions can be

segmented by means of a look-up table (LUT).(A.

de la Escalera, 1997). Computation time needed to

obtain red-component image in HSI space is ten

times bigger than in RGB space.

In this work, the RGB color space has been

chosen, so that relations between components are

used to highlight red-colored and blue-colored

regions in the image and reduce the light-

dependency of the RGB space in the perception of

the color, as is described by (6).



















≤−

>−−







≤−

>−−

min

brbr

grgr

ffif

ffifff

ffif

ffifff

(6)

Figure 4: two-dimensional distribution of points, and maximums in the distribution.

source Image

50 100 150 200 250 300 350

100

150

Edge Image by Canny method, labeled

50 100 150 200 250 300 350

100

150

Regions

100 200 300

100

150

Red Image

100 200 300

100

150

(b)

(d)

Regions

100 200 300

100

150

Red Image

100 200 300

100

150

(b)

(d)

source Image

100 200 300

100

150

Blue Image

100 200 300

100

150

(a)

(c)

source Image

100 200 300

100

150

Blue Image

100 200 300

100

150

(a)

(c)

Figure 5: source images, regions of interest, blobs in the red image and blobs in the blue image

ICINCO 2004 - ROBOTICS AND AUTOMATION

Where f

, f

, and f

are, respectively, the functions

that give the red, green, and blue levels of each point

of the image, and r is the highlight red-colored

region. Similar method to obtain the highlight blue-

colored regions is used.

The most important reason why RGB is used is

that it allows to speed up the detection process while

preserving the detection robustness at the same time.

4.1 Blob-based analysis

The red and blue components of the regions of

interest already obtained are calculated and

thresholded, yielding the blobs that can be seen in

Figure 5. These blobs, corresponding to road signs,

must fulfill the size and aspect constraints described

by (8) and (9):

min

≥ (8)

maxmin

),max(

≥

−

≥ (9)

Where A

is the minimum area that a blob must

have and h

and l

are the blob’s width and height

respectively. Resultant blobs are useful to place a

road sign-searching template on the image.

5 TEMPLATE MATCHING

A ring-shaped template T is placed in the edge

image I, in the centre of the blob obtained, for

prohibition and obligation signs, which are circular.

The matching-measurements matrix D(T; I) (

D. M.

Gavrila,1999)

is determined by the pixel values of I

which lie under the high level pixels of the T. These

pixel values form a distribution of distances from the

template points to the nearest contour in the image.

The lower these distances are, the better the match

between image and template at this location.

The sequence to fit the template is as follows:

• The template is moved over the edge-image

I, along the eight directions indicated in

Figure 6 (Ritter and Wilson, 2001), while

the sum of all the points remaining inside the

ring at the shifted position is bigger than the

sum of all the points inside the ring in the

previous position.

• Radius R and r of the ring are set following

the same criteria as in the ring-centre

adjustment.

• The first two steps are repeated at each shift

step.

• For a candidate object (region) to be

considered as a road sign the sum of the

points in the ring must be greater than a

certain threshold value (10).

threshold

Uyxt

∑

≥),( (10)

Figure 7 shows the results obtained by using the

described method. The image on the right, two signs

have been detected, an obligatory-direction blue one

and a Stop-one which is not circular but can be fitted

using the circular based assumption, the detecting

method provides color information, blue for

obligation signs and red for prohibition and stop

ones. A smaller obligatory-direction road sign has

not been detected, because it is still too far from the

ego-vehicle. In the image on the left, a speed-limit

sign has been detected, and an informative sign as

well, but the template has only been fitted to the first

one because the informative sign has been rejected.

)

Figure 6: Circular ring template, an

searching directions.

VISION-BASED TRAFFIC SIGN DETECTION FOR ASSISTED DRIVING OF ROAD VEHICLES

6 CONCLUSIONS AND FUTURE

WORK

An algorithm for real-time detection of traffic signs

is carried out based on vertical projections of edge

pixels, shape analysis and template-based detection.

The algorithm has been empirically tested on real

road images, and aims at assisting human drivers in

automatic recognition of traffic signs in order to

ensure traffic rules fulfilment. Our future work

involves robust detection of road signs under

adverse weather conditions, as well as the use of a

neural network for fine grane classification and

validation of the detected road signs.

ACKNOWLEDGEMENTS

This work has been supported by the Comisión

Interministerial de Ciencia Y Tecnología (CICYT-

Spain) by means of Research Project DPI2002-

04064-C05-04.

REFERENCES

D. M. Gavrila. Traffic Sign Recognition Revisited.

Proceedings of the 21st DAGM Symposium für

Mustererkennung. Pp. 86-93. Springer Verlag 1999.

A. de la Escalera, J. M. Armingol, M. Mata. Traffic Sign

Recognition and Analysis for Intelligent Vehicles.

Image and Vision Computing, Vol. 11, N. 3, pp. 247-

258 (2003).

R. C. Gonzalez and R. E. Woods, Digital Image

Processing, 2nd ed. Reading, MA: Addison-Wesley,

1993.

J. Canny. “A Computional approach to Edge-Detection,”

IEEE Transactions on pattern Analysis and Machine

Intelligence, vol 8, pp. 679-700, 1986.

T. Hibi, Vision based extraction and recognition of road

sign region from natural color image, by using HSL

and coordinates transformation, 29th International

Symposium on Automotive Technology and

Automation, Robotics, Motion and Machine Vision in

the Automotive Industries, ISATA June (1996).

G. Piccioli, E. de Micheli, P. Parodia, M. Campani, Robust

method for road sign detection and recognition, Image

and Vision Computing 14 (3) (1996) 209–223.

A. de la Escalera, L. Moreno, M. A. Salichs, J. M.

Armingol. Road Traffic Sign Detection and

Classification. IEEE Transactions on Industrial

Electronics, Vol. 44, N. 6, pp. 848-859 (1997).

Gerhard X. Ritter and Joseph N. Wilson. Handbook of

Computer Vision Algorithms in Image Algebra. 2001

by CRC Press LLC.

Source Image

50 100 150 200 250 300 350

100

150

Detection

50 100 150 200 250 300 350

100

150

Source Image

50 100 150 200 250 300 350

100

150

Detection

50 100 150 200 250 300 350

100

150

Figure 7: examples of traffic sign detection.

ICINCO 2004 - ROBOTICS AND AUTOMATION