ROBUST ROAD SIGNS SEGMENTATION IN COLOR IMAGES

Bishesh Khanal, Sharib Ali and D

esir

e Sidib

Universit

e de Bourgogne, Laboratoire Le2i, UMR CNRS 5158, 12 rue de la Fonderie, 71200, Le Creusot, France

Keywords:

Road Sign Detection, Color Segmentation, Color Constancy, Log-chromaticity Color Space.

Abstract:

This paper presents an efﬁcient method for road signs segmentation in color images. Color segmentation

of road signs is a difﬁcult task due to variations in the image acquisition conditions. Therefore, a color con-

stancy algorithm is usually applied prior to segmentation, which increases the computation time. The proposed

method is based on a log-chromaticity color space which shows good invariance properties to changing illumi-

nation. Thus, the method is simple and fast since it does not require color constancy algorithms. Experiments

with a large dataset and comparison with other approaches, show the robustness and accuracy of the method

in detecting road signs in various conditions.

1 INTRODUCTION

The detection and identiﬁcation of road signs is an

important component of any Intelligent Transporta-

tion System (ITS). It is useful in highway mainte-

nance, sign inventory or driver support systems. A

full road sign recognition system is composed of two

main stages, detection and recognition (Bascon et al.,

2007). The goal of the ﬁrst stage is to identify poten-

tial road signs in the image, i.e. the possible regions

that represent road signs characteristics, using either

color or shape information. In the second stage, the

potential regions are further analyzed to determine the

correct sign and its meaning.

In this paper we focus on the detection stage of

road signs in still images or videos. Road sign detec-

tion in natural scene images is a challenging task due

to the complexity of the scene’s environment. The

challenges include changes of illumination, shadows,

occlusions, size variation, and the presence of object

similar in color.

Color and shape are the two main features used to

detect and recognize road signs. Color as a low-level

feature offers many advantages such as robustness to

occlusions, scale variation and geometric transforma-

tions. However, the color of an object depends on

the illumination conditions, the camera parameters

and the reﬂectance properties of the object. A good

road signs detector must then be robust against illu-

mination variations in uncontrolled environment and

a color constancy algorithm is usually used as a pre-

processing step. However, this increases the compu-

tation time of the detection method.

In this paper, a fast and robust road sign detection

method based on color segmentation is proposed. The

method is based on the invariance properties of the

log-chromaticity color space. The log-chromaticity

color space shows two important properties. Firstly,

a surface color seen under different illuminant col-

ors tends to lie on a straight line in this space. Sec-

ondly, for a given camera, all these lines are parallel

to each other for different surface colors. Thus an il-

lumination invariant representation of images can be

obtained in log-chromaticity color space (Finlayson

et al., 2004). We take advantage of this invariant prop-

erty to develop an efﬁcient road signs segmentation

algorithm which is robust against illumination varia-

tions and shadows. This robust and fast algorithm can

be used as main input for a shape recognition and road

signs interpretation module of an ITS.

The paper is organized as follows: Section 2 de-

scribes previous work on road signs segmentation

based on color information. Section 3 introduces the

proposed method for road signs detection, and exper-

imental results are shown in Section 4. Finally, some

conclusions are derived in the last section.

2 RELATED WORK

The two main features used to detect road signs are

the color (red, yellow, blue, green, and white) and

the shape (circle, triangle, rectangle, and octagon) of

road signs. Road signs are speciﬁcally designed to

have a distinctive color from the surrounding envi-

307

Khanal B., Ali S. and Sidibé D..

ROBUST ROAD SIGNS SEGMENTATION IN COLOR IMAGES.

DOI: 10.5220/0003802103070310

In Proceedings of the International Conference on Computer Vision Theory and Applications (VISAPP-2012), pages 307-310

ISBN: 978-989-8565-03-7

 2012 SCITEPRESS (Science and Technology Publications, Lda.)

ronment in order to help drivers to recognize them

easily and quickly. Therefore, several detection meth-

ods are based on color segmentation. Among differ-

ent segmentation approaches, image thresholding is

the simplest method and is used in a lot of works.

For example, (Benallal and Meunier, 2003) use a sim-

ple threshold formula applied to the red color chan-

nel to detect red road signs in RGB color space.

However, RGB space is very sensitive to illumina-

tion changes and trafﬁc scenes tend to have varying

illumination. Therefore, a color space transformation

from RGB space to another color space is usually

performed. Several color spaces are used including

HSV (Paclik et al., 2000), HSI (Escalera et al., 2003;

Fang et al., 2003), CIECAM97 (Gao et al., 2006) and

IHSI (Fleyeh, 2004). The main assumption in these

works is that the hue and saturation components are

less sensitive to changes in illumination compared to

the luminance component. Hence, the luminance is

discarded in the segmentation algorithm. Neverthe-

less, most of the color based methods can deal only

with slight variations in lighting conditions.

(Fleyeh, 2004) compares three segmentation

methods for road signs detection in the Improved

Hue, Luminance and Saturation (IHLS) color space.

The ﬁrst method uses an adapted threshold which is

computed for each image based on its mean lumi-

nance value. The second method uses region grow-

ing, and the third method, similar to the method de-

veloped by (Escalera et al., 2003), is based on lookup

tables where the thresholds are obtained from a set

of training images. The author concludes that the

segmentation method using adaptive threshold gives

the best results. Though this method, using dynamic

thresholding can adapt to the image content, it fails

to correctly segment road signs in different lighting

conditions. For example, it cannot detect a road sign

acquired by night, in a raining day or in very bright

sunlight.

To achieve robustness against illumination

changes, a color constancy algorithm can be used as

a preprocessing step. For example, (Fleyeh, 2005)

uses the local color shifts method of (Ebner, 2004) to

normalize the images before applying the dynamic

thresholding method. (Le et al., 2010) use the same

color constancy algorithm and (Zakir et al., 2011)

use the Gray-World algorithm to normalize the

images. In the experiments reported by the authors

of these work, applying a color constancy algorithm

leads to an improvement in the detection accuracy.

Nevertheless, using color constancy also increases

the overall computation time of the detection method

and, thus, limits its application for real-time systems.

In this paper, we present a robust road signs

segmentation method which achieves very good

detection results while avoiding the use of color

constancy. The method is based on the invariance

properties of the log-chromaticity color space.

3 ROBUST ROAD SIGNS

SEGMENTATION

3.1 Log-chromaticity Color Space

Properties

The log-chromaticity color space (LCCS) is a 2D

space obtained by taking the logarithm of ratios of

color channels. For example, log(R/G) and log(B/G)

form a 2D LCCS. The illumination invariance prop-

erty is based on the image formation model given by:

= σ

Ω

E(λ)S(λ)Q

(λ)dλ, c ∈ {R, G,B}; (1)

where, I

is the color intensity at a pixel for the color

channel c, σ is the Lambertian shading, E is the illu-

mination power spectral distribution, S is the surface

spectral reﬂectance function and, Q

is the camera

sensor sensitivity function. The integral is computed

over the visible spectrum Ω.

Under the assumptions of Planckian lighting,

Lambertian surface and a narrowband camera, equa-

tion 1 becomes:

= σIk

−5

−

T λ

S(λ

, (2)

where T is the illuminant temperature and k

and

are constants containing the Planck constant, the

Boltzman constant and the speed of light in vac-

uum (Finlayson et al., 2004).

If, we consider the chromatic components by tak-

ing the ratios, for example, of Red and Blue channels

w.r.t. Green channel, then, from equation 2 it is clear

that intensity and shading information are removed.

To remove the nonlinearity, we take the natural loga-

rithm of the ratios and obtain:

= log(I

) = log(s

)+(e

−e

)/T, c ∈ {R,B};

(3)

with s

= k

−5

S(λ

and e

= −k

/λ

This last equation shows that in the LCCS all the

color values of a surface seen under different illu-

minants fall on a straight line. The direction of this

line, given by the vector (e

− e

), is independent of

the surface reﬂectance function. As a consequence,

different surface characteristics will produce different

lines in the LCCS. However, all these lines are paral-

lel, since they share the same slope (independent from

VISAPP 2012 - International Conference on Computer Vision Theory and Applications

308

the surface). An invariant image can be formed by

projecting the chromatic components into a direction

orthogonal to these parallel lines.

3.2 Road Signs Detection in

Log-Chromaticity Color Space

As stated in Section 2, an important characteristic of

any road signs segmentation method is its ability to

achieve correct detection in the presence of varying

illumination conditions.

The invariance properties of the LCCS can be ex-

ploited to develop a robust road signs detector. The

motivation of our approach is that a characteristic

color of road signs, red for instance, under different

lighting conditions tend to form a distinct cluster in

the LCCS. Indeed, intensity and shading information

are removed when projecting the color of a pixel in

LCCS. Furthermore, when the illumination is varied,

the color values of a surface moves on a straight line

in LCCS. On this line, the red color for instance, will

occupy a small range. Hence, we can deﬁne a simple

segmentation rule based on the boundaries of the clus-

ter formed by red signs color in LCCS. The bound-

aries are obtained from a training set of pixels that

are manually selected from various road signs images

chosen in order to represent different lighting condi-

tions.

A pixel in an image is classiﬁed as a road sign

pixel if its projection in the LCCS satisﬁes:

a ≤ log(R/G) ≤ b and c ≤ log(B/G) ≤ d, (4)

where (a, b,c,d) are the thresholds. For red signs,

for instance, these thresholds are found to be equal

to (a,b,c, d) ≡ (0.5,2.1,−0.9, 0.8).

The experiments described in Section 4 show that

this simple rule leads to very good detection results in

various conditions, providing an efﬁcient solution for

road signs segmentation.

4 EXPERIMENTS

To evaluate the performance of the proposed road

signs segmentation method we apply it on a dataset

of 389 images containing 567 road signs of various

shapes and types. The dataset is a subset of the

one used by (Bascon et al., 2007) and shows im-

ages of road signs in different conditions such as

bright sunlight and electric lighting by night. We also

compare our method with the dynamic thresholding

method (Fleyeh, 2005) and its combination with two

popular color constancy algorithms: the Gray-World

method (Barnard et al., 2002) and the comprehensive

(a)

(b)

(c)

(d)

(e)

Figure 1: Example of segmentattion results. (a) Origi-

nal images. (b) Segmentation with the dynamic thresh-

olding method. (c) Segmentation with dynamic thresh-

old and Gray-World constancy. (d) Segmentation with dy-

namic threshold and comprehensive normalization color

constancy. (e) Segmentation with the proposed method.

normalization color constancy algorithm (Finlayson

et al., 1998).

Figure 1 shows some segmentation results ob-

tained with the different methods. As can be seen

in ﬁgure 1(b) and ﬁgure 1(c), the dynamic thresh-

olding method and its combination with the Gray-

World color constancy algorithm perform poorly.

Some clearly visible road signs are not detected. On

the contrary, the use of the comprehensive normal-

ization color constancy algorithm together with dy-

namic thresholding, ﬁgure 1(d), and the proposed

method based on log-chromaticity color space, ﬁg-

ure 1(e), give very good results. However, the pro-

posed method does not need any color constancy al-

gorithm and is, thus, much faster.

Detection results using the entire dataset are sum-

marised in Table 1. From the 567 road signs, 552

are correctly detected by the proposed method which

results in a correct detection rate of more than 97%.

The standard dynamic thresholding method achieves

very poor performance as it correctly detects less than

half of the road signs (correct detection rate of 46%).

Adding the Gray-World color constancy algorithm

slightly improves the results, from 46% to 52%, while

ROBUST ROAD SIGNS SEGMENTATION IN COLOR IMAGES

309

Table 1: Summary of detection results.

Detection rate Computation

(%) time per image (s)

DT 46.2 0.249

DT + GW 52.02 0.546

DT + CN 87.83 29.01

LCCS 97.35 0.093

DT = dynamic thresholding

DT+GW = dynamic threshold with Gray-World color constancy

DT+CN = dynamic threshold with comprehensive normalization

color constancy

LCCS = proposed method in log-chromaticity color space.

increasing the computation time by a factor two.

The comprehensive normalization color con-

stancy algorithm gives good results with a correct de-

tection rate of more than 87%. However, the pro-

cessing time of comprehensive normalization is very

high, which limits its use in real-time applications.

On the contrary, the proposed method based on log-

chromaticity color space achieves the best detection

results in the least computation time as shown in Ta-

ble 1. Note that the given computation time are obtain

with non optimized MATLAB codes using a 2.4 GHz

CPU and images of size 640 x 480.

The proposed method fails to correctly detect road

signs when the image is acquired in complete dark

condition, the road sign being only illuminated by the

headlights of the moving car. In such a case, all meth-

ods fail to detect the road signs.

5 CONCLUSIONS

In this paper, an efﬁcient and robust road signs seg-

mentation method based on color information is pro-

posed. The method is based on ﬁnding boundaries

of clusters formed by distinctive road signs color in

the log-chromaticity color space. Based on invari-

ance properties of this color space, we propose an ef-

ﬁcient segmentation method and experiments with a

large dataset show that the proposed method is par-

ticularly robust against severe illumination changes

for images taken under various conditions. Compar-

ison with other approaches based on color constancy

algorithms show that the proposed method achieves

the best segmentation results while requiring the least

computation time. The simplicity and the robustness

of the method make it suitable for real-time appli-

cations such as on-board drivers assistance systems.

Our future work include using the proposed method

as a ﬁrst step of a road signs recognition system.

REFERENCES

Barnard, K., Martin, L., Coath, A., and Funt, B. (2002). A

Comparison of Computational Color Constancy Algo-

rithms - Part II: Experiments with Image Data. IEEE

Transactions on Image Processing, 11(9):985–996.

Bascon, S. M., Arroyo, S. L., Jimenez, P. G., Moreno,

H. G., and Ferreras, F. L. (2007). Road Sign Detection

and Recognition Based on Support Vector Machines.

IEEE Transactions on Intelligent Transportation Sys-

tem, 8(2):264–278.

Benallal, M. and Meunier, J. (2003). Real-time Color Seg-

mentation of Road Signs. In Proceedings of Canadian

Conference on Electrical and Computer Engineering

(IEEE CCECE 03), pages 1823–1826.

Ebner, M. (2004). Color Constancy Using Local Color

Shifts. In Proceedings of the European Conference

on Computer Vision, pages 276–287.

Escalera, A., Armingol, J. M., and Mata, M. (2003). Trafﬁc

Sign Recognition and Analysis for Intelligent Vehi-

cles. Image and Vision Computing, 21:247–258.

Fang, C. Y., Chen, S. W., and Fuh, C. S. (2003). Road

Sign Detection and Tracking. IEEE Transactions on

Vehicular Technology, 52(5):1329–1341.

Finlayson, G., Drew, M., and Lu, C. (2004). Intrinsic Im-

ages by Entropy Minimization. In Proceedings of

the European Conference on Computer Vision, pages

582–595.

Finlayson, G. D., Schiele, B., and Crowley, J. L. (1998).

Comprehensive Colour Image Normalization . In Pro-

ceedings of the European Conference on Computer Vi-

sion, pages 475–490.

Fleyeh, H. (2004). Color Detection and Segmentation for

Road and Trafﬁc Signs. In Proceedings of the 2004

IEEE Conference on Cybernetics and Intelligent Sys-

tems, pages 808–813.

Fleyeh, H. (2005). Trafﬁc Signs Color Detection and Seg-

mentation in Poor Light Conditions. In MV A2005

IAPR Conference on Machine Vision Applications,

pages 306–309.

Gao, X., Podladchikova, L., Shaposhnikov, D., Hong, K.,

and Shevtsova, N. (2006). Recognition of trafﬁc signs

based on their colour and shape features extracted us-

ing human vision models. Journal of Visual Commu-

nication and Image Representation, 17(4):675–685.

Le, T. T., Tran, S. T., Mita, S., and Nguyen, T. D. (2010).

Real Time Trafﬁc Sign Detection Using Color and

Shape-Based Features. In Proceedings of Intelligent

Information and Database Systems, Second Interna-

tional Conference, pages 268–278.

Paclik, P., Novovicova, J., Pudil, P., and Somol, P. (2000).

Road Sign Classifﬁcation Using Laplace Kernel Clas-

siﬁer. Pattern Recognition Letters, 21:1165–1173.

Zakir, U., Zafar, I., and Edirisinghe, E. A. (2011). Road

Sign Detection and Recognition by Using Local En-

ergy based Shape Histogram (LESH). International

Journal of Image Processing, 4(6):567–583.

VISAPP 2012 - International Conference on Computer Vision Theory and Applications

310