TRAFFIC LIGHT RECOGNITION USING CIRCULAR

SEPARABILITY FILTER

Shodai Horima and Kazunori Onoguchi

Graduate School of Science and Technology, Hirosaki University, 3 Bunkyo-cho, Hirosaki, Aomori, 036-8561, Japan

Keywords: Traffic light detection, Traffic light recognition, Color identification, Circular Separability Filter, ITS.

Abstract: This paper proposes the camera-based approach to recognize the traffic light for driver assistance. The

circular separability filter applied to RGB images extracts the area of the traffic light. The separability has

large value in the boundary where the intensity between two areas changes like the step and it doesn't

depend on the intensity difference (height of the step). Scanning the circular mask in each RGB image, the

separability is calculated. The separability becomes large in an area where a color is homogeneous and a

shape is similar to the circle. Therefore, the pixel with large separability is selected as the candidate of the

traffic light. Unlike the conventional method which calculates the circularity from the binarized region, the

proposed method can identify the traffic light whose outline is indistinct and whose radius is small. At first,

the proposed method removes the region where the saturation is low and the brightness is extremely low or

high because there is few possibility that the traffic light is included in these regions. Next, the circular

mask is scanned in each RGB image captured from the on-vehicle color camera and the separability

between the inside circle and the outside ring is calculated. The maximum value of separability calculated in

RGB images is selected as the separability of each pixel. Pixels with large separability are detected as the

candidate region of the traffic light. Finally, the candidate region around which inactive traffic lamps exist is

identified as the traffic light. Experiments recognizing various traffic lights under various weathers and time

show the effectiveness of the proposed method.

1 INTRODUCTION

More than 700,000 traffic accidents a year still occur

in Japan though the number of traffic accidents tend

to decrease recently. Because older drivers will also

increase, it is expected that the risk of the traffic

accident will rise in the future. To deal with this

situation, many driving support technologies have

been developed as part of Intelligent Transport

Systems (ITS). It is important to decrease the traffic

accident in the intersection because more than half

of traffic accidents occur in intersections. In the

intersection, overlooking or misidentifying a traffic

light caused the serious accident. Therefore, the

driving support system which rouses the attention or

avoids danger by showing an aspect of the traffic

light to the driver is very useful.

It requires large cost and large time to construct

the road-to-vehicle communication system

transmitting an aspect of the traffic light to vehicles

by the telecommunication facility. Therefore, a lot of

methods to recognize the color of the active traffic

light in images captured from the on-vehicle camera

have been proposed. Because an active traffic light

is usually a red, yellow or green bright region, most

of conventional methods first convert the RGB color

space to some color spaces so as to detect candidate

regions with specific colors of traffic lights. Then,

candidate regions are detected in the converted

image by the binarization and the morphological

operation. Finally, traffic lights are identified by

verifying information around candidate regions, e.g.,

their contours. M. R. Yelal et al. (M. R. Yelal, 2006)

proposed the method using the La*b* color space.

This method detects only traffic lights with simple

background, e.g., clear sky because traffic lights are

identified by verifying edge information around

candidate regions. L. Tsinas et al. (L. Tsinas, 1996)

proposed the method using the HSI color space. This

method causes a lot of false detection because

candidate regions of traffic lights are verified by

only the size of the region. Several methods

identifying the traffic light from the circularity of the

candidate region were proposed because the outline

277

Horima S. and Onoguchi K. (2012).

TRAFFIC LIGHT RECOGNITION USING CIRCULAR SEPARABILITY FILTER.

In Proceedings of the 1st International Conference on Pattern Recognition Applications and Methods, pages 277-283

DOI: 10.5220/0003741402770283

 SciTePress

Figure 1: Circular separability fileter. (a) Conventional

filter. (b) Proposed filter.

of the traffic light is a circle. K. Lu et al. (K. Lu,

2005) evaluated the circularity of the traffic light by

the difference between the maximum distance and

the minimum distance from the center of the

candidate region to the contour point. M. Omach et

al. (M. Omachi, 2009) evaluated the circularity of

the candidate region by the Hough Transform. J.

Park et al. (J. Park, 2009) distinguished whether or

not the candidate region is the traffic light by the

Haralick's circularity and intensities in right and left

neighbor regions. D. Nienhuser et al. (D. Nienhuser,

2010) use a morphological operator to extract

circular regions. These methods estimate the

circularity of the candidate region obtained by

binarizing the color conversion image. Therefore,

they are very sensitive to the threshold for the

binarization. Moreover, it is difficult to identify the

traffic light whose shape is distorted in an image

because of the brightness saturation or whose view

is small because of the distance. Though some

methods (A. Nakano, 2010) recognize the traffic

light by learning a variety of traffic light images, it is

difficult to detect the traffic light in the complicated

background under various brightness.

This paper proposes the method to recognize the

traffic light by applying the circular separability

filter (K. Fukui, 1988), (K. Fukui, 1995) to RGB

images. The circular separability filter outputs large

value in a round area where color is homogeneous.

The separability is calculated by scanning the

circular mask in each RGB image and the pixel with

high separability is detected as the candidate of the

traffic light. Unlike the conventional method which

calculates the circularity from the binarized region,

the proposed method can identify the traffic light

with indistinct outline and small radius.

At first, the proposed method converts the RGB

color space to the HLS color space and the region

where the saturation is low and the brightness is

extremely low or high is removed from the image.

(a) (b)

Figure 2: Preprocessing. (a) Input image. (b)

Preprocessing image.

Next, the circular mask shown in Fig.1(b) is

scanned in each RGB image while changing its

radius. The pixel with large separability is detected

as the candidate of the traffic light. The circular

separability filter has been used the pupil detection

(K. Fukui, 1988). This method used the double ring

mask shown in Fig.1(a) because the purpose is to

find the round contour regardless of its internal

pattern. This mask tends to detect the traffic sign or

the signboard as the traffic light wrongly. Finally, it

is examined whether the inactive traffic lights exist

around the candidate region. If inactive traffic lights

are detected in the right and left or top and bottom of

the candidate region, it is identified as a traffic light.

2 TRAFFIC LIGHT DETECTION

2.1 Preprocessing

The saturation of the active traffic light is usually

high. Moreover, the black region or the white region

can be excluded from the candidate of the traffic

light. Therefore, the region where the saturation is

low and the brightness is extremely low or high is

removed from the image.

The RGB color space is converted to the HLS

color space. In the HLS color space, the hue H

represented as the value between 0 and 360 degrees.

The saturation S

and the brightness B

are

represented as the value between 0 and 1. The pixel

satisfying the condition (1) is excluded from the

candidate of the traffic light.

: S

< THs or B

< THb or B

> 1 - THb (1)

Figure 3: Typical traffic lights in Japan.

ICPRAM 2012 - International Conference on Pattern Recognition Applications and Methods

278

THs and THb are thresholding parameters decided

experimentally. In the experiment, THs was set to

0.33 and THb was set to 0.12. Fig.2(b) shows the

image after preprocessing. White or black areas,

e.g., clouds in the sky are removed from the input

image shown in Fig.2(a).

2.2 Circular Separability Filter

As shown in Fig.3, the frame of the traffic light is

white or black typically in Japan. Our method

detects the circular oundary between the traffic lamp

and the surrounding white or black region by the

circular separability filter. The circular mask shown

in Fig. 1 (b) consists of the inside round area R

and

the outside ring area R

. The separability η is given

η =

()()

11 mm

PPnPPn −+−

()

∑

−

(2)

where n

is the number of pixels in R

, n

is the

number of pixels in R

, N is the total number of n

and n

, P

is the intensity of the pixel i, P

is the

average intensity in R

, P

is the average intensity in

and P

is the average intensity in the total region

of R

and R

. The separability η has a value within a

range from 0.0 to 1.0. It approaches 1.0 when the

intensity between two areas changes like the step

and it approaches 0.0 when the intensity between

two areas changes gradually.

The separabirity η between R

and R

becomes large

when the intensity of the pixel in R

is similar, the

intensity of the pixel in R

is also similar and the

intensity in R

is different from the intensity in R

Therefore, the separability η becomes large when the

circular boundary between R

and R

corresponds to

the contour of the traffic lamp. Traffic signs or

signboards often have some textures in the inside.

The circular separability filter (Fig. 1 (b)) proposed

in this paper can suppress to detect round contours

of traffic signs or signboards wrongly.

The separability is unstable in the hue image

because hue in a white or black region is not decided

correctly. Therefore, our method calculates the

separability in each RGB image.

Figure 4: Search area.

2.3 The Candidate Region of the

Traffic Light

As shown in Fig.4, the image above the vanishing

line is divided equally into n search areas S

(i=1~n)

because traffic lights usually exist above the

vanishing line in an image. To search for the

candidate of the traffic light, the circular mask is

scanned in each area S

while changing the radius.

Far traffic lights exist in a lower area S

than near

ones. The radius of the traffic lamp is smaller in a

lower area S

. Therefore, the variable range of the

radius in a lower area is set smaller than a upper

area. In experiments, we divided the image above

the vanishing line into twenty search areas S

(i=1~20) as shown in Fig.4. S

is the top area and S

is the bottom area. The radius in S

is changed

within the range from 15 pixels to 20 pixels. On the

other hand, the radius in S

is changed within the

range from 4 pixels to 9 pixels. In each area S

twenty positions or less with the large separability η

are selected as the candidate region C

of the traffic

light. The radius of the circular mask whose

separability is the maximum is chosen as the radius

of C

. In Fig.5(a), candidate regions whose

separability are high are shown in purple round

regions. Some mis-detection regions appear in

candidate regions.

TRAFFIC LIGHT RECOGNITION USING CIRCULAR SEPARABILITY FILTER

279

Figure 5: Verification. (a) Candidate regions whose

separability are high. (b) Candidate regions after color

verification. (c) Candidate regions after evaluating E

Figure 6: The position of the candidate region and inactive

areas. (a) Blue in horizontal type is active. (b) Yellow in

horizontal type is active. (c) Red in horizontal type is

active. (d) Blue in vertical type is active. (e) Yellow in

vertical type is active. (f) Red in vertical type is active.

2.4 The Verification of the Candidate

Region

Each candidate region Ci is verified by color and the

existence of inactive traffic lamps.

2.4.1 Verification of color

The RGB image is converted to the HSV color space

and the average hue value H

is calculated in the

candidate region C

. The candidate region Ci not

satisfying the condition (2) is deleted from candidate

regions because the color of the traffic light is near

red, yellow, or green. It is difficult to distinguish red

or yellow by using only hue value because yellow

hue is close to red hue. Then, our method

discriminates red or yellow by using the average

blue value Rm and the average green value Gm, in

addition to Hm.

Green : 140 < H

< 200

Yellow : 10 < H

< 60 and |B

- G

| > 120

Red : 0 < H

< 10

10 < H

< 60 and |B

- G

| < 120

(3)

2.4.2 Verification of Two Neighboring

Region

If the candidate region is the true traffic light, two

inactive traffic lamps are sure to exist around the

candidate region. The arrangement of them can be

expected because the color of the active traffic lamp

has been decided by color verification. As shown in

Fig.3, there are two types of traffic lights in Japan.

One is the vertical type and the other is the

horizontal type. When the color of the candidate

region C

is green, inactive two lamps E

and E

ICPRAM 2012 - International Conference on Pattern Recognition Applications and Methods

280

exist in the right side or the upper part of C

shown in Fig.6 (a) and (d). When the color of the

candidate region C

is yellow, one inactive lamp E

exists in the left side or the lower part and another

inactive lamp E

exists in the right side or the upper

part, as shown in Fig.6 (b) and (e). When the color

of the candidate region C

is red, two inactive lamps

and E

exist in the left side or the lower part, as

shown in Fig.6 (c) and (f).

The location and the radius of E

and E

are

estimated from those of C

. In E

and E

, the circular

separability and the average brightness in the HSV

color space are examined because an inactive traffic

lamp is a round region where with low intensity.

The evaluation value K

given by the equation (4) is

estimated to verify the existence of the inactive

traffic lamp.

= Bη

+ (1 -AVb

) (4)

In equation (4), Bη

is the circular separability in

(i= 1 or 2) and AVb

is the average brightness in

(i= 1 or 2).

If E

is an inactive traffic lamp, K

is large

because Bη

of the inactive traffic lamp is large and

AVb

of it is small. If either K

or K

is low, this

region is excluded from the candidate regions. In

remaining candidate region, the evaluation value E

given by the equation (5) is estimated. The candidate

region whose E

is large is identified as the traffic

light.

= η + kAV

+ K

(5)

In equation (5), η is the circular separability of the

candidate region, k is the constant parameter larger

than 1.0, AV

is the average of the saturation in the

candidate region. In experiments, k was adjusted to

2.0.

Figure 5(c) shows the candidate region whose E

is large. Only traffic light is detected exactly.

3 EXPERIMENTS

Experiments have been conducted to recognize

various traffic lights in images captured from the on-

vehicle camera. The focal length is 16 millimeters

and the image size is 1600 × 1200 pixels. Figures 7

and 8 show some recognition results. Traffic lights

in complicated backgrounds are recognized correctly

in Fig.7 (a), (b), (c) and (d). Traffic lights at the

backlight are recognized in Fig.8 (a). It is difficult to

recognize traffic lights in Fig.8 (b), (c) and (d)

because the brightness of active lamps is very low.

Table 1: Evaluation result.

10 pixels or

4 pixels or

Detected traffic lights 47 301

The number of true positive 46 275

True positive rate(%) 97.8 91.4

The number of false positive 0 16

False positive rate(%) 0 5.3

The number of false negative 1 10

False negativerate(%) 2.2 3.3

(a)

(b)

(c)

(d)

Figure 7: Some experimental results (1).

TRAFFIC LIGHT RECOGNITION USING CIRCULAR SEPARABILITY FILTER

281

However, the proposed method can recognize them

correctly because the circular sparability catches

slight contrast between a traffic lamp and a frame

around it. In fig.8 (e), a part of the traffic light is

lacked because of the reflection caused by direct

sunshine. However, the active red light is detected

correctly. In experiments conducted under a variety

of weathers and time, active traffic lamps whose

radius are four pixels or more in images have been

recognized. Table 1 shows the evaluation result of

the proposed method. For the traffic light whose

radius is ten pixels or more, the recognition rate is

97.8%. For four pixels in a radius, it is 91.4%.

(a)

(b)

(c)

(d)

(e)

(f)

Figure 8: Some experimental results (2).

4 CONCLUSIONS

This paper proposed the method for the traffic light

recognition using the on-vehicle camera. The

candidate of the traffic light is detected by the

circular separability filter applied in each RGB

image. Unlike the conventional method using the

circularity calculated from the binarized candidate

ICPRAM 2012 - International Conference on Pattern Recognition Applications and Methods

282

region, the proposed method can identify the traffic

light whose outline is indistinct and whose radius is

small because the separability doesn't depend on the

intensity difference. The candidate region around

which inactive traffic lamps exist is identified as the

traffic light. In experements,we comfirmed that

traffic lights whose radius is four pixels or more

were recognized by the accuracy of 91.4 %. In the

future, we will improve the performance and the

processing time by tracking detected regions.

Moreover, the proposed method will be evaluated by

comparison with several different approaches.

This work belongs to “Development of energy

saving ITS technologies” which Hirosaki Univ.

contracted with NEDO.

REFERENCES

M. R. Yelal, S. Sai, G. R. Shaffer and A. K. Kumar,

"Color-based Signal Light Tracing in Real-time Video",

Proceedings of AVSS'06, 2006.

L. Tsinas and V. Graefe, "Real-Time Recognition of

Signaling Lights in Road Traffic". Proceedings of

MVA'96, pp.71-74, 1996.

K. Lu, C. Wang and S. Chen, "Traffic Light Recognition",

Proceedings of CVGIP'05, pp.263-270, 2005.

M. Omachi and S. Omachi, "Traffic Light Detection with

Color and Edge Information", Proceedings of

ICCSIT'09, pp.284-287, 2009.

J. Park and C. Jeong, "Real-time Signal Light Detection",

International Journal of Signal Processing, Image

Processing and Pattern Recognition, Vol.2, No.2,

pp.1-10, 2009.

D. Nienhuser, M. Drescher and J. M. Zollner, ``Visual

State Estimation of Traffic Lights using Hidden

Markov Models,'' Proceedings of ITSC2010,

pp.1705-1710, 2010.

K. Fukui and O Yamaguchi, "Facial freature point

extraction method based on combination of shape

extraction and pattern matching", Systems and

Computers in Japan, Vol. 29, No. 6, 1998 (Translated

from Denshi Joho Tsushin Gakkai Ronbunshi, Vol.

J80-D-II, No. 8, August 1997, pp. 2170.2177).

K. Fukui, "Edge Extraction Method Based on Separability

of Image Features," IEICE Transactions of

Information and Systems vol.E78-D, no.12, pp1533-

1538, 1995. 21.

A. Nakano, H. Koyasu and H. Maekawa, "Detection and

Recognition of Traffic Signal Using Machine

Learning," The Special Interest Group Notes of IPSJ,

2010-CVIM-172(26), pp. 1-7, 2010(in Japanese).

TRAFFIC LIGHT RECOGNITION USING CIRCULAR SEPARABILITY FILTER

283