ity of signs. Extension of these automated traffic-
sign inventory systems with a subsign recognition
module would increase the inventory generation effi-
ciency, and would decrease the required manual inter-
action. However, subsign detection and recognition is
rather difficult, as the subsign contents vary greatly
and sometimes contain arbitrary texts and/or custom
symbols. Furthermore, these signs are smaller than
normal road signs and consist of less discriminative
colors, as they are usually white. Also, the captur-
ing quality and conditions are varying, as the captur-
ings are made outdoors during different weather con-
ditions and from driving vehicles, typically with large
inter-capturing distances at higher driving speeds.
In literature, few publications report on the recog-
nition of subsigns. In (Hamdoun et al., 2008), rectan-
gle detection is employed in a region below road signs
to retrieve present supplementary signs, followed by
a classification stage to solely retrieve exit-lane sub-
signs. Rectangle detection is also exploited in (Nien-
huser et al., 2010), where found rectangles are classi-
fied using a two-stage cascade, aiming at discrimina-
tion between both subsign and non-subsign rectangles
and between 4 different subsign types. In (Puthon
et al., 2012), a region growing approach is described
and compared against several other techniques, where
the proposed method achieves a correct detection rate
over 70%.
This paper describes a generic and learning-based
approach for both detection and classification of sub-
signs. The work forms an extension to our exist-
ing traffic-sign inventory system (Hazelhoff et al.,
2012), but due to the generic nature of our algorithm,
it is also applicable to other, similar systems such
as (Maldonado-Bascon et al., 2007), (Maldonado-
Bascon et al., 2008), (Overett and Petersson, 2011),
(Timofte et al., 2009) and (Timofte et al., 2011)). In-
stead of treating the complimentary signs as an addi-
tional sign class, and thereby searching the complete
image for complimentary signs, the output of existing
road-sign inventory systems, like any of the above-
mentioned systems, is exploited. This narrows the
search area to the regions below the identified traf-
fic signs, which increases robustness, since objects
with a similar appearance compared to subsigns (i.e.
white rectangles) occur frequently in real-world situ-
ations. The subsign recognition system exploits both
the single-image detections and tracked detections (in
this paper referred to as 3D signs) given by existing
inventory systems.
The system starts by detection of subsigns in a
fixed region below each of the detected signs, as
the vast majority of subsigns are located below road
signs. Afterwards, the subsign detection results ob-
(a) (b) (c) (d) (e)
Figure 2: Example of a 3D sign, consisting of multiple de-
tections of the same traffic sign tracked over multiple con-
secutive capturings.
tained for each detection of a 3D sign, are combined
to improve robustness. When a subsign is found, the
corresponding pixel regions are extracted, which are
then subject to classification, to retrieve either the
subsign type or a subsign-with-text code. This system
is evaluated on a large, real-world dataset containing
3, 104 signs (397 signs with subsign), with 29 differ-
ent subsign types for classification. It will be shown
that subsign detection is indeed possible with reason-
able performance, even with a generic concept.
The remainder of this paper is organized as fol-
lows. Section 2 contains the system overview of our
subsign detection and classification system, which is
described in detail in Sect. 3. The performed exper-
iments and results are found in Sect. 4, followed by
the conclusions in Sect. 5.
2 SYSTEM OVERVIEW
The system for automatic recognition of subsigns de-
scribed in this paper operates on 3D signs detected
by our traffic-sign inventory system (Hazelhoff et al.,
2012). These 3D signs consist of multiple detections
of the same road sign, tracked over consecutive image
frames. An example of an input 3D sign is shown in
Fig. 2. The system overview of the subsign recogni-
tion system is depicted in Fig. 3, and the four primary
modules are briefly described below.
1. Single-image detection: The region below each
detection given by the inventory system is divided
in overlapping windows. Each window is de-
scribed based on densely extracted SIFT descrip-
tors, which are subject to classification with a lin-
ear Support Vector Machine (SVM). The maxi-
mum SVM output of one out of all windows is
returned for each analyzed detection.
2. Multiview Detection: The single-image detection
results are combined to determine the presence of
a subsign for each 3D sign.
3. Subsign Localization: When a subsign is found
for the 3D sign, the pixel region corresponding to
the subsign, is retrieved for each detection with a
positive SVM output during the single-image de-
tection stage.
SubsignDetectionandClassificationSystemforAutomatedTraffic-signInventorySystems
263