PROBABILISTIC MODELING AND FUSION FOR IMAGE FEATURE
EXTRACTION WITH APPLICATIONS TO LICENSE PLATE
DETECTION
Rami Al-Hmouz, Subhash Challa and Duc Vo
Networked Sensors Technologies Lab, University Of Technology Sydney, Sydney, Austrlia
Keywords:
LPR, Plate Location, Extraction, Data Fusion.
Abstract:
The paper proposes a novel feature fusion concept for object extraction. The image feature extraction process
is modeled as a feature detection problem in noise. The geometric features are probabilistically modeled
and detected under various detection thresholds. These detection results are then fused within the Bayesian
framework to obtain the final features for further processing. Along with a probabilistic model, pixels voting
algorithm is also tested through binary threshold variation. The performance of these approaches is compared
with the traditional approaches of image feature extraction in the context of automatic license plate detection
problem.
1 INTRODUCTION
Geometric features in the a image such as area,
perimeter length etc, can determine the position of
any object in the camera field of view. Weather con-
dition and illumination have a significant impact on
selecting the appropriate threshold and consequently
on geometric features extraction. This paper proposes
novel probabilistic fusion methods to improve such
feature extraction.
We are specifically focused on methods of fea-
ture extraction that first convert the color or Gray
Scale image into a binary image before feature de-
tection/extraction. A binary image, is obtained from
thresholding a Gray scale image; a threshold must be
selected to label some intensity values as a white in-
tensity and some as a black intensity. The selection of
the threshold is critical to the quality of binary image
and in terms of feature extraction. A new probabilis-
tic fusion method for feature and subsequent object
extraction is introduced in this paper and its potential
in the context of license plate extraction application is
demonstrated.
Automatic license plate recognition (ALPR) is a
promising field of research, it has numerous appli-
cation areas such as unattended parking lots (Sirithi-
naphong and Chamnongthai, 1998), security control
of restricted areas (Yamaguchi et al., 1999), traffic
law enforcement(Davieset et al., 1990) and automatic
toll collection (Lotufo et al., 1990). ALPR techniques
vary from one application to another according to
their different working environments. In general, LPR
algorithms are usually composed of three processing
steps, namely, extraction of the region of the license
plate, segmentation of characters from the plate re-
gion, and recognition of each character.
Extracted License plate region is the most impor-
tant and challenging step of the plate recognition, be-
cause it is the first step and the next steps of LPR de-
pend on it. Generally, there are three method of ex-
tracting the plate from an image:
1. Plate features: The colored image is converted
into a gray level image in which a threshold is
selected to obtain the image in a black and white
format. A rectangular region with plate features is
the candidate region which needs to be located. A
good contrast of image pixels and a carefully se-
lected threshold are necessary in order to the plate
to be appeared in the image in standard plate fea-
tures such features include shape symmetry (Kim
and Chien, 2001), height to width ratio (Naito et
al., 2000), color (Kim et al., 1996), area and pixels
density.
398
Al-Hmouz R., Challa S. and Vo D. (2007).
PROBABILISTIC MODELING AND FUSION FOR IMAGE FEATURE EXTRACTION WITH APPLICATIONS TO LICENSE PLATE DETECTION.
In Proceedings of the Second International Conference on Computer Vision Theory and Applications - IFP/IA, pages 398-403
Copyright
c
 SciTePress

2. Edge detection(Parker and Federl, 1996): the
process depends on either the edges of the plate
or the edges of the characters. Any edge detec-
tion technique can be used to locate the edges,as
a rule,the plate and characters have many vertical
edges. A threshold should be chosen carefully for
various illuminants and weather conditions.
3. Signature analysis: the plate region is classified
based on plate signature. The signature could be
a big variation in the pixels brightness in some
rows of the input image (Barroso et al., 1997) or
it could be any text seen in the image (Dlagnekov,
2005).
All of the previous methods work perfectly in the ab-
sence of uncertainties. Noise is produced because of:
• Various colors for the characters and plate back-
ground.
• Weather condition and different illuminations.
• Non character symbols and dirt on license plate
• Non uniform lighting across the plate.
• Image processing noise.
Combining multiple classifiers has been long pursued
for improving the accuracy of single classifiers (Rah-
man and Fairhurst, 2003). A classifier fusion-based
detection algorithm was introduced by (Huang and
Guo, 2003) to extract the optimal features form the
candidate plate region.
In this paper, a novel technique of object ex-
traction based on modeling of geometric features is
proposed. Geometric features are model as random
variables and the fusion result of features will be a
measure of detection and extraction of an object in
the seen image. Multiple binary images for a Gray
scale image are obtained form multiple thresholds,
Bayes’ rule is used to update the posterior under the
assumption of normal distribution of all uncertain-
ties. Also, a deterministic method of object extrac-
tion based on geometric features is presented,pixels
vote for the area that appear the most in the same con-
text of threshold variation. Both methods are tested
on license plate extraction application under various
illumination conditions.
The rest of the paper is organized as follows: sec-
tion 2 Plate extraction algorithm, section 3 explains
Plate extractionmodel Algorithm, section 4 shows the
experimental results and finally section 5 concludes
the paper.
2 PLATE EXTRACTION
ALGORITHM
The first step in the LPR system is acquiring frames
from a digital camera. Light illumination and the
quality of the selected frame are having an effect on
all over LPR system and especially on the first part
of plate extraction; however some image process-
ing techniques can be used to estimate the illumi-
nate (Finlayson et al.,2001)or enhance image equality
(Rajaram et al., 2006) before the image got passed to
the next module of plate extraction.
2.1 Image Processing
Initially, the colored input image is converted into a
Gray scale image, then the Gray image is normalized
to increase the contrast between the plate back ground
and other areas around it as shown in Figure 1. A se-
lected threshold is chosen to convert the image into a
binary format, OTSU technique (Otsu, 1979) can be
used to find the optimal threshold. However, Thresh-
olding the image could be ended up with unconnected
plate region or totally missing the plate region.
(a) (b)
(c) (d)
Figure 1: Image conversion (a)Original. (b)Grey scale.
(c)Normalization gray. (d)Binary.
Multiple thresholds are used to convert the im-
age into black and white in order to guarantee that
the plate region will be appeared at least once in the
view, also plates come with various colors, it is hard
for the plate region to appear under different illumi-
nants, therefore varying the thresholds will help the
plate to appear as a connected region at least once in
the binary images.
2.2 Plate Extraction
Most plates in New South Wales (NSW) Australia
come in a rectangular shape with a dark black frame

around them which help to distinguish the plate area
as seen in Figure 2.
Figure 2: NSW plate types.
The most common features that identify the plate
are:
1. length/width ratio (u).
2. back ground area (a).
3. character/plate area density (d).
The binary image consists of black and white re-
gions,the white regions appear as connected areas
among the black regions. The nominated plate is
among the white areas if the threshold has been cho-
sen carefully. Most studies which use this method ex-
amine whether the connected region features fall in
the standard license plate features or not, then again,
noise will affect on those features as well as regions
having the same plate features such as the front lights
have an affect on selecting the right plate as can be
seen in Figure 3.
(a) (b)
(c) (d)
Figure 3: Thresholding image (a)Original. (b)t1. (c)t2.
(d)t3.
The probability of each connected region that can
be selected as a plate region under the assumption
that features are under the influence of normally dis-
tributed noise, is calculated by using a three-degree
Gaussian distribution:
p(z) =
1
(2π)
1/2
(detV)
1/2
exp

−
1
2
(z− s)
T
V
−1
(z− s)

(1)
where:
z =


u
a
d


is vector of random variables (measurements of re-
gion features).
s =


m
u
m
a
m
d


is the average standard plate features,
m =
1
N
N
∑
j= 1
Max(i)
j
+ Min(i)
j
2
,..N = 100, i = u, a, d.
N is number of plates used to find the mean of
plate area and
V =


σ
2
u
0 0
0 σ
2
a
0
0 0 σ
2
d


is the Variance,
σ
2
i
=
1
N
N
∑
i=1
(i− m
i
)
2
...i = u, a,d.
The region that have the highest probability will
be selected as the candidate plate region,the selected
region features are almost similar to the standard plate
features. Vector z can be expand to include other re-
gion features such as the orientation and perimeter of
the connected regions.
3 PLATE EXTRACTION MODEL
ALGORITHM
Let R
1
,R
2
,...,R
n
be the connected regions and
t
1
,t
2
,...,t
m
be the thresholds been used to convert the
image into black and white then:
p(R
n
/t
m
) =
1
(2π)
1/2
(detV)
1/2
exp

−
1
2
(R
n
− s)
T
V
−1
(R
n
− s)

(2)
The probability of the pixel v
ij
is
P(v
ij
/t
m
) =

p(R
n
/t
m
)/∆ ifv
ij
∈ R
n
Min(p(R
n
/t
m
))
2
/∆ otherwise
(3)
∆ is the normalization all over the image pixels.
The probabilities of the regions are assigned to the
pixels that belong to the same regions, for example of

a region R
1
has a probability 0.1 then all pixels in-
side this region will have the same probability and the
other pixels who do not have regions will have a small
probability which is less than any connected region
probability Min(p(R
n
/t
m
)), this small probability is
assigned to these pixel in order avoid multiplication
by zero. The highest pixels probabilities locate the li-
cense plate region and from the indices the plate can
be simply extracted.
The plate region could be defected because of the
noise problem, threshold t
1
will produce P(v
ij
/t
1
)
, correspondingly threshold t
2
informs with another
P(v
ij
/t
2
), it is like a new sensor inform with a new
data. The most probable plate region will appear sev-
eral times when using several thresholds. The updated
posterior P(v
ij
/t
1
,t
2
) can be calculated using Bayes’
rule:
P(v
ij
/t
1
,t
2
) =
P(t
1
/v
ij
,t
2
)· P(v
ij
/t
2
)
p(t
1
,t
2
)
(4)
The noise pixels are identical independently distrib-
uted,then
P(v
ij
/t
1
,t
2
) =
P(v
ij
/t
1
)P(v
ij
/t
2
)
∆
(5)
Recursive updating is simplified assuming con-
ditional independence of the measurements (Pearl,
1998) which implies
P(v
ij
/t
1
,t
2
,....,t
n
) = ∆
n
∏
γ=1
P(v
ij
/t
γ
) (6)
Other methods of plate extraction for example
edge detection technique can be used to inform with
another connected regions, Instead of converting the
image into a binary format, the edge is detected in or-
der to form connected regions and then the connected
region probabilities are calculated as previous method
using equation 1 and the update using equation 6.
4 EXPERIMENTAL RESULTS
The algorithm has been tested on 500 colored images.
All images tested on 480*640 pixels obtained form
Mobitx (Mobotix) camera which located in a car park,
the image were taken from different colors and sizes
of NSW plates. The plate region features statistics are
shown in Figure 4.
The algorithm successfully locates the license
plate in 98%. Using various thresholds and updat-
ing the posterior boosts the result accuracy. Figure
1000 2000 3000 4000 5000 6000 7000 8000
0
1
2
3
x 10
−4
area
p
(a)
0 0.2 0.4 0.6 0.8 1
6
7
8
9
10
11
12
x 10
−4
width/length
p
(b)
0 0.2 0.4 0.6 0.8 1
0
1
2
3
4
5
6
7
8
x 10
−3
density
p
(c)
Figure 4: Plate features distributions (a)Area.
(b)width/length. (c)Density.
5 shows the binary images of different threshold val-
ues, at a certain threshold there would be several areas
that could be nominated as the candidate plate. Once
updating the results with each other, some area will
be killed and disappear and others will be boosted
and come into view. When comparing this results
with traditional method of plate extraction using the
threshold t4, the method will fail to extract the plate
region,becuase the plate area region doesnt form un-
der that particular thershold value. The regions prob-
abilities are assigned to thier pixels as seen in Figure
6, after udating the posterior P(v
ij
), The pixels with
highest probabilities will be selected as the candidate
plate.
In order for the plate region to be appeared more
than any region in the image, the thresholds should
be in the range of the optimal threshold. If this was
not the case, other regions will appear more and their
pixel probabilities will be higher than the plate region
pixels probabilities, as a result, another region will
be picked up as a candidate region. In most cases,
thresholds in the range between 0.15 to .45 in 8 bit

(a) (b)
(c) (d)
Figure 5: Binary images (a)t1. (b)t2. (c)t3. (b)t4.
(a)
(b)
Figure 6: Posterior pixels probabilities (a)P(v/t
1
,t
2
).
(b)P(v/t
1
,t
2
t
3
,t
4
).
gray-level will be sufficient for plate region to ap-
pear at least once. Moreover, in some cases when the
probability of the plate region is not high enough, the
normalization all over the image pixels will kill this
probabilities and become almost as other pixels prob-
abilities. Yet, instead of assigning probabilities to the
plate pixels and fuse the result using Bayes’ rule, a
deterministic method can be used to locate the plate,
and fusion is achieved through pixels voting. For each
threshold, the plate area is tested using the traditional
method of plate extraction using plate feature and the
candidate plate region will vote for the most seen pix-
els, the pixels with highest votes form the different
thresholds will be picked up as the plate region. In
Figure 7, two regions appear to be a candidate region
for the plate but one of them has more votes than the
other, the region with more vote is the candidate plate
region.
The miss located number plates are all of that the
connected regions of the plate area which do not take
the plate shape because of indistinguishable frame
around the plate; the area of a connected plate re-
gion might not be formed under any threshold value.
In addition, when the plate back ground color is al-
most the same the vehicle’s color in which the plate
regions could not be formed. Therefore, using another
method to locate the license plate is essential to locate
the plate correctly and yet the results from both meth-
ods can be fused to enhance the performance for these
special cases, as well as for the case when the frame
is clearly visible around the plate.
(a)
(b)
Figure 7: (a)Original. (b)Pixels voting.
Both algorithms have high accuracy of plate ex-
traction due to the variation of the threshold values
which allow the plate to show up in the binary im-
age.Furthermore, more tests can be carried out on the
plate region to confirmthe previous tests. The number
of character can be counted after the plate is extracted,
also whether if they are in the line or not. If this test
fails, the next highest probability/vote region will be
tested. If the test still fails, then different threshold or
another algorithm should be examined.

5 CONCLUSION AND FUTURE
WORK
In this paper, new robust techniques of license plate
detection and extraction have been presented. Mul-
tiple thresholds are used to convert the image into
binary images, each binary image acts as a sensor
that informs with new data, and results from binary
images are fused using a probabilistic approach and
a deterministic approach. In the probabilistic ap-
proach, the plate geometric features are modeled as
random variables which are normally distributed, the
probability of connected regions is calculated through
fusion of region features. In the deterministic ap-
proach, the candidate plate region is extracted for
every threshold, the pixels which are appeared the
most will vote for the plate region. Geometric fea-
tures are tested for both methods, results showed the
efficiency for both methods and how they outperform
traditional methods of plate extraction.
For future work, other methods of plate extraction
will be examined, an appropriate probabilistic model
needed to be investigated in order both results can be
fused together throughout updating the posterior with
new data.
REFERENCES
Barroso, J. Bulas-Cruz, J. and Dagless, E. L. (1997). Real
Time Number Plate Reading. In 4th IFAC Workshop
on Algorithms and Architectures for Real-time Con-
trol, Portugal.
Davies,P. Emmott, N. and Ayland, N. (1990). License plate
recognition technology for toll violation enforcement
In Inst. Elect. Eng. Colloquium Image Analysis for
Transport Applications, pp. 7/1-7/5.
Dlagnekov, L (2005). Video-based Car Surveillance: Li-
cense Plate, Make, and Model Recognition. A Mas-
ters of Science in Computer Science thesis. UNIVER-
SITY OF CALIFORNIA, SAN DIEGO.
Finlayson, G. Hordley, S. and Hubel, P. (2001). Color by
correlation: A simple,unifying framework for color
constancy. In IEEE Transactions on Pattern Analysis
and Machine Intelligence, 23(11):12091221.
Huang, Z. and Guo, Y . (2003). Classifier Fusion Based
Vehicle License Plate Detection Algorithm. In pro-
ceedings of the Second International Conference on
Machine Learning and Cybernetics , pp. 2984-2989.
Kim, D. S. and Chien,S. I. (2001). Automatic car license
plate extraction using modified generalized symme-
try transform and image warping. In Proc. IEEE Int.
Symp. Industrial Electronics, vol. 3, pp. 2022-2027.
Kim,S. K. Kim,D. W. and Kim, H. J.(1996). A recog-
nition of vehicle license plate using a genetic algo-
rithm based segmentation. In Proc. Int. Conf. Image
Processing, vol. 2, pp. 661-664.
Lotufo, R. A. Morgan, A.D. and Johnson,A. S. (1990). Au-
tomatic numberplate recognition. In Inst. Elect. Eng.
Colloquium on Image Analysis for Transport Applica-
tions, pp. 6/1-6/6.
Mobotix. Retrieved November 10, 2006, from.
http://www.mobotix-camera.com/.
Naito,T. Tsukada,T. Yamada,K. Kozuka,K. and Yamamoto,
S. (2000). Robust license-plate recognition method
for passing vehicles under outside environment. In
IEEE Trans. Veh. Technol., vol. 49, pp. 2309-2319.
Otsu, N. (1979). A threshold selection method from gray
level histograms. In IEEE Transactions on Systems,
Man, and Cybernetics, vol. SMC-9, pp. 62-66.
Parker, J. R. and Federl, P. (1996). An approach to li-
cense plate recognition. In Computer Science Tech-
nical reports, University of Calgary, Alberta Canada,
Vol. 591-11.
Pearl, J (1998). Probabilistic Reasoning in Intelligent Sys-
tems. Morgan Kaufmann,San Francisco, CA,USA.
Rahman,A.F.R. and Fairhurst,M.C. (2003). Multiple clas-
sifier decision combination strategies for character
recognition: a review. In Int. J. Document Analysis
and Recognition, 5(4): 166-194.
Rajaram, S. Gupta M.S. Petrovic, N.,3 and Huang, T.S.
(2006). Learning-Based Nonparametric Image Super-
Resolution. In EURASIP Journal on Applied Signal
Processing Volume 2006, Article ID 51306, Pages 111
DOI 10.1155/ASP/2006/5130.
Sirithinaphong, T and Chamnongthai,K. (1998). The recog-
nition of car license plate for automatic parking sys-
tem. In 5th Int. Symp, Signal Processing and its Ap-
plications, pp. 455-457.
Yamaguchi, K. Nagaya, Y. Ueda, K. Nemoto,H. and Nak-
agawa,M. (1999). A method for identifying specific
vehicles using template matching. In Proc. IEEE Int.
Conf. Intelligent Transportation Systems,pp.8-13.