Car License Plate Extraction from Video Stream in

Complex Environment

Giorgio Grasso, Giuseppe Santagati

Facoltà di Scienze,

Università degli Studi di Messina,

Salita Sperone, Messina, Italy

Abstract. Th

e recognition of car license plates has a variety of applications

ranging from surveillance, to access and traffic control, to law enforcement.

Today a number of algorithms have been developed to extract car license plate

numbers from imaging data. In general there two class of systems, one

operating on triggered high speed cameras, employed in speed limit

enforcement, and one based on video cameras mainly used in various

surveillance systems (car-park access, gate monitoring, etc). A complete

automatic plate recognition system, consists of two main processing phases: the

extraction of the plate region from the full image; optical character recognition

(OCR) to identify the license plate number. This paper focuses on dynamic

multi-method image analysis for the extraction of car license plate regions,

from live video streams. Three algorithms have been deviced, implemented and

tested on city roads, to automatically extract sub-images containing car plates

only. The first criterion is based on the ratio between the height and width of

the plate, which has, for each type of plate, a standard value; the second

criterion is based on the eccentricity of the image on the two dimensions, i.e.

the projection histogram of plate number pixels onto the reference axes of the

image; the third criterion is based on the intensity histogram of the image. For

each criterion a likelihood is defined, which reaches its maximum when the

tested sub-image is close to the standard value for the type of plate considered.

The tuning of the methods has been carried on several video streams taken

during travel on busy city roads. The results for the overall recognition rate on

single frames is around 65%, whereas the multi-frame recognition rate is

around 85%. The significant value for the performance of the method is the

latter, as typically a license plate is visible in 5-10 frames. Based on three

parameters ranking, the same system can potentially distinguish and identify a

wide range of license plate types.

1 Introduction

Car license plate automatic recognition has an ever-increasing importance in many

fields of application. Law enforcement has gained greatly from the availability of

systems able to detect autonomously suspicious car plate numbers. There are also a

large number of examples in surveillance, for premises monitoring, gate access

control, car parking automatic management, etc. Differently from usual optical

Grasso G. and Santagati G. (2005).

Car License Plate Extraction from Video Stream in Complex Environment.

In Proceedings of the 5th International Workshop on Pattern Recognition in Information Systems, pages 73-80

DOI: 10.5220/0002565600730080

 SciTePress

character recognition (OCR) applications (e.g. documents archiving, postal code

reading, etc), which occur in controlled environments and lighting conditions, the

recognition of car plates is generally applied to imaging data collected in highly

complex sceneries [1,2]. In general such a system has to operate day and night, with

varying visibility conditions, analyzing images containing a large number of

unwanted objects of different nature (e.g. buildings, traffic signs, people, etc). In

addition the scene to be analyzed may contain more than one car [1,3,5].

The most important phase of car plate recognition is the extraction of the plate

region from the full scene frames. Subsequently OCR is applied, being this

technology in a mature state and quite reliable. Of course the reliability of OCR

algorithms rely on good quality images, not containing noise coming from unwanted

information [4].

This paper reports a novel car plate extraction method, based on three independent

feature matching criteria. In order to tackle the problem three parameters have been

identified as representative of a particular license plate type: the ratio between height

and width of the plate; the number of rows and columns where the characters are

located; the ratio between the plate number area and the plate background area. The

standard values of all the three features, defined for each car plate type, are compared

with the values computed for each sub-image analyzed, to construct a likelihood

ranking. The ranking gives an indication of how likely it is that a sub-image contains

a car plate and only a car plate of a particular type (e.g. national, foreign, front or

back, etc).

Experiments have been conducted on 34.5 minutes of video streams, recorded on

high traffic city roads. The data has been divided into two subsets, one used for

training of the system, the second one for testing. A total of 25 car plates have been

considered for training and a total of 40 car plates have been used during the

performance tests. Video streams where recorded on a standard digital camcorder,

with full PAL resolution at framerate, using MPEG2 compression.

The developed system can recognize car plates in a variety of lighting conditions

and a broad range of sub-image sizes, starting from 70x20 pixels (corresponding to

less than 2% of the frame area).

2 Image Segmentation

The car plate recognition process requires a first step of image segmentation, to

extract homogeneous regions within single frames. This is a necessary phase that

partitions the acquired image into several sub-images, to be taken into account as

candidate car plates.

In this paper a gradient based segmentation algorithm has been employed, which

uses the Canny [6] method to extract edges from imaging data. A thresholding

procedure is then used to remove dark areas of the image, given that plates show

usually high values of intensity. This process results in a binary image where white

pixels are the ones corresponding to brighter areas in the original frame. After

thresholding and edge extraction a seeded region growing (SRG) [7] strategy is used

to identify uniform image areas. A typical result of a complete segmentation for a

frame captured during experiments is shown in Fig. 1. The top left image (a) shows

the original data, the top right image (b) the threshold result, the bottom left image the

edge detection output (c), the bottom right image (d) the segmentation result.

(a) (b)

Figure 1. Example of image segmentation. Top left original image (a); top right thresholded

image (b); bottom left edge detection result (c); bottom right segmentation output (d).

Once the image frame is segmented into homogeneous regions - solid colors in Fig

1. (d) - the subsequent feature extraction procedure starts, to identify amongst

candidate areas of the image, the ones corresponding to car license plates. A detail of

the feature recognition strategies is reported in the following section.

3 Feature Extraction

The second step towards the definition of a car plate type is to identify univocal

features representing it. What is known about car plates is that they have a rectangular

shape, they contain a certain number of characters with a specified font and size,

distributed over a fixed number of rows and columns. The above features constitute a

reliable indication of a specific car license plate type, exception made for customized

car plates. An additional feature useful for identification is the background/foreground

colors, which is some cases can be different from white/black.

Starting from the above definition of what constitutes a car plate, three identifying

features have been defined: the ratio between height and width of a plate; the number

of rows and columns over which the plate digits distribute, which are described by the

projection eccentricity; the ratio between the areas covered in the plate by digits and

the area of background, defined through the intensity histogram.

A brief description of the methods used to implement the feature extraction for the

three criteria above mentioned is reported in the following.

Aspect ratio

Given a particular type of car plate, to be identified, its height and width are

measured and their ratio is computed to give a dimensionless characteristic number.

The absolute value of the difference between the height/width ratio measured on each

sub-image, and the characteristic number is defined as

∆

R. From all

∆

R measured on

the training set of images, a maximum value is obtained

∆

max

. A parameter is then

defined that gives a measure of how close to the actual value the aspect ratio of each

sub-image is, with respect to a specific plate type:

= 1 -

∆

R /

∆

max

Digit distribution

A second identifying feature is the location of characters included in the car plate.

Two integer numbers can be defined as the number of rows Nr and the number of

columns Nc over which characters align in the plate. In our case all digits and letters

spread across a single row and 7 columns, giving the following values for Nr = 1, Nc

= 7 (see Fig. 2). Using the eccentricity projection of digit pixels, an histogram can be

constructed, where peaks correspond to individual characters and valleys correspond

to the separation between adjacent characters. To obtain Nr

and

the eccentricity

histograms have to be processed and the number of peaks and valleys extracted. For

this purpose a threshold value is applied to the smoothed histograms, as a fraction of

the average value of each histogram, this to construct a binary vector. Then the

number of transitions from zero to one in the binary vector is counted.

∆

Figure 2. Italian license plate example, with its eccentricity histograms reported in the x, y

axes.

Once Nr

and

Nc are computed for a sub-image, they are compared to the actual

values for the specific car plate type considered, and the absolute value of the

differences ∆Nr and ∆Nc extracted. From the training set of data, the maximum

values of these quantities are derived, ∆Nr

max

and ∆Nc

max

. Two parameters can be

defined, giving a measure of how close to the actual case the distribution of characters

in a given sub-image is:

= 1 - ∆Nr / ∆Nr

max

= 1 - ∆Nc / ∆Nc

max

Coverage ratio

Given a particular license plate type, which corresponds to a fixed background

color and a distribution of digits with specific font type and size, the coverage ratio of

is confined in a fixed range. In order to account for variations of the coverage related

to varying character sequences and to allow for noisy data, a characteristic intensity

histogram can be constructed from experimental training data.

This characteristic histogram is a representation of the “average” value for the

coverage ratio. To construct such an average histogram the sub-images containing

license plates are extracted manually from the data and their histograms computed.

The result of averaging over all 25 sample license plate is shown in Fig. 3. It is

noticeable as the histogram shows a broad distribution of dark pixels, present in lower

number, and a more peaked bright pixel distribution, corresponding to the plate

background.

From the analysis of the histogram shape it is evident that the two pixel classes are

present in the images, digits and background, which spread over a rather wide range

of intensities. This is mainly due to the changing environmental conditions during

experiments and data acquisition noise.

Figure 3. Average intensity histogram computed on experimental sample Italian license plate

images.

The intensity histogram for a single plate candidate sub-image is derived, stretched

to cover the dynamic range of the average histogram (see above), with which is then

compared. From the comparison a metric distance is extracted as the sum of the

absolute value of the differences computed over all intensities, to obtain the coverage

ratio difference

∆

CR. The maximum value of

∆

CR is computed on the training set of

data giving

∆

max

. A parameter can then be defined, which gives a measure of how

close to the actual scenario the coverage of characters with respect to the background

of a candidate plate sub-image is:

= 1 -

∆

max

∆

max

4 Classification Methods

The recognition process starts from the segmented image as input and proceeds to

compute the features of each sub-image identified, in order to compare them with the

standard values defined for a specific car plate type.

Each of the four parameters defined in the previous section, D

and D

has values ranging from zero to one. The latter corresponding to the perfect

correspondence of the analyzed sub-image to the specific plate type considered.

Once the four feature values for a sub-image are extracted they have to be turned

into homogeneous parameters and combined into a single recognition score. To

achieve this the training car plate set is analyzed to construct statistical distributions

of each feature.

The distribution widths are used to define the weight of each feature in the

combined recognition score. When data distribute over a wide range it means that the

corresponding feature has a low discriminating capability, thus it should not

contribute substantially to the recognition process.

In the present case, however, all the feature normalized distributions show

comparable widths and they are, as a consequence, all significant in the recognition

process. For the case of Italian license plates the parameter D

has been neglected

because of the single row digit distribution characterizing the standard plate.

Data are shown in Fig. 4 for the training set used during the calibration of the

system. Fig. 4 refers to the aspect ratio D

distribution (a), digit distribution on

columns D

(b), coverage ratio D

(c).

(a) (b) (c)

Figure 4. Distributions of values for the three features used in the recognition process (see

text).

From the training set of images and the above distributions, with mean and

variance reported in Table 1, the following formula is defined for the recognition

score R:

R =

with

δ = 1

5 Experimental results

Apart from the training set a total of 34.5 min of video streams have been analyzed.

From this data 2362 single sub-images have been extracted, according to the

procedure described in section 2. All sub-images have been processed according to

the algorithm presented in sections 3 and 4, giving an overall recognition performance

of 65% on single frames and of 85% on sequences of 5 frames. In Table 1 a summary

of the experimental results are presented.

Table 1. Results of the plate classification experiments.

Video stream Environmental

conditions

# of actual

plates

Average # of

frames/plate

Recog. rate on

single frame

Recog. rate on

sequences

movie1.avi daylight 13 54 72% 90%

movie2.avi daylight 16 48 74% 91%

movie3.avi night 11 49 54% 65%

The lower performance of the algorithm, observed in data collected at night, is due

both to the lower visibility of the car plates and to the slow and noisy response of the

camera.

The values for the weights of the feature parameters in defined section 4 have

been, for the experiments conducted on the collected video streams, of α = 0.07, β =

0, γ = 0.75 and δ = 0.18. The parameter β has value zero because the car license plate

type analyzed has only one row of characters, thus it does not have a relevant

influence on the recognition process.

The obtain the final classification the recognition score R parameter has been

thresholded to 0.5 – i.e. all sub-images with a computed value of R higher than 0.5 are

considered car plate images. This is consistent with the assumption that the parameter

R is an indirect measure of the probability that a sub-image represents a car plate. In

this view when the value of R is higher the ½, there more than 50% chance that the

sub-image is the one containing only the car plate.

Table 2 shows some examples of recognized car license plates, together with the

corresponding values of recognition score R.

Table 2. Example recognized car license plates

License plate sub-

image

0.68

0.52

0.54

6 Conclusion

This paper describes a novel method for car license plate automatic extraction from

video streams. Experiments were presented for a set of 34.5 min of video data,

showing a good performance for the recognition of car plate locations within outdoor

sceneries. The algorithm allows the analysis of both single frames and sequences, the

latter giving a greatly improved performance. This is extremely useful for a great deal

of applications of the method, where single plate are visible in sequences of

successive video frames.

The system can be trained to recognize a variety of plate types, and can be

extended to do multi-type recognition, through a ranking procedure on the single

recognition parameter defined R.

Another possible extension of the method, able to track plates on image sequences,

is in the field of motion dynamics, extremely useful, for example, in speed limit

enforcement applications.

Though preliminary, the results shown here are promising for the definition of a

robust method for car plate recognition.

References

1. Mi-Ae K., Young-Mo K.: License Plate Surveillance System Using Weighted

Template Matching. 32nd Applied Image Pattern Recognition Workshop, Image Data

Fusion, Washington, DC, USA (2003) 269-274

2. Choudhury A. Rahman, Wael M. Badawy, Ahmad Radmanesh: A Real Time

Vehicle's License Plate Recognition System. IEEE Conference on Advanced Video

and Signal Based Surveillance, Miami, FL, (2003)

3. Kwang-Baek K., Si-Woong J., Cheol-Ki K.: Recognition of Car License Plate by

Using Dynamical Thresholding Method and Enhanced Neural Networks. Computer

Analysis of Images and Patterns, 10th International Conference, Groningen, The

Netherlands, (2003) 309-319

4. Yuntao C., Qian H.: Extracting characters of license plates from video sequences.

Mach. Vis. Appl. 10(5/6) (1998) 308-320

5. Eun Ryung Lee, Pyeoung Kee Kim, Hang Joon Kim: Automatic Recognition of a Car

License Plate using Color Image Processing. International Conference on Image

Processing, Austin, Texas, USA (1994) 301-305

6. Canny J.: A Computational Approach to Edge Detection. IEEE Transactions on

Pattern Analysis and Machine Intelligence, 8(6) (1986)

7. Adams, R., Bischof, L.: Seeded region growing. IEEE Trans. On PAMI, 16(6) (1994)

641-647