License plate character segmentation using likelihood maximization

• US 9,014,432 B2
• Filed: 05/04/2012
• Issued: 04/21/2015
• Est. Priority Date: 05/04/2012
• Status: Active Grant

• Current Assignee: Conduent Business Services, LLC (Conduent, Inc.)
• Original Assignee: Xerox Corporation (Xerox Holdings Corp.)
• Inventors: Fan, Zhigang, Zhao, Yonghui, Burry, Aaron Michael, Kozitsky, Vladimir
• Primary Examiner(s): Koziol, Stephen R
• Assistant Examiner(s): SCHWARTZ, RAPHAEL M
• Application Number:
  US13/464,357
• Publication Number:
  US 20130294652A1
• Time in Patent Office: 1,082 Days
• Field of Search: None
• US Class Current: 382/105
• CPC Class Codes: G06V 20/625 License plates
• CPC Class Codes: G06V 20/63 Scene text, e.g. street names

Abstract

A method determines a license plate layout configuration. The method includes generating at least one model representing a license plate layout configuration. The generating includes segmenting training images each defining a license plate to extract characters and logos from the training images. The segmenting includes calculating values corresponding to parameters of the license plate and features of the characters and logos. The segmenting includes estimating a likelihood function specified by the features using the values. The likelihood function measures deviations between an observed plate and the model. The method includes storing a layout structure and the distributions for each of the at least one model. The method includes receiving as input an observed image including a plate region. The method includes segmenting the plate region and determining a license plate layout configuration of the observed plate by comparing the segmented plate region to the at least one model.

Claims

1. A method for determining a license plate layout configuration, the method comprising:

generating at least one model representing a license plate layout configuration, the generating including;

for training images each including a sample license plate, segmenting each character and logo from the at least one sample license plate into a bounding box representation,

calculating values corresponding with each bounding box representation of the at least one sample license plate, and

estimating a likelihood function for defining a distribution of the values

storing a license plate layout configuration and the distribution of values for each of the at least one model;

classifying an observed license plate in a captured image, the classifying including;

segmenting the observed license plate into segments using varying thresholds to extract segments, wherein each threshold applied to the observed license plate produces the segments making up a possible layout model;

estimating features for the segments extracted for each possible layout model;

identifying candidate models by comparing measurements for the features against the distributions stored for the models;

calculating a likelihood for each determined match between a layout structure of the each candidate model and the stored models, wherein the likelihood is calculated as a sum of a marginal (log) likelihood using the equation;

log p(x|s,m)=Σ

_i log f_i(x|s,m),

where p(x|s, m) is the likelihood given the segmentation s under model m and observation x and f_i(x|s, m) is marginal distributions for feature i under model m;

determining the model having the maximum likelihood; and

associating the observed license plate as having the license plate layout configuration associated with the model having the maximum likelihood.

2. The method according to claim 1, wherein the estimating the likelihood function includes:

building a histogram for each feature; and

,

estimating the likelihood function as normalized histograms.

3. The method according to claim 1, wherein the segmenting the plate region includes:

(a) producing a binary map of the plate region by applying a threshold for binarization;

(b) determining object connected components relative to background components, wherein the object components include candidates for a character or a logo;

(c) matching the determined black connected components against the at least one model;

(d) determining if the matching score for the model is below a pre-set threshold; and

(e) determining the model that is a closest match to the plate region in the observed image.

4. The method according to claim 3, wherein the segmenting the plate region further includes:

measuring height and width features of each determined black connected component.

5. The method according to claim 3, wherein the segmenting the plate region further includes:

varying the threshold for binarization and repeating (b) through (e).

6. The method according to claim 1, wherein the segmenting the plate region includes:

(a) producing a binary map of the plate region;

(b) determining object components relative to background components;

(c) performing a projection in a vertical region;

(d) locating gaps between components by applying a threshold for valley detection;

(e) dividing the region into a sub-region at the gap;

(f) determining a collection of black pixels in each sub-region as including a candidate for a character or logo;

(g) matching the collections of black pixels against the at least one model;

(h) determining if a matching score for the model is below a pre-set threshold; and

(i) determining a model that is a closest match to the plate region in the observed image.

7. The method according to claim 6, wherein the segmenting the plate region further includes:

varying the threshold of valley detecting and repeating (d)-(i).

8. The method according to claim 1, wherein the calculating the values includes character dimensions, character positions, logo dimensions, and logo positions.

9. The method according to claim 1, wherein the one of the dimension and the relative position is selected from the group consisting of:

character width;

character height;

distance between centers of characters in a group of characters;

a difference between left and right margins of the license plate;

logo height;

logo width;

logo positions relative to neighboring characters; and

,

a combination of the above.

10. The method according to claim 9, wherein the left margin is calculated as a distance between center of the first character of the plate to the left edge and the right margin is calculated as a distance between the center of the last character of the plate and the right edge.

11. A system for determining a license plate layout configuration, the system comprising:

a training device including a processor for generating at least one model representing a license plate layout configuration, the training device being adapted to;

for training images of sample license plates, segment each character and logo from at least one sample license plate into a bounding box representation,

calculate values corresponding to with each bounding box representation of the at least one sample license plate, and,

estimate a likelihood function for defining a distribution of the values;

a storage device in communication with the training device, the storage device being adapted to store a license plate layout configuration and the distribution for each model;

a segmentation device in communication with the storage device, wherein upon receipt of an observed image including a plate region, the segmentation device is adapted to;

segment the plate region into segments using varying thresholds to extract the segments, wherein each threshold applied to the plate region produces the segments making up a possible layout model;

estimate features for the segments extracted for each possible layout model;

identify candidate models by comparing measurements for the features against the distributions stored for the models;

calculate a likelihood for each determined match between a layout structure of the each candidate model and the stored models, wherein the likelihood is calculated as a sum of a marginal (log) likelihood using the equation;

log p(x|s,m)=Σ

_i log f_i(x|s,m),

where p (x|s, m) is the likelihood given the segmentation s under model m and observation x and f_i (x|s, m) is marginal distributions for feature i under model m;

determine the model having the maximum likelihood; and

associate the license plate layout configuration as belonging to that of the model having the maximum likelihood.

12. The system according to claim 11, wherein the training device estimates the likelihood function as normalized histograms built for each of the features.

13. The system according to claim 11, wherein the segmentation device is adapted to:

produce a binary map of the plate region;

use the binary map to determine object connected components relative to background components;

match the determined black connected components against the at least one model;

determine if a matching score for the at least one model is below a pre-set threshold; and

,

determine the at least one model that is a closest match to the plate region in the observed image.

14. The system according to claim 13, wherein the segmentation device is adapted to produce and use at least a second binary map by varying a threshold for binarization.

15. The system according to claim 13, wherein the segmentation device is further adapted to:

measure height and width features of each determined black connected component.

16. The system according to claim 13, wherein after producing the binary map, the segmentation device is further adapted to:

perform a projection in the vertical region;

locate gaps between components by applying a threshold for valley detection;

divide the region into a sub-region at the gap; and

,

determine a collection of black pixels in each sub-region as including a candidate for a character or logo.

17. The system according to claim 16, wherein the gaps can be located by varying the threshold of valley.

18. A method for segmenting an observed image of a license plate for determining a layout configuration of the license plate, the method including:

providing at least one model representing a license plate layout configuration;

receiving as input an observed image including a plate region;

producing binary maps of the plate region by applying at least two binary thresholds;

using the binary maps to determine object connected components relative to background connected components;

matching the determined black connected components against the at least one model to identify at least one candidate model, the matching including;

estimating features for each, and

identifying the at least one candidate model by comparing measurements for the features against distributions stored for the at least one model;

in response to a matching score for the at least one candidate model below a pre-set threshold, calculating a likelihood for each determined match between a layout structure of the at least one candidate model and the at least one model, wherein the likelihood is calculated as a sum of a marginal (log) likelihood using the equation;

log p(x|s,m)=Σ

_i log f_i(x|s,m),

where p (x|s, m) is the likelihood given the segmentation s under model m and observation x and f_i(x|s, m) is marginal distributions for feature i under model m;

determining the at least one model having the maximum likelihood; and

associating the license plate layout configuration as belonging to that of the at least one model having the maximum likelihood.

19. A computer program product comprising non-transitory computer-readable recording media which encodes instructions for performing the method of claim 18.

20. A system for determining a license plate configuration, the system comprising:

a segmentation device comprising memory which stores instructions for performing the method of claim 18 and a processor, in communication with the memory for executing the instructions.