Robust character segmentation for license plate images

• US 8,934,676 B2
• Filed: 07/02/2012
• Issued: 01/13/2015
• Est. Priority Date: 05/04/2012
• Status: Active Grant

• Current Assignee: Conduent Business Services, LLC (Conduent, Inc.)
• Original Assignee: Xerox Corporation (Xerox Holdings Corp.)
• Inventors: Burry, Aaron Michael, Fillion, Claude, Kozitsky, Vladimir
• Primary Examiner(s): Repko, Jason M
• Assistant Examiner(s): Potts, Ryan P
• Application Number:
  US13/539,739
• Publication Number:
  US 20130294654A1
• Time in Patent Office: 925 Days
• Field of Search: None
• US Class Current: 382/105
• CPC Class Codes: G06V 20/62 Text, e.g. of license plates, overlay texts or captions on TV images
• CPC Class Codes: G06V 20/625 License plates
• CPC Class Codes: G06V 30/10 Character recognition
• CPC Class Codes: G06V 30/148 Segmentation of character regions

Abstract

A method and system for achieving accurate segmentation of characters with respect to a license plate image within a tight bounding box image. A vehicle image can be captured by an image capturing unit and processed utilizing an ALPR unit. A vertical projection histogram can be calculated to produce an initial character boundary (cuts) and local statistical information can be employed to split a large cut and insert a missing character. The cut can be classified as a valid and/or a suspect character and the suspect character can be analyzed. The suspect character can be normalized and passed to an OCR module for decoding and generating a confidence quote with every conclusion. The non-character images can be rejected at the OCR level by enforcing a confidence threshold. An adjoining suspect narrow character can be combined and the OCR confidence of the combined character can be assessed.

Claims

1. A license plate character segmentation method, said method comprising:

determining an initial character boundary of a vehicle image utilizing a midpoint of a segment along a vertical projection histogram in order to partition said initial character boundary from said vehicle image;

classifying said initial character boundary as a valid character boundary or a suspect character boundary;

analyzing said suspect character boundary and enabling robust performance over a plurality of state logos, fonts, and character spacing;

receiving a character image and a label and identifying said character image utilizing said label as a likely character or not;

creating a copy of an input character label array as outlabel and initializing all values of said outlabel to zero;

setting each element in said outlabel to one for a character identified as a non-character that is touching a boundary until a valid character is encountered;

assigning said outlabel element to two for each non-character surrounded by said valid character;

setting said outlabel element to three for each set of at least two non-characters surrounded by said valid character and combining at least two images and copying to a first location for each set of element in said outlabel set to three and also set outlabel for second location back to one; and

normalizing and passing said suspect character boundary to an optical character recognition subsystem for decoding and generating a confidence quote for each suspect character boundary of said vehicle image analyzed and thereafter modifying each suspect character boundary decoded based on a set of confidence thresholds.

2. The method of claim 1 wherein said modification of each suspect character boundary comprises rejecting a non-character image by enforcing a confidence threshold of said set of confidence thresholds.

3. The method of claim 1 wherein said modification of each suspect character boundary comprises accepting a non-character image by enforcing a confidence threshold of said set of confidence thresholds.

4. The method of claim 1 wherein said modification of each suspect character boundary comprises combining an adjoining suspect narrow character with said each suspect character boundary to create a combined character and assessing said optical character recognition confidence of said combined character to achieve enhanced character segmentation accuracy.

5. The method of claim 1 further comprising:

splitting a single, wide initial character boundary into two by placing a boundary at a minimum histogram value in a region near a center of said wide initial character boundary; and

replacing a large gap caused by omission of a special character with a missing image data.

6. The method of claim 1 further comprising training an image-based classifier to identify presence of a valid letter and/or a number and applying said image-based classifier to each segment of said initial character boundary.

7. The method of claim 1 further comprising identifying and addressing at least one segmentation error.

8. The method of claim 7 wherein addressing said at least one segmentation error further comprises:

obtaining an optical character recognition identification and said confidence quote by running each character image labeled as zero, two, and three;

comparing said confidence quote to said threshold for each character image labeled as three, if greater, change from one to three and output said optical character recognition identification as part of a computer code; and

selecting a lowest confidence neighbor for each character image labeled as two, merging two character images and passing to said optical character recognition subsystem to obtain said optical character recognition identification and said confidence quote for said two character images merged.

9. The method of claim 8 further comprising:

changing said label from one to four and output merged identification as part of said computer code if said confidence quote of said merged image is greater than said threshold and greater than said confidence quote of said character labeled as two and greater than its neighbor; and

comparing said confidence of said character labeled as two to said threshold if said merged image confidence does not meet a requirement otherwise passing said identification as part of said computer code.

10. The method of claim 1 wherein said segmentation method returns a bitmap of a segmented image along with a label identifying said bitmap that is likely a valid character image.

11. A license plate character segmentation system, said system comprising:

a processor;

a data bus coupled to said processor; and

a computer-usable medium embodying computer code, said computer-usable medium being coupled to said data bus, said computer code comprising instructions executable by said processor and configured for;

determining an initial character boundary of a vehicle image utilizing a midpoint of a segment along a vertical projection histogram in order to partition said initial character boundary from said vehicle image;

classifying said initial character boundary cut as a valid character boundary or a suspect character boundary;

analyzing said suspect character boundary and enabling robust performance over a plurality of state logos, fonts, and character spacing;

receiving a character image and a label and identifying said character image utilizing said label as a likely character or not;

creating a copy of an input character label array as outlabel and initializing all values of said outlabel to zero;

setting each element in said outlabel to one for a character identified as a non-character that is touching a boundary until a valid character is encountered;

assigning said outlabel element to two for each non-character surrounded by said valid character;

setting said outlabel element to three for each set of at least two non-characters surrounded by said valid character and combining at least two images and copying to a first location for each set of element in said outlabel set to three and also set outlabel for second location back to one; and

normalizing and passing said suspect character boundary to an optical character recognition subsystem for decoding and generating a confidence quote for each suspect character boundary of said vehicle image analyzed and thereafter modifying each suspect character boundary decoded based on a set of confidence thresholds.

12. The system of claim 11 wherein said modification of each suspect character boundary comprises rejecting a non-character image by enforcing a confidence threshold of said set of confidence thresholds.

13. The system of claim 11 wherein said modification of each suspect character boundary comprises accepting a non-character image by enforcing a confidence threshold of said set of confidence thresholds.

14. The system of claim 11 wherein said modification of each suspect character boundary comprises combining an adjoining suspect narrow character with said each suspect character boundary to create a combined character and assessing said optical character recognition confidence of said combined character to achieve enhanced character segmentation accuracy.

15. A non-transitory processor-readable medium storing code representing instructions to cause a process to perform a process of license plate character segmentation, said code comprising code to:

determine an initial character boundary of a vehicle image utilizing a midpoint of a segment along a vertical projection histogram in order to partition said initial character boundary from said vehicle image;

classify said initial character boundary as a valid character boundary or a suspect character boundary;

analyze said suspect character and enable robust performance over a plurality of state logos, fonts, and character spacing;

receive a character image and a label and identifying said character image utilizing said label as a likely character or not;

create a copy of an input character label array as outlabel and initializing all values of said outlabel to zero;

set each element in said outlabel to one for a character identified as a non-character that is touching a boundary until a valid character is encountered;

assign said outlabel element to two for each non-character surrounded by said valid character;

set said outlabel element to three for each set of at least two non-characters surrounded by said valid character and combining at least two images and copying to a first location for each set of element in said outlabel set to three and also set outlabel for second location back to one; and

normalize and pass said suspect character boundary to an optical character recognition subsystem for decoding and generating a confidence quote for each suspect character boundary of said vehicle image analyzed and thereafter modifying each suspect character boundary decoded based on a set of confidence thresholds.