Character segmentation method for vehicle license plate recognition

• US 6,473,517 B1
• Filed: 09/15/1999
• Issued: 10/29/2002
• Est. Priority Date: 09/15/1999
• Status: Expired due to Fees

• Current Assignee: Siemens Corp. (Siemens AG)
• Original Assignee: Siemens Corporate Research Incorporated (Siemens AG)
• Inventors: Neubauer, Claus, Tyan, Jenn-Kwei
• Primary Examiner(s): Boudreau, Leo
• Assistant Examiner(s): AZARIAN, SEYED H
• Application Number:
  US09/396,952
• Time in Patent Office: 1,140 Days
• Field of Search: 382/105, 382/104, 382/205, 382/177, 382/278, 382/159, 382/173, 382/176, 382/291, 382/204, 382/218, 382/178, 340/907, 340/933, 340/10.1, 704/251, 375/240.1, 375/240.24, 375/240.02, 345/810, 345/840, 345/520, 345/427
• US Class Current: 382/105
• CPC Class Codes: G06V 20/625 License plates
• CPC Class Codes: G06V 30/10 Character recognition
• CPC Class Codes: G06V 30/153 using recognition of characters or words

Abstract

A method for segmenting and recognizing license plates, in accordance with the present invention includes capturing an image of a license plate and preprocessing the image to prepare the image for segmentation and recognition. Forward splitting regions of the license plate image into suspected characters regions, the suspected character regions are recognized in the license plate image and a confidence score is provided based on a probability of a correct match. If the suspected characters have a confidence score below a threshold value, backward merging is employed which merges adjacent suspected character regions. The backward merged suspected character regions in the license plate are recognized. If the backward merged suspected character regions have a confidence score below the threshold value, the backward merging and recognizing steps are repeated to improve the confidence score.

Claims

1. A method for segmenting and recognizing license plates comprising the steps of:

capturing an image of a license plate;

preprocessing the image to prepare the image for segmentation and recognition;

forward splitting regions of the license plate image into suspected characters regions;

recognizing suspected character regions in the license plate image and providing a confidence score based on a probability of a correct match;

if the suspected characters have a confidence score below a threshold value, backward merging adjacent suspected character regions;

recognizing the backward merged suspected character regions in the license plate; and

if the backward merged suspected character regions have a confidence score below the threshold value, repeating the backward merging and recognizing steps.

2. The method as recited in claim 1, wherein the step of recognizing suspected character regions in the license plate image includes determining regions with a confidence score below a threshold, and the step of forward splitting regions includes the step of segmenting touching or overlapping characters in adjacent suspected character regions by employing a discrimination function based on contour projections of the characters.

3. The method as recited in claim 2, wherein the step of segmenting touching or overlapping characters is recursively applied to achieve the threshold value of a confidence score.

4. The method as recited in claim 1, wherein the step of recognizing suspected character regions in the license plate image and providing a confidence score based on a probability of a correct match includes the step of recognizing characters in the license plate image by employing at least one of a neural network and template matching.

5. The method as recited in claim 1, wherein the step of recognizing the backward merged suspected character regions in the license plate includes the step of recognizing characters in the license plate image by employing at least one of a neural network and template matching.

6. The method as recited in claim 1, wherein the step of preprocessing the image to prepare the image for segmentation and recognition includes the steps of:

detecting pretilt in the license plate image by detecting edges and selecting feature points to determine a projection array on the edges;

selecting the projection array having a highest cost to determine an angle of rotation; and

correcting the license plate image in accordance with the angle of rotation.

7. The method as recited in claim 1, wherein the step of preprocessing the image to prepare the image for segmentation and recognition includes the steps of:

refining a position of the license plate by;

determining a height of characters on the license plate image by employing a projection function;

computing a spatial variance to determine a center position of the characters;

employing the center position and the height of the characters to reduce a size of a region of interest by detecting edges of a plate border and computing a length of the edges to identify the region of interest.

8. The method as recited in claim 1, wherein the step of preprocessing the image to prepare the image for segmentation and recognition includes the steps of

providing a projection profile of pixel intensities across vertical lines of pixels in the image;

filtering the projection profile; and

identifying locations of characters in the image depicted by area below a threshold value in the filtered projection profile.

9. The method as recited in claim 1, further comprising the step of outputting recognized characters of the license plate image.

10. The method as recited in claim 1, further comprising the step of comparing recognized characters and character blocks to predetermined license plate codes and conventions to check accuracy of recognition.

11. A method for segmenting and recognizing license plates comprising the steps of:

capturing an image of a license plate;

preprocessing the image to prepare the image for segmentation and recognition;

forward splitting regions of the license plate image into suspected characters regions by segmenting touching or overlapping characters in adjacent suspected character regions by employing a discrimination function based on contour projections of the characters, when the suspected characters have a confidence score below a threshold value;

recognizing suspected character regions in the license plate image and providing a confidence score based on a probability of a correct match;

if the suspected characters have a confidence score below a threshold value other than due to touching or overlapping characters, backward merging adjacent suspected character regions;

recognizing the backward merged suspected character regions in the license plate;

if the backward merged suspected character regions have a confidence score below the threshold value, repeating the backward merging and recognizing steps;

comparing recognized characters and character blocks to predetermined license plate codes and conventions to check accuracy of recognition; and

outputting recognized characters of the license plate image.

12. The method as recited in claim 11, wherein the step of segmenting touching or overlapping characters is recursively applied to achieve the threshold value of a confidence score.

13. The method as recited in claim 11, wherein the step of recognizing suspected character regions in the license plate image and providing a confidence score based on a probability of a correct match includes the step of recognizing characters in the license plate image by employing at least one of a neural network and template matching.

14. The method as recited in claim 11, wherein the step of recognizing the backward merged suspected character regions in the license plate includes the step of recognizing characters in the license plate image by employing at least one of a neural network and template matching.

15. The method as recited in claim 11, wherein the step of preprocessing the image to prepare the image for segmentation and recognition includes the steps of:

detecting pretilt in the license plate image by detecting edges and selecting feature points to determine a projection array on the edges;

selecting the projection array having a highest cost to determine an angle of rotation; and

correcting the license plate image in accordance with the angle of rotation.

16. The method as recited in claim 11, wherein the step of preprocessing the image to prepare the image for segmentation and recognition includes the steps of:

refining a position of the license plate by;

determining a height of characters on the license plate image by employing a projection function;

computing a spatial variance to determine a center position of the characters;

employing the center position and the height of the characters to reduce a size of a region of interest by detecting edges of a plate border and computing a length of the edges to identify the region of interest.

17. The method as recited in claim 11, wherein the step of preprocessing the image to prepare the image for segmentation and recognition includes the steps of

providing a projection profile of pixel intensities across vertical lines of pixels in the image;

filtering the projection profile; and

identifying locations of characters in the image depicted by area below a threshold value in the filtered projection profile.

18. A program storage device readable by machine, tangibly embodying a program of instructions executable by the machine to perform method steps for segmenting and recognizing license plates, the method steps comprising:

capturing an image of a license plate;

preprocessing the image to prepare the image for segmentation and recognition;

forward splitting regions of the license plate image into suspected characters regions;

recognizing suspected character regions in the license plate image and providing a confidence score based on a probability of a correct match;

if the suspected characters have a confidence score below a threshold value, backward merging adjacent suspected character regions;

recognizing the backward merged suspected character regions in the license plate; and

if the backward merged suspected character regions have a confidence score below the threshold value, repeating the backward merging and recognizing steps.

19. The program storage device as recited in claim 18, wherein the step of recognizing suspected character regions in the license plate image includes determining regions with a confidence score below a threshold, and the step of forward splitting regions includes the step of segmenting touching or overlapping characters in adjacent suspected character regions by employing a discrimination function based on contour projections of the characters.

20. The program storage device as recited in claim 19, wherein the step of segmenting touching or overlapping characters is recursively applied to achieve the threshold value of a confidence score.

21. The program storage device as recited in claim 18, wherein the step of recognizing suspected character regions in the license plate image and providing a confidence score based on a probability of a correct match includes the step of recognizing characters in the license plate image by employing at least one of a neural network and template matching.

22. The program storage device as recited in claim 18, wherein the step of recognizing the backward merged suspected character regions in the license plate includes the step of recognizing characters in the license plate image by employing at least one of a neural network and template matching.

23. The program storage device as recited in claim 18, wherein the step of preprocessing the image to prepare the image for segmentation and recognition includes the steps of:

detecting pretilt in the license plate image by detecting edges and selecting feature points to determine a projection array on the edges;

selecting the projection array having a highest cost to determine an angle of rotation; and

correcting the license plate image in accordance with the angle of rotation.

24. The program storage device as recited in claim 18, wherein the step of preprocessing the image to prepare the image for segmentation and recognition includes the steps of:

refining a position of the license plate by;

determining a height of characters on the license plate image by employing a projection function;

computing a spatial variance to determine a center position of the characters;

employing the center position and the height of the characters to reduce a size of a region of interest.

25. The program storage device as recited in claim 18, wherein the step of preprocessing the image to prepare the image for segmentation and recognition includes the steps of

providing a projection profile of pixel intensities across vertical lines of pixels in the image;

filtering the projection profile; and

identifying locations of characters in the image depicted by area below a threshold value in the filtered projection profile.

26. The program storage device as recited in claim 18, further comprising the step of outputting recognized characters of the license plate image.

27. The program storage device as recited in claim 18, further comprising the step of comparing recognized characters and character blocks to predetermined license plate codes and conventions to check accuracy of recognition.