Methods and systems for accurately recognizing vehicle license plates

• US 10,706,330 B2
• Filed: 09/14/2016
• Issued: 07/07/2020
• Est. Priority Date: 10/01/2015
• Status: Active Grant

• Current Assignee: IntelliVision Technologies Corp. (Melrose Industries Plc)
• Original Assignee: IntelliVision Technologies Corp. (Melrose Industries Plc)
• Inventors: Gope, Chandan, Gupta, Gagan, Jindal, Nitin, Agarwal, Amit
• Primary Examiner(s): Sun, Jiangeng
• Application Number:
  US15/265,026
• Publication Number:
  US 20170300786A1
• Time in Patent Office: 1,392 Days
• Field of Search: None
• CPC Class Codes: G06F 18/24 Classification techniques
• CPC Class Codes: G06T 5/20 using local operators
• CPC Class Codes: G06T 5/70 Denoising; Smoothing
• CPC Class Codes: G06V 20/56 exterior to a vehicle by using sensors mounted on the vehicle
• CPC Class Codes: G06V 20/625 License plates
• CPC Class Codes: G06V 30/19173 Classification techniques

Abstract

The present invention discloses methods, systems and computer programmable products for detecting license plates and recognizing characters in the license plates. The system receives an image and identifies one or more regions including a license plate. The one or more regions are converted into a plurality of binarized images, which are then filtered to remove noise. Next, one or more clusters of characters are identified in the plurality of binarized images. The one or more clusters of characters are analyzed to recognize a set of characters, wherein each character in the set includes a confidence value.

Claims

1. A license plate detection and recognition system with temporal validation, said system comprising:

a license plate detection unit;

a character detection unit;

a character recognition unit;

a post-processor;

a processor, a memory element coupled to the processor, and encoded instructions, when implemented by the processor, cause the system to;

detect one or more regions within one and/or a plurality of captured or stored image frames comprising a license plate within a detected vehicle by the license plate detection unit;

detect one or more clusters of characters in each of a filtered binarized image of the detected regions and send the one or more detected clusters to the character recognition unit for recognition of characters;

send the recognized characters to the post-processor for temporal validation, wherein the post-processor considers placement of the recognized characters across a plurality of image frames and a set of characters that denote a different alignment or placement in a minority of frames are flagged as outliers and discarded; and

compute a weighted probability for each of any remaining recognized characters with an ambiguity (at least two character possibilities) by multiplying a number of frames that recognized the character/total number of frames by a confidence value in percentage for each of the at least two character possibilities and validate the character with the higher weighted probability, thereby resolving the ambiguity.

2. The system for detecting and recognizing a license plate of claim 1, further comprising receiving the image frames from at least one of an image capturing device, a network and a memory unit.

3. The system for detecting and recognizing a license plate of claim 1, wherein the image frame is a frame of a video.

4. The system of claim 1, further comprising a license plate verification unit configured to detect the one or more regions in the image based on at least one of the group comprising color, edges, transition in edges, shape, size, orientation, Histogram of Gradients (HoGs) and a machine-learning based classifier.

5. The system of claim 1, wherein each of a horizontal profile and a vertical profile of the binarized image is generated based on one or more transition points identified by scanning the binarized image, wherein each of the one or more transition points represents one of a transition from black to white and a transition from white to black.

6. The system of claim 1, further comprising a character recognition unit that recognizes a set of characters using a classifier, and wherein the classifier is based on at least one of a supervised and an unsupervised machine-learning network.

7. The system of claim 1, wherein the temporal validation assigns a higher weight to image frames that are at least one of significant or of a higher image quality in computing the weighted probability.

8. The system of claim 7, wherein the significance is pre-determined and the higher image quality is determined based on image sharpness.

9. The system of claim 1, further comprising a license plate detection unit for detecting license plates of at least different sizes or formats.

10. The system of claim 1, wherein the license plate detection unit is further configured to create a list of thresholds based on a variable adaptive threshold (T), wherein the list of thresholds comprises N values in the range of {f*T, g*T}, where f<

1 and g>

1.

11. The system of claim 1, further comprising a set of parameters comprising at least one of a group comprising a mean, a median, a mode and a standard deviation of pixel intensity in a window.

12. The system of claim 1, further configured to detect one or more vehicles in the image.

13. A license plate detection and recognition system using temporal validation to resolve any ambiguity of a recognized character, said system comprising:

a post-processor;

a processor, a memory element coupled to the processor, and encoded instructions, when implemented by the processor, cause the system to;

send recognized characters from a detected one or more clusters of characters in each of a filtered binarized image of detected regions within one and/or a plurality of captured or stored images frames comprising at least a portion of a license plate to the post-processor for temporal validation, wherein the post-processor considers placement of the recognized characters across a plurality of image frames and a set of characters that denote a different alignment or placement in a minority of frames are flagged as outliers and discarded; and

compute a weighted probability for each of any remaining recognized characters with an ambiguity (at least two character possibilities) by multiplying a number of frames that recognized the character/total number of frames by a confidence value in percentage for each of the at least two character possibilities and validate the character with the higher weighted probability, thereby resolving the ambiguity.

14. The system of claim 13, further comprising a license plate verification unit configured to detect the one or more regions in the image based on at least one of the group comprising color, edges, transition in edges, shape, size, orientation, Histogram of Gradients (HoGs) and a machine-learning based classifier.

15. The system of claim 13, wherein each of a horizontal profile and a vertical profile of the binarized image is generated based on one or more transition points identified by scanning the binarized image, wherein each of the one or more transition points represents one of a transition from black to white and a transition from white to black.

16. The system of claim 13, further comprising a character recognition unit that recognizes a set of characters using a classifier, and wherein the classifier is based on at least one of a supervised and an unsupervised machine-learning network.

17. The system of claim 13, wherein the post processor is further configured to validate the set of characters based on at least one of the group comprising spatial arrangement of characters, frequency of occurrence of characters and a set of pre-determined rules.

18. The system of claim 13, wherein the license plate detection unit is further configured to create a list of thresholds based on a variable adaptive threshold (T), wherein the list of thresholds comprises N values in the range of {f*T, g*T}, where f<

1 and g>

1.

19. The system of claim 13, further comprising a set of parameters comprising at least one of a group comprising a mean, a median, a mode and a standard deviation of pixel intensity in a window.

20. A license plate detection and recognition method using temporal validation to resolve any ambiguity of a recognized character, said method comprising the steps of:

detecting one or more regions within one and/or a plurality of captured or stored image frames comprising a license plate within a detected vehicle by the license plate detection unit;

detecting one or more clusters of characters in each of a filtered binarized image of the detected regions and send the one or more detected clusters to the character recognition unit for recognition of characters;

sending the recognized characters to the post-processor for temporal validation, wherein the post-processor considers placement of the recognized characters across a plurality of image frames and a set of characters that denote a different alignment or placement in a minority of frames are flagged as outliers and discarded; and

computing a weighted probability for each of any remaining recognized characters with an ambiguity (at least two character possibilities) by multiplying a number of frames that recognized the character/total number of frames by a confidence value in percentage for each of the at least two character possibilities and validating the character with the higher weighted probability, thereby resolving the ambiguity.