Automated license plate recognition system and method using human-in-the-loop based adaptive learning

• US 9,092,979 B2
• Filed: 12/14/2010
• Issued: 07/28/2015
• Est. Priority Date: 12/14/2010
• Status: Active Grant

• Current Assignee: Conduent Business Services, LLC (Conduent, Inc.)
• Original Assignee: Xerox Corporation (Xerox Holdings Corp.)
• Inventors: Burry, Aaron M., Paul, Peter
• Primary Examiner(s): Bali, Vikkram
• Application Number:
  US12/967,760
• Publication Number:
  US 20120148105A1
• Time in Patent Office: 1,687 Days
• Field of Search: 382/103, 382/105, 382/155, 382/159, 382/161, 382/173, 382/176, 382/177
• US Class Current: 1/1
• CPC Class Codes: G06V 10/987 with the intervention of an operator
• CPC Class Codes: G06V 20/625 License plates
• CPC Class Codes: G08G 1/0175 by photographing vehicles, e.g. when violating traffic rules

Abstract

An automated license plate recognition (ALPR) system and method using a human-in-the-loop based adaptive learning approach. One or more images with respect to an automotive vehicle can be segmented in order to determine a license plate of the automotive vehicle within a scene. An optical character recognition (OCR) engine loaded with an OCR algorithm can be further adapted to determine a character sequence of the license plate based on a training data set. A confidence level with respect to the images can be generated in order to route a low confidence image to an operator for obtaining a human interpreted image. The parameters with respect to the OCR algorithm can be adjusted based on the human interpreted image and the actual image of the license plate. A license plate design can be then incorporated into the OCR engine in order to automate the process of recognizing the license plate with respect to the automotive vehicle in a wide range of transportation related applications.

Claims

1. A method for automatically recognizing a license plate character sequence of a license plate of an automotive vehicle, said method comprising:

training a recognition algorithm associated with a license plate optical character recognition engine by adjusting recognition algorithm parameters based on a training data set comprising variations in training data license plate images and corresponding training data ground truth license plate character sequences;

segmenting at least one image of an automotive vehicle to determine a license plate image of a license plate located on said automotive vehicle within a scene;

determining an automatically recognized license plate character sequence in said license plate image utilizing said license plate optical character recognition engine;

generating a confidence level for said automatically recognized license plate character sequence with respect to said license plate image, wherein said confidence level is based on a signal-to-noise ratio in said license plate image and other image artifact quantifications;

comparing said confidence level to a lower limit;

routing said license plate image to a human operator if said confidence level is below said lower limit;

obtaining a human interpreted license plate character sequence from a human operator after said human operator examines said license plate image; and

adjusting at least one parameter of said recognition algorithm associated with said license plate optical character recognition engine based on said human interpreted license plate character sequence and said license plate image in order to automatically recognize said license plate character sequence of said license plate of said automotive vehicle in a wide range of transportation applications.

2. The method of claim 1 wherein said at least one image of said automotive vehicle contains a license plate with a license plate design differing from the license plate designs of said training data license plate images within said training data set.

3. The method of claim 1 further comprising adjusting said at least one parameter of said algorithm in a workflow to incorporate said human interpreted license plate character sequence and said image of said license plate into a training data set of said optical character recognition engine.

4. The method of claim 1 further comprising processing said at least one image of said automotive vehicle by applying a filtering approach associated with said image processing unit.

5. The method of claim 1 further comprising processing said at least one image of said automotive vehicle by applying a noise reduction and deblurring approach associated with said image processing unit.

6. The method of claim 1 further comprising generating said confidence level of said automatically recognized license plate character sequence based on a distance between at least one alternate license plate character sequence.

7. The method of claim 2 wherein said different license plate design is the license plate design of a different state.

8. The method of claim 1 wherein said at least one image of said automotive vehicle is obtained under at least one of lighting, vibration, and weather conditions differing from image capture conditions associated with said training data license plate images within said training data set.

9. The method of claim 1 wherein said license plate image segmented from said at least one image of said automotive vehicle contains at least one of blur, contrast, rotation, and scale image characteristics differing from said training data license plate images within said training data set.

10. The method of claim 1 wherein said license plate image segmented from said at least one image of said automotive vehicle contains occlusion characteristics differing from said training data license plate images within said training data set.

11. The method of claim 1 wherein said recognition algorithm further comprises correlation comparisons relative to at least one of the following:

a reference database; and

data indicative of nearest neighbor distance measures.

12. The method of claim 11 wherein said human interpreted license plate character sequence and said license plate image segmented from said at least one image of said automotive vehicle are added to said reference database.

13. The method of claim 1 wherein said at least one image of said automotive vehicle is obtained using a camera whose detection characteristics are at least one of:

visible light monochrome, visible light color, near infrared, short wave infrared, and long wave infrared.

14. The method of claim 1 wherein said license plate image comprises an image of a portion of a license plate corresponding to one character.

15. A system for automatically recognizing a license plate character sequence of a license plate of an automotive vehicle, said system comprising:

a processor;

a data bus coupled to said processor; and

a computer-usable medium embodying computer code, said computer-usable medium being coupled to said data bus, said computer program code comprising instructions executable by said processor and configured for;

training a recognition algorithm associated with a license plate optical character recognition engine by adjusting recognition algorithm parameters based on a training data set comprising variations in training data license plate images and corresponding training data ground truth license plate character sequences;

segmenting at least one image of an automotive vehicle to determine a license plate image of a license plate located on said automotive vehicle within a scene;

determining an automatically recognized license plate character sequence in said license plate image utilizing said license plate optical character recognition engine;

generating a confidence level for said automatically recognized license plate character sequence with respect to said license plate image, wherein said confidence level is based on a signal-to-noise ratio in said license plate image and other image artifact quantifications;

comparing said confidence level to a lower limit;

routing said license plate image to a human operator if said confidence level is below said lower limit;

obtaining a human interpreted license plate character sequence from a human operator after said human operator examines said license plate image; and

adjusting at least one parameter of said recognition algorithm associated with said license plate optical character recognition engine based on said human interpreted license plate character sequence and said license plate image in order to automatically recognize said license plate character sequence of said license plate of said automotive vehicle in a wide range of transportation applications.

16. The system of claim 15 wherein said at least one image of said automotive vehicle contains a license plate with a license plate design differing from the license plate designs of said training data license plate images within said training data set.

17. The system of claim 15 wherein said instructions are further configured for adjusting said at least one parameter of said algorithm in a workflow to incorporate said human interpreted license plate character sequence and said image of said license plate into a training data set of said optical character recognition engine.

18. The system of claim 15 wherein said instructions are further configured for processing said at least one image of said automotive vehicle by applying a filtering approach associated with said image processing unit.

19. The system of claim 15 wherein said instructions are further configured for processing said at least one image of said automotive vehicle by applying a noise reduction and deblurring approach associated with said image processing unit.

20. A system for automatically recognizing a license plate character sequence of a license plate of an automotive vehicle, said system comprising:

a processor;

a data bus coupled to said processor; and

a computer-usable medium embodying computer code, said computer-usable medium being coupled to said data bus, said computer program code comprising instructions executable by said processor and configured for;

training a recognition algorithm associated with a license plate optical character recognition engine by adjusting recognition algorithm parameters based on a training data set comprising variations in training data license plate images and corresponding training data ground truth license plate character sequences;

segmenting at least one image of an automotive vehicle to determine a license plate image of a license plate located on said automotive vehicle within a scene;

generating a confidence level for said automatically recognized license plate character sequence with respect to said license plate image, wherein said confidence level is based on a signal-to-noise ratio in said license plate image and other image artifact quantifications;

determining an automatically recognized license plate character sequence in said license plate image utilizing said license plate optical character recognition engine; and

adjusting at least one parameter of said recognition algorithm associated with said license plate optical character recognition engine based on a human interpreted license plate character sequence and said license plate image in order to automatically recognize said license plate character sequence of said license plate of said automotive vehicle in a wide range of transportation applications.