Intelligent automatic license plate recognition for electronic tolling environments

• US 10,019,640 B2
• Filed: 09/01/2016
• Issued: 07/10/2018
• Est. Priority Date: 06/24/2016
• Status: Active Grant

• Current Assignee: Accenture Global Solutions Limited (Accenture PLC)
• Original Assignee: Accenture Global Solutions Limited (Accenture PLC)
• Inventors: Almeida, Ricardo Andre Santos Gomes de, Pinto, Antonio Ricardo Ruano, Figueira, Romeu Rodrigues
• Primary Examiner(s): Baker, Charlotte M
• Application Number:
  US15/254,859
• Publication Number:
  US 20170372161A1
• Time in Patent Office: 677 Days
• Field of Search: 382105, 34028602, 340506, 340507, 700 3, 700 82
• CPC Class Codes: G06F 18/256 of results relating to different input data, e.g. multimodal recognition
• CPC Class Codes: G06V 10/811 the classifiers operating on different input data, e.g. multi-modal recognition
• CPC Class Codes: G06V 20/54 of traffic, e.g. cars on the road, trains or boats
• CPC Class Codes: G06V 20/62 Text, e.g. of license plates, overlay texts or captions on TV images
• CPC Class Codes: G06V 20/625 License plates
• CPC Class Codes: G06V 20/63 Scene text, e.g. street names
• CPC Class Codes: G07C 9/20 involving the use of a pass

Abstract

An intelligent automatic license plate recognition (IALPR) system implements technical solutions that improve the accuracy of automatic license plate recognition. The IALPR analyzes an image of a vehicle proximate to a toll collection point using optical character recognition (OCR), and determines candidate license plate identifications based, at least in part, on the corresponding OCR confidence level. The IALPR can also perform fingerprinting for candidate license plate images and matching analysis with a knowledge base, resulting in additional confidence levels. The IALPR can also perform behavioral analysis on the candidate license plate identifications, including trip context analysis, historical behavioral analysis, or other analytics. The IALPR can generate an overall confidence level for the candidate license plate identifications responsive to the OCR and vehicle fingerprint confidence levels and the behavioral analysis. This enhanced analysis helps the IALPR reduce the number of incorrect license plate identifications and reduce the need for human review.

Claims

1. A method comprising:

receiving, at a processor, an image including a representation of a license plate of a vehicle when the vehicle is proximate to a current toll collection point;

performing, by the processor, a first optical character recognition (OCR) process on the image to determine a candidate license plate string;

determining, by the processor, a candidate license plate identification associated with the candidate license plate string;

determining, by the processor, an OCR confidence level for the candidate license plate identification based on the first OCR process;

performing, by the processor, a behavioral analysis of geospatial vehicle position data related to the candidate license plate identification;

generating, by the processor, a behavioral analysis result based on the analysis of the geospatial vehicle position data;

determining, by the processor, an overall confidence level for the candidate license plate identification responsive to the OCR confidence level and the behavioral analysis result;

comparing the overall confidence level to a minimum confidence threshold; and

if the determined overall confidence level is above the minimum confidence threshold, communicating the candidate license plate information as a correctly identified license plate identification to a transaction processing system to thereby facilitate billing and collection of a toll associated with the current toll collection point.

2. The method of claim 1, where:

performing the behavioral analysis comprises performing a trip context analysis, the trip context analysis comprising;

searching for a related toll event occurring at a related toll collection point, the related toll collection point with respect to the current toll collection point, and having a license plate identification matching the candidate license plate identification; and

increasing the overall confidence level for the candidate license plate identification when;

the related toll event is found, and

the related toll event occurred within a travel time range,

the travel time range based on an average travel time between the current toll collection point and the related toll collection point.

3. The method of claim 1 where:

performing the behavioral analysis comprises performing a historical behavioral analysis, the historical behavioral analysis comprising;

searching for historical toll events occurring at the current toll collection point, the historical toll events associated with a license plate identification matching the candidate license plate identification;

determining, by the processor, a historical daily toll event pattern associated with the license plate identification, the determining responsive to the historical toll events;

determining, by the processor, whether a toll event corresponding to the image occurred within a daily toll time range of the historical daily toll event pattern for the license plate identification; and

increasing the overall confidence level for the candidate license plate identification when the toll event corresponding to the image occurred within the daily toll time range.

4. The method of claim 1 further comprising:

determining, by the processor, a physical vehicle attribute of the vehicle in the image;

comparing, by the processor, the physical vehicle attribute to a pre-determined vehicle attribute associated with the candidate license plate identification; and

increasing the overall confidence level for the candidate license plate identification when the physical vehicle attribute matches the pre-determined vehicle attribute.

5. The method of claim 1 further comprising determining the overall confidence level using a statistical model of random variables.

6. The method of claim 1 further comprising:

performing, by the processor, a second OCR process on the image to determine a second candidate license plate string, the second OCR process different from the first OCR process;

determining, by the processor, a second candidate license plate identification associated with the second candidate license plate string;

determining, by the processor, a second OCR confidence level for the second candidate license plate identification based on the second OCR process; and

determining, by the processor, the overall confidence level for the candidate license plate identification based on the OCR confidence level and the second OCR confidence level.

7. The method of claim 1 further comprising:

performing, by the processor, a fingerprint process on the license plate of the image to determine a current license plate fingerprint;

comparing, by the processor, the current license plate fingerprint to stored license plate fingerprints;

determining, by the processor, a closest license plate fingerprint to the current license plate fingerprint, the determining responsive to the comparing;

determining, by the processor, a fingerprint candidate license plate identification, the fingerprint candidate license plate identification responsive to the closest license plate fingerprint; and

determining, by the processor, a fingerprint confidence level for the fingerprint candidate license plate identification responsive to the closest license plate fingerprint.

8. The method of claim 7 further comprising:

determining an intermediate confidence level for the candidate license plate identification, the determining responsive to the OCR confidence level and the fingerprint confidence level; and

determining, by the processor, that the intermediate confidence level does not exceed an intermediate confidence level threshold.

9. The method of claim 1, wherein if the determined overall confidence level does not exceed the minimum confidence threshold, the method further comprises:

displaying the image at the toll collection point on a graphical user interface in communication with the processor;

receiving, from the graphical user interface, an external license plate identification for the image; and

updating a license plate database with the external license plate identification.

10. The method of claim 1 further comprising:

generating, by the processor, a derivative candidate license plate string from the candidate license plate string;

determining, by the processor, a derivative candidate license plate identification responsive to the derivative candidate license plate string;

determining, by the processor, a derivative confidence level for the derivative candidate license plate identification;

performing, by the processor, the behavioral analysis on the derivative candidate license plate identification; and

determining, by the processor, a derivative overall confidence level for the derivative candidate license plate identification responsive to the derivative confidence level and the behavioral analysis.

11. A system comprising:

a communication interface configured to receive;

a candidate license plate identification associated with an image of a vehicle proximate to a current toll collection point during a current toll event, and

an identification confidence level associated with the candidate license plate identification;

behavioral analysis circuitry coupled to the communication interface, the behavioral analysis circuitry configured to;

perform a behavioral analysis of geospatial vehicle positioning data related to the candidate license plate identification to generate behavioral information;

search for a related toll event occurring at a related toll collection point with respect to the current toll collection point and having a license plate identification matching the candidate license plate identification;

determine a current travel time from a time difference between the current toll event and the related toll event;

determine an average travel time between the current toll collection point and the related toll collection point; and

determine a degree of difference between the current travel time and the average travel time; and

probability evaluation circuitry configured to;

determine an overall confidence level associated with the candidate license plate identification responsive to the candidate license plate identification, the identification confidence level, and the behavioral information;

compare the overall confidence level to a minimum confidence threshold; and

if the determined overall confidence level is above the minimum confidence threshold, control the communication interface to communicate the candidate license plate information as a correctly identified license plate identification to a transaction processing system to thereby facilitate billing and collection of a toll associated with the current toll collection point.

12. The system of claim 11 wherein the probability evaluation circuitry further comprises a statistical model of random variables.

13. The system of claim 11 wherein the behavioral analysis circuitry is further configured to:

search for historical toll events occurring at the current toll collection point with a license plate identification matching the candidate license plate identification;

determine a historical daily toll event pattern including a daily toll time range for the license plate identification responsive to the historical toll events; and

determine whether the current toll event occurred within the daily toll time range.

14. The system of claim 11 wherein the communication interface is further configured to receive a physical vehicle attribute of the vehicle in the image;

the system further comprising vehicle attribute analysis circuitry configured to;

determine whether the physical vehicle attribute matches a pre-determined vehicle attribute associated with a license plate identification matching the candidate license plate identification; and

determine a physical vehicle attribute matching confidence level; and

the probability evaluation circuitry further configured to alter the overall confidence level for the candidate license plate identification responsive to determining that the physical vehicle attribute matches the pre-determined vehicle attribute and responsive to the physical vehicle attribute matching confidence level.

15. A non-transitory computer-readable medium including instructions that, when executed by circuitry elements, cause the circuitry elements to perform a method comprising:

receiving, via a communication interface, a candidate license plate identification associated with an image of a vehicle proximate to a current toll collection point during a current toll event and an identification confidence level associated with the candidate license plate identification;

performing, by behavioral analysis circuitry coupled to the communication interface, a behavioral analysis process on the candidate license plate identification to generate behavioral information;

determining, by probability evaluation circuitry coupled to the behavioral analysis circuitry, an overall confidence level associated with the candidate license plate identification responsive to the candidate license plate identification, the identification confidence level, and the behavioral information;

searching, by the behavioral analysis circuitry, for historical toll events occurring at the current toll collection point with a license plate identification matching the candidate license plate identification;

determining, by the behavioral analysis circuitry, a historical daily toll event pattern including a daily toll time range for the license plate identification responsive to the historical toll events;

determining, by the behavioral analysis circuitry, whether the current toll event occurred within the daily toll time range;

altering, by the probability evaluation circuitry, the overall confidence level for the candidate license plate identification responsive to determining whether the current toll event occurred within the daily toll time range;

comparing the overall confidence level to a minimum confidence threshold; and

if the determined overall confidence level is above the minimum confidence threshold, communicating the candidate license plate information as a correctly identified license plate identification to a transaction processing system to thereby facilitate billing and collection of a toll associated with the current toll collection point.

16. The computer-readable medium of claim 15, wherein the method further comprises determining, by the probability evaluation circuitry, the overall confidence level associated with the candidate license plate identification using a statistical model of random variables.

17. The computer-readable medium of claim 15, wherein the method further comprises:

searching, by the behavioral analysis circuitry, for a related toll event occurring at a related toll collection point with respect to the current toll collection point and having a license plate identification matching the candidate license plate identification;

determining, by the behavioral analysis circuitry, a current travel time from a time difference between the current toll event and the related toll event;

determining, by the behavioral analysis circuitry, an average travel time between the current toll collection point and the related toll collection point;

determining, by the behavioral analysis circuitry, a degree of difference between the current travel time and the average travel time; and

altering, by the probability evaluation circuitry, the overall confidence level for the candidate license plate identification responsive to finding the related toll event and responsive to determining the degree of difference between the current travel time and the average travel time.

18. The computer-readable medium of claim 15, wherein the method further comprises:

receiving, by the communication interface, a physical vehicle attribute of the vehicle in the image;

determining, by vehicle attribute analysis circuitry, whether the physical vehicle attribute matches a pre-determined vehicle attribute associated with the candidate license plate identification;

determining, by the vehicle attribute analysis circuitry, a physical vehicle attribute matching confidence level; and

altering, by the probability evaluation circuitry, the overall confidence level for the candidate license plate identification responsive to determining that the physical vehicle attribute matches the pre-determined vehicle attribute and responsive to determining the physical vehicle attribute matching confidence level.

19. A system comprising:

a communication interface configured to receive;

a candidate license plate identification associated with an image of a vehicle proximate to a current toll collection point during a current toll event;

an identification confidence level associated with the candidate license plate identification; and

a physical vehicle attribute of the vehicle in the image;

vehicle attribute analysis circuitry configured to;

determine whether the physical vehicle attribute matches a pre-determined vehicle attribute associated with a license plate identification matching the candidate license plate identification; and

determine a physical vehicle attribute matching confidence level; and

behavioral analysis circuitry coupled to the communication interface, the behavioral analysis circuitry configured to;

perform a behavioral analysis of geospatial vehicle positioning data related to the candidate license plate identification to generate behavioral information;

probability evaluation circuitry configured to;

determine an overall confidence level associated with the candidate license plate identification responsive to the candidate license plate identification, the identification confidence level, and the behavioral information;

alter the overall confidence level for the candidate license plate identification responsive to determining that the physical vehicle attribute matches the pre-determined vehicle attribute and responsive to the physical vehicle attribute matching confidence level;

compare the overall confidence level to a minimum confidence threshold; and

if the determined overall confidence level is above the minimum confidence threshold, control the communication interface to communicate the candidate license plate information as a correctly identified license plate identification to a transaction processing system to thereby facilitate billing and collection of a toll associated with the current toll collection point.