Shadow detection and removal in license plate images

• US 10,026,004 B2
• Filed: 07/08/2016
• Issued: 07/17/2018
• Est. Priority Date: 07/08/2016
• Status: Active Grant

• Current Assignee: Conduent Business Services, LLC (Conduent, Inc.)
• Original Assignee: Conduent Business Services, LLC (Conduent, Inc.)
• Inventors: Mizes, Howard, Kozitsky, Vladimir
• Primary Examiner(s): Harandi, Siamak
• Application Number:
  US15/205,146
• Publication Number:
  US 20180012101A1
• Time in Patent Office: 739 Days
• CPC Class Codes: G06T 2207/20104 Interactive definition of region of interest [ROI]
• CPC Class Codes: G06T 5/94 based on local image properties, e.g. for local contrast enhancement
• CPC Class Codes: G06T 7/0002 Inspection of images, e.g. flaw detection
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 20/625 License plates
• CPC Class Codes: G06V 30/10 Character recognition
• CPC Class Codes: G06V 30/155 Removing patterns interfering with the pattern to be recognised, such as ruled lines or underlines

Abstract

A method, system, and apparatus for license plate relighting comprises collecting an image of a license plate, performing license plate recognition on the image of the license plate; calculating a confidence metric for the license plate recognition; and performing a shadow detection and relighting method if the confidence metric is below a predetermined threshold, comprising identifying a shaded region of said license plate, determining if the shaded region is actually shaded, and relighting the actually shaded region.

Claims

1. A method for license plate image shadow detection and relighting comprising:

collecting an image of a license plate with an image capturing device;

identifying a candidate region of interest containing said license plate;

identifying a potentially shaded region of said license plate by identifying a boundary between said potentially shaded region and an unshaded region;

determining if said shaded region is actually shaded; and

relighting said actually shaded region.

2. The method of claim 1 wherein identifying a potentially shaded region of said license plate further comprises:

fitting a tight bound box around said license plate in said image of said license plate;

removing fine structures from said image of said license plate;

binarizing said image of said license plate;

identifying edges in said image of said license plate;

selecting a longest edge from said identified edges in said image of said license plate as an edge defining a boundary between said shaded region of said license plate and an unshaded region of said license plate; and

translating said longest edge.

3. The method of claim 2 wherein said fine structures are removed with an ordered statistic filter.

4. The method of claim 2 wherein identifying edges in said image of said license plate further comprises:

identifying edges primarily oriented in an expected direction of a shadow boundary.

5. The method of claim 4 wherein said expected direction of said shadow boundary is determined according to a relative orientation of a vehicle with said license plate and an expected position of the sun according to a geolocation on the earth and a time of day.

6. The method of claim 1 wherein determining if said shaded region is actually shaded further comprises:

extracting at least two features from said image of said license plate;

combining said at least two features; and

classifying said combined at least two features using a pre-trained shadow classifier in order to determine if a shadow is present in said image of said license plate.

7. The method of claim 1 wherein relighting said actually shaded region further comprises:

creating a shadow mask; and

applying a relighting application to said shadow masked region.

8. The method of claim 7 wherein said relighting application comprises:

calculating a mean RGB value for said shadow masked region;

calculating a mean RGB value for an unmasked region; and

scaling pixels in said shadow masked region according to the ratio of said mean RGB value for said shadow masked region and said mean RGB value of said unmasked region.

9. A system for license plate image shadow detection and relighting comprising:

a video acquisition module configured to collect image data of a license plate;

a processor; and

a computer-usable medium embodying computer code, said computer-usable medium being coupled to said processor, said computer code comprising non-transitory instruction media executable by said processor configured for;

identifying a candidate region of interest containing said license plate;

identifying a potentially shaded region of said license plate by identifying a boundary between said potentially shaded region and an unshaded region in said candidate region of interest containing said license plate;

determining if said shaded region is actually shaded; and

relighting said actually shaded region.

10. The system of claim 9 wherein identifying a potentially shaded region of said license plate further comprises:

fitting a tight bound box around said license plate in said image of said license plate;

removing fine structures from said image of said license plate;

binarizing said image of said license plate;

identifying edges in said image of said license plate;

selecting a longest edge from said identified edges in said image of said license plate as an edge defining a boundary between said shaded region of said license plate and an unshaded region of said license plate; and

translating said longest edge.

11. The system of claim 10 further comprising an order static filter for removing said fine structures.

12. The system of claim 10 wherein identifying edges in said image of said license plate further comprises:

identifying edges primarily oriented in an expected direction of a shadow boundary.

13. The system of claim 12 wherein said expected direction of said shadow boundary is determined according to a relative orientation of a vehicle with said license plate and an expected position of the sun according to a geolocation on the earth and a time of day.

14. The system of claim 9 wherein determining if said shaded region is actually shaded further comprises:

extracting at least two features from said image of said license plate;

combining said at least two features; and

classifying said combined at least two features using a pre-trained shadow classifier in order to determine if a shadow is present in said image of said license plate.

15. The system of claim 9 wherein relighting said actually shaded region further comprises:

creating a shadow mask; and

applying a relighting application to said shadow masked region.

16. The system of claim 15 wherein said relighting application comprises:

calculating a mean RGB value for said shadow masked region;

calculating a mean RGB value for an unmasked region; and

scaling pixels in said shadow masked region according to the ratio of said mean RGB value for said shadow masked region and said mean RGB value of said unmasked region.

17. A method for license plate image recognition comprising:

collecting an image of a license plate with an image capturing device;

performing license plate recognition on said image of said license plate;

calculating a confidence metric for said license plate recognition; and

performing a shadow detection and relighting comprising;

identifying a candidate region of interest containing said license plate;

identifying a potentially shaded region of said license plate by identifying a boundary between said potentially shaded region and an unshaded region;

determining if said shaded region is actually shaded;

relighting said actually shaded region; and

performing a second license plate recognition on said image of said license plate if said confidence metric is below a predetermined threshold.

18. The method of claim 17 wherein identifying a potentially shaded region of said license plate further comprises:

fitting a tight bound box around said license plate in said image of said license plate;

converting said image of said license plate to black and white;

removing structures from said image of said license plate;

binarizing said image of said license plate;

identifying edges in said image of said license plate;

selecting a longest edge from said identified edges in said image of said license plate as an edge defining a boundary between said shaded region of said license plate and an unshaded region of said license plate;

smoothing said selected longest edge from said identified edges in said image of said license plate; and

translating said smoothed longest edge.

19. The method of claim 17 wherein determining if said shaded region is actually shaded further comprises:

extracting at least two features from said image of said license plate;

combining said at least two features; and

classifying said combined at least two features using a pre-trained shadow classifier in order to determine if a shadow is present in said image of said license plate.

20. The method of claim 17 wherein relighting said actually shaded region further comprises:

creating a shadow mask; and

applying a relighting application to said shadow masked region.