ROBUST CROPPING OF LICENSE PLATE IMAGES

• US 20130272579A1
• Filed: 04/17/2012
• Published: 10/17/2013
• Est. Priority Date: 04/17/2012
• Status: Active Grant

• Current Assignee: Conduent Business Services, LLC (Conduent, Inc.)
• Original Assignee: Xerox Corporation (Xerox Holdings Corp.)
• Inventors: Burry, Aaron Michael, Fillion, Claude, Kozitsky, Vladimir, Bala, Raja, Fan, Zhigang
• Granted Patent: US 9,760,789 B2
• Time in Patent Office: Days
• US Class Current: 382/105
• CPC Class Codes: G06V 20/625 License plates
• CPC Class Codes: G06V 20/63 Scene text, e.g. street names

Abstract

A method, system, and computer-usable tangible storage device for robustly cropping and accurately recognizing license plates to account for noise sources and interfering artifacts are disclosed. License plate images and sub-images can be tightly cropped utilizing an image-based classifier and gradient-based cropping. An image-based classifier can identify the location of valid characters within the image. Because of a number of noise sources, such as, for example, residual plate rotation and shear in the characters within the image, the image-based classifier performs a “rough” identification of the image boundaries. An additional processing step utilizing gradient-based cropping is performed to fine-tune the license plate image boundaries. Gradient-based cropping eliminates unwanted border artifacts that could substantially impact the segmentation and license plate character recognition results.

Claims

1. A method for robust cropping of a license plate image, said method comprising:

identifying a first detail of said license plate image utilizing an image-based classifier;

identifying a second detail of said license plate image utilizing a gradient-based classifier; and

utilizing said identified first detail and said identified second detail to accurately recognize and identify a license plate in said license plate image.

2. The method of claim 1 wherein identifying said first detail of said license plate image utilizing said image-based classifier further comprises roughly identifying a location of a character on said license plate image.

3. The method of claim 1 wherein said first detail comprises at least one of:

a character, a valid character, a symbol, a special symbol, a logo, a license plate border, a license plate border with poor contrast, no detectable plate border, and extraneous edge information outside of said license plate.

4. The method of claim 1 wherein said first detail comprises a noise source, said noise source comprising at least one of:

residual plate rotation, shear in a character within said license plate image, a heavy shadow, non-uniform illumination, challenging optical geometry, tilt, shear, projective distortion, a license plate frame, a license plate sticker, a license plate frame that covers or partially covers a character on said license plate, a license plate sticker that covers or partially covers a character on said license plate, partial occlusion of a character, a trailer hitch ball, poor contrast, general license plate image noise, and salt and pepper noise.

5. The method of claim 1 wherein identifying said second detail of said license plate image utilizing said gradient-based classifier further comprises fine-tuning said second detail for robustness for a license plate edge effect, a license plate frame, a license plate border artifact, and excessive character shear to improve segmentation and character recognition results.

6. The method of claim 5 wherein said second detail comprises at least one of:

a character, a valid character, a symbol, a special symbol, a logo, a license plate border, a license plate border with poor contrast, no detectable plate border, and extraneous edge information outside of said license plate.

7. The method of claim 1 further comprising:

tightly cropping said license plate image to create a license plate sub-image utilizing said image-based classifier and said gradient-based classifier, wherein tightly cropping said license plate image comprises horizontally cropping said license plate image to account for a noise source and an interfering artifact in said license plate image;

at least one of removing a logo from said license plate sub-image, removing a symbol from said license plate sub-image, and flagging an artifact from said license plate sub-image before submitting said license plate sub-image for character recognition;

identifying a license plate edge artifact utilizing said gradient-based classifier on said license plate sub-image to accurately determine a license plate border artifact for removal without incorrectly discarding a valid license plate character;

capturing said license plate image; and

localizing said license plate image.

8. The method of claim 1 further comprising:

extracting a tight bounding box license plate sub-image to reduce noise present in said license plate image;

extracting a license plate character from said tight bounding box license plate sub-image via at least one of;

tilt removal, vertical cropping, and horizontal cropping; and

segmenting said license plate character in said tight bounding box license plate sub-image for a robust, highly accurate, computationally efficient solution to recognize said license plate character and identify said license plate.

9. The method of claim 1 further comprising identifying said first detail and said second detail utilizing a Sparse Network of Winnows classifier, wherein said Sparse Network of Winnows classifier accounts for at least one of:

varying distances between characters on said license plate, a license plate border, a symbol, a special symbol, and logos on said license plate.

10. A system for robust cropping of a license plate image, comprising:

a processor;

a data bus coupled to said processor; and

a computer-usable tangible storage device storing computer program code, said computer program code comprising program instructions executable by said processor for robust cropping of a license plate image, said program instructions comprising;

program instructions to identify a first detail of said license plate image utilizing an image-based classifier;

program instructions to identify a second detail of said license plate image utilizing a gradient-based classifier; and

program instructions to utilize said identified first detail and said identified second detail to accurately recognize and identify a license plate in said license plate image.

11. The system of claim 10 wherein said program instruction to identify said first detail of said license plate image utilizing said image-based classifier further comprises program instructions to roughly identify a location of a character on said license plate image, wherein said first detail comprises at least one of:

a character, a valid character, a symbol, a special symbol, a logo, a license plate border, a license plate border with poor contrast, no detectable plate border, and extraneous edge information outside of said license plate.

12. The system of claim 10 wherein said first detail comprises a noise source, said noise source comprising at least one of:

residual plate rotation, shear in a character within said license plate image, a heavy shadow, non-uniform illumination, challenging optical geometry, tilt, shear, projective distortion, a license plate frame, a license plate sticker, a license plate frame that covers or partially covers a character on said license plate, a license plate sticker that covers or partially covers a character on said license plate, partial occlusion of a character, a trailer hitch ball, poor contrast, general license plate image noise, and salt and pepper noise.

13. The system of claim 10 wherein said program instructions to identify said second detail of said license plate image utilizing said gradient-based classifier further comprises program instructions to fine-tune said second detail for robustness for a license plate edge effect, a license plate frame, a license plate border artifact, and excessive character shear to improve segmentation and character recognition results, wherein said second detail comprises at least one of:

a character, a valid character, a symbol, a special symbol, a logo, a license plate border, a license plate border with poor contrast, no detectable plate border, and extraneous edge information outside of said license plate.

14. The system of claim 10 further comprising:

program instructions to tightly crop said license plate image to create a license plate sub-image utilizing said image-based classifier and said gradient-based classifier, wherein tightly cropping said license plate image comprises horizontally cropping said license plate image to account for a noise source and an interfering artifact in said license plate image;

at least one of program instructions to remove a logo from said license plate sub-image, program instructions to remove a symbol from said license plate sub-image, and program instructions to flag an artifact from said license plate sub-image before submitting said license plate sub-image for character recognition;

program instructions to identify a license plate edge artifact utilizing said gradient-based classifier on said license plate sub-image to accurately determine a license plate border artifact for removal without incorrectly discarding a valid license plate character;

program instructions to capture said license plate image; and

program instructions to localize said license plate image.

15. The system of claim 10 further comprising:

program instructions to extract a tight bounding box license plate sub-image to reduce noise present in said license plate image;

program instructions to extract a license plate character from said tight bounding box license plate sub-image via at least one of;

tilt removal, vertical cropping, and horizontal cropping; and

program instructions to segment said license plate character in said tight bounding box license plate sub-image for a robust, highly accurate, computationally efficient solution to recognize said license plate character and identify said license plate.

16. The system of claim 10 further comprising program instructions to identify said first detail and said second detail utilizing a Sparse Network of Winnows classifier, wherein said Sparse Network of Winnows classifier accounts for at least one of:

varying distances between characters on said license plate, a license plate border, a symbol, a special symbol, and logos on said license plate.

17. A computer-usable tangible storage device storing computer program code, said computer program code comprising program instructions executable by a processor for robust cropping of a license plate image, said program instructions comprising:

program instructions to identify a first detail of said license plate image utilizing an image-based classifier;

program instructions to identify a second detail of said license plate image utilizing a gradient-based classifier; and

program instructions to utilize said identified first detail and said identified second detail to accurately recognize and identify a license plate in said license plate image.

18. The computer-usable tangible storage device of claim 17 wherein said program instruction to identify said first detail of said license plate image utilizing said image-based classifier further comprises program instructions to roughly identify a location of a character on said license plate image, wherein said first detail comprises at least one of:

a character, a valid character, a symbol, a special symbol, a logo, a license plate border, a license plate border with poor contrast, no detectable plate border, and extraneous edge information outside of said license plate, wherein said first detail comprises a noise source, said noise source comprising at least one of;

residual plate rotation, shear in a character within said license plate image, a heavy shadow, non-uniform illumination, challenging optical geometry, tilt, shear, projective distortion, a license plate frame, a license plate sticker, a license plate frame that covers or partially covers a character on said license plate, a license plate sticker that covers or partially covers a character on said license plate, partial occlusion of a character, a trailer hitch ball, poor contrast, general license plate image noise, and salt and pepper noise.

19. The computer-usable tangible storage device of claim 17 wherein said program instructions to identify said second detail of said license plate image utilizing said gradient-based classifier further comprises program instructions to fine-tune said second detail for robustness for a license plate edge effect, a license plate frame, a license plate border artifact, and excessive character shear to improve segmentation and character recognition results, wherein said second detail comprises at least one of:

a character, a valid character, a symbol, a special symbol, a logo, a license plate border, a license plate border with poor contrast, no detectable plate border, and extraneous edge information outside of said license plate.

20. The computer-usable tangible storage device of claim 17 further comprising:

program instructions to tightly crop said license plate image to create a license plate sub-image utilizing said image-based classifier and said gradient-based classifier, wherein tightly cropping said license plate image comprises horizontally cropping said license plate image to account for a noise source and an interfering artifact in said license plate image;

at least one of program instructions to remove a logo from said license plate sub-image, program instructions to remove a symbol from said license plate sub-image, and program instructions to flag an artifact from said license plate sub-image before submitting said license plate sub-image for character recognition;

program instructions to identify a license plate edge artifact utilizing said gradient-based classifier on said license plate sub-image to accurately determine a license plate border artifact for removal without incorrectly discarding a valid license plate character;

program instructions to capture said license plate image;

program instructions to localize said license plate image;

program instructions to extract a tight bounding box license plate sub-image to reduce noise present in said license plate image;

program instructions to extract a license plate character from said tight bounding box license plate sub-image via at least one of;

tilt removal, vertical cropping, and horizontal cropping;

program instructions to segment said license plate character in said tight bounding box license plate sub-image for a robust, highly accurate, computationally efficient solution to recognize said license plate character and identify said license plate; and

program instructions to identify said first detail and said second detail utilizing a Sparse Network of Winnows classifier, wherein said Sparse Network of Winnows classifier accounts for at least one of;

varying distances between characters on said license plate, a license plate border, a symbol, a special symbol, and logos on said license plate.