Robust cropping of license plate images

• US 9,760,789 B2
• Filed: 04/17/2012
• Issued: 09/12/2017
• Est. Priority Date: 04/17/2012
• Status: Active Grant

• Current Assignee: Conduent Business Services, LLC (Conduent, Inc.)
• Original Assignee: Conduent Business Services, LLC (Conduent, Inc.)
• Inventors: Burry, Aaron Michael, Fillion, Claude, Kozitsky, Vladimir, Bala, Raja, Fan, Zhigang
• Primary Examiner(s): Koziol, Stephen R
• Assistant Examiner(s): SCHWARTZ, RAPHAEL M
• Application Number:
  US13/448,976
• Publication Number:
  US 20130272579A1
• Time in Patent Office: 1,974 Days
• CPC Class Codes: G06V 20/625 License plates
• CPC Class Codes: G06V 20/63 Scene text, e.g. street names

Abstract

A method, system, and computer-usable tangible storage device for robustly cropping and accurately recognizing license plates to account for noise sources and interfering artifacts are disclosed. License plate images and sub-images can be tightly cropped utilizing an image-based classifier and gradient-based cropping. An image-based classifier can identify the location of valid characters within the image. Because of a number of noise sources, such as, for example, residual plate rotation and shear in the characters within the image, the image-based classifier performs a “rough” identification of the image boundaries. An additional processing step utilizing gradient-based cropping is performed to fine-tune the license plate image boundaries. Gradient-based cropping eliminates unwanted border artifacts that could substantially impact the segmentation and license plate character recognition results.

Claims

1. A method for cropping of a license plate image, said method comprising:

identifying a first plurality of details of said license plate image utilizing an image-based classifier that serves as a valid character detector to indicate a subset of said license plate image where character images likely reside wherein said first plurality of details comprise at least one of;

a character, a valid license plate character, a symbol, a logo, a license plate border, a license plate border with poor contrast, and extraneous edge information outside of said license plate;

resizing said subset of said license plate image where character images likely reside to a standard vertical dimension associated with a classifier template;

identifying candidate character locations of said license plate image and determining if said candidate character locations exceed a minimum number of characters;

performing a first cropping of said license plate image including removing tilt from and vertically cropping said license plate image;

identifying a second detail of said license plate image utilizing a gradient-based classifier on said candidate character locations and performing a second cropping of said first cropping of said license plate image based on detected character locations, further comprising a horizontal cropping via projective segmentation of said license plate image based on said detected character locations, if said candidate character locations exceed said minimum number of characters; and

utilizing said identified first plurality of details and said identified second detail to recognize and identify a license plate in said license plate image;

further comprising identifying said first plurality of details and said second detail utilizing a Sparse Network of Winnows classifier, wherein said Sparse Network of Winnows classifier accounts for the following;

varying distances between characters on said license plate, a license plate border, a symbol, and logos on said license plate.

2. The method of claim 1 wherein said plurality of details comprise a at least one noise source, said at least one noise source comprising at least one of:

plate rotation, shear in a character within said license plate image, a shadow, non-uniform illumination, tilt, shear, projective distortion, a license plate frame, a license plate sticker, a license plate frame that covers or partially covers a character on said license plate, a license plate sticker that covers or partially covers a character on said license plate, partial occlusion of a character, a trailer hitch ball, and salt and pepper noise.

3. The method of claim 1 wherein identifying said second detail of said license plate image utilizing said gradient-based classifier further comprises fine-tuning said second detail for robustness for a license plate frame, a license plate border artifact, and character shear to improve segmentation and character recognition results.

4. The method of claim 3 wherein said second detail comprises a plurality of details comprising at least one of:

a character, a valid license plate character, a symbol, a license plate special symbol, a logo, a license plate border, a license plate border with poor contrast, no detectable plate border, and extraneous edge information outside of said license plate.

5. The method of claim 1 further comprising:

cropping said license plate image with said image-based classifier and said gradient-based classifier to create a license plate sub-image, wherein cropping said license plate image comprises horizontally cropping said license plate image to account for a noise source and an interfering artifact in said license plate image;

at least one of removing a logo from said license plate sub-image, removing a symbol from said license plate sub-image, and flagging an artifact from said license plate sub-image before submitting said license plate sub-image for character recognition;

identifying a license plate edge artifact utilizing said gradient-based classifier on said license plate sub-image to determine a license plate border artifact for removal without incorrectly discarding a valid license plate character;

capturing said license plate image; and

localizing said license plate image.

6. The method of claim 1 further comprising:

extracting a bounding box license plate sub-image to reduce noise present in said license plate image;

extracting a license plate character from said bounding box license plate sub-image via the following;

tilt removal, vertical cropping, and horizontal cropping; and

segmenting said license plate character in said bounding box license plate sub-image to recognize said license plate character and identify said license plate.

7. A system for cropping of a license plate image, comprising:

a processor that communicates electronically with a memory; and

a non-transitory computer-usable medium embodying computer program code stored in said memory, said non-transitory computer-usable medium capable of communicating with said processor, said computer program code comprising instructions executable by said processor and configured for;

identifying a first plurality of details of said license plate image utilizing an image-based classifier that serves as a valid character detector to indicate a subset of said license plate image where character images likely reside wherein said first plurality of details comprise at least one of;

a character, a valid license plate character, a symbol, a special license plate symbol, a logo, a license plate border, a license plate border with poor contrast, and extraneous edge information outside of said license plate;

resizing said subset of said license plate image where character images likely reside to a standard vertical dimension associated with a classifier template;

identifying candidate character locations of said license plate image and determining if said candidate character locations exceed a minimum number of characters;

performing a first cropping of said license plate image including removing tilt from and vertically cropping said license plate image;

identifying a second detail of said license plate image utilizing a gradient-based classifier on said candidate character locations and performing a second cropping of said first cropping of said license plate image based on detected character locations, further comprising a horizontal cropping via projective segmentation of said license plate image based on said detected character locations, if said candidate character locations exceed said minimum number of characters; and

utilizing said identified first plurality of details and said identified second detail to recognize and identify a license plate in said license plate image;

further comprising identifying said first plurality of details and said second detail utilizing a Sparse Network of Winnows classifier, wherein said Sparse Network of Winnows classifier accounts for the following;

varying distances between characters on said license plate, a license plate border, a symbol, and logos on said license plate.

8. The system of claim 7 wherein said instructions are further configured to fine-tune said second detail for robustness for a license plate frame, a license plate border artifact, and character shear to improve segmentation and character recognition results, wherein said second detail comprises a plurality of details comprising at least one of:

a character, a valid license plate character, a symbol, a special license plate symbol, a logo, a license plate border, a license plate border with poor contrast, no detectable plate border, and extraneous edge information outside of said license plate.

9. The system of claim 7 wherein said instructions are further configured for:

cropping said license plate image to create a license plate sub-image utilizing said image-based classifier and said gradient-based classifier, wherein cropping said license plate image comprises horizontally cropping said license plate image to account for a noise source and an interfering artifact in said license plate image;

removing a logo from said license plate sub-image, program instructions to remove a symbol from said license plate sub-image, and program instructions to flag an artifact from said license plate sub-image before submitting said license plate sub-image for character recognition;

identifying a license plate edge artifact utilizing said gradient-based classifier on said license plate sub-image to determine a license plate border artifact for removal without incorrectly discarding a valid license plate character;

capturing said license plate image; and

localizing said license plate image.

10. The system of claim 7 wherein said instructions are further configured for:

extracting a bounding box license plate sub-image to reduce noise present in said license plate image;

extracting a license plate character from said bounding box license plate sub-image via at least one of;

tilt removal, vertical cropping, and horizontal cropping; and

segmenting said license plate character in said bounding box license plate sub-image to recognize said license plate character and identify said license plate.