Method and system for detecting and recognizing text in images

• US 8,977,072 B1
• Filed: 12/13/2012
• Issued: 03/10/2015
• Est. Priority Date: 01/23/2008
• Status: Active Grant

• Current Assignee: A9.com Incorporated (Amazon.com, Inc.)
• Original Assignee: A9.com Incorporated (Amazon.com, Inc.)
• Inventors: Manmatha, Raghavan, Ruzon, Mark A.
• Primary Examiner(s): ENTEZARI, MICHELLE M
• Application Number:
  US13/713,776
• Time in Patent Office: 817 Days
• Field of Search: 382/229
• US Class Current: 382/283
• CPC Class Codes: G06F 18/25 Fusion techniques
• CPC Class Codes: G06V 20/62 Text, e.g. of license plates, overlay texts or captions on TV images
• CPC Class Codes: G06V 30/10 Character recognition
• CPC Class Codes: G06V 30/15 Cutting or merging image elements, e.g. region growing, watershed or clustering-based techniques
• CPC Class Codes: G06V 30/153 using recognition of characters or words
• CPC Class Codes: G06V 30/155 Removing patterns interfering with the pattern to be recognised, such as ruled lines or underlines
• CPC Class Codes: G06V 30/162 Quantising the image signal
• CPC Class Codes: G06V 30/164 Noise filtering
• CPC Class Codes: G06V 30/224 of printed characters having additional code marks or containing code marks

Abstract

Various embodiments of the present invention relate to a method, system and computer program product for detecting and recognizing text in the images captured by cameras and scanners. First, a series of image-processing techniques is applied to detect text regions in the image. Subsequently, the detected text regions pass through different processing stages that reduce blurring and the negative effects of variable lighting. This results in the creation of multiple images that are versions of the same text region. Some of these multiple versions are sent to a character-recognition system. The resulting texts from each of the versions of the image sent to the character-recognition system are then combined to a single result, wherein the single result is detected text.

Claims

1. A computer-implemented method comprising:

under the control of one or more computer systems configured with executable instructions,

obtaining an input image that includes at least one image variation;

filtering and segmenting the input image;

creating a mask of the image from a plurality of bounding boxes, each bounding box at least partially enclosing a connected component;

intersecting the filtered and segmented input image with the mask to produce a first output image;

separately processing the filtered and segmented input image to create a binary chip for each of the plurality of bounding boxes to produce a second output image;

separately recognizing text in the first output image and each binary chip of the second output image using an optical character recognizer; and

combining the separately recognized text from the first output image and the second output image to produce a single output.

2. The computer-implemented method of claim 1, further comprising:

separately processing the input image to produce a third output image using a different processing technique than used to produce the first output image and the second output image, wherein the recognized text from the first output image, the second output image, and the third output image are combined using a majority vote process to select portions from the first output image, the second output image, and the third output image.

3. The computer-implemented method of claim 1, wherein analyzing the separately recognized text from the first output image and the second output image comprises taking a logical OR of the first output image and second output image.

4. The computer-implemented method of claim 1, wherein the at least one image variation includes at least one of noise, blur, or a lighting variation.

5. A computer-implemented method comprising:

under the control of one or more computer systems configured with executable instructions,

processing an input image using a first process to produce a first output image, the first output image including a plurality of bounding boxes, each bounding box at least partially enclosing a connected component;

separately processing the first output using a second process to generate a binary chip of each bounding box to produce a second output image;

recognizing text in the first output image and in the binary chip of each bounding box; and

comparing the recognized text from the plurality of bounding boxes of the first output image to corresponding binary chips of the second output image to produce a consensus output for text in the input image.

6. The computer-implemented method of claim 5, wherein the input image includes one or more detected text regions and wherein each binary chip includes a detected text region.

7. The computer-implemented method of claim 5, further comprising:

forming a gray-level chip from the input image; and

processing the formed gray-level chip to form each binary chip.

8. The computer-implemented method of claim 5, further comprising:

separately processing the input image to produce a third output image using a different processing technique used to produce the first output image and the second output image, wherein the recognized text from the first output image, the second output image, and the third output image are combined using a majority vote process to select portions from the first output image, the second output image, and the third output image.

9. The computer-implemented method of claim 5, wherein the first output image and the second output image include text and wherein the consensus output includes text from the first output image and the second output image.

10. The computer-implemented method of claim 5, wherein the input image includes at least one of noise, blur, or a lighting variation.

11. The computer-implemented method of claim 10, wherein processing the input image includes reducing at least one of the noise, blur, or the lighting variation of the input image.

12. A computing system, comprising:

at least one processor; and

memory including instructions that, when executed by the at least one processor, cause the computing system to;

process an input image using a first process to produce a first output image, the first output image including a plurality of bounding boxes, each bounding box at least partially enclosing a connected component;

separately process the first output image using a second process to generate a binary chip of each bounding box to produce a second output image;

recognize text in the first output image and in the binary chip of each bounding box; and

compare the recognized text from the plurality of bounding boxes of the first output image to corresponding binary chips of the second output image to produce a consensus output for text in the input image.

13. The computing system of claim 12, wherein the instructions, when executed by the at least one processor, further causes the computing system to:

detect one or more text regions by at least filtering and segmenting the input image and intersecting the filtered and segmented input image with a mask created from a plurality of bounding boxes, each bounding box at least partially enclosing a connected component, each connected component including a plurality of pixels comprising the image and connected based at least in part on a predetermined pixel intensity and predefined distance between the pixels.

14. The computing system of claim 12, wherein the instructions, when executed by the at least one processor, further causes the computing system to:

determine one or more text portions of the input image based at least in part upon whether pixel values for pixels in a respective region are above or below a threshold value for a respective region; and

connect text portions using at least one character connection algorithm.

15. The computing system of claim 12, wherein analyzing the recognized text from the first output image and the second output image comprises taking a logical OR of the first output and second output.

16. A non-transitory computer-readable storage medium storing instructions that, when executed by a processor, cause the processor to:

obtain an input image that includes at least one image defect;

process the input image using a first processing technique to produce a first output image, the first output image including a plurality of bounding boxes, each bounding box at least partially enclosing a connected component;

separately process the first output image using a second process to generate a binary chip of each bounding box to produce a second output image;

recognize text in the first output image and in the binary chip of each bounding box; and

compare the recognized text from the plurality of bounding boxes of the first output image to corresponding binary chips of the second output image to produce a consensus output for text in the input image.

17. The non-transitory computer-readable storage medium of claim 16, wherein the instructions that, when executed by the processor, further cause the processor to:

separately processing the input image to produce a third output image using a different processing technique used to produce the first output image and the second output image, wherein the recognized text from the first output, the second output image, and the third output image are combined using a majority vote process to select portions from the first output image, the second output image, and the third output image.

18. The non-transitory computer-readable storage medium of claim 16, wherein processing the input image includes reducing at least one of noise, blur, or lighting variation of the input image.

19. The non-transitory computer-readable storage medium of claim 16, wherein analyzing the recognized text from the first output image and the second output image comprises taking a logical OR of the first output image and second output image.