Image pre-processing for reducing consumption of resources

• US 8,965,117 B1
• Filed: 12/17/2013
• Issued: 02/24/2015
• Est. Priority Date: 12/17/2013
• Status: Expired due to Fees

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Rybakov, Oleg, Yuan, Chang, Lish, Christopher John, Jia, Junxiong, Nambiar, Rakesh Madhavan, Benitez, Matias Omar Gregorio
• Primary Examiner(s): Bayat, Ali
• Application Number:
  US14/109,204
• Time in Patent Office: 434 Days
• Field of Search: 382/162, 382/164, 382/165, 382/190, 382/197, 382/173, 382/170, 382/195, 382/175, 382/305, 348/453, 235/470, 235/454
• US Class Current: 382/165
• CPC Class Codes: G06F 18/24 Classification techniques
• CPC Class Codes: G06F 18/285 Selection of pattern recognition techniques, e.g. of classifiers in a multi-classifier system
• CPC Class Codes: G06T 5/70 Denoising; Smoothing
• CPC Class Codes: G06V 30/19113 Selection of pattern recognition techniques, e.g. of classifiers in a multi-classifier system
• CPC Class Codes: G06V 30/224 of printed characters having additional code marks or containing code marks
• CPC Class Codes: G06V 30/2247 Characters composed of bars, e.g. CMC-7
• CPC Class Codes: G06V 30/413 Classification of content, e.g. text, photographs or tables

Abstract

Embodiments of the subject technology provide methods and systems of image pre-processing for improving the accuracy of optical character recognition (OCR) and reducing the power consumption on a given computing device (e.g., mobile computing device). The subject technology, in some examples, classifies an image received from a camera of a mobile computing device into one or more classes: 1) normal background, 2) textured background, 3) image with text, 4) image with barcode, 5) image with QR code, and/or 6) image with clutter or “garbage.” Based on the classes associated with the image, the subject technology may forgo certain image processing operations, when the image is not associated with a particular class, in order to save resources (e.g., CPU cycles, battery power, memory usage, etc.) on the mobile computing device.

Claims

1. A computer-implemented method comprising:

acquiring an input image via a camera of a mobile computing device;

determining an amount of texture in the input image;

performing a smoothing operation on the input image to produce a smoothed image;

creating a binarized image from the smoothed image;

identifying a plurality of contours using the binarized image, wherein each contour of the plurality of contours corresponds to a glyph of a plurality of glyphs;

computing a plurality of features for a first glyph of the plurality of glyphs, wherein the features are computed using at least one of the input image, the smoothed image, or the binarized image;

computing a plurality of scores for the first glyph using at least a portion of the plurality of features, wherein each score of the plurality of scores corresponds to a type of glyph;

creating a group of glyphs comprising the first glyph using the plurality of scores for the first glyph;

determining a type for the group of glyphs, wherein the type comprises text, barcode, or QR code; and

causing a recognition operation to be performed using at least a portion of the input image, at least a portion of the smoothed image, or at least a portion of the binarized image, wherein the recognition operation corresponds to the type of the group.

2. The computer-implemented method of claim 1, wherein performing the smoothing operation comprises:

performing a downscaling operation on the input image to produce a downscaled image; and

performing an upscaling operation on the downscaled image to produce an upscaled image.

3. The computer-implemented method of claim 1, wherein determining an amount of texture in the input image comprises:

determining a plurality of intersections between a row of the image and a smoothed version of the row of the image;

determining a plurality of amplitudes corresponding to differences between the row of the image and the smoothed version of the row of the image;

using the plurality of amplitudes to determine an amount of texture in the image.

4. The computer-implemented method of claim 1, wherein a first contour corresponds to the first glyph and wherein the plurality of features comprises one or more of a gradient of the first contour, a perimeter of the first contour, a size of the first glyph, a color of a portion of the input image corresponding to the first glyph, or a stroke width.

5. A system comprising:

a processor; and

a memory device including instructions that, when executed by the processor, cause the processor to;

receive an image;

create a binarized image from the image;

identify a plurality of contours using the binarized image, wherein each contour of the plurality of contours corresponds to a glyph of a plurality of glyphs;

compute a plurality of features for a first glyph of the plurality of glyphs;

compute a plurality of scores for the first glyph using at least a portion of the plurality of features, wherein each score of the plurality of scores corresponds to a type of glyph;

create a group of glyphs comprising the first glyph using the plurality of scores for the first glyph;

determine a type for the group of glyphs;

cause a recognition operation to be performed for the group, wherein the recognition operation corresponds to the type of the group.

6. The system of claim 5, wherein the instructions further cause the processor to:

determine whether texture is present in the image; and

smooth the image to reduce the texture.

7. The system of claim 6, wherein the instructions further cause the processor to determine whether texture is present in the image by:

determining a plurality of intersections between a row of the image and a smoothed version of the row of the image;

determining a plurality of amplitudes corresponding to differences between the row of the image and the smoothed version of the row of the image;

using the plurality of amplitudes to determine whether texture is present in the image.

8. The system of claim 7, wherein using the plurality of amplitudes to determine whether texture is present in the image comprises:

creating a distribution of the plurality of amplitudes; and

comparing a sum of at least a portion of the distribution to a threshold.

9. The system of claim 8, wherein the instructions further cause the processor to:

create a second group of glyphs;

determine a second type for the second group of glyphs;

determine a first priority for the first group and a second priority for the second group, wherein the first priority is higher than the second priority; and

cause a second recognition operation to be performed, wherein the second recognition operation corresponds to the second type, and wherein the second recognition operation begins after the recognition operation begins.

10. The system of claim 5, wherein a first contour corresponds to the first glyph and wherein the plurality of features comprises one or more of a gradient of the first contour, a perimeter of the first contour, a size of the first glyph, a color of a portion of the input image corresponding to the first glyph, or a stroke width.

11. The system of claim 5, wherein the instructions further cause the processor to:

compute a plurality of scores for the first glyph by using a model for each type of glyph and using a state vector machine or a classification forest.

12. The system of claim 5, wherein the instructions further cause the processor to:

create a group of glyphs by using an unsupervised clustering procedure.

13. A non-transitory computer-readable medium including instructions stored therein that, when executed by at least one computing device, cause the at least one computing device to:

receive an image;

create a binarized image from the image;

identify a plurality of contours using the binarized image, wherein each contour of the plurality of contours corresponds to a glyph of a plurality of glyphs;

compute a plurality of features for a first glyph of the plurality of glyphs;

compute a plurality of scores for the first glyph using at least a portion of the plurality of features, wherein each score of the plurality of scores corresponds to a type of glyph;

create a group of glyphs comprising the first glyph using the plurality of scores for the first glyph;

determine a type for the group of glyphs; and

cause a recognition operation to be performed for the group, wherein the recognition operation corresponds to the type of the group.

14. The non-transitory computer-readable medium of claim 13, wherein the instructions further cause the at least one computing device to:

determine whether texture is present in the image; and

smooth the image to reduce the texture.

15. The non-transitory computer-readable medium of claim 14, wherein the instructions further cause the at least one computing device to determine whether texture is present in the image by:

determining a plurality of intersections between a row of the image and a smoothed version of the row of the image;

determining a plurality of amplitudes corresponding to differences between the row of the image and the smoothed version of the row of the image;

using the plurality of amplitudes to determine whether texture is present in the image.

16. The non-transitory computer-readable medium of claim 15, wherein using the plurality of amplitudes to determine whether texture is present in the image comprises:

creating a distribution of the plurality of amplitudes; and

comparing a sum of at least a portion of the distribution to a threshold.

17. The non-transitory computer-readable medium of claim 16, wherein the instructions further cause the at least one computing device to:

create a second group of glyphs;

determine a second type for the second group of glyphs;

determine a first priority for the first group and a second priority for the second group, wherein the first priority is higher than the second priority; and

cause a second recognition operation to be performed, wherein the second recognition operation corresponds to the second type, and wherein the second recognition operation begins after the recognition operation begins.

18. The non-transitory computer-readable medium of claim 13, wherein a first contour corresponds to the first glyph and wherein the plurality of features comprises one or more of a gradient of the first contour, a perimeter of the first contour, a size of the first glyph, a color of a portion of the input image corresponding to the first glyph, or a stroke width.

19. The non-transitory computer-readable medium of claim 13, wherein the instructions further cause the at least one computing device to:

compute a plurality of scores for the first glyph by using a model for each type of glyph and using a state vector machine or a classification forest.

20. The non-transitory computer-readable medium of claim 13, wherein the instructions further cause the at least one computing device to:

create a group of glyphs by using an unsupervised clustering procedure.