Optimizing pre-processing times for faster response

• US 9,262,689 B1
• Filed: 12/18/2013
• Issued: 02/16/2016
• Est. Priority Date: 12/18/2013
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Sikka, Avnish, Ramos, David Paul, Hart, Matthew Daniel, Liu, Yue, McConville, Emilie Noelle
• Primary Examiner(s): Couso, Yon
• Application Number:
  US14/133,347
• Time in Patent Office: 790 Days
• Field of Search: 382/164
• US Class Current: 1/1
• CPC Class Codes: G06V 20/62 Text, e.g. of license plates, overlay texts or captions on TV images
• CPC Class Codes: G06V 30/10 Character recognition

Abstract

Embodiments of the subject technology provide for determining a region of a first acquired image based at least on a viewing mode and a set of respective positions of graphical elements to decrease the pre-processing time and perceived latency for the first image. One or more regions of text in the first image are detected, and a set of regions of text that overlap with the region of the image is determined and pre-processed. The subject technology may then pre-process an entirety of a subsequent image (e.g., to pick up missing text from the region of the first image). Thus, additional OCR results may be provided to the user by using the subsequent image(s) and merging subsequent results with previous results from the first image.

Claims

1. A computer-implemented method comprising:

acquiring an image via a camera of a mobile computing device;

displaying a representation of the image in a user interface of the mobile computing device;

determining a viewing mode of the mobile computing device;

displaying graphical elements in the user interface, the graphical elements overlapping a portion of the representation of the image;

determining positions of the graphical elements in the user interface;

determining, using the positions, an area of the representation that is not overlapped by graphical elements;

determining a portion of the image corresponding to the area of the representation;

identifying at least one region of text within the portion; and

performing a pre-processing operation on the at least one region of text to prepare the image for optical character recognition.

2. The computer-implemented method of claim 1, further comprising:

providing OCR results for the at least one region of text for display on the mobile computing device.

3. The computer-implemented method of claim 1, further comprising:

adjusting a size of the at least one region of text.

4. The computer-implemented method of claim 1, wherein the pre-processing operation comprises at least one of detecting a set of glyphs in the at least one region of text, removing one or more non-text features in the at least one region of text, and performing binarization of the at least one region of text.

5. The computer-implemented method of claim 2, further comprising:

receiving second OCR results from OCR performed on an acquired second image;

merging the OCR results and the second OCR results into third OCR results; and

providing the third OCR results for display on the mobile computing device.

6. The computer-implemented method of claim 1, further comprising determining the area of the representation based at least on a language of a user.

7. A system comprising:

a display;

a processor; and

a memory device including instructions that, when executed by the processor, cause the processor to perform a set of operations comprising;

acquiring an image by the system;

displaying a representation of the image on the display;

displaying a graphical element on the display, the graphical element overlapping the representation;

determining an area of the representation that is not overlapped by the graphical element, wherein the area includes a center portion of the representation;

determining a portion of the image corresponding to the area of the representation;

determining at least one text region that at least partially overlaps with the portion; and

performing a pre-processing operation on the at least one text region before performing at least one recognition process.

8. The system of claim 7, wherein the pre-processing operation comprises at least one of removing one or more non-text features in the at least one text region, and performing binarization of the at least one text region by changing the at least one text region to only include black and white colors.

9. The system of claim 7, wherein the at least one recognition process comprises optical character recognition.

10. The system of claim 7, wherein:

determining an area of the representation is based at least on a viewing mode of the system, the viewing mode of the system comprising either a portrait viewing mode or a landscape viewing mode, and

the area comprises at least a first set of distances away from a top border or a bottom border of the representation when the viewing mode is in the portrait viewing mode, and the area comprises at least a second set of distances away from a left border or a right border when the viewing mode is in the landscape viewing mode.

11. The system of claim 7, wherein:

determining an area of the representation is based at least on a position of the graphical element.

12. The system of claim 7, further comprising determining the area of the representation is based at least on a language of a user.

13. The system of claim 7, further comprising determining the area of the representation is based at least on a set of distances away from at least one of a top border, bottom border, left border and right border of the representation.

14. The system of claim 7, wherein the memory device includes further instructions that, when executed by the processor, cause the processor to perform a further set of operations comprising:

providing OCR results for the at least one text region for display on the display.

15. The system of claim 7, wherein the memory device includes further instructions that, when executed by the processor, cause the processor to perform a further set of operations comprising:

adjusting a size of the at least one text region.

16. The system of claim 15, wherein adjusting a size of the at least one text region comprises:

resizing the size of the at least one text region to match a height or width of a respective size of the area of the representation.

17. The system of claim 14, wherein the memory device includes further instructions that, when executed by the processor, cause the processor to perform a further set of operations comprising:

receiving second OCR results from OCR performed on an acquired second image;

merging the OCR results and the second OCR results into third OCR results; and

providing the third OCR results for display on the system.

18. A non-transitory computer-readable medium including instructions stored therein that, when executed by at least one computing device, cause the at least one computing device to perform a set of operations comprising:

acquiring an image by the at least one computing device;

displaying a representation of the image in a display of the at least one computing device;

displaying a graphical element on the display, the graphical element overlapping the representation;

region of the determining an area of the representation that is not overlapped by the graphical element, wherein the area includes a center portion of the representation;

determining a portion of the image corresponding to the area of the representation

determining at least one text region that at least partially overlaps with the portion; and

performing a pre-processing operation on the at least one text region before performing at least one recognition process.

19. The non-transitory computer-readable medium of claim 18, wherein the non-transitory computer-readable medium includes further instructions stored therein that, when executed by the at least one computing device, cause the at least one computing device to perform a further set of operations comprising:

providing OCR results for the at least one text region for display on the at least one computing device.

20. The non-transitory computer-readable medium of claim 19, wherein the image region after the set of operations comprising:

receiving second OCR results from OCR performed on an acquired second image;

merging the OCR results and the second OCR results into third OCR results; and

providing the third OCR results for display on the least one computing device.