Adaptive thresholding for image recognition

• US 8,606,011 B1
• Filed: 06/07/2012
• Issued: 12/10/2013
• Est. Priority Date: 06/07/2012
• Status: Expired due to Fees

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Ivanchenko, Volodymyr V., Heller, Geoffrey Scott, Suplee, Richard Howard III, Bibireata, Daniel
• Primary Examiner(s): Do, Anh
• Application Number:
  US13/491,398
• Publication Number:
  US 20130330003A1
• Time in Patent Office: 551 Days
• Field of Search: 382/141, 382/149, 382/176, 382/194, 382/199, 348/127, 356/124, 494/7, 210/512.1, 210/782, 250/559.05, 250/559.42
• US Class Current: 382/176
• CPC Class Codes: G06V 20/63 Scene text, e.g. street names
• CPC Class Codes: G06V 30/1478 of characters or characters lines
• CPC Class Codes: H04W 4/50 Service provisioning or reconfiguring

Abstract

Various approaches for providing textual information to an application, system, or service are disclosed. In particular, various embodiments enable a user to capture an image with a camera of a portable computing device. The computing device is capable of taking the image and processing it to recognize, identify, and/or isolate the text in order to forward the text to an application or function. The application or function can then utilize the text to perform an action in substantially real-time. The text may include an email, phone number, URL, an address, and the like and the application or function may be dialing the phone number, navigating to the URL, opening an address book to save contact information, displaying a map to show the address, and so on. Adaptive thresholding can be used to account for variations across an image, in order to improve the accuracy and efficiency of text recognition processes.

Claims

1. A computer-implemented method comprising:

under control of one or more computer systems configured with executable instructions,

obtaining an image captured by a camera of a portable computing device, the image including pixel values for a plurality of pixels of the camera;

identifying one or more potential edge locations within a region of the image by analyzing one or more image intensity gradients within the region;

analyzing pixel values of a set of pixels on opposing sides of the one or more potential edge locations to determine a character pixel value and a background pixel value for the region;

based at least in part on the character pixel value and the background pixel value for the region, determining a character pixel value threshold for the region; and

determining one or more character portions within the region based at least in part upon whether pixel values for pixels in the region are above or below the character pixel value threshold.

2. The computer-implemented method of claim 1, further comprising:

associating adjacent pixels of the plurality of pixels having a pixel value at least meeting the character pixel value threshold to form at least a portion of the one or more character portions using a connected component algorithm; and

providing at least a portion of the one or more character portions to a character recognition engine.

3. The computer-implemented method of claim 2, further comprising:

analyzing the one or more character portions using the character recognition engine to recognize at least one character associated with the one or more character portions.

4. The computer-implemented method of claim 3, further comprising:

identifying a character pattern corresponding to the recognized at least one character;

determining an application on the portable computing device associated with the character pattern; and

sending the recognized at least one character to the determined application.

5. A computer-implemented method comprising:

under control of one or more computer systems configured with executable instructions,

analyzing one or more intensity gradients of a region in an image to identify one or more potential edge locations, the image including pixel values for a plurality of pixels;

analyzing pixel values for a set of pixels within a determined distance of the one or more potential edge locations;

determining a character pixel value threshold for the region based at least in part upon the pixel values within the determined distance of each of the one or more potential edge locations; and

determining one or more character portions based at least in part upon a portion of the set of pixels having a pixel value at least meeting the character pixel value threshold.

6. The computer-implemented method of claim 5, further comprising:

analyzing the one or more character portions using a character recognition algorithm to recognize at least one character associated with the one or more character portions.

7. The computer-implemented method of claim 5, wherein an edge location corresponds to a pixel intensity gradient or pixel saturation gradient having a value at least meeting the character pixel value threshold within the determined distance.

8. The computer-implemented method of claim 5, wherein analyzing the pixel values includes analyzing a first set of pixels on a first side of the one or more potential edge locations and analyzing a second set of pixels on a second side of the one or more potential edge locations.

9. The computer-implemented method of claim 5, wherein the pixel values correspond to at least one of a grayscale pixel intensity value, a color pixel intensity value, a pixel saturation value, or a pixel saturation level.

10. The computer-implemented method of claim 5, further comprising:

determining related individual character elements of the one or more character portions using a text line construction algorithm.

11. The computer-implemented method of claim 5, further comprising:

associating pixels of the plurality of pixels having a pixel value at least meeting the character pixel value threshold using a connected component algorithm;

applying a stroke width transform algorithm to the associated pixels; and

providing results of applying the stroke width transformation algorithm to a character recognition algorithm.

12. The computer-implemented method of claim 10, further comprising:

determining a dominant direction of the individual character elements after determining related individual character elements.

13. The computer-implemented method of claim 10, further comprising:

determining whether the individual character elements satisfy at least one predefined character constraint; and

removing individual character elements from consideration when an individual character element does not satisfy the at least one of the predefined character constraints.

14. The computer-implemented method of claim 13, wherein the predefined character constraints include at least one of spacing between character elements, ratio of height to width of adjacent character elements, or vertical alignment of character elements.

15. A portable computing device, comprising:

a processor; and

a memory device including instructions that, when executed by the processor, cause the processor to;

analyze one or more intensity gradients of a region in an image to identify one or more potential character edge locations;

analyze pixel values for a set of pixels within a determined distance of the one or more potential edge locations;

determine a character pixel value threshold for the region based at least in part upon the pixel values within the determined distance of each of the one or more edge locations; and

determine one or more character portions based at least in part upon a portion of the set of pixels having a pixel value at least meeting the character pixel value threshold.

16. The portable computing device of claim 15, wherein the instructions, when executed by the processor, further enabling the portable computing device to:

analyze the character portions using a character recognition algorithm to recognize at least one character associated with the one or more character portions.

17. The portable computing device of claim 16, wherein the instructions, when executed by the processor, further enabling the portable computing device to:

obtain at least one additional image; and

analyze the at least one additional image when at least one character is unable to be recognized.

18. The portable computing device of claim 16, wherein the instructions, when executed by the processor, further enabling the portable computing device to:

identify a character pattern corresponding to the recognized at least one character;

determine an application on the portable computing device associated with the character pattern; and

provide the recognized at least one character to the determined application.

19. A non-transitory computer-readable storage medium storing instructions that, when executed by a processor, cause the processor to:

analyze one or more intensity gradients of a region in an image to identify one or more potential edge locations, the image including pixel values for a plurality of pixels;

analyze pixel values for a set of pixels of the image within a determined distance of the one or more potential edge locations;

determine a character pixel value threshold for the region based at least in part upon the pixel values within the determined distance of each of the one or more potential edge locations; and

determine one or more character portions based at least in part upon a portion of the set of pixels having a pixel value at least meeting the character pixel value threshold.

20. The non-transitory computer-readable storage medium of claim 19, storing instructions that, when executed by a processor, cause the processor to further:

analyze the one or more character potions using a character recognition algorithm to recognize at least one character associated with the one or more character portions.

21. The non-transitory computer-readable storage medium of claim 19, wherein the pixel values correspond to at least one of a grayscale pixel intensity value, a color pixel intensity value, a pixel saturation value, or a pixel saturation level.

22. The non-transitory computer-readable storage medium of claim 19, wherein analyzing the pixel values includes analyzing a first set of pixels on a first side of the one or more potential edge locations and analyzing a second set of pixels on a second side of the one or more potential edge locations.

23. The non-transitory computer-readable storage medium of claim 19, storing instructions that, when executed by a processor, cause the processor to further:

convert the image to a gray-scale image before analyzing the one or more intensity gradients.

24. The non-transitory computer-readable storage medium of claim 19, storing instructions that, when executed by a processor, cause the processor to further:

remove portions of the region with pixel values below the character pixel value threshold.

25. The non-transitory computer-readable storage medium of claim 19, storing instructions that, when executed by a processor, cause the processor to further:

associating adjacent pixels of the plurality of pixels having a pixel value at least meeting the character pixel value threshold to form at least a portion of a character using a connected component algorithm.

26. The non-transitory computer-readable storage medium of claim 25, storing instructions that, when executed by a processor, cause the processor to further:

apply a stroke width transform algorithm to the at least a portion of a character; and

provide results of applying the stroke width transform algorithm to a character recognition algorithm.