Text image quality based feedback for improving OCR

• US 9,317,764 B2
• Filed: 03/15/2013
• Issued: 04/19/2016
• Est. Priority Date: 12/13/2012
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Baheti, Pawan Kumar, Bisain, Abhijeet S., Soundararajan, Rajiv, Gore, Dhananjay Ashok
• Primary Examiner(s): Mehta, Bhavesh
• Assistant Examiner(s): Youssef, Menatoallah
• Application Number:
  US13/843,637
• Publication Number:
  US 20140168478A1
• Time in Patent Office: 1,131 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06V 20/63 Scene text, e.g. street names
• CPC Class Codes: G06V 30/10 Character recognition
• CPC Class Codes: G06V 30/1456 based on user interactions

Abstract

An electronic device and method capture multiple images of a scene of real world at a several zoom levels, the scene of real world containing text of one or more sizes. Then the electronic device and method extract from each of the multiple images, one or more text regions, followed by analyzing an attribute that is relevant to OCR in one or more versions of a first text region as extracted from one or more of the multiple images. When an attribute has a value that meets a limit of optical character recognition (OCR) in a version of the first text region, the version of the first text region is provided as input to OCR.

Claims

1. A method to improve text recognition by using multiple images of identical text, the method comprising:

capturing a plurality of images of a scene of real world at a plurality of zoom levels, said scene of real world containing text of one or more sizes;

extracting from each of the plurality of images, one or more text regions;

analyzing an attribute that is relevant to OCR in one or more versions of a first text region as extracted from one or more of said plurality of images;

when the attribute has a value that meets a limit of optical character recognition (OCR) in a version of the first text region, providing the version of the first text region as input to OCR;

when the value of the attribute does not meet the limit of OCR, calculating a new zoom level at which the attribute of the first text region meets the limit of OCR, and storing at least an identification of the first text region in a list;

repeating the providing or the calculating, with other text regions extracted from the plurality of images;

using the list to identify a maximum zoom level that retains all text regions in the list within a field of view of a camera; and

based on the maximum zoom level, generating feedback to capture at least one additional image.

2. The method of claim 1 wherein:

the attribute comprises height of each region in the one or more text regions.

3. The method of claim 1 wherein:

the extracting comprises checking for presence of a line of pixels of a common binary value in the one or more text regions.

4. The method of claim 1 wherein:

the extracting comprises checking a variance in width of a stroke of a character in the one or more text regions.

5. The method of claim 1 wherein said using of the list to identify the maximum zoom level comprises:

checking if an extreme x-coordinate of the first text region is greater than w/zoom_level, wherein w is a width of the first text region and zoom_level is a level of zoom at which an image comprising the first text region is captured by the camera;

when the extreme x-coordinate is greater than w/zoom_level, further checking whether a number of images within a field of view is equal to a length of the list; and

when the number of images within the field of view is equal to the length of the list, setting at least one of a flag or a variable.

6. The method of claim 1 wherein said using of the list to identify the maximum zoom level comprises:

checking if an extreme y-coordinate of the first text region is greater than h/zoom_level, wherein h is a height of the first text region and zoom_level is a level of zoom at which an image comprising the first text region is captured by the camera;

when the extreme y-coordinate is greater than h/zoom_level, further checking whether a number of images within a field of view is equal to a length of the list; and

when the number of images within the field of view is equal to the length of the list, setting at least one of a flag or a variable.

7. The method of claim 1 wherein:

the plurality of images are captured in a sequence successively one after another.

8. The method of claim 7 wherein:

the plurality of images are captured prior to said extracting.

9. The method of claim 7 wherein:

said plurality of images are automatically captured in response to a single user input.

10. The method of claim 1 wherein:

a feature in the scene of real world not captured in said at least one additional image comprising an enlarged version of the first text region is captured in an image in the plurality of images that comprises a smaller version of the first text region.

11. The method of claim 1 further comprising:

providing a user with said feedback generated based on the maximum zoom level.

12. The method of claim 1 further comprising:

analyzing an attribute that is relevant to OCR in one or more versions of a second text region as extracted from one or more of said plurality of images; and

when the attribute has a value that meets a limit of optical character recognition (OCR) in a version of the second text region, providing the version of the second text region as input to OCR.

13. The method of claim 12 further comprising:

outputting text recognized in said first and second regions.

14. At least one non-transitory computer readable storage media comprising a plurality of instructions to be executed by at least one processor to obtain multiple images for use in text recognition, the plurality of instructions comprising:

first instructions to capture a plurality of images of a scene of real world at a plurality of zoom levels, said scene of real world containing text of one or more sizes;

second instructions to extract from each of the plurality of images, one or more text regions;

third instructions to analyze an attribute that is relevant to OCR in one or more versions of a first text region as extracted from one or more of said plurality of images;

fourth instructions to provide the version of the first text region as input to OCR, when the attribute has a value that meets a limit of optical character recognition (OCR) in a version of the first text region;

fifth instructions to calculate a new zoom level at which the attribute of the first text region meets the limit of OCR and store at least an identification of the first text region in a list, when the value of the attribute not meeting the limit of OCR;

sixth instructions to repeatedly execute the fourth instructions and the fifth instructions, with other text regions extracted from the plurality of images;

seventh instructions to use the list to identify a maximum zoom level that retains all text regions in the list within a field of view of a camera; and

based on the maximum zoom level, eighth instructions to generate feedback to capture at least one additional image.

15. The at least one non-transitory computer readable storage media of claim 14 wherein:

the attribute comprises height of each region in the one or more text regions.

16. The at least one non-transitory computer readable storage media of claim 14 wherein:

the second instructions comprise instructions to check for presence of a line of pixels of a common binary value in the one or more text regions.

17. The at least one non-transitory computer readable storage media of claim 14 wherein:

the second instructions comprise instructions to check a variance in width of a stroke of a character in the one or more text regions.

18. The at least one non-transitory computer readable storage media of claim 14 further comprising:

fifth instructions to check if an extreme x-coordinate of the first text region is greater than w/zoom_level, wherein w is a width of the first text region and zoom_level is a level of zoom at which an image comprising the first text region is captured by the camera;

sixth instructions to be executed when the extreme x-coordinate is greater than w/zoom_level, to further check whether a number of images within a field of view is equal to a length of the list; and

seventh instructions to be executed when the number of images within the field of view is equal to the length of the list, to set at least one of a flag or a variable.

19. The at least one non-transitory computer readable storage media of claim 14 further comprising:

fifth instructions to check if an extreme y-coordinate of the first text region is greater than h/zoom_level, wherein h is a height of the first text region and zoom_level is a level of zoom at which an image comprising the first text region is captured by the camera;

sixth instructions to be executed when the extreme y-coordinate is greater than h/zoom_level, to further check whether a number of images within a field of view is equal to a length of the list; and

seventh instructions to be executed when the number of images within the field of view is equal to the length of the list, to set at least one of a flag or a variable.

20. The at least one non-transitory computer readable storage media of claim 14 wherein:

the plurality of images are captured in a sequence successively one after another.

21. The at least one non-transitory computer readable storage media of claim 14 wherein:

a feature in the scene of real world not captured in said at least one additional image comprising an enlarged version of the first text region is captured in an image in the plurality of images that comprises a smaller version of the first text region.

22. A mobile device to decode text in real world images, the mobile device comprising:

a camera;

a memory operatively connected to the camera to receive at least an image therefrom, the image comprising one or more text regions;

at least one processor operatively connected to the memory to execute a plurality of instructions stored in the memory;

wherein the plurality of instructions cause the at least one processor to;

capture a plurality of images of a scene of real world at a plurality of zoom levels, said scene of real world containing text of one or more sizes;

extract from each of the plurality of images, one or more text regions;

analyze an attribute that is relevant to OCR in one or more versions of a first text region as extracted from one or more of said plurality of images; and

when the attribute has a value that meets a limit of optical character recognition (OCR) in a version of the first text region, provide the version of the first text region as input to OCR;

when the value of the attribute does not meet the limit of OCR, calculate a new zoom level at which the attribute of the first text region meets the limit of OCR, and storing at least an identification of the first text region in a list;

repeat execution of instructions to provide or instructions to calculate, with other text regions extracted from the plurality of images;

use the list to identify a maximum zoom level that retains all text regions in the list within a field of view of a camera; and

based on the maximum zoom level, generate feedback to capture at least one additional image.

23. The mobile device of claim 22 wherein:

the attribute comprises height of each region in the one or more text regions.

24. The mobile device of claim 22 wherein:

the instructions that cause the at least one processor to extract comprise instructions to check for presence of a line of pixels of a common binary value in the one or more text regions.

25. The mobile device of claim 22 wherein the at least one processor is further configured to:

check a variance in width of a stroke of a character in the one or more text regions.

26. The mobile device of claim 22 wherein:

the plurality of images are captured in a sequence successively one after another.

27. The mobile device of claim 22 wherein:

a feature in the scene of real world not captured in said at least one additional image comprising an enlarged version of the first text region is captured in an image in the plurality of images that comprises a smaller version of the first text region.

28. A mobile device comprising:

a camera configured to capture a plurality of images of a scene of real world at a plurality of zoom levels, said scene of real world containing text of one or more sizes;

a memory coupled to the camera for storing the plurality of images;

means, coupled to the memory, for extracting from each of the plurality of images, one or more text regions;

means for analyzing an attribute that is relevant to OCR in one or more versions of a first text region as extracted from one or more of said plurality of images;

responsive to the attribute having a value that meets a limit of optical character recognition (OCR) in a version of the first text region, means for providing the version of the first text region as input to OCR;

responsive to the value of the attribute not meeting the limit of OCR, means for calculating a new zoom level at which the attribute of the first text region meets the limit of OCR, and storing at least an identification of the first text region in a list;

means for repeatedly invoking the means for providing or the means for calculating, with other text regions extracted from the plurality of images;

means for using the list to identify a maximum zoom level that retains all text regions in the list within a field of view of a camera; and

based on the maximum zoom level, means for generating feedback to capture at least one additional image.