AUTOMATIC CORRECTION OF SKEW IN NATURAL IMAGES AND VIDEO

• US 20140022406A1
• Filed: 03/14/2013
• Published: 01/23/2014
• Est. Priority Date: 07/19/2012
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Baheti, Pawan Kumar, Barman, Kishor K., Acharya, Hemanth P.
• Granted Patent: US 9,076,242 B2
• Time in Patent Office: Days
• US Class Current: 348/222.1
• CPC Class Codes: G06T 3/608 by skew deformation, e.g. two-pass or three-pass rotation
• CPC Class Codes: G06V 20/62 Text, e.g. of license plates, overlay texts or captions on TV images
• CPC Class Codes: G06V 30/10 Character recognition
• CPC Class Codes: G06V 30/1478 of characters or characters lines
• CPC Class Codes: H04N 23/64 Computer-aided capture of images, e.g. transfer from script file into camera, check of taken image quality, advice or proposal for image composition or decision on when to take image

Abstract

An electronic device and method use a camera to capture an image of an environment outside followed by identification of regions therein. A subset of the regions is selected, based on attributes of the regions, such as aspect ratio, height, and variance in stroke width. Next, a number of angles that are candidates for use as skew of the image are determined (e.g. one angle is selected for each region. based on peakiness of a histogram of the region, evaluated at different angles). Then, an angle that is most common among these candidates is identified as the angle of skew of the image. The just-described identification of skew angle is performed prior to classification of any region as text or non-text. After skew identification, at least all regions in the subset are rotated by negative of the skew angle, to obtain skew-corrected regions for use in optical character recognition.

Claims

1. A method to correct skew in an image of a scene of real world, the method comprising:

automatically selecting a subset of regions, from among a set of regions identified in the image, at least based on attributes of regions in the subset of regions;

automatically determining a plurality of candidate angles of skew, at least based on presence of a line of pixels in a region in the subset of regions;

automatically identifying as an angle of skew of the image, a most common candidate angle among the plurality of candidate angles; and

automatically rotating by negative of the angle of skew of the image, at least the subset of regions;

wherein the automatically selecting, the automatically determining, the automatically identifying and the automatically rotating are performed by at least one processor.

2. The method of claim 1 wherein:

the automatically selecting comprises sorting the set of regions based on aspect ratio.

3. The method of claim 1 wherein:

the automatically selecting comprises comparing against a predetermined threshold H, a height of the region in the subset of regions.

4. The method of claim 1 wherein:

the automatically selecting comprises comparing a variance in width of a portion of the region against a predetermined threshold V.

5. The method of claim 4 wherein:

the width used in comparing is a stroke width of the region.

6. The method of claim 1 wherein:

the automatically determining comprises preparing a histogram of counters of pixels in the region in the subset of regions, to identify for the region in the subset of regions a peak and repeating the preparing of a histogram at multiple angles relative to the region, to detect a largest peak and to identify the angle at which the largest peak is detected as a candidate angle.

7. The method of claim 1 wherein:

the automatically determining comprises checking whether at a candidate angle, one or more attributes of projection profiles meet at least one test for presence of a straight line of pixels in the region.

8. The method of claim 1 wherein:

the automatically selecting, the automatically determining, the automatically identifying and the automatically rotating are performed prior to classification of the subset of regions as text or non-text.

9. At least one non-transitory computer readable storage media comprising a plurality of instructions to be executed by at least one processor to correct skew in an image of a scene of real world, the plurality of instructions comprising:

first instructions to select a subset of regions, from among a set of regions identified in the image, at least based on attributes of regions in the subset of regions;

second instructions to determine a plurality of candidate angles of skew, at least based on presence of a line of pixels in a region in the subset of regions;

third instructions to identify as an angle of skew of the image, a most common candidate angle among the plurality of candidate angles; and

fourth instructions to rotate by negative of the angle of skew of the image, at least the subset of regions.

10. The at least one non-transitory computer readable storage media of claim 9 wherein:

the first instructions comprise instructions to sort the set of regions based on aspect ratio.

11. The at least one non-transitory computer readable storage media of claim 9 wherein:

the first instructions comprise instructions to compare against a predetermined threshold H, a height of the region in the subset of regions.

12. The at least one non-transitory computer readable storage media of claim 9 wherein:

the first instructions comprise instructions to compare a variance in width of a portion of the region against a predetermined threshold V.

13. The at least one non-transitory computer readable storage media of claim 12 wherein:

the width used by the instructions to compare is a stroke width of the region.

14. The at least one non-transitory computer readable storage media of claim 9 wherein:

the second instructions comprise instructions to prepare a histogram of counters of pixels in the region in the subset of regions, to identify for the region in the subset of regions a peak and repeat execution of the instructions to prepare a histogram at multiple angles relative to the region to detect a largest peak and to identify the angle at which the largest peak is detected as a candidate angle.

15. The at least one non-transitory computer readable storage media of claim 9 wherein:

the second instructions comprise instructions to check whether at a candidate angle, one or more attributes of projection profiles meet at least one test for presence of a straight line of pixels in the region.

16. The at least one non-transitory computer readable storage media of claim 9 wherein:

the first instructions, the second instructions, the third instructions, and the fourth instructions are configured to be executed prior to classification of the subset of regions as text or non-text.

17. A mobile device to decode text in real world images, the mobile device comprising:

a camera;

a memory operatively connected to the camera to receive at least an image therefrom, the image comprising one or more text regions;

at least one processor operatively connected to the memory to execute a plurality of instructions stored in the memory;

wherein the plurality of instructions cause the at least one processor to;

select a subset of regions, from among a set of regions identified in the image, at least based on attributes of regions in the subset of regions;

determine a plurality of candidate angles of skew, at least based on presence of a line of pixels in a region in the subset of regions;

identify as an angle of skew of the image, a most common candidate angle among the plurality of candidate angles; and

rotate by negative of the angle of skew of the image, at least the subset of regions.

18. The mobile device of claim 17 wherein the at least one processor is further configured to:

sort the set of regions based on aspect ratio.

19. The mobile device of claim 17 wherein the at least one processor is further configured to:

compare against a predetermined threshold H, a height of the region in the subset of regions.

20. The mobile device of claim 17 wherein the at least one processor is further configured to:

compare a variance in width of a portion of the region against a predetermined threshold V.

21. The mobile device of claim 20 wherein:

the width used by the at least one processor to compare is a stroke width of the region.

22. The mobile device of claim 17 wherein:

the plurality of instructions comprise instructions to prepare a histogram of counters of pixels in the region in the subset of regions, to identify for the region in the subset of regions a peak and repeat execution of the instructions to prepare a histogram at multiple angles relative to the region to detect a largest peak and to identify the angle at which the largest peak is detected as a candidate angle.

23. The mobile device of claim 17 wherein:

the plurality of instructions comprise instructions to check whether at a candidate angle, one or more attributes of projection profiles meet at least one test for presence of a straight line of pixels in the region.

24. The mobile device of claim 17 wherein:

the plurality of instructions are configured to be executed prior to classification of the subset of regions as text or non-text.

25. A mobile device comprising:

a camera to capture an image of an environment outside the mobile device;

a memory coupled to the camera for storing the image;

means, coupled to the memory, for identifying a set of regions in the image;

wherein each region in the set of regions is identified based on variation of intensities of pixels;

means for selecting a subset of regions, from among the set of regions identified in the image, at least based on attributes of regions in the subset of regions;

means for determining a plurality of candidate angles of skew, at least based based on pixels in the regions in the subset of regions;

means for identifying as an angle of skew of the image, a most common candidate angle among the plurality of candidate angles, by use of at least one processor; and

means for rotating by negative of the angle of skew of the image, at least the subset of regions.