Method of improving contrast for text extraction and recognition applications

• US 9,171,224 B2
• Filed: 09/10/2013
• Issued: 10/27/2015
• Est. Priority Date: 07/04/2013
• Status: Expired due to Fees

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Barman, Kishor K., Acharya, Hemanth P., Baheti, Pawan Kumar
• Primary Examiner(s): Allison, Andrae S
• Application Number:
  US14/023,306
• Publication Number:
  US 20150010233A1
• Time in Patent Office: 777 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06V 10/20 Image preprocessing

Abstract

An electronic device and method receive (for example, from a memory), a grayscale image of a scene of real world captured by a camera of a mobile device. The electronic device and method also receive a color image from which the grayscale image is generated, wherein each color pixel is stored as a tuple of multiple components. The electronic device and method determine a new intensity for at least one grayscale pixel in the grayscale image, based on at least one component of a tuple of a color pixel located in correspondence to the at least one grayscale pixel. The determination may be done conditionally, by checking whether a local variance of intensities is below a predetermined threshold in a subset of grayscale pixels located adjacent to the at least one grayscale pixel, and selecting the component to provide most local variance of intensities.

Claims

1. A method of using color images to enhance grayscale images used in text extraction, the method comprising:

receiving a color image of a scene of real world captured by a camera of a mobile device;

receiving a first grayscale image generated from the color image;

checking whether a plurality of groups of pixels meet a predetermined test for low contrast;

wherein the predetermined test is performed on color pixels in the color image or on grayscale pixels in the first grayscale image or on any characteristics thereof in combination;

in response to the predetermined test being met by a first group, the first group being comprised in the plurality of groups of pixels, using at least one component in a tuple of a first color pixel in the color image to determine for a first grayscale pixel in the first group, a first new intensity which is to replace a first existing intensity of the first grayscale pixel in the first grayscale image;

in response to the predetermined test being not met by a second group, the second group being comprised in the plurality of groups of pixels, retaining unchanged a second existing intensity of a second grayscale pixel in the first grayscale image; and

storing in a memory, a second grayscale image formed as a composite, by combination of at least the first new intensity for the first grayscale pixel and the second existing intensity of the second grayscale pixel.

2. The method of claim 1 wherein said at least one component in the tuple is hereinafter a first tuple component, the method further comprising:

in response to the predetermined test being met by a third group, the third group being comprised in the plurality of groups of pixels in the first grayscale image, using a second tuple component in the tuple of a third color pixel in the color image to determine for a third grayscale pixel in the third group, a third new intensity which is to replace a third existing intensity for the third grayscale pixel in the first grayscale image.

3. The method of claim 1 wherein:

the predetermined test comprises comparison of variance in intensities of a group of pixels with a predetermined limit.

4. The method of claim 1 wherein:

the tuple has N color channels;

a color channel is automatically selected for having highest contrast among the N color channels; and

an intensity of the color channel that is automatically selected is used in determining the first new intensity.

5. The method of claim 1 wherein:

the checking is performed at least in response to failure to detect after binarization, any word of text in a region that comprises the first group and the second group in the first grayscale image.

6. The method of claim 1 wherein the predetermined test is hereinafter a first test, the method further comprising:

checking whether a second test for presence of a peak is met, in a histogram of intensities of grayscale pixels located in a rectangular block that encloses within the second grayscale image, a location corresponding to the first grayscale pixel in the first grayscale image.

7. The method of claim 6 wherein the rectangular block is hereinafter a first rectangular block, the method further comprising:

when the second test is not met, binarizing the first rectangular block in the second grayscale image, based on a threshold identified by use of the histogram of intensities of grayscale pixels; and

when the second test is met, retaining unchanged a second rectangular block in the first grayscale image corresponding to the first rectangular block in the second grayscale image.

8. The method of claim 6 wherein:

the second test is checked at least when confidence of decoding of a word in the rectangular block is not above a limit.

9. The method of claim 6 wherein the peak is hereinafter a first peak, the method further comprising:

checking whether a third test for presence of a second peak is met, in a histogram of counters along a height of the rectangular block.

10. A non-transitory computer-readable storage media comprising a plurality of instructions to use color images to enhance grayscale images used in text extraction, which, when executed by a processor perform steps in a method comprising:

receiving a color image of a scene of real world captured by a camera of a mobile device;

receiving a first grayscale image generated from the color image;

checking whether a plurality of groups of pixels meet a predetermined test for low contrast;

wherein the predetermined test is performed on color pixels in the color image or on grayscale pixels in the first grayscale image or on any characteristics thereof in combination;

responsive to the predetermined test being met by a first group, the first group being comprised in the plurality of groups of pixels, using at least one component in a tuple of a first color pixel in the color image to determine for a first grayscale pixel in the first group, a first new intensity which is to replace a first existing intensity of the first grayscale pixel in the first grayscale image;

responsive to the predetermined test being not met by a second group, the second group being comprised in the plurality of groups of pixels, retaining unchanged a second existing intensity of a second grayscale pixel in the first grayscale image; and

storing in a memory, a second grayscale image formed as a composite, by combination of at least the first new intensity for the first grayscale pixel and the second existing intensity of the second grayscale pixel.

11. The non-transitory computer-readable storage media of claim 10 wherein said at least one component in the tuple is hereinafter a first tuple component, the method further comprising:

in response to the predetermined test being met by a third group, the third group being comprised in the plurality of groups of pixels in the first grayscale image, using a second tuple component in the tuple of a third color pixel in the color image to determine for a third grayscale pixel in the third group, a third new intensity which is to replace a third existing intensity for the third grayscale pixel in the first grayscale image.

12. The non-transitory computer-readable storage media of claim 10 wherein:

the predetermined test comprises comparison of variance in intensities of a group of pixels with a predetermined limit.

13. The non-transitory computer-readable storage media of claim 10 wherein:

the tuple has N color channels;

a color channel is automatically selected for having highest contrast among the N color channels; and

an intensity of the color channel that is automatically selected is used in determining the first new intensity.

14. The non-transitory computer-readable storage media of claim 10 wherein:

the checking is performed at least in response to failure to detect after binarization, any word of text in a region that comprises the first group and the second group in the first grayscale image.

15. The non-transitory computer-readable storage media of claim 10 wherein the predetermined test is hereinafter a first test, the method further comprising:

checking whether a second test for presence of a peak is met, in a histogram of intensities of grayscale pixels located in a rectangular block that encloses within the second grayscale image, a location corresponding to the first grayscale pixel in the first grayscale image.

16. The non-transitory computer-readable storage media of claim 15 wherein the rectangular block is hereinafter a first rectangular block, the method further comprising:

when the second test is not met, binarizing the first rectangular block in the second grayscale image, based on a threshold identified by use of the histogram of intensities of grayscale pixels; and

when the second test is met, retaining unchanged a second rectangular block in the first grayscale image corresponding to the first rectangular block in the second grayscale image.

17. The non-transitory computer-readable storage media of claim 15 wherein:

the second test is checked at least when confidence of decoding of a word in the rectangular block is not above a limit.

18. A mobile device comprising:

a camera;

a memory operatively connected to the camera to receive at least an image therefrom;

a processor operatively connected to the memory to execute a plurality of instructions stored in the memory, configured to;

supply a color image of a scene of real world captured by the camera and a first grayscale image generated from the color image;

check whether a plurality of groups of pixels meet a predetermined test for low contrast;

wherein the predetermined test is performed on color pixels in the color image or on grayscale pixels in the first grayscale image or on any characteristics thereof in combination;

responsive to the predetermined test being met by a first group, the first group being comprised in the plurality of groups of pixels, use at least one component in a tuple of a first color pixel in the color image to determine for a first grayscale pixel in the first group, a first new intensity which is to replace a first existing intensity of the first grayscale pixel in the first grayscale image;

responsive to the predetermined test being not met by a second group, the second group being comprised in the plurality of groups of pixels, retaining unchanged a second existing intensity of a second grayscale pixel in the first grayscale image; and

store in the memory, formed as a composite, by combination of at least the first new intensity for the first grayscale pixel and the second existing intensity of the second grayscale pixel.

19. The mobile device of claim 18 wherein:

said at least one component in the tuple is hereinafter a first tuple component; and

the processor is further configured to be responsive to the predetermined test being met by a third group, the third group being comprised in the plurality of groups of pixels in the first grayscale image, to use a second tuple component in the tuple of a third color pixel in the color image to determine for a third grayscale pixel in the third group, a third new intensity which is to replace a third existing intensity for the third grayscale pixel in the first grayscale image.

20. The mobile device of claim 18 wherein:

the predetermined test comprises comparison of variance in intensities of a group of pixels with a predetermined limit.

21. The mobile device of claim 18 wherein:

the tuple has N color channels;

a color channel is automatically selected for having highest contrast among the N color channels; and

an intensity of the color channel that is automatically selected is used in determining the first new intensity.

22. An apparatus for identifying regions of text, the apparatus comprising:

a processor;

a memory coupled to the processor, the memory storing a color image of a scene of real world captured by a camera of a mobile device, and a first grayscale image generated from the color image;

means for checking whether a plurality of groups of pixels meet a predetermined test for low contrast;

wherein the predetermined test is performed on color pixels in the color image or on grayscale pixels in the first grayscale image or on any characteristics thereof in combination;

means, responsive to the predetermined test being met by a first group, the first group being comprised in the plurality of groups of pixels, for using at least one component in a tuple of a first color pixel in the color image to determine for a first grayscale pixel in the first group, a first new intensity which is to replace a first existing intensity of the first grayscale pixel in the first grayscale image;

means, responsive to the predetermined test being not met by a second group, the second group being comprised in the plurality of groups of pixels, for retaining unchanged a second existing intensity of a second grayscale pixel in the first grayscale image;

means for storing in the memory, a second grayscale image formed as a composite, by combination of at least the first new intensity for the first grayscale pixel and a second existing intensity of a second grayscale pixel.

23. The apparatus of claim 22 wherein:

the predetermined test comprises comparison of variance in intensities of a group of pixels with a predetermined limit.

24. The apparatus of claim 22 wherein:

the tuple has N color channels;

a color channel is automatically selected for having highest contrast among the N color channels; and

an intensity of the color channel that is automatically selected is used in determining the first new intensity.