Recognizing text in images using ranging data

• US 8,031,940 B2
• Filed: 06/29/2006
• Issued: 10/04/2011
• Est. Priority Date: 06/29/2006
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Vincent, Luc, Ulges, Adrian
• Primary Examiner(s): Repko; Jason M
• Assistant Examiner(s): Edwards; Patrick L
• Application Number:
  US11/479,957
• Publication Number:
  US 20080002893A1
• Time in Patent Office: 1,923 Days
• Field of Search: 382/176, 382/321, 382/294, 382/299
• US Class Current: 382/176
• CPC Class Codes: G06V 20/63 Scene text, e.g. street names
• CPC Class Codes: G06V 30/10 Character recognition

Abstract

Methods, systems, and apparatus including computer program products for recognizing text in images are provided. In one implementation, a computer-implemented method for recognizing text in an image is provided. The method includes receiving a plurality of images. The method also includes processing the images to detect a corresponding set of regions of the images, each image having a region corresponding to each other image region, as potentially containing text. The method further includes combining the regions to generate an enhanced region image and performing optical character recognition on the enhanced region image.

Claims

1. A computer-implemented method for recognizing text in an image, comprising:

processing an image to divide the image into one or more regions;

detecting one or more features in each region;

determining for each region whether it is a candidate text region potentially containing text using the detected features;

receiving ranging data associated with the image;

generating a planar map of the image using the ranging data;

comparing the candidate text regions with the planar map;

eliminating candidate text regions that are not located on a single plane;

enhancing the candidate text regions that were not eliminated to generate an enhanced image; and

performing optical character recognition on the enhanced image.

2. The method of claim 1, further comprising:

normalizing the image to increase contrast in the image.

3. The method of claim 1 in which the image is a high dynamic range image, the method further comprising:

receiving a plurality of versions of a first image in which each version of the first image has a different exposure level; and

compositing the plurality of first image versions to form the high dynamic range image.

4. The method of claim 1, where enhancing the candidate text regions comprises:

generating a superresolution version of each candidate text region.

5. The method of claim 4, where generating the superresolution version of a candidate text region comprises:

receiving a plurality of images, each including a version of the candidate text region;

supersampling the candidate text region version from each image of the plurality of images;

aligning each supersampled candidate text region version from the plurality of images to a high resolution grid; and

compositing the aligned images to create the superresolution image.

6. The method of claim 1, wherein receiving ranging data comprises receiving ranging data from range sensors.

7. The method of claim 6, wherein the ranging data comprises a distance from a camera position to a point in the image.

8. The method of claim 7, further comprising eliminating candidate text regions that do not meet a predetermined distance criteria.

9. The method of claim 6, wherein generating a planar map of the image using the ranging data comprises decomposing the image into a planar region and a non-planar region.

10. A computer program product, encoded on a computer-readable storage device, operable to cause data processing apparatus to perform operations comprising:

processing the image to divide the image into one or more regions;

detect one or more features in each region;

determining for each region whether it is a candidate text region potentially containing text using the detected features;

receiving ranging data associated with the image;

generating a planar map of the image using the ranging data;

comparing the candidate text regions with the planar map;

eliminating candidate text regions that are not located on a single plane;

enhancing the candidate text regions to generate an enhanced image; and

performing optical character recognition on the enhanced image.

11. The computer program product of claim 10, where enhancing the candidate text regions comprises:

generating a superresolution version of each identified candidate text region.

12. The computer program product of claim 11, where generating the superresolution version of a detected candidate text region comprises:

receiving a plurality of images, each including a version of the candidate text region;

supersampling the candidate text region from each image of the plurality of images;

aligning each supersampled image from the plurality of images to a high resolution grid; and

compositing the aligned images to create a single superresolution image.

13. The computer program product of claim 10, further comprising:

normalizing the image to increase contrast in the image.

14. The computer program product of claim 10 in which the image is a high dynamic range image, the method further comprising:

receiving a plurality of versions of a first image in which each version of the first image has a different exposure level; and

compositing the plurality of first image versions to form the high dynamic range image.

15. The computer program product of claim 10, wherein receiving ranging data comprises receiving ranging data from range sensors.

16. The computer program product of claim 15, wherein the ranging data comprises a distance from a camera position to a point in the image.

17. The computer program product of claim 16, further comprising eliminating candidate text regions that do not meet a predetermined distance criteria.

18. The computer program product of claim 15, wherein generating a planar map of the image using the ranging data comprises decomposing the image into a planar region and a non-planar region.

19. A system comprising:

one or more processors configured to execute instructions in order to perform the following operations;

processing an image to divide the image into one or more regions;

detecting one or more features in each region;

determining for each region whether it is a candidate text region potentially containing text using the detected features;

receiving ranging data associated with the image;

generating a planar map of the image using the ranging data;

comparing the candidate text regions with the planar map;

eliminating candidate text regions that are not located on a single plane;

enhancing the candidate text regions that were not eliminated to generate an enhanced image; and

performing optical character recognition on the enhanced image.

20. The system of claim 19 in which the operations further comprise:

normalizing the image to increase contrast in the image.

21. The system of claim 19 in which the image is a high dynamic range image, and in which the operations further comprise:

receiving a plurality of versions of a first image in which each version of the first image has a different exposure level; and

compositing the plurality of first image versions to form the high dynamic range image.

22. The system of claim 19, in which enhancing the candidate text regions comprises generating a superresolution version of each candidate text region.

23. The system of claim 22, where generating the superresolution version of a candidate text region comprises:

receiving a plurality of images, each including a version of the candidate text region;

supersampling the candidate text region version from each image of the plurality of images;

aligning each supersampled candidate text region version from the plurality of images to a high resolution grid; and

compositing the aligned images to create the superresolution image.

24. The system of claim 19, wherein receiving ranging data comprises receiving ranging data from range sensors.

25. The system of claim 24, wherein the ranging data comprises a distance from a camera position to a point in the image.

26. The system of claim 25, further comprising eliminating candidate text region that do not meet a predetermined distance criteria.

27. The system of claim 24, wherein generating a planar map of the image using the ranging data comprises decomposing the image into a planar region and a non-planar region.