Enhancing text in images

• US 20080002914A1
• Filed: 06/29/2006
• Published: 01/03/2008
• Est. Priority Date: 06/29/2006
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Vincent, Luc, Ulges, Adrian
• Granted Patent: US 7,953,295 B2
• Time in Patent Office: Days
• US Class Current: 382/299
• CPC Class Codes: G06T 3/4053 based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
• CPC Class Codes: G06V 20/63 Scene text, e.g. street names
• CPC Class Codes: G06V 30/10 Character recognition
• CPC Class Codes: G06V 30/164 Noise filtering

Abstract

Methods, systems, and apparatus including computer program products for enhancing text in images are provided. In one implementation, a computer-implemented method is provided. The method includes receiving a plurality of images each image including a corresponding version of an identified candidate text region and aligning each candidate text region from the plurality of images to a high resolution grid. The method also includes compositing the aligned candidate text regions to create a single superresolution image and performing character recognition on the superresolution image to identify text.

Claims

1. A computer-implemented method comprising:

receiving a plurality of images each image including a corresponding version of an identified candidate text region;

aligning each candidate text region from the plurality of images to a high resolution grid;

compositing the aligned candidate text regions to create a single superresolution image; and

performing character recognition on the superresolution image to identify text.

2. The method of claim 1, where receiving a plurality of images includes receiving images having features associated with text and features associated with non-text image content.

3. The method of claim 2, where identifying the candidate text region of a received image of the plurality of images comprises:

dividing the image into a plurality of regions;

detecting features in each region of the plurality of regions;

analyzing detected features in each region; and

determining whether the detected features indicate text.

4. The method of claim 3, further comprising:

receiving ranging data associated with the received image;

generating a planar map of the received image using the ranging data;

comparing the identified candidate text region with the planar map; and

eliminating the identified candidate text region if the candidate text region is not located on a single plane.

5. The method of claim 1, where the receive images correspond to images of a scene collected from different perspectives.

6. The method of claim 1, further comprising:

supersampling the candidate text regions from each image of the plurality of images;

7. The method of claim 6, where supersampling the candidate text regions is performed using bicubic splines.

8. The method of claim 1, where aligning the candidate text regions comprises aligning pixels of the candidate text regions using block matching.

9. The method of claim 1, where aligning the candidate text regions includes using range data associated with each image of the plurality of images.

10. The method of claim 1, where compositing the aligned candidate text regions comprises combining pixels from each candidate text region including:

determining a median value for each pixel value; and

combining the median values for aligned pixels.

11. A computer program product, encoded on a computer-readable medium, operable to cause data processing apparatus to perform operations comprising:

receiving a plurality of images each including a corresponding version of an identified candidate text region;

aligning each candidate text region from the plurality of images to a high resolution grid;

compositing the aligned candidate text regions to create a single superresolution image; and

performing character recognition on the superresolution image to identify text.

12. The computer program product of claim 11, where receiving a plurality of images includes receiving images having features associated with text and features associated with non-text image content.

13. The computer program product of claim 11, where identifying the candidate text region of a received image of the plurality of images comprises:

dividing the image into a plurality of regions;

detecting features in each region of the plurality of regions;

analyzing detected features in each region; and

determining whether the detected features indicate text.

14. The computer program product of claim 12, further comprising:

receiving ranging data associated with the received image;

generating a planar map of the received image using the ranging data;

comparing the identified candidate text region with the planar map; and

eliminating the identified candidate text region if the candidate text region is not located on a single plane.

15. The computer program product of claim 11, where the receive images correspond to images of a scene collected from different perspectives.

16. The computer program product of claim 11, further comprising:

supersampling the candidate text regions from each image of the plurality of images.

17. The computer program product of claim 16, where supersampling the candidate text regions is performed using bicubic splines.

18. The computer program product of claim 11, where aligning the candidate text regions comprises aligning pixels of the candidate text regions using block matching.

19. The computer program product of claim 11, where aligning the candidate text regions includes using range data associated with each image of the plurality of images.

20. The computer program product of claim 11, where compositing the aligned candidate text regions comprises combining pixels from each candidate text region including:

determining a median value for each pixel value; and

combining the median values for aligned pixels.

21. A system comprising:

means for receiving a plurality of images each including a corresponding version of an identified candidate text region;

means for aligning each candidate text region from the plurality of images to a high resolution grid;

means for compositing the aligned candidate text regions to create a single superresolution image; and

means for performing character recognition on the superresolution image to identify text.