Enhancing text in images

• US 7,953,295 B2
• Filed: 06/29/2006
• Issued: 05/31/2011
• Est. Priority Date: 06/29/2006
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Ulges, Adrian, Vincent, Luc
• Primary Examiner(s): Desire; Gregory M
• Application Number:
  US11/479,115
• Publication Number:
  US 20080002914A1
• Time in Patent Office: 1,797 Days
• Field of Search: 382/175, 382/176, 382/181, 382/229
• US Class Current: 382/299
• CPC Class Codes: G06T 3/4053 based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
• CPC Class Codes: G06V 20/63 Scene text, e.g. street names
• CPC Class Codes: G06V 30/10 Character recognition
• CPC Class Codes: G06V 30/164 Noise filtering

Abstract

Methods, systems, and apparatus including computer program products for enhancing text in images are provided. In one implementation, a computer-implemented method is provided. The method includes receiving a plurality of images each image including a corresponding version of an identified candidate text region and aligning each candidate text region from the plurality of images to a high resolution grid. The method also includes compositing the aligned candidate text regions to create a single superresolution image and performing character recognition on the superresolution image to identify text.

Claims

1. A computer-implemented method comprising:

receiving, by a computing system, a plurality of images each image including a corresponding version of an identified candidate text region;

aligning, by the computing system, each candidate text region from the plurality of images to a high resolution grid;

compositing, by the computing system, the aligned candidate text regions to create a single superresolution image; and

performing, by the computing system, character recognition on the superresolution image to identify text.

2. The method of claim 1, where receiving a plurality of images includes receiving images having features associated with text and features associated with non-text image content.

3. The method of claim 2, further comprising identifying the candidate text region of a received image of the plurality of images by:

dividing the image into a plurality of regions;

detecting features in each region of the plurality of regions;

analyzing detected features in each region; and

determining whether the detected features indicate text.

4. The method of claim 3, further comprising:

receiving ranging data associated with the received image;

generating a planar map of the received image using the ranging data;

comparing the identified candidate text region with the planar map; and

eliminating the identified candidate text region if the candidate text region is not located on a single plane.

5. The method of claim 1, where the received images correspond to images of a scene collected from different perspectives.

6. The method of claim 1, further comprising:

supersampling the candidate text regions from each image of the plurality of images.

7. The method of claim 6, where supersampling the candidate text regions is performed using bicubic splines.

8. The method of claim 1, where aligning the candidate text regions comprises aligning pixels of the candidate text regions using block matching.

9. The method of claim 1, where aligning the candidate text regions includes using range data associated with each image of the plurality of images.

10. The method of claim 1, where compositing the aligned candidate text regions comprises combining pixels from each candidate text region including:

determining a median value for each pixel value; and

combining the median values for aligned pixels.

11. A computer program product, encoded on a computer-readable medium, operable to cause data processing apparatus to perform operations comprising:

receiving a plurality of images each including a corresponding version of an identified candidate text region;

aligning each candidate text region from the plurality of images to a high resolution grid;

compositing the aligned candidate text regions to create a single superresolution image; and

performing character recognition on the superresolution image to identify text.

12. The computer program product of claim 11, where receiving a plurality of images includes receiving images having features associated with text and features associated with non-text image content.

13. The computer program product of claim 12, further comprising:

receiving ranging data associated with the received image;

generating a planar map of the received image using the ranging data;

comparing the identified candidate text region with the planar map; and

eliminating the identified candidate text region if the candidate text region is not located on a single plane.

14. The computer program product of claim 11, further comprising identifying the candidate text region of a received image of the plurality of images by:

dividing the image into a plurality of regions;

detecting features in each region of the plurality of regions;

analyzing detected features in each region; and

determining whether the detected features indicate text.

15. The computer program product of claim 11, where the received images correspond to images of a scene collected from different perspectives.

16. The computer program product of claim 11, further comprising:

supersampling the candidate text regions from each image of the plurality of images.

17. The computer program product of claim 16, where supersampling the candidate text regions is performed using bicubic splines.

18. The computer program product of claim 11, where aligning the candidate text regions comprises aligning pixels of the candidate text regions using block matching.

19. The computer program product of claim 11, where aligning the candidate text regions includes using range data associated with each image of the plurality of images.

20. The computer program product of claim 11, where compositing the aligned candidate text regions comprises combining pixels from each candidate text region including:

determining a median value for each pixel value; and

combining the median values for aligned pixels.

21. A system comprising:

means for receiving a plurality of images each including a corresponding version of an identified candidate text region;

means for aligning each candidate text region from the plurality of images to a high resolution grid;

means for compositing the aligned candidate text regions to create a single superresolution image; and

means for performing character recognition on the superresolution image to identify text.

22. A system comprising:

one or more data processing apparatus configured to perform operations comprising;

receiving a plurality of images each including a corresponding version of an identified candidate text region;

aligning each candidate text region from the plurality of images to a high resolution grid;

compositing the aligned candidate text regions to create a single superresolution image; and

performing character recognition on the superresolution image to identify text.

23. The system of claim 22, where receiving a plurality of images includes receiving images having features associated with text and features associated with non-text image content.

24. The system of claim 23, further comprising:

receiving ranging data associated with the received image;

generating a planar map of the received image using the ranging data;

comparing the identified candidate text region with the planar map; and

eliminating the identified candidate text region if the candidate text region is not located on a single plane.

25. The system of claim 22, further comprising identifying the candidate text region of a received image of the plurality of images by:

dividing the image into a plurality of regions;

detecting features in each region of the plurality of regions;

analyzing detected features in each region; and

determining whether the detected features indicate text.

26. The system of claim 22, where the received images correspond to images of a scene collected from different perspectives.

27. The system of claim 22, further comprising:

supersampling the candidate text regions from each image of the plurality of images.

28. The system of claim 27, where supersampling the candidate text regions is performed using bicubic splines.

29. The system of claim 22, where aligning the candidate text regions comprises aligning pixels of the candidate text regions using block matching.

30. The system of claim 22, where aligning the candidate text regions includes using range data associated with each image of the plurality of images.

31. The system of claim 22, where compositing the aligned candidate text regions comprises combining pixels from each candidate text region including:

determining a median value for each pixel value; and

combining the median values for aligned pixels.