Using extracted image text

• US 9,269,013 B2
• Filed: 05/30/2014
• Issued: 02/23/2016
• Est. Priority Date: 06/29/2006
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Vincent, Luc, Ulges, Adrian
• Primary Examiner(s): Desire, Gregory M
• Application Number:
  US14/291,331
• Publication Number:
  US 20140270549A1
• Time in Patent Office: 634 Days
• US Class Current: 1/1
• CPC Class Codes: G01S 19/51 Relative positioning
• CPC Class Codes: G06F 16/5846 using extracted text
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 3/4053 based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/33 using feature-based methods
• CPC Class Codes: G06V 20/56 exterior to a vehicle by using sensors mounted on the vehicle
• CPC Class Codes: G06V 20/63 Scene text, e.g. street names
• CPC Class Codes: G06V 30/10 Character recognition

Abstract

Methods, systems, and apparatus including computer program products for using extracted image text are provided. In one implementation, a computer-implemented method is provided. The method includes receiving an input of one or more image search terms and identifying keywords from the received one or more image search terms. The method also includes searching a collection of keywords including keywords extracted from image text, retrieving an image associated with extracted image text corresponding to one or more of the image search terms, and presenting the image.

Claims

1. A computer-implemented method comprising:

receiving a plurality of different images of a first scene, wherein each image has a different exposure level;

compositing two or more images into a composite image using the plurality images;

detecting one or more features in each of one or more regions of the composite image;

determining for each region of the composite image whether the region is a candidate text region potentially containing text based on the detected one or more features; and

generating text by performing optical character recognition on a plurality of the regions determined to contain text.

2. The method of claim 1, further comprising:

increasing contrast of the composite image including normalizing pixel values in the composite image.

3. The method of claim 2 wherein normalizing pixel values in the composite image comprises:

computing a mean and variance of pixel values in the composite image; and

scaling pixel values in the composite image according to the computed mean and variance.

4. The method of claim 1, further comprising:

generating a superresolution version of the candidate text region.

5. The method of claim 4, wherein generating the superresolution version of the candidate text region comprises:

obtaining a plurality of versions of the candidate text region, each version obtained from a corresponding image of the plurality of images;

aligning the versions of the candidate text region to a high resolution grid; and

compositing the aligned versions of the candidate text region to generate the superresolution version of the candidate text region.

6. The method of claim 5, further comprising:

supersampling the obtained versions of a particular candidate text region from each image of the plurality of images.

7. The method of claim 5, wherein aligning the versions of the candidate text region comprises aligning pixels of the versions of the candidate text region using block matching.

8. The method of claim 5 wherein aligning the versions of the candidate text region comprises:

receiving ranging data and movement information associated with each version of the candidate text region; and

aligning the versions of the candidate text region based at least in part on the received ranging data and movement information.

9. The method of claim 5, wherein compositing the aligned versions of the candidate text region comprises combining pixels from each version of the candidate text region including:

computing a median value of pixels in each version of the candidate region; and

combining the computed median values for corresponding pixels in the aligned versions of the candidate text region.

10. The method of claim 5, wherein compositing the image comprises performing a high dynamic range process to generate a high dynamic range image.

11. A system comprising:

one or more data processing apparatus; and

a data store storing instructions that, when executed by the one or more data processing apparatus, cause the one or more data processing apparatus to perform operations comprising;

receiving a plurality of different images of a first scene, wherein each image has a different exposure level;

compositing two or more images into a composite image using the plurality images;

detecting one or more features in each of one or more regions of the composite image;

determining for each region of the composite image whether the region is a candidate text region potentially containing text based on the detected one or more features; and

generating text by performing optical character recognition on a plurality of the regions determined to contain text.

12. The system of claim 11, wherein the operations further comprise:

increasing contrast of the composite image including normalizing pixel values in the composite image.

13. The system of claim 12 wherein normalizing pixel values in the composite image comprises:

computing a mean and variance of pixel values in the composite image; and

scaling pixel values in the composite image according to the computed mean and variance.

14. The system of claim 11, wherein the operations further comprise:

generating a superresolution version of the candidate text region.

15. The system of claim 14, wherein generating the superresolution version of the candidate text region comprises:

obtaining a plurality of versions of the candidate text region, each version obtained from a corresponding image of the plurality of images;

aligning the versions of the candidate text region to a high resolution grid; and

compositing the aligned versions of the candidate text region to generate the superresolution version of the candidate text region.

16. The system of claim 15, wherein the operations further comprise:

supersampling the obtained versions of a particular candidate text region from each image of the plurality of images.

17. The system of claim 15, wherein aligning the versions of the candidate text region comprises aligning pixels of the versions of the candidate text region using block matching.

18. The system of claim 15 wherein aligning the versions of the candidate text region comprises:

receiving ranging data and movement information associated with each version of the candidate text region; and

aligning the versions of the candidate text region based at least in part on the received ranging data and movement information.

19. The system of claim 15, wherein compositing the aligned versions of the candidate text region comprises combining pixels from each version of the candidate text region including:

computing a median value of pixels in each version of the candidate region; and

combining the computed median values for corresponding pixels in the aligned versions of the candidate text region.

20. A computer readable medium storing instructions that, when executed by one or more data processing apparatus, cause the one or more data processing apparatus to perform operations comprising:

receiving a plurality of different images of a first scene, wherein each image has a different exposure level;

compositing two or more images into a composite image using the plurality images;

detecting one or more features in each of one or more regions of the composite image;

determining for each region of the composite image whether the region is a candidate text region potentially containing text based on the detected one or more features; and

generating text by performing optical character recognition on a plurality of the regions determined to contain text.