Using extracted image text

• US 9,542,612 B2
• Filed: 01/13/2016
• Issued: 01/10/2017
• Est. Priority Date: 06/29/2006
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Vincent, Luc, Ulges, Adrian
• Primary Examiner(s): Desire, Gregory M
• Application Number:
  US14/994,638
• Publication Number:
  US 20160125254A1
• Time in Patent Office: 363 Days
• US Class Current: 1/1
• CPC Class Codes: G01S 19/51 Relative positioning
• CPC Class Codes: G06F 16/5846 using extracted text
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 3/4053 based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/33 using feature-based methods
• CPC Class Codes: G06V 20/56 exterior to a vehicle by using sensors mounted on the vehicle
• CPC Class Codes: G06V 20/63 Scene text, e.g. street names
• CPC Class Codes: G06V 30/10 Character recognition

Abstract

Methods, systems, and apparatus including computer program products for using extracted image text are provided. In one implementation, a computer-implemented method is provided. The method includes receiving an input of one or more image search terms and identifying keywords from the received one or more image search terms. The method also includes searching a collection of keywords including keywords extracted from image text, retrieving an image associated with extracted image text corresponding to one or more of the image search terms, and presenting the image.

Claims

1. A computer-implemented method comprising:

receiving a plurality of different images of a first scene, wherein each image has a different exposure level;

generating a high dynamic range image using the plurality images;

detecting one or more features in each of one or more regions of the high dynamic range image;

determining for each region of the high dynamic range image whether the region is a candidate text region potentially containing text based on the detected one or more features; and

generating text by performing optical character recognition on a plurality of the regions determined to contain text.

2. The method of claim 1, further comprising:

increasing contrast of the high dynamic range image including normalizing pixel values in the high dynamic range image.

3. The method of claim 2, wherein normalizing pixel values in the high dynamic range image comprises:

computing a mean and variance of pixel values in the high dynamic range image; and

scaling pixel values in the high dynamic range image according to the computed mean and variance.

4. The method of claim 1, further comprising:

generating a superresolution version of the candidate text region.

5. The method of claim 4, wherein generating the superresolution version of the candidate text region comprises:

obtaining a plurality of versions of the candidate text region, each version obtained from a corresponding image of the plurality of images;

aligning the versions of the candidate text region to a high resolution grid; and

compositing the aligned versions of the candidate text region to generate the superresolution version of the candidate text region.

6. The method of claim 5, further comprising:

supersampling the obtained versions of a particular candidate text region from each image of the plurality of images.

7. The method of claim 5, wherein aligning the versions of the candidate text region comprises aligning pixels of the versions of the candidate text region using block matching.

8. The method of claim 5, wherein aligning the versions of the candidate text region comprises:

receiving ranging data and movement information associated with each version of the candidate text region; and

aligning the versions of the candidate text region based at least in part on the received ranging data and movement information.

9. The method of claim 5, wherein compositing the aligned versions of the candidate text region comprises combining pixels from each version of the candidate text region including:

computing a median value of pixels in each version of the candidate region; and

combining the computed median values for corresponding pixels in the aligned versions of the candidate text region.

10. A system, comprising:

one or more processing devices; and

a memory storage apparatus in data communication with the one or more processing devices and storing instructions that cause the one or more processing devices to perform operations comprising;

receiving a plurality of different images of a first scene, wherein each image has a different exposure level;

generating a high dynamic range image using the plurality images;

detecting one or more features in each of one or more regions of the high dynamic range image;

determining for each region of the high dynamic range image whether the region is a candidate text region potentially containing text based on the detected one or more features; and

generating text by performing optical character recognition on a plurality of the regions determined to contain text.

11. The system of claim 10, the operations further comprising:

increasing contrast of the high dynamic range image including normalizing pixel values in the high dynamic range image.

12. The system of claim 11, wherein normalizing pixel values in the high dynamic range image comprises:

computing a mean and variance of pixel values in the high dynamic range image; and

scaling pixel values in the high dynamic range image according to the computed mean and variance.

13. The system of claim 10, the operations further comprising:

generating a superresolution version of the candidate text region.

14. The system of claim 13, wherein generating the superresolution version of the candidate text region comprises:

obtaining a plurality of versions of the candidate text region, each version obtained from a corresponding image of the plurality of images;

aligning the versions of the candidate text region to a high resolution grid; and

compositing the aligned versions of the candidate text region to generate the superresolution version of the candidate text region.

15. The system of claim 14, the operations further comprising:

supersampling the obtained versions of a particular candidate text region from each image of the plurality of images.

16. The system of claim 14, wherein aligning the versions of the candidate text region comprises aligning pixels of the versions of the candidate text region using block matching.

17. The system of claim 14, wherein aligning the versions of the candidate text region comprises:

receiving ranging data and movement information associated with each version of the candidate text region; and

aligning the versions of the candidate text region based at least in part on the received ranging data and movement information.

18. The system of claim 14, wherein compositing the aligned versions of the candidate text region comprises combining pixels from each version of the candidate text region including:

computing a median value of pixels in each version of the candidate region; and

combining the computed median values for corresponding pixels in the aligned versions of the candidate text region.

19. A memory storage apparatus storing instructions that cause the one or more processing devices to perform operations comprising:

receiving a plurality of different images of a first scene, wherein each image has a different exposure level;

generating a high dynamic range image using the plurality images;

detecting one or more features in each of one or more regions of the high dynamic range image;

determining for each region of the high dynamic range image whether the region is a candidate text region potentially containing text based on the detected one or more features; and

generating text by performing optical character recognition on a plurality of the regions determined to contain text.