Using extracted image text

• US 8,503,782 B2
• Filed: 01/13/2012
• Issued: 08/06/2013
• Est. Priority Date: 06/29/2006
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Vincent, Luc, Ulges, Adrian
• Primary Examiner(s): Desire, Gregory M
• Application Number:
  US13/350,726
• Publication Number:
  US 20120114241A1
• Time in Patent Office: 571 Days
• Field of Search: 382/168, 382/190, 382/218, 382/219, 382/305, 382/306, 715/968
• US Class Current: 382/177
• CPC Class Codes: G01S 19/51 Relative positioning
• CPC Class Codes: G06F 16/5846 using extracted text
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 3/4053 based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/33 using feature-based methods
• CPC Class Codes: G06V 20/56 exterior to a vehicle by using sensors mounted on the vehicle
• CPC Class Codes: G06V 20/63 Scene text, e.g. street names
• CPC Class Codes: G06V 30/10 Character recognition

Abstract

Methods, systems, and apparatus including computer program products for using extracted image text are provided. In one implementation, a computer-implemented method is provided. The method includes receiving an input of one or more image search terms and identifying keywords from the received one or more image search terms. The method also includes searching a collection of keywords including keywords extracted from image text, retrieving an image associated with extracted image text corresponding to one or more of the image search terms, and presenting the image.

Claims

1. A computer-implemented method comprising:

receiving one or more search terms;

identifying, in a collection of images, an image associated with the one or more search terms, the one or more search terms having been extracted from two or more adjacent candidate text regions in the image, the candidate text regions identified as regions of the image containing text; and

providing the image in response to receiving the search terms,

wherein the one or more search terms that were extracted from the image were extracted using a text recognition process including;

processing the image to divide the image into one or more regions;

detecting one or more features in each region;

detecting one or more candidate text regions including the two or more adjacent candidate text regions, wherein candidate text regions are regions potentially containing text;

enhancing the candidate text regions to generate an enhanced image; and

performing optical character recognition on the enhanced image.

2. The method of claim 1, wherein enhancing the candidate text regions to generate an enhanced image comprises compositing corresponding regions in a plurality of images to create a single superresolution image.

3. The method of claim 1, wherein identifying, in a collection of images, an image associated with the one or more search terms comprises giving a greater weight to matching search terms that were extracted from the candidate text regions.

4. The method of claim 1, further comprising:

providing one or more advertisements with the provided image.

5. The method of claim 4, further comprising:

identifying the one or more advertisements based at least in part on the received search terms.

6. The method of claim 4, further comprising:

identifying the one or more advertisements based at least in part on the one or more search terms extracted from the candidate text regions.

7. The method of claim 1, further comprising:

presenting a visual identifier in the text of the provided image, wherein the visual identifier corresponds to the received search terms.

8. The method of claim 7, wherein presenting the visual identifier includes highlighting text in the provided image.

9. A system comprising:

one or more data processing apparatus; and

a computer-readable storage device having stored thereon instructions that, when executed by the one or more data processing apparatus, cause the one or more data processing apparatus to perform operations comprising;

receiving one or more search terms;

identifying, in a collection of images, an image associated with the one or more search terms, the one or more search terms having been extracted from two or more adjacent candidate text regions in the image, the candidate text regions identified as regions of the image containing text; and

providing the image in response to receiving the search terms,

wherein the one or more search terms that were extracted from the image were extracted using a text recognition process including;

processing the image to divide the image into one or more regions;

detecting one or more features in each region;

detecting one or more candidate text regions including the two or more adjacent candidate text regions, wherein candidate text regions are regions potentially containing text;

enhancing the candidate text regions to generate an enhanced image; and

performing optical character recognition on the enhanced image.

10. The system of claim 9, wherein enhancing the candidate text regions to generate an enhanced image comprises compositing corresponding regions in a plurality of images to create a single superresolution image.

11. The system of claim 9, wherein identifying, in a collection of images, an image associated with the one or more search terms comprises giving a greater weight to matching search terms that were extracted from the candidate text regions.

12. The system of claim 9, wherein the operations further comprise:

providing one or more advertisements with the provided image.

13. The system of claim 12, wherein the operations further comprise:

identifying the one or more advertisements based at least in part on the received search terms.

14. The system of claim 12, wherein the operations further comprise:

identifying the one or more advertisements based at least in part on the one or more search terms extracted from the candidate text regions.

15. The system of claim 9, wherein the operations further comprise:

presenting a visual identifier in the text of the provided image, wherein the visual identifier corresponds to the received search terms.

16. The system of claim 15, wherein presenting the visual identifier includes highlighting text in the provided image.