Using extracted image text

• US 8,098,934 B2
• Filed: 06/29/2006
• Issued: 01/17/2012
• Est. Priority Date: 06/29/2006
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Vincent, Luc, Ulges, Adrian
• Primary Examiner(s): Desire, Gregory M
• Application Number:
  US11/479,155
• Publication Number:
  US 20080002916A1
• Time in Patent Office: 2,028 Days
• Field of Search: 382/164, 382/168, 382/171, 382/173, 382/176, 382/177, 382/190, 382/229, 382/299, 382/305
• US Class Current: 382/177
• CPC Class Codes: G01S 19/51 Relative positioning
• CPC Class Codes: G06F 16/5846 using extracted text
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 3/4053 based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/33 using feature-based methods
• CPC Class Codes: G06V 20/56 exterior to a vehicle by using sensors mounted on the vehicle
• CPC Class Codes: G06V 20/63 Scene text, e.g. street names
• CPC Class Codes: G06V 30/10 Character recognition

Abstract

Methods, systems, and apparatus including computer program products for using extracted image text are provided. In one implementation, a computer-implemented method is provided. The method includes receiving an input of one or more image search terms and identifying keywords from the received one or more image search terms. The method also includes searching a collection of keywords including keywords extracted from image text, retrieving an image associated with extracted image text corresponding to one or more of the image search terms, and presenting the image.

Claims

1. A computer-implemented method comprising:

receiving an image and data identifying a geographic location corresponding to where the image was captured;

extracting text from within the image;

indexing the extracted text with the data identifying the geographic location;

receiving a request and using the indexed extracted text and the data identifying the geographic location to determine that the image satisfies the request;

providing search results relevant to the geographic location and the image in response to the request; and

in which receiving, extracting, indexing, and presenting are performed by one or more data processing apparatuses.

2. The method of claim 1, where extracting text from within the image comprises:

processing the image to divide the image into one or more regions;

detecting one or more features in each region;

determining for each region whether it is a candidate text region potentially containing text using the detected features in the region;

enhancing the candidate text regions to generate an enhanced image; and

performing optical character recognition on the enhanced image.

3. The method of claim 1, where providing search results relevant to the geographic location includes providing data identifying the geographic location associated with the indexed text.

4. The method of claim 1, where the request includes one or more keyword search terms.

5. The method of claim 1, where the request includes an address associated with the geographic location.

6. The method of claim 1, further comprising:

providing a map of the geographic location associated with the image.

7. The method of claim 1, further comprising:

providing one or more advertisements with the provided image.

8. The method of claim 7, where the provided one or more advertisements are determined according to content of the request.

9. The method of claim 7, where the provided one or more advertisements are determined using the extracted text of the presented image.

10. A machine-readable storage device having stored thereon instructions, which, when executed by data processing apparatus, cause the data processing apparatus to perform operations comprising:

receiving an image and data identifying a geographic location corresponding to where the image was captured;

extracting text from within the image;

indexing the extracted text with the data identifying the geographic location;

receiving a request and using the indexed extracted text and the data identifying the geographic location to determine that the image satisfies the request;

providing search results relevant to the geographic location and the image in response to the request.

11. The storage device of claim 10, where extracting text from within the image comprises:

processing the image to divide the image into one or more regions;

detecting one or more features in each region;

determining for each region whether it is a candidate text region potentially containing text using the detected features in the region;

enhancing the candidate text regions to generate an enhanced image; and

performing optical character recognition on the enhanced image.

12. The storage device of claim 10, where providing search results relevant to the geographic location includes providing data identifying the geographic location associated with the indexed text.

13. The storage device of claim 10, where the user request includes one or more keyword search terms.

14. The storage device of claim 10, where the user request includes an address associated with the geographic location.

15. The storage device of claim 10, wherein the operations further comprise:

providing a map of the geographic location associated with the image.

16. The storage device of claim 10, wherein the operations further comprise:

providing one or more advertisements with the provided image.

17. The storage device of claim 16, where the provided one or more advertisements are determined according to content of the request.

18. The storage device of claim 16, where the provided one or more advertisements are determined using the extracted text of the provided image.

19. A system comprising:

one or more data processing apparatuses configured to perform operations comprising;

receiving an image and data identifying a geographic location corresponding to where the image was captured;

extracting text from within the image;

indexing the extracted text with the data identifying the geographic location;

receiving a request and using the indexed extracted text and the data identifying the geographic location to determine that the image satisfies the request;

providing search results relevant to the geographic location and the image in response to the request.

20. The system of claim 19, where extracting text from within the image comprises:

processing the image to divide the image into one or more regions;

detecting one or more features in each region;

determining for each region whether it is a candidate text region potentially containing text using the detected features in the region;

enhancing the candidate text regions to generate an enhanced image; and

performing optical character recognition on the enhanced image.

21. The system of claim 19, where providing search results relevant to the geographic location includes providing data identifying the geographic location associated with the indexed text.

22. The system of claim 19, where the request includes one or more keyword search terms.

23. The system of claim 19, where the request includes an address associated with the geographic location.

24. The system of claim 19, wherein the operations further comprise:

providing a map of the geographic location associated with the image.

25. The system of claim 19, wherein the operations further comprise:

providing one or more advertisements with the provided image.

26. The system of claim 25, where the provided one or more advertisements are determined according to content of the request.

27. The system of claim 25, where the provided one or more advertisements are determined using the extracted text of the provided image.