Using extracted image text

• US 9,760,781 B2
• Filed: 05/03/2017
• Issued: 09/12/2017
• Est. Priority Date: 06/29/2006
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Ulges, Adrian, Vincent, Luc
• Primary Examiner(s): Desire, Gregory M
• Application Number:
  US15/585,767
• Publication Number:
  US 20170236012A1
• Time in Patent Office: 132 Days
• CPC Class Codes: G01S 19/51 Relative positioning
• CPC Class Codes: G06F 16/5846 using extracted text
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 3/4053 based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/33 using feature-based methods
• CPC Class Codes: G06V 20/56 exterior to a vehicle by using sensors mounted on the vehicle
• CPC Class Codes: G06V 20/63 Scene text, e.g. street names
• CPC Class Codes: G06V 30/10 Character recognition

Abstract

Methods, systems, and apparatus including computer program products for using extracted image text are provided. In one implementation, a computer-implemented method is provided. The method includes receiving an input of one or more image search terms and identifying keywords from the received one or more image search terms. The method also includes searching a collection of keywords including keywords extracted from image text, retrieving an image associated with extracted image text corresponding to one or more of the image search terms, and presenting the image.

Claims

1. A computer-implemented method, comprising:

receiving an image of a street scene depicting a first location for which first global positioning system coordinates is known and depicting a second location for which second global positioning system coordinates is known;

identifying a set of intermediate points between the first location and the second location depicted within the image;

determining, based on the first global positioning system coordinates and the second global positioning coordinates, and for each intermediate point in the set of intermediate points, global positioning coordinates indicative of a location of the intermediate point, wherein the global positioning coordinates are for each intermediate point are different from the first global positioning system coordinates and the second global positioning coordinates; and

associating, with the image, the global positioning coordinates indicative of the locations of the intermediate points with the intermediate points of the image.

2. The computer implemented method of claim 1, wherein determining global positioning coordinates indicative of a location of the intermediate point comprises linearly interpolating between the first global positioning system coordinates and the second global positioning coordinates based on a relative position of the intermediate point from the first location depicted in the scene and the second location depicted in the scene.

3. The computer implemented method of claim 1, wherein:

identifying a set of intermediate points comprises identifying vertical lines in the image; and

determining global positioning coordinates indicative of a location of the intermediate point comprises linearly interpolating between the first global positioning system coordinates and the second global positioning coordinates based on a relative position of the vertical line from the first location depicted in the scene and the second location depicted in the scene.

4. The computer implemented method of claim 1, wherein the first location for which first global positioning system coordinates is known is a first street address and the second location for which second global positioning system coordinates is known is a second street address.

5. The computer implement method of claim 1, wherein:

the image comprises a first image merged with a second image; and

the first image depicts the first location and the second image depicts the second location.

6. A data processing apparatus, comprising:

one or more processors; and

a computer readable memory device storing instructions executable by the one or more processors and that upon such execution cause the one or more processors to perform operations comprising;

receiving an image of a street scene depicting a first location for which first global positioning system coordinates is known and depicting a second location for which second global positioning system coordinates is known;

identifying a set of intermediate points between the first location and the second location depicted within the image;

determining, based on the first global positioning system coordinates and the second global positioning coordinates, and for each intermediate point in the set of intermediate points, global positioning coordinates indicative of a location of the intermediate point, wherein the global positioning coordinates are for each intermediate point are different from the first global positioning system coordinates and the second global positioning coordinates; and

associating, with the image, the global positioning coordinates indicative of the locations of the intermediate points with the intermediate points of the image.

7. The data processing apparatus of claim 6, wherein determining global positioning coordinates indicative of a location of the intermediate point comprises linearly interpolating between the first global positioning system coordinates and the second global positioning coordinates based on a relative position of the intermediate point from the first location depicted in the scene and the second location depicted in the scene.

8. The data processing apparatus of claim 6, wherein:

identifying a set of intermediate points comprises identifying vertical lines in the image; and

determining global positioning coordinates indicative of a location of the intermediate point comprises linearly interpolating between the first global positioning system coordinates and the second global positioning coordinates based on a relative position of the vertical line from the first location depicted in the scene and the second location depicted in the scene.

9. The data processing apparatus of claim 6, wherein the first location for which first global positioning system coordinates is known is a first street address and the second location for which second global positioning system coordinates is known is a second street address.

10. The data processing apparatus of claim 6, wherein:

the image comprises a first image merged with a second image; and

the first image depicts the first location and the second image depicts the second location.

11. A computer readable memory device storing instructions executable by one or more processors and that upon such execution cause the one or more processors to perform operations comprising:

receiving an image of a street scene depicting a first location for which first global positioning system coordinates is known and depicting a second location for which second global positioning system coordinates is known;

identifying a set of intermediate points between the first location and the second location depicted within the image;

determining, based on the first global positioning system coordinates and the second global positioning coordinates, and for each intermediate point in the set of intermediate points, global positioning coordinates indicative of a location of the intermediate point, wherein the global positioning coordinates are for each intermediate point are different from the first global positioning system coordinates and the second global positioning coordinates; and

associating, with the image, the global positioning coordinates indicative of the locations of the intermediate points with the intermediate points of the image.

12. The computer readable memory device of claim 11, wherein determining global positioning coordinates indicative of a location of the intermediate point comprises linearly interpolating between the first global positioning system coordinates and the second global positioning coordinates based on a relative position of the intermediate point from the first location depicted in the scene and the second location depicted in the scene.

13. The computer readable memory device of claim 11, wherein:

identifying a set of intermediate points comprises identifying vertical lines in the image; and

determining global positioning coordinates indicative of a location of the intermediate point comprises linearly interpolating between the first global positioning system coordinates and the second global positioning coordinates based on a relative position of the vertical line from the first location depicted in the scene and the second location depicted in the scene.

14. The computer readable memory device of claim 11, wherein the first location for which first global positioning system coordinates is known is a first street address and the second location for which second global positioning system coordinates is known is a second street address.

15. The computer readable memory device of claim 11, wherein:

the image comprises a first image merged with a second image; and

the first image depicts the first location and the second image depicts the second location.