Inferring locations from an image

• US 8,131,118 B1
• Filed: 01/31/2008
• Issued: 03/06/2012
• Est. Priority Date: 01/31/2008
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Jing, Yushi, Yuan, Jian
• Primary Examiner(s): Couso, Yon
• Application Number:
  US12/023,763
• Time in Patent Office: 1,496 Days
• Field of Search: 382/100, 382/217, 382/218, 382/305, 709/207, 709/204, 709/223, 348/113, 348/116, 455/456.1, 455/457, 701/200, 701/207, 701/208
• US Class Current: 382/305
• CPC Class Codes: G01C 21/20 Instruments for performing navigational calculations G01C21/24, G01C21/26 take precedence
• CPC Class Codes: G01C 21/30 Map- or contour-matching
• CPC Class Codes: G01S 5/16 using electromagnetic waves other than radio waves
• CPC Class Codes: G06F 16/583 using metadata automatically derived from the content
• CPC Class Codes: G06F 18/24 Classification techniques
• CPC Class Codes: G06V 20/20 in augmented reality scenes

Abstract

Systems and methods are disclosed for determining the location where an image was captured. In general, a device such as a smartphone may capture one or more images from a location, such as images of buildings, street signs and the like, and a central system may compare the submitted images to images in an image library to identify matches. The location of the match may then be provided back to the smartphone.

Claims

1. A computer-implemented method comprising:

receiving, at a computer server system, an image from a remote electronic device;

comparing, with the computer sever system information from the received image with corresponding information from a plurality of stored images, the comparison comprising;

(a) calculating a level of difference between a first set of feature points in the received image and a second set of feature points in the plurality of stored images; and

(b) identifying at least one image within the plurality of stored images that satisfies a threshold level of difference between the first set of feature points and the second set of feature points;

extracting, with the computer server system, location information from the at least one identified image to determine location information to associate with the received image;

detecting a difference in illumination of an object in the received image and, based on the difference and a time of day the image was captured, inferring a user position relative to the object; and

transmitting, from the computer server system to the remote device, information identifying a geographic location, the information identifying the geographic location based on the extracted location information and the inferred user position.

2. The method of claim 1, wherein the compared information from the images comprises information from multiple sets of feature points in the images.

3. The method of claim 1, further comprising transmitting a prompt requesting a user to submit information on a name of a roadway or other textual geographic description, and using resulting supplied information to identify location-related information for the received image.

4. The method of claim 1, further comprising generating a match quality indicator for a match between the received image and a stored image, and assigning location information from the stored image to the received image if the quality indicator exceeds a determined level.

5. The method of claim 1, further comprising using information about the remote device location from a source other than the remote device to determine the location for the received image.

6. The method of claim 1, further comprising transmitting to the remote device information relating to events occurring near the identified location.

7. The method of claim 1, further comprising analyzing image characteristics within the received image to determine a user position relative to a subject depicted in the received image.

8. The method of claim 7, wherein the user position is represented by a facing direction of said user relative to the subject depicted in the received image.

9. The method of claim 7, wherein the user position is represented by a proximate distance indicator to a specific geographical location depicted in the received image.

10. A system comprising:

one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising;

receiving, at a computer server system, an image from a remote electronic device;

comparing, with the computer server system, information from the received image with corresponding information from a plurality of stored images, the comparison including;

(a) calculating a level of difference between a first set of feature points in the received image and a second set of feature points in the plurality of stored images; and

(b) identifying at least one image within the plurality of stored images that satisfies a threshold level of difference between the first set of feature points and the second set of feature points;

extracting, with the computer server system, location information from the at least one identified image to determine location information to associate with the received image;

detecting a difference in illumination of an object in the received image and, based on the difference and a time of day the image was captured, inferring a user position relative to the object; and

transmitting, from the computer server system to the remote device, information identifying a geographic location, the information identifying the geographic location based on the extracted location information and the inferred user position.

11. The system of claim 10, wherein the one or more computers automatically return local search results for the identified geographic locations.

12. The system of claim 10, wherein the one or more computers automatically provide business information pertaining to the geographically-associated items.

13. A computer-readable medium having instructions encoded thereon, which, when executed by a processor, cause the processor to perform operations comprising:

receiving, at a computer server system, an image from a remote electronic device;

comparing, with the computer server system, information from the received image with corresponding information from a plurality of stored images, the comparison comprising;

(a) calculating a level of difference between a first set of feature points in the received image and a second set of feature points in the plurality of stored images; and

(b) identifying at least one image within the plurality of stored images that satisfies a threshold level of difference between the first set of feature points and the second set of feature points;

extracting, with the computer server system, location information from the at least one identified image to determine location information to associate with the received image;

detecting a difference in illumination of an object in the received image and, based on the difference and a time of day the image was captured, inferring a user position relative to the object; and

transmitting, from the computer server system to the remote device, information identifying a geographic location, the information identifying the geographic location based on the extracted location information and the inferred user position.

14. The medium of claim 13, wherein the compared information from the images comprises information from multiple sets of feature points in the images.

15. The medium of claim 13, wherein the operations further comprise transmitting a prompt requesting a user to submit information on a name of a roadway or other textual geographic description, and using resulting supplied information to identify location-related information for the received image.

16. The medium of claim 13, wherein the operations further comprise transmitting to the remote device data for displaying a map with the location for the received image marked on the map.

17. The medium of claim 13, wherein the operations further comprise receiving a series of images in a video and comparing a plurality of images in the series to stored images to determine a geographic location corresponding to the received images.

18. The medium of claim 13, wherein the operations further comprise generating a match quality indicator for a match between the received image and a stored image, and assigning location information from the stored image to the received image if the quality indicator exceeds a determined level.

19. The medium of claim 13, wherein the operations further comprise using information about the remote device location from a source other than the remote device to determine the location for the received image.

20. The medium of claim 13, wherein the operations further comprise transmitting to the remote device information relating to events occurring near the identified location.