System and method of determining map coordinates from images

• US 8,577,604 B1
• Filed: 06/10/2011
• Issued: 11/05/2013
• Est. Priority Date: 07/07/2010
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Ogale, Abhijit, Lafon, Stephane, Frome, Andrea
• Primary Examiner(s): Black, Thomas
• Assistant Examiner(s): SMITH, ISAAC G
• Application Number:
  US13/157,831
• Time in Patent Office: 879 Days
• Field of Search: 701/200, 701/414, 701/428, 701/438, 701/439, 701/500, 701/522, 701/532, 701/533, 701/450, 701/469, 701/523, 707/737, 707/769, 340/990, 345/419, 345/420, 345/441, 345/620, 348/169, 348/231.6, 348/345, 348/42, 348/48, 382/103, 382/104, 382/153, 382/154, 382/182, 382/276, 382/284, 382/289, 382/305, 382/310, 382/107, 382/190, 382/298, 382/294
• US Class Current: 701/450
• CPC Class Codes: G01C 21/26 specially adapted for navigation in a road network
• CPC Class Codes: G01C 21/34 Route searching; Route guidance
• CPC Class Codes: G01C 21/3476 using point of interest [POI] information, e.g. a route passing visible POIs
• CPC Class Codes: G01C 21/3602 Input other than that of destination using image analysis, e.g. detection of road signs, lanes, buildings, real preceding vehicles using a camera
• CPC Class Codes: G01C 21/3811 Point data, e.g. Point of Interest [POI]
• CPC Class Codes: G01C 21/3837 Data obtained from a single source
• CPC Class Codes: G01S 17/08 for measuring distance only indirect measurement G01S17/46; active triangulation systems G01S17/48
• CPC Class Codes: G01S 19/51 Relative positioning
• CPC Class Codes: G01S 5/16 using electromagnetic waves other than radio waves

Abstract

Systems and methods provide approximations of latitude and longitude coordinates of objects, for example a business, in street level images. The images may be collected by a camera. An image of a business is collected along with GPS coordinates and direction of the camera. Depth maps of the images may be generated, for example, based on laser depth detection or displacement of the business between two images caused by a change in the position of the camera. After identifying a business in one or more images, the distance from the camera to a point or area relative to the business in the one or more images may be determined based on the depth maps. Using this distance and the direction of the camera which collected the one or more images and GPS coordinates of the camera, the approximate GPS coordinates of the business may be determined.

Claims

1. A method comprising:

accessing a plurality of images, each image of the plurality being associated with geographic location information and direction information identifying a view point of a camera that captured the image;

identifying a portal in an image of the plurality of images, wherein the portal is associated with a point of interest identified by a location of a map marker in map data;

accessing surface depth data identifying one or more distances from the geographic location of the camera that captured the image to objects of the image;

determining a distance from the camera that captured the image to the portal in the image based on the surface depth data;

determining, by a processor, an approximate geographic location of the portal based on the determined distance and the geographic location information and direction information associated with the image; and

using the approximate geographic location of the portal to adjust the location of the map marker in the map data.

2. The method of claim 1, wherein the approximate geographic location of the portal is latitude and longitude coordinates.

3. The method of claim 1, further comprising:

identifying a second image of the plurality of images having the portal therein;

determining a second approximate geographic location of the portal using the second image; and

determining a third approximate geographic location of the portal based on the second approximate geographic location of the portal and the approximate geographic location of the portal, and wherein the location of the map marker is adjusted further based on the third approximate geographic location.

4. The method of claim 1, further comprising adjusting the location of a second marker in the map data based on the adjusted location of the map marker in the map data, the second marker corresponding to a second point of interest proximate to the point of interest.

5. The method of claim 1, further comprising:

receiving, from a client device, a request for turn by turn directions, the request including a first location and a destination location;

when the destination location corresponds to the point of interest,

generating a set of turn by turn directions based on the adjusted the location of the map marker in the map data; and

transmitting the turn by turn directions to the client device.

6. The method of claim 1, wherein the portal includes an entry to a building and provides access to the point of interest.

7. The method of claim 1, wherein the portal includes text on a building.

8. The method of claim 1, wherein the portal includes a logo on a building.

9. A device comprising:

memory storing the plurality of images, each of the plurality of images being associated with geographic location information and direction information identifying a view point of a camera that captured the image;

a processor coupled to the memory, the processor being configured to;

identify the portal in and image of the plurality of images, wherein the portal is associated with a point of interest identified by a location of a map marker in map data;

access surface depth data identifying one or more distances from the geographic location of the camera that captured the image to objects of the image;

determine a distance from the camera that captured the image to the portal in the image based on the surface depth data;

determine an approximate geographic location of the portal based on the determined distance and the geographic location information and direction information associated with the selected image; and

use the approximate geographic location of the portal to adjust the location of the map marker in the map data.

10. The device of claim 9, wherein the approximate geographic location of the portal is latitude and longitude coordinates.

11. The device of claim 9, wherein the processor is further configured to:

identify a second image of the plurality images having the portal therein;

determine a second approximate geographic location of the portal using the second image; and

determine a third approximate geographic location of the portal based on the second approximate geographic location of the portal and the approximate geographic location of the portal, and wherein the location of the map marker is adjusted further based on the third approximate geographic location.

12. The device of claim 9, wherein the processor is further configured to adjust the location of a second marker in the map data based on the adjusted location of the map marker in the map data, the second marker corresponding to a second point of interest proximate to the point of interest.

13. The device of claim 11, wherein the processor is further configured to:

receive, from a client device, a request for turn by turn directions, the request including a first location and a destination location;

when the destination location corresponds to the point of interest,

generate a set of turn by turn directions based on the adjusted the location of the map marker in the map data; and

transmit the turn by turn directions to the client device.

14. A non-transitory, tangible computer-readable storage medium on which computer readable instructions of a program are stored, the instructions, when executed by a processor, cause the processor to perform a method, the method comprising:

accessing a plurality of images, each image of the plurality being associated with geographic location information and direction information identifying a view point of a camera that captured the image;

identifying a portal in an image of the plurality of images, wherein the portal is associated with a point of interest and provides access to the point of interest, and the point of interest is identified a map marker in map data;

accessing surface depth data identifying one or more distances from the geographic location of the camera that captured the image to objects of the image;

determining a distance from the camera that captured the image to the portal in the image based on the surface depth data;

determining an approximate geographic location of the portal based on the determined distance and the geographic location information and direction information associated with the selected image; and

using the approximate geographic location of the portal to adjust the location of the map marker in the map data.

15. The tangible computer-readable storage medium of claim 14, wherein the approximate geographic location of the portal is latitude and longitude coordinates.

16. The tangible computer-readable storage medium of claim 14, wherein the method further comprises:

identifying a second image of the plurality of images having the portal therein;

determining a second approximate geographic location of the portal using the second image; and

determining a third approximate geographic location of the portal based on the second approximate geographic location of the portal and the approximate geographic location of the portal, and wherein the location of the map marker is adjusted further based on the third approximate geographic location.

17. The tangible computer-readable storage medium of claim 14, wherein the method further comprises adjusting the location of a second marker in the map data based on the adjusted location of the marker in the map data, the second marker corresponding to a second point of interest proximate to the point of interest.

18. The tangible computer-readable storage medium of claim 16, wherein the method further comprises:

receiving, from a client device, a request for turn by turn directions, the request including a first location and a destination location;

when the destination location corresponds to the point of interest associated with the portal,

generating a set of turn by turn directions based on the adjusted location of the marker in the map data; and

transmitting the turn by turn directions to the client device.

19. The tangible computer-readable storage medium of claim 14, wherein the portal includes a building number of a building.

20. The tangible computer-readable storage medium of claim 14, wherein the portal includes text on a building.