Image-based localization for addresses

• US 8,688,368 B2
• Filed: 04/09/2007
• Issued: 04/01/2014
• Est. Priority Date: 04/09/2007
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Ofek, Eyal, Mishra, Pragyana K.
• Primary Examiner(s): Tran, Dalena
• Application Number:
  US11/784,863
• Publication Number:
  US 20080249702A1
• Time in Patent Office: 2,549 Days
• Field of Search: 701/201, 701/400, 701408-409, 701/426, 701/438, 701445-447, 382/103, 382/113, 382/173, 382/180, 382/191, 382/190, 382/100, 382/220, 702/2, 702/5, 702/7, 702/15, 702/16
• US Class Current: 701/409
• CPC Class Codes: G06F 16/29 Geographical information databases
• CPC Class Codes: G06F 16/58 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06T 7/74 involving reference images or patches
• CPC Class Codes: G06V 20/39 Urban scenes
• CPC Class Codes: H04N 1/32101 Display, printing, storage or transmission of additional information, e.g. ID code, date and time or title
• CPC Class Codes: H04N 2201/3253 Position information, e.g. geographical position at time of capture, GPS data
• CPC Class Codes: H04N 2201/3274 Storage or retrieval of prestored additional information

Abstract

The present localization system determines a precise location for an address based on images. Some implementations identify the precise location using one image. Other implementations identify the precise location using multiple images. While still other implementations identify the precise location by interpolation, which uses precise locations of addresses nearby that were previously localized using one or more images. The images used in determining the precise locations have a visual feature associated with the address appearing in the image.

Claims

1. A computer-readable media having computer-executable instructions that when executed, perform a method comprising:

obtaining an entity from a first database, the entity being associated with an address and a visual feature that is viewable at street level;

identifying, from a set of street level images that are each associated with respective positional data, a subset of the street level images captured in proximity to the address of the obtained entity;

analyzing each street level image in the identified subset to determine whether the visual feature appears in that street level image; and

determining a precise location for the entity based on a number of street level images in which the visual feature appears.

2. The computer-readable media recited in claim 1, wherein the number equals one and determining the precise location for the address is based on the positional data associated with the one matched street level image.

3. The computer-readable media recited in claim 1, wherein the number equals more than one and determining the precise location for the address includes triangulation using at least two of the street level images in which the visual feature appeared to identify the precise location.

4. The computer-readable media recited in claim 1, wherein the number equals zero and determining the precise location for the address includes interpolation using at least one previously determined precise location for another entity with a nearby address.

5. The computer-readable media recited in claim 1, wherein locating the set of street level images includes searching an image database in which the set of street level images are organized by associated positional data.

6. The computer-readable media recited in claim 1, further comprising storing the precise location in a second database.

7. The computer-readable media recited in claim 6, further comprising storing a matched street level image in the second database in association with the precise location for the address, the matched street level image being one of the street level images in which the visual feature appears.

8. The computer-readable media recited in claim 7, further comprising displaying a symbol on a computer-generated map, the symbol representing the precise location for the address.

9. The computer-readable media recited in claim 8, further comprising displaying the matched street level image in conjunction with the computer-generated map.

10. The computer-readable media recited in claim 1, wherein the first database comprises a yellow page directory.

11. The computer-readable media recited in claim 1, wherein the visual feature comprises a logo.

12. The computer-readable media recited in claim 1, wherein the visual feature comprises a sign.

13. The computer-readable media recited in claim 1, wherein the visual feature comprises a house number plate.

14. A computer readable storage medium storing a computer-implemented mapping application comprising:

obtaining a precise location for an entity, the precise location being derived by

obtaining, from a first database, an entity associated with an address and a visual feature that is viewable at street level,

analyzing each one of a set of street level images captured near the address associated with the obtained entity to determine whether the visual feature appears in that street level image, each street level image associated with respective positional data, and

determining the precise location based on the positional data of each street level image identified in the analyzing; and

placing a symbol on a computer-generated map based on the precise location for the entity.

15. The computer-implemented mapping application recited in claim 14, wherein at least one of the street level images displays a visual feature associated with the entity.

16. The computer-implemented mapping application recited in claim 14, wherein the precise location for the entity is retrieved from a database.

17. The computer-implemented mapping application recited in claim 14, wherein the precise location for the entity is determined in response to a request to map the address.

18. A computer-implemented method comprising:

ascertaining whether a visual feature is present within a street level image, the visual feature including a feature that is viewable at street level, the street level image being associated with positional data that identifies a location where the street level image was captured; and

determining, after the ascertaining, a precise location for an entity associated with the visual feature based on the positional data, when the visual feature is determined to be present within the street level image.

19. The computer-implemented method recited in claim 18, wherein determining the precise location comprises calculating a distance from the location where the street level image was captured to the entity and adding the distance to the positional data.

20. The computer-implemented method recited in claim 18, wherein the positional data comprises a geographic coordinates obtained via a global positioning device.