Visual assessment of landmarks

• US 8,060,302 B2
• Filed: 03/31/2009
• Issued: 11/15/2011
• Est. Priority Date: 03/31/2009
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Epshtein, Boris, Ofek, Eyal
• Primary Examiner(s): O'Connor; Gerald J.
• Assistant Examiner(s): WINSTON III, EDWARD B
• Application Number:
  US12/416,115
• Publication Number:
  US 20100250126A1
• Time in Patent Office: 959 Days
• Field of Search: 701/2, 701/201, 701/207, 701/209
• US Class Current: 701/209
• CPC Class Codes: G01C 21/3602 Input other than that of destination using image analysis, e.g. detection of road signs, lanes, buildings, real preceding vehicles using a camera
• CPC Class Codes: G01C 21/3644 Landmark guidance, e.g. using POIs or conspicuous other objects

Abstract

Visual images may be used to detect, or verify the existence of, landmarks. In one example, the landmarks may be used to provide driving or walking directions. A set of records may be examined to identify the street addresses of businesses or other entities. Additionally, street-level images may be captured with a camera, and the locations at which the images were captured may be recorded. The images may be evaluated with an Optical Character Recognition (OCR) process to determine what words appear in the images. The words in the image are compared with the names of entities whose addresses are near where the image was captured. If the words match the entity name, then a sign identifying the entity is presumed to be visible along a particular route, and the entity may be used as a landmark in a set of walking or driving directions.

Claims

1. One or more computer-readable storage media that store executable instructions that, when executed by a computer, cause the computer to perform acts comprising:

receiving a record that comprises a name of an entity and a street address of said entity;

receiving a plurality of images;

identifying a set of said images that were captured within a first distance of a location of said entity, said location being determined from said street address;

performing a character recognition process on said set of images to identify words that appear in said set of images;

comparing said words to said name;

determining, based on comparison of said words with said name that said entity appears in an image that is in said set of set of images; and

providing directions to a person, wherein said directions include said entity as a landmark;

said comparing comprising;

calculating Levenshtein distances between words in said set of images and words in said name; and

determining that one of said Levenshtein distances is less that a quantity.

2. The one or more computer-readable storage media of claim 1, wherein said comparing further comprises:

determining how many words in one image in said set match a number of words in said name.

3. The one or more computer-readable storage media of claim 1, wherein said comparing further comprises:

determining which words in one image in said set are not found within said first distance of said location of said entity; and

determining that one of the words in one image in said set, that are not found within said first distance of said location of said entity, matches one of the words in said name.

4. The one or more computer-readable storage media of claim 1, further comprising:

determining which of said images in said set contains words that have a smallest Levenshtein distance to said name.

5. The one or more computer-readable storage media of claim 1, further comprising:

determining which images in said set were captured from a street whose name matches a street that is part of said street address.

6. The one or more computer-readable storage media of claim 1, further comprising:

rating said entity'"'"'s quality as a landmark based on one or more criteria.

7. A system for assessing landmarks, the system comprising:

a database that stores records of entities, said records identifying names and addresses of said entities;

a landmark detector that receives a record of an entity that is one of said entities, and that further receives a plurality of images, said landmark detector identifying, based on geographic locations at which said images were captured, which of said images were captured within a distance from said entity, said landmark detector identifying words that appear in each of said images, said landmark detector determining, based on a comparison of words in said images with words in a name of said entity, which of said images said entity appears in, said landmark detector rating said entity'"'"'s quality as a landmark based on one or more criteria, said landmark detector determining that one of said images contains a first word that appears in a name of said entity by calculating a Levenshtein distance between words in said name and said first word, and by determining that said Levenshtein distance is lower than an amount; and

a navigation application that provides directions using said entity as a landmark, said navigation application choosing said entity as a landmark based on said entity'"'"'s quality as a landmark as rated by said landmark detector.

8. The system of claim 7, wherein said landmark detector rates said entity'"'"'s quality as a landmark based on said entity being a particular type of business.

9. The system of claim 7, wherein said landmark detector rates said entity'"'"'s quality as a landmark based on said entity being at or near an intersection.

10. The system of claim 7, wherein said landmark detector rates said entity'"'"'s quality as a landmark based on a size of a sign that appears with said entity, as determined from one of said images.

11. The system of claim 7, wherein said landmark detector rates said entity'"'"'s quality as a landmark based on visibility of said entity as determined from analysis of one of said images.

12. The system of claim 7, wherein said landmark detector rates said entity'"'"'s qualities as a landmark based on a name of said entity having a first word that is distinct in that said first word appears in a sign associated with said entity and that does not appear in any other sign within said distance from said entity.

13. The system of claim 7, wherein said application comprises a web application that provides maps and directions.

14. The system of claim 7, further comprising:

an automobile navigation device, said application being part of said automobile navigation device.

15. A method of using landmarks in directions, the method comprising:

using a processor to perform acts comprising;

receiving a plurality of image frames, each of said image frames comprising an image and a location at which said image was captured;

receiving a first set of records, each of said records comprising a name of an entity and a street address of said entity;

identifying a second set of said plurality of image frames that were captured within a distance of a street address of a first entity, said first entity being included in said first set of records;

determining which images in the images frames in said second set were captured on a street that matches said street address, the images captured on a street that matches said street address being in a third set;

determining that a first one of the images in said third set contains words that match a name of said first entity;

calculating Levenshtein distances between words in a name of said first entity and words in a fourth set of words that appear in said first one of the images;

based on one of said Levenshtein distances being lower than an amount, determining that said first entity appears in said first one of the images; and

providing directions from a first point to a second point, said directions including, as a landmark, said first entity.

16. The method of claim 15, further comprising:

using a processor to perform acts comprising;

identifying a word in a name of said first entity that does not appear in a name of any entity located within a distance from said first entity;

determining that said first one of said images contains said word.

17. The method of claim 15, further comprising:

using a processor to perform acts comprising;

identifying a fourth set of words that appear in said first one of the images in said third set;

determining that said fourth set contains either all of the words in a name of said first entity or all but one of the words in the name of said first entity.