Automated location estimation using image analysis

• US 8,144,920 B2
• Filed: 03/15/2007
• Issued: 03/27/2012
• Est. Priority Date: 03/15/2007
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Kansal, Aman, Zhao, Feng
• Primary Examiner(s): Lu, Tom Y
• Assistant Examiner(s): CONWAY, THOMAS A
• Application Number:
  US11/686,733
• Publication Number:
  US 20080226130A1
• Time in Patent Office: 1,839 Days
• Field of Search: None
• US Class Current: 382/100
• CPC Class Codes: G01C 21/20 Instruments for performing navigational calculations G01C21/24, G01C21/26 take precedence
• CPC Class Codes: G01C 21/206 specially adapted for indoor navigation
• CPC Class Codes: G01S 5/16 using electromagnetic waves other than radio waves
• CPC Class Codes: G06F 16/29 Geographical information databases
• CPC Class Codes: G06F 16/58 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/583 using metadata automatically derived from the content
• CPC Class Codes: G06F 16/5866 using information manually generated, e.g. tags, keywords, comments, manually generated location and time information
• CPC Class Codes: G06F 16/587 using geographical or spatial information, e.g. location
• CPC Class Codes: G06F 16/9537 Spatial or temporal dependent retrieval, e.g. spatiotemporal queries
• CPC Class Codes: G06V 20/10 Terrestrial scenes scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59

Abstract

An implementation of automated location estimation using image analysis is described. In this implementation, an image of a place is obtained and matched with previously stored images. The matching may be achieved by employing methods based on key feature extraction algorithm, color histogram analysis, pattern matching or other image comparison techniques. Upon determining a match, the location information associated with the matched previously stored images provides the location. The location information may be in the form of location tags or location keywords and the location information may be used by the user or other applications for the purposes of location determination. The above technique also allows for the user to enter location information to improve accuracy. The location information may also be assigned to the previously stored images residing in local and remote databases for users and applications to automatically assign information or keywords to images.

Claims

1. One or more computer storage media devices comprising computer-readable instructions that when executed, perform acts comprising:

acquiring an image of a current location;

iteratively comparing the image in a plurality of different formats with two or more previously stored images in corresponding formats to identify a subset of potential matches using a first of a plurality of different image analysis techniques, and to identify one or more matching images of the subset using a second of the plurality of different image analysis techniques;

identifying an location of a place of the image based on the comparisons performed using the plurality of different image analysis techniques; and

assigning, to the image, location information associated with the identified location.

2. One or more computer storage media devices as recited in claim 1, wherein the acts further comprise prompting a user to enter one or more location keywords if the image does not match the one or more previously stored images.

3. One or more computer storage media devices as recited in claim 1, wherein the acts further comprise:

requesting the image from a user by a location-based service to verify a location of a user;

computing the location of a place in the image using other estimation methods, wherein the other estimation methods comprise location estimation methods utilizing one or more of global positioning systems and cellular network systems; and

verifying the location computed using the other estimation methods by comparing the location with the location information assigned to the image to determine that modified location information has not been provided using the other estimation methods.

4. One or more computer storage media devices as recited in claim 1, further comprising:

computing the location of a place in the image using other estimation methods, wherein the other estimation methods comprise location estimation methods utilizing one or more of global positioning systems and cellular network systems; and

combining the location of the place computed by using the other estimation methods with the location information assigned to the image.

5. A system comprising:

the one or more computer storage media devices as recited in claim 1; and

one or more processors to execute the computer-readable instructions stored on the computer storage media devices.

6. One or more computer storage media devices as recited in claim 1, wherein the acts further comprise tagging the image with one or more location keywords provided by a user, wherein the one or more location keywords may be included in location information of the image.

7. One or more computer storage media devices as recited in claim 6, wherein the acts further comprise adding the one or more location keywords as location information of the previously stored image upon a determination that the previously stored image does not have the location information.

8. One or more computer storage media devices as recited in claim 1, wherein the act of comparing the image to a previously stored image using the plurality of different image analysis techniques comprises:

converting the image into a plurality of different formats for the plurality of different image analysis techniques; and

comparing the image in the plurality of different formats with the previously stored image using the plurality of different image analysis techniques.

9. One or more computer storage media devices as recited in claim 8, wherein:

the plurality of different formats allows matching of two images of a location taken under differing conditions;

the plurality of different formats comprise one or more of a format based on key features, a format based on color histogram, and a format based on pattern matching; and

the act of identifying the location comprises determining a weighted sum of results of the comparing in a first format of the plurality of different formats and results of the comparing in a second format of the plurality of different formats.

10. One or more computer storage media devices as recited in claim 8, wherein the acts further comprise:

retrieving the previously stored image from a database via a network, and

wherein the network comprises an internet, a cellular network and a satellite network.

11. A portable computing device comprising:

a memory;

one or more processors;

an image acquisition component to acquire an image of a place;

a location module, which is stored in the memory and executable on the one or more processors, to estimate a location of the place based on the image acquired by the image acquisition component;

a network interface to interact with a remote database having previously stored images; and

the location module, to verify a location computed using other estimation methods, the other estimation methods utilizing one or more of global positioning systems and cellular network systems, the verification by comparing the location estimated based on the image acquired and the location computed using other estimation methods to determine that modified location information has not been provided using the other estimation methods.

12. A portable computing device as recited in claim 11, wherein the location module comprises:

a converter module to convert the image into a plurality of different formats for a plurality of different comparison techniques;

a matcher module to match the image in the plurality of different formats with the one or more previously stored images in corresponding formats; and

an assignor module to assign, to the image, location information of the one or more previously stored images that matched the image based on comparison using the plurality of different formats.

13. A portable computing device as recited in claim 11, wherein the location module further comprises a user input module to accept location keywords from a user, wherein the location information of the image comprises the location keywords.

14. A portable computing device as recited in claim 11, wherein the portable computing device further comprises:

a local database to store the images and the location information; and

a user interface to display the location information.

15. A portable computing device as recited in claim 11, wherein the portable computing device comprises at least one of:

a camera, a communication device, a personal digital assistant, a portable music player, or a portable computing device.

16. A method comprising:

obtaining an image;

matching, by a processor configured by executable instructions, the image with

one or more previously stored images, the matching based on comparisons performed using a plurality of different image analysis techniques, the matching comprising;

converting the image into a plurality of different formats for a plurality of different comparison techniques; and

iteratively comparing the image in the plurality of different formats with two or more of the previously stored images in corresponding formats to identify a subset of potential matches using a first of the plurality of comparison techniques, and to identify one or more matching ones of the subset using a second of the plurality of comparison techniques; and

identifying a location of a place of the image based on the iteratively comparing of the image in the plurality of different formats to the subset of the two or more previously stored images; and

assigning location information, of the image, to the one or more previously stored images that match the image if the one or more previously stored images do not have the location information.

17. A method as recited in claim 16, further comprising tagging the image with the location information that has been provided by a user, wherein the location information comprises at least one of a location tag and a location keyword.

18. A method as recited in claim 16, further comprising assigning, to the image, location information of the one or more previously stored images that match the image if the image does not have the location information.

19. A method as recited in claim 16, further comprising:

computing location of a place in the image using other estimation methods, wherein the other estimation methods comprise location estimation methods utilizing one or more of global positioning systems and cellular network systems; and

assigning the location of the place computed by using the other estimation methods to the one or more previously stored images that match the image.

20. A method comprising:

acquiring an image of a place;

estimating, by a processor configured by executable instructions, a location of the place based on the image acquired;

computing a location of the place in the image acquired using other estimation methods, the other estimation methods utilizing one or more of global positioning systems and cellular network systems; and

verifying the location computed using the other estimation methods by comparing the location estimated based on the image acquired with the location computed using other estimation methods to determine that modified location information has not been provided using the other estimation methods.