Location estimation using image analysis

• US 9,222,783 B2
• Filed: 02/21/2012
• Issued: 12/29/2015
• Est. Priority Date: 03/15/2007
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Kansal, Aman, Zhao, Feng
• Primary Examiner(s): CONWAY, THOMAS A
• Application Number:
  US13/401,582
• Publication Number:
  US 20120148091A1
• Time in Patent Office: 1,407 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G01C 21/20 Instruments for performing navigational calculations G01C21/24, G01C21/26 take precedence
• CPC Class Codes: G01C 21/206 specially adapted for indoor navigation
• CPC Class Codes: G01S 5/16 using electromagnetic waves other than radio waves
• CPC Class Codes: G06F 16/29 Geographical information databases
• CPC Class Codes: G06F 16/58 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/583 using metadata automatically derived from the content
• CPC Class Codes: G06F 16/5866 using information manually generated, e.g. tags, keywords, comments, manually generated location and time information
• CPC Class Codes: G06F 16/587 using geographical or spatial information, e.g. location
• CPC Class Codes: G06F 16/9537 Spatial or temporal dependent retrieval, e.g. spatiotemporal queries
• CPC Class Codes: G06V 20/10 Terrestrial scenes scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59

Abstract

An implementation of location estimation using image analysis is described. In this implementation, an image of a place is obtained and matched with previously stored images. The matching may be achieved by employing methods based on key feature extraction algorithm, color histogram analysis, pattern matching or other image comparison techniques. Upon determining a match, the location information associated with the image provides the location. The location information may be in the form of location tags or location keywords and the location information may be used by the user or other applications for the purposes of location determination. The technique allows for the user to enter location information. The location information may be assigned to the previously stored images residing in local and remote databases for users and applications to assign information or keywords to images.

Claims

1. One or more computer storage media comprising computer-readable instructions that when executed, perform acts comprising:

acquiring an image of a current location;

iteratively comparing the image in a plurality of different formats with two or more previously stored images in corresponding formats to identify a subset of potential matches using a first of a plurality of different image analysis techniques, and to identify one or more matching images of the subset using a second of the plurality of different image analysis techniques, the two or more previously stored images include assigned location information;

computing location information of a place in the acquired image using one or more estimation methods, wherein the one or more estimation methods comprise one or more location estimation methods utilizing one or more of global positioning systems and cellular network systems; and

assigning to the one or more matching images a combination of the location information computed for the acquired image and the location information previously assigned to the one or more matching images.

2. One or more computer-readable media as recited in claim 1, further comprising assigning to the one or more matching images the location information in response to determining that the one or more matching images have conflicting location information.

3. One or more computer-readable media as recited in claim 1, further comprising:

storing the location information for future assignment; and

assigning to one or more additional matching images the location information.

4. One or more computer-readable media as recited in claim 1, further comprising transmitting the location information to a database for later reference, assignment, or a combination of both.

5. One or more computer-readable media as recited in claim 1, wherein the inputted location information comprises a location tag, a location keyword, or a combination of both.

6. One or more computer-readable media as recited in claim 1, wherein the act of comparing the image to the two or more previously stored images using the plurality of different image analysis techniques comprises:

converting the image into a plurality of different formats for the plurality of different image analysis techniques; and

comparing the image in the plurality of different formats with the previously stored image using the plurality of different image analysis techniques.

7. A portable computing device comprising:

memory to store one or more images with location information;

one or more processors;

an image acquisition component to acquire an image of a place;

a location module, stored in the memory and executable on the one or more processors, to estimate a location of the place based on the image acquired by the image acquisition component and to associate the estimated location of the place with the acquired image;

an assignor module, stored in the memory and executable on the one or more processors, to assign, to the one or more previously stored images that match the acquired image, a combination of the estimated location information associated with the acquired image and the location information previously assigned to the one or more previously stored images; and

a network interface to interact with a remote database having previously stored images to assemble the acquired image and the estimated location of the place.

8. The portable computing device as recited in claim 7, wherein the estimated location of the place is based on information collected from a global positioning system, a cellular network system, or a combination of both.

9. The portable computing device as recited in claim 7, further comprising:

a converter module to convert the image and one or more previously stored images into one or more comparable formats; and

a matcher module to match the image in the one or more comparable formats with the one or more previously stored images in the one or more comparable formats.

10. The computing device as recited in claim 7, wherein the location module further comprises:

a user input module to accept the inputted location information from the user.

11. The portable computing device as recited in claim 10, wherein the inputted location information from the user comprises a location tag, a location keyword, or a combination of both.

16. The portable computing device as recited in claim 7, wherein the assignor module assigns the location information associated with the acquired image to the one or more previously stored images that matched the image in response to determining that the one or more matching images do not have the location information.

12. A method comprising:

obtaining an image;

matching, by a processor configured by executable instructions, the image with one or more previously stored images, the one or more previously stored images include assigned location information, the matching based on comparisons performed using a plurality of different image analysis techniques, the matching comprising;

converting the image into a plurality of different formats for a plurality of different comparison techniques; and

iteratively comparing the image in the plurality of different formats with two or more of the previously stored images in corresponding formats to identify a subset of potential matches using a first of the plurality of comparison techniques, and to identify one or more matching ones of the subset using a second of the plurality of comparison techniques;

computing a location of a place in the image using estimation methods, wherein the estimation methods comprise location estimation methods utilizing one or more of global positioning systems and cellular network systems; and

assigning a combination of the location of the place in the image computed by using the estimation methods and the previously assigned location information to the one or more previously stored images that match the image.

13. The method as recited in claim 12, further comprising tagging the image with the location information that has been provided by a user, wherein the location information comprises at least one of a location tag and a location keyword.

14. The method as recited in claim 12, further comprising:

prompting a user to provide additional location information comprising at least one of a location tag and a location keyword; and

tagging the image and to the one or more previously stored images that match the image with the additional location information that has been provided by a user.

15. The methods as recited in claim 12, further comprising:

assigning, to the one or more previously stored images that match the image, location information of the image in response to determining that the one or more previously stored images that match the image have conflicting location information.

17. The methods as recited in claim 12, further comprising:

assigning to the one or more matching images the location information associated with the acquired image in response to determining that the one or more matching images do not have the location information.