Location estimation using image analysis

• US 9,710,488 B2
• Filed: 11/25/2015
• Issued: 07/18/2017
• Est. Priority Date: 03/15/2007
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Kansal, Aman, Zhao, Feng
• Primary Examiner(s): Sherali, Ishrat I
• Application Number:
  US14/951,948
• Publication Number:
  US 20160154821A1
• Time in Patent Office: 601 Days
• Field of Search: 382100, 382305
• CPC Class Codes: G01C 21/20 Instruments for performing navigational calculations G01C21/24, G01C21/26 take precedence
• CPC Class Codes: G01C 21/206 specially adapted for indoor navigation
• CPC Class Codes: G01S 5/16 using electromagnetic waves other than radio waves
• CPC Class Codes: G06F 16/29 Geographical information databases
• CPC Class Codes: G06F 16/58 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/583 using metadata automatically derived from the content
• CPC Class Codes: G06F 16/5866 using information manually generated, e.g. tags, keywords, comments, manually generated location and time information
• CPC Class Codes: G06F 16/587 using geographical or spatial information, e.g. location
• CPC Class Codes: G06F 16/9537 Spatial or temporal dependent retrieval, e.g. spatiotemporal queries
• CPC Class Codes: G06V 20/10 Terrestrial scenes scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59

Abstract

An implementation of location estimation using image analysis is described. In this implementation, an image of a place is obtained and matched with previously stored images. The matching may be achieved by employing methods based on key feature extraction algorithm, color histogram analysis, pattern matching or other image comparison techniques. Upon determining a match, the location information associated with the image provides the location. The location information may be in the form of location tags or location keywords and the location information may be used by the user or other applications for the purposes of location determination. The technique allows for the user to enter location information. The location information may be assigned to the previously stored images residing in local and remote databases for users and applications to assign information or keywords to images.

Claims

1. One or more computer storage devices storing computer-readable instructions that, when executed by one or more processors, perform acts comprising:

acquiring a first image of a location;

iteratively comparing the first image in a first format and a second format with two or more second images in the first format and the second format by;

identifying, using a first image analysis technique on the first image in the first format and the two or more second images in the first format, a subset of potential matching images from the two or more second images, wherein the two or more second images are stored in one or more databases associated with one or more location service devices; and

identifying, using a second image analysis technique on the first image in the second format and the subset of potential matching images in the second format, one or more third images from the subset of potential matching images that match the first image;

determining that the one or more third images that match the first image lack location information;

prompting, based at least in part on determining that the one or more third images that match the first image lack location information, a user to input location information for the first image;

assigning the location information to the first image; and

updating the one or more databases by storing the first image and the location information in the one or more databases for future use by the one or more location service devices in determining a location associated with future images.

2. The one or more computer storage devices of claim 1, the acts further comprising assigning the location information to the one or more third images that match the first image.

3. The one or more computer storage devices of claim 1, the acts further comprising assigning the location information to the one or more third images in response to determining that the one or more third images are lacking location information.

4. The one or more computer storage devices of claim 1, the acts further comprising assigning to the one or more third images the location information in response to determining that the one or more third images have conflicting location information.

5. The one or more computer-readable storage devices of claim 1, the acts further comprising:

receiving additional matching images that match with the first image; and

assigning the location information to one or more additional matching images.

6. The one or more computer storage devices of claim 1, wherein the input comprises at least one of a location tag or a location keyword.

7. The one or more computer storage devices of claim 1, wherein the comparing the first image to the two or more second images comprises:

converting the first image into a first format for the first image analysis technique; and

comparing the first image in the first format with the two or more second images using the first image analysis technique.

8. A method comprising:

obtaining a first image;

matching, by one or more processors associated with a location service and configured by executable instructions, the first image with one or more second images stored in one or more databases associated with the location service, the matching being based at least in part on comparisons between the first image and each of the one or more second images, the matching comprising;

converting the first image into a first format for a first image comparison technique and into a second format for a second image comparison technique; and

iteratively comparing the first image with two or more of the second images to;

identify, using the first image comparison technique on the first image in the first format and the two or more of the second images in the first format, a subset of potential image matches, and

identify, using the second image comparison technique on the first image in the second format and the subset of potential image matches in the second format, one or more third images from the subset of potential image matches that match the first image;

determining that the one or more third images that match the first image lack location information;

prompting, based at least in part on determining that the one or more third images that match the first image lack location information, a user to input location information for the image;

assigning the location information to the first image; and

updating the one or more databases associated with the location service by storing the first image and the location information in the one or more databases for future use by the location service in determining a location associated with future images.

9. The method of claim 8, further comprising:

assigning the location information to the one or more third images of the subset of potential images that match the first image.

10. The method of claim 8, further comprising:

determining that the one or more third images have conflicting location information; and

in response to determining that the one or more third images have conflicting location information, assigning the location information to the one or more third images.

11. The method of claim 8, further comprising:

determining that the one or more third images have insufficient location information; and

supplementing the insufficient location information with the location information.

12. The method of claim 8, further comprising determining that an alternative location estimation method is not available, the alternative location estimation method comprising at least one of a global positioning system (GPS) location estimation method or a cellular network location estimation method.

13. The method of claim 8, wherein the location information comprises at least one of a location tag or a location keyword.

14. The method of claim 8, further comprising computing a location of a place in the image using estimation methods, the estimation methods comprising location estimation methods utilizing one or more of global positioning systems and cellular network systems.

15. A system comprising:

one or more processors;

memory communicatively coupled to the one or more processors;

an image acquisition component stored in the memory that, when executed by the one or more processors, acquires first image of an environment of a computing device associated with the system;

a location module stored in the memory that, when executed by the one or more processors, performs acts comprising;

iteratively comparing the first image in a first format and a second format with two or more second images in the first format and the second format to identify one or more third images that match the first image using one or more image comparison techniques, wherein the two or more second images are stored in one or more databases;

determining that the one or more third images that match the first image lack location information or have insufficient location information;

prompting, based at least in part on determining that the one or more third images that match the first image lack location information or have insufficient location information, a user to input location information for the first image;

assigning the location information to the first image; and

updating the one or more databases by storing the first image and the location information in the one or more databases for future use by the location module in determining a location associated with future images.

16. The system of claim 15, the acts further comprising:

assigning the location information to the one or more third images.

17. The system of claim 15, the acts further comprising:

determining that the one or more third images have conflicting location information; and

in response to determining that the one or more third images have conflicting location information, assigning the location information to the one or more third images.

18. The system of claim 15, the acts further comprising:

determining that the one or more third images have insufficient location information; and

supplementing the insufficient location information with the location information.

19. The system of claim 15, the acts further comprising:

determining that an alternative location estimation method is not available, the alternative location estimation method comprising at least one of a global positioning system (GPS) location estimation method or a cellular network location estimation method; and

wherein the prompting the user to input the location information is based at least in part on determining that the alternative location estimation method is not available.

20. The system of claim 15, wherein the location information includes at least one of an indication of a room of a building corresponding to the environment of the computing device, or an indication of a floor of the building.