DEVICE LOCALIZATION USING CAMERA AND WIRELESS SIGNAL

• US 20140357290A1
• Filed: 05/31/2013
• Published: 12/04/2014
• Est. Priority Date: 05/31/2013
• Status: Abandoned Application

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Grabner, Michael, Eade, Ethan, Nister, David
• Application Number:
  US13/907,741
• Publication Number:
  US 20140357290A1
• Time in Patent Office: Days
• US Class Current: 455/456.1
• CPC Class Codes: G01S 5/02529 not involving signal parameters, i.e. only involving identifiers
• CPC Class Codes: G01S 5/16 using electromagnetic waves other than radio waves
• CPC Class Codes: G06T 2207/30244 Camera pose
• CPC Class Codes: G06T 7/74 involving reference images or patches
• CPC Class Codes: H04W 64/00 Locating users or terminals or network equipment for network management purposes, e.g. mobility management

Abstract

A source wireless fingerprint is associated with a source image. One or more eligible cataloged wireless fingerprints having a threshold similarity to the source wireless fingerprint are found. Similarly, one or more eligible cataloged images having a threshold similarity to the source image are found. A current location of a device that acquires the source wireless fingerprint and source image is inferred as a chosen cataloged location of a chosen eligible cataloged wireless fingerprint and a chosen eligible cataloged image.

Claims

1. On a computing device, a method of device localization, the method comprising:

receiving a source wireless fingerprint identified by a device at a location;

receiving a source image associated with the source wireless fingerprint, the source image captured by the device at the location;

finding one or more eligible cataloged wireless fingerprints having a threshold similarity to the source wireless fingerprint, each cataloged wireless fingerprint associated with a cataloged location and a cataloged image;

finding one or more eligible cataloged images having a threshold similarity to the source image, each cataloged image associated with a cataloged location and a cataloged wireless fingerprint; and

inferring a current location of the device as a chosen cataloged location of a chosen eligible cataloged wireless fingerprint and a chosen eligible cataloged image.

2. The method of claim 1, wherein each cataloged image is associated with a cataloged orientation, and further comprising inferring a current orientation of the device as a chosen cataloged orientation of the chosen eligible cataloged image.

3. The method of claim 2, further comprising cataloging the inferred current orientation as a cataloged orientation with the source image and the source wireless fingerprint.

4. The method of claim 2, wherein the current orientation and the current location collectively define a six-degree-of-freedom pose of the device.

5. The method of claim 1, wherein the source wireless fingerprint indicates signal strength for each of a plurality of different wireless access points.

6. The method of claim 5, wherein finding one or more eligible wireless fingerprints having a threshold similarity to the source wireless fingerprint includes comparing signal strengths of the source wireless fingerprint with corresponding signal strengths of a cataloged wireless fingerprint.

7. The method of claim 1, wherein finding one or more eligible wireless fingerprints includes filtering the one or more cataloged wireless fingerprints for one or more candidate wireless fingerprints having a threshold similarity to the source wireless fingerprint, and finding one or more eligible images includes determining which of the one or more candidate wireless fingerprints has an associated cataloged image most closely matching the source image.

8. The method of claim 1, wherein finding one or more eligible images includes filtering the one or more cataloged images for one or more candidate images having a threshold similarity to the source image, and finding one or more eligible wireless fingerprints includes determining which of the one or more candidate images has an associated cataloged wireless fingerprint most closely matching the source wireless fingerprint.

9. The method of claim 1, further comprising cataloging the inferred current location as a cataloged location with the source image and the source wireless fingerprint.

10. The method of claim 1, wherein the device includes a camera and a wireless input.

11. The method of claim 1, wherein the source image is a still image.

12. The method of claim 1, wherein the source image is a video image.

13. The method of claim 1, wherein the chosen eligible cataloged image is chosen for having a greatest similarity to the source image.

14. On a computing device, a method of device localization, the method comprising:

receiving a source wireless fingerprint identified by a device at a location;

receiving a source image associated with the source wireless fingerprint, the source image captured by the device at the location;

filtering one or more cataloged wireless fingerprints for one or more candidate wireless fingerprints having a threshold similarity to the source wireless fingerprint, each cataloged wireless fingerprint associated with a cataloged location and a cataloged image;

choosing which cataloged image associated with a candidate wireless fingerprint most closely matches the source image; and

inferring a current location of the device as a chosen cataloged location of a chosen cataloged image.

15. The method of claim 14, wherein each cataloged image is associated with a cataloged orientation, and further comprising inferring a current orientation of the device as a chosen cataloged orientation of the chosen cataloged image.

16. The method of claim 15, wherein the current orientation and the current location collectively define a six degree of freedom pose of the device.

17. The method of claim 14, wherein the source wireless fingerprint indicates signal strength for each of a plurality of different wireless access points.

18. The method of claim 14, wherein the device includes a camera and a wireless input.

19. The method of claim 14, further comprising cataloging the inferred current location as a cataloged location with the source image and the source wireless fingerprint.

20. A computing system configured for device localization, the system comprising:

a logic machine;

a storage machine holding instructions executable by the logic machine to;

receive a source wireless fingerprint identified by a device at a location;

receive a source image associated with the source wireless fingerprint, the source image captured by the device at the location and being associated with a six-degree-of-freedom pose;

filter one or more cataloged wireless fingerprints for one or more candidate wireless fingerprints having a threshold similarity to the source wireless fingerprint, each cataloged wireless fingerprint associated with a cataloged location and a cataloged image;

choose which cataloged image associated with a candidate wireless fingerprint most closely matches the source image; and

infer a current six-degree-of-freedom pose of the device as a chosen six-degree-of-freedom pose of a chosen cataloged image.