ENHANCED NAVIGATION THROUGH MULTI-SENSOR POSITIONING

• US 20130142384A1
• Filed: 12/06/2011
• Published: 06/06/2013
• Est. Priority Date: 12/06/2011
• Status: Abandoned Application

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Ofek, Eyal
• Application Number:
  US13/311,941
• Publication Number:
  US 20130142384A1
• Time in Patent Office: Days
• US Class Current: 382/103
• CPC Class Codes: G06F 18/253 of extracted features
• CPC Class Codes: G06T 2207/30232 Surveillance
• CPC Class Codes: G06T 7/292 Multi-camera tracking
• CPC Class Codes: G06V 10/255 Detecting or recognising potential candidate objects based on visual cues, e.g. shapes
• CPC Class Codes: G06V 10/806 of extracted features
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: H04N 7/18 Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
• CPC Class Codes: H04W 4/023 using mutual or relative location information between multiple location based services [LBS] targets or of distance thresholds
• CPC Class Codes: H04W 4/024 Guidance services
• CPC Class Codes: H04W 4/029 Location-based management or tracking services

Abstract

Enhanced navigation and positional metadata are provided based upon position determination utilizing data provided by multiple different systems of sensors. Infrastructure, or fixed sensor, data provides an initial location determination of humans and user-specific sensors that are co-located with their respective users provides identification of the users whose locations were determined. Navigation instructions provided based on the determined locations are enhanced by additional sensor data that is received from other user-specific sensors that are co-located with the users. Additionally, user privacy can be maintained by only utilizing sensor data authorized by the user, or by publishing fixed sensor data, identifying locations and movements of users, but not their identity, thereby enabling a user'"'"'s computing device to match such information to the information obtained from user-specific sensors to determine the user'"'"'s location.

Claims

1. One or more computer-readable media comprising computer-executable instructions for determining users'"'"' physical locations, the computer-executable instructions performing steps comprising:

detecting movement of unidentified users in one or more image feeds received from one or more fixed location imaging devices, whose location remains invariant;

correlating the detected movement with one or more motion sensor data from one or more mobile computing devices carried by one or more users;

identifying at least some of the unidentified users based on the correlating;

determining physical locations for the identified users from the one or more image feeds received from the one or more fixed location imaging devices;

receiving image feeds from mobile computing devices carried by at least some of the identified users whose physical locations were determined; and

performing the detecting, the correlating, the identifying and the determining with the received image feeds from the mobile computing devices.

2. The computer-readable media of claim 1, wherein the identifying performed with the received image feeds from the mobile computing devices identifies previously unidentified users.

3. The computer-readable media of claim 1, wherein the determining physical locations performed with the received image feeds from the mobile computing devices determines a more accurate physical location than that previously determined.

4. The computer-readable media of claim 1, comprising further computer-executable instructions for providing navigational instructions to a first user from among the identified users whose physical locations were determined, the navigational instructions being based on the determined location of the first user.

5. The computer-readable media of claim 4, wherein the navigational instructions direct the first user to an item determined to be of interest to the first user that is in a same venue as the fixed location imaging devices.

6. The computer-readable media of claim 4, comprising further computer-executable instructions for providing navigational instructions to a second user, also from among the identified users whose physical locations were determined, the navigational instructions being based on the determined physical location of the second user, the navigational instructions to the second user being provided concurrently with the navigational instructions to the first user, wherein the navigational instructions provided to the first user guide the first user to the second user, and wherein further the navigational instructions provided to the second user guide the second user to the first user.

7. The computer-readable media of claim 4, wherein the first user is one of the at least some of the identified users that are carrying the mobile computing devices from which image feeds are being received, and wherein further the computer-executable instructions for providing the navigational instructions comprise computer-executable instructions for generating heads-up navigational instructions in accordance with an orientation of the first user, as evidenced by the image feed being generated by the first user'"'"'s mobile computing device, the generated heads-up navigational instructions to be superimposed over the image feed being generated by the first user'"'"'s mobile computing device.

8. The computer-readable media of claim 7, comprising further computer-executable instructions for providing a heads-up graphical representation of a second user in accordance with a determined physical location of the second user and in accordance with the orientation of the first user, as evidenced by the image feed being generated by the first user'"'"'s mobile computing device, the heads-up graphical representation of the second user to be superimposed over the image feed being generated by the first user'"'"'s mobile computing device.

9. The computer-readable media of claim 1, wherein every one of the one or more users, from whose mobile computing devices the one or more sensor data is received, have previously authorized usage of the one or more sensor data to locate the one or more users.

10. A method for determining users'"'"' physical locations, the method comprising the steps of:

detecting movement of unidentified users in one or more image feeds received from one or more fixed location imaging devices, whose location remains invariant;

correlating the detected movement with one or more motion sensor data from one or more mobile computing devices carried by one or more users;

identifying at least some of the unidentified users based on the correlating;

determining physical locations for the identified users from the one or more image feeds received from the one or more fixed location imaging devices;

receiving image feeds from mobile computing devices carried by at least some of the identified users whose physical locations were determined; and

performing the detecting, the correlating, the identifying and the determining with the received image feeds from the mobile computing devices.

11. The method of claim 10, wherein the identifying performed with the received image feeds from the mobile computing devices identifies previously unidentified users.

12. The method of claim 10, wherein the determining physical locations performed with the received image feeds from the mobile computing devices determines a more accurate physical location than that previously determined.

13. The method of claim 10, further comprising the steps of providing navigational instructions to a first user from among the identified users whose physical locations were determined, the navigational instructions being based on the determined location of the first user.

14. The method of claim 13, wherein the navigational instructions direct the first user to an item determined to be of interest to the first user that is in a same venue as the fixed location imaging devices.

15. The method of claim 13, further comprising the steps of providing navigational instructions to a second user, also from among the identified users whose physical locations were determined, the navigational instructions being based on the determined physical location of the second user, the navigational instructions to the second user being provided concurrently with the navigational instructions to the first user, wherein the navigational instructions provided to the first user guide the first user to the second user, and wherein further the navigational instructions provided to the second user guide the second user to the first user.

16. The method of claim 13, wherein the first user is one of the at least some of the identified users that are carrying the mobile computing devices from which image feeds are being received, and wherein further the providing the navigational instructions comprises generating heads-up navigational instructions in accordance with an orientation of the first user, as evidenced by the image feed being generated by the first user'"'"'s mobile computing device, the generated heads-up navigational instructions to be superimposed over the image feed being generated by the first user'"'"'s mobile computing device.

17. The method of claim 16, further comprising the steps of providing a heads-up graphical representation of a second user in accordance with a determined physical location of the second user and in accordance with the orientation of the first user, as evidenced by the image feed being generated by the first user'"'"'s mobile computing device, the heads-up graphical representation of the second user to be superimposed over the image feed being generated by the first user'"'"'s mobile computing device.

18. The method of claim 10, wherein every one of the one or more users, from whose mobile computing devices the one or more sensor data is received, have previously authorized usage of the one or more sensor data to locate the one or more users.

19. A mobile computing device comprising:

a motion sensor detecting the mobile computing device'"'"'s motion;

an image sensor capturing an image feed of an area surrounding the mobile computing device;

a network interface transmitting the motion detected by the motion sensor and the image feed captured by the image sensor; and

a display displaying navigational instructions from a current location of the mobile computing device to a destination, the current location of the mobile computing device having been determined by reference to the mobile computing device'"'"'s motion as detected by the motion sensor.

20. The mobile computing device of claim 20, wherein the display further displays the image feed being captured by the image sensor and a heads-up graphical representation of a second user superimposed over the image feed in accordance with a determined physical location of the second user and in accordance with the orientation of the mobile computing device, as evidenced by the image feed, and wherein further the displayed navigational instructions comprise heads-up navigational instructions that are also superimposed over the image feed in accordance with the orientation of the mobile computing device, as evidenced by the image feed.