Object tracking

• US 9,443,414 B2
• Filed: 08/07/2012
• Issued: 09/13/2016
• Est. Priority Date: 08/07/2012
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Scavezze, Mike, Scott, Jason, Steed, Jonathan, McIntyre, Ian, Krauss, Aaron, McCulloch, Daniel, Latta, Stephen
• Primary Examiner(s): YANG, WEI WEN
• Application Number:
  US13/569,003
• Publication Number:
  US 20140044305A1
• Time in Patent Office: 1,498 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G08B 13/19613 Recognition of a predetermined image pattern or behaviour pattern indicating theft or intrusion
• CPC Class Codes: G08B 21/24 Reminder alarms, e.g. anti-loss alarms
• CPC Class Codes: H04B 1/385 Transceivers carried on the body, e.g. in helmets
• CPC Class Codes: H04B 2001/3866 carried on the head
• CPC Class Codes: H04M 1/724097 Worn on the head
• CPC Class Codes: H04M 1/72412 using two-way short-range wireless interfaces
• CPC Class Codes: H04W 4/025 using location based information parameters
• CPC Class Codes: H04W 68/005 Transmission of information for alerting of incoming communication

Abstract

Embodiments are disclosed herein that relate to the automatic tracking of objects. For example, one disclosed embodiment provides a method of operating a mobile computing device having an image sensor. The method includes acquiring image data, identifying an inanimate moveable object in the image data, determining whether the inanimate moveable object is a tracked object, if the inanimate moveable object is a tracked object, then storing information regarding a state of the inanimate moveable object, detecting a trigger to provide a notification of the state of the inanimate moveable object, and providing an output of the notification of the state of the inanimate moveable object.

Claims

1. A method of operating a mobile computing device, the mobile computing device comprising an image sensor, the method comprising:

acquiring image data;

identifying an inanimate moveable object in the image data;

determining whether the inanimate moveable object is a tracked object based at least on previous user interactions with the inanimate moveable object;

if the inanimate moveable object is a tracked object, then storing information regarding a state of the inanimate moveable object, the state including a most recent physical location of the object within a use environment;

detecting a trigger to provide a notification of the state of the inanimate moveable object;

providing an output of the notification of the state of the inanimate moveable object, the notification including instructions to guide a user to the stored most recent physical location of the object within the use environment; and

when the object is no longer located at the stored most recent location, providing an output of one or more suggested current locations of the object.

2. The method of claim 1, further comprising determining whether the inanimate moveable object is a tracked object before storing information regarding the state of the inanimate moveable object.

3. The method of claim 2, further comprising receiving a user input assigning the inanimate moveable object as a tracked object and/or assigning a user-selected importance score to the inanimate moveable object.

4. The method of claim 2, further comprising assigning the inanimate moveable object an importance score based upon the previous user interactions with the inanimate moveable object, and designating the inanimate moveable object as a tracked object based upon the importance score meeting a threshold importance score.

5. The method of claim 4, wherein the importance score is assigned based at least partially upon a number of user interactions with the inanimate moveable object.

6. The method of claim 4, wherein the importance score is assigned based at least partially upon one or more of a location of the inanimate moveable object and a time at which a user interaction with the inanimate moveable object occurs.

7. The method of claim 6, wherein the importance score is assigned at least partially upon one or more of the location of the inanimate moveable object being designated as an important location, a proximity of the inanimate moveable object to other tracked objects, and a temporal proximity between a user interaction with the inanimate moveable object and a change in user context.

8. The method of claim 1, wherein the one or more suggested current locations of the object are based on historical locations of the object and/or user behavior.

9. The method of claim 1, wherein the state of the inanimate moveable object further comprises a value of a variable physical property of the inanimate moveable object, and wherein detecting the trigger further comprises detecting that the value of the variable physical property meets a threshold condition.

10. The method of claim 1, wherein detecting the trigger comprises detecting one or more of a time, a location, and an object contextually associated with the inanimate moveable object.

11. The method of claim 1, wherein detecting the trigger comprises receiving a user input requesting information regarding the state.

12. The method of claim 1, further comprising storing the information regarding the state by sending the information to a remote service via a network, and wherein the state includes GPS coordinates of a most recent physical location of the object within the use environment.

13. The method of claim 1, wherein the image data is acquired from one or more of an image sensor and a remote service, and wherein detecting the trigger to provide the notification of the state of the inanimate moveable object comprises detecting the trigger at a later time than a time at which the information regarding the state of the inanimate moveable object is stored.

14. A mobile computing device, comprising:

a sensor subsystem comprising an image sensor;

a display subsystem;

a logic subsystem configured to execute instructions; and

a data-holding subsystem comprising instructions stored thereon that are executable by the logic subsystem to;

receive image data from the image sensor;

detect via the image data a plurality of user interactions with a moveable inanimate physical object;

assign an importance score to the inanimate moveable physical object based upon the plurality of user interactions with the inanimate moveable physical object;

if the importance score meets a threshold importance score, then store information regarding a value of a variable physical property of the inanimate moveable physical object;

detect a trigger to provide a notification of the value of the variable physical property of the inanimate moveable physical object; and

provide an output of the notification of the value of the variable physical property of the inanimate moveable physical object.

15. The mobile computing device of claim 14, wherein the instructions are executable to detect the trigger by detecting that the value of the variable physical property meets a threshold condition.

16. The mobile computing device of claim 14, wherein the instructions are executable to detect the trigger by detecting a user input requesting information regarding the value of the variable physical property of the inanimate moveable physical object, wherein the mobile computing device is a head-mounted display device configured to be worn by a user, and wherein the image data is captured by the image sensor as the user moves about an environment.