Motion modeling in visual tracking

• US 9,767,609 B2
• Filed: 02/12/2014
• Issued: 09/19/2017
• Est. Priority Date: 02/12/2014
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Eade, Ethan, Ebstyne, Michael John, Schaffalitzky, Frederick, Steedly, Drew
• Primary Examiner(s): Brier, Jeffrey A
• Application Number:
  US14/179,384
• Publication Number:
  US 20150228118A1
• Time in Patent Office: 1,315 Days
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00

Abstract

Embodiments are disclosed that relate to determining a pose of a device. One disclosed embodiment provides a method comprising receiving sensor information from one or more sensors of the device, and selecting a motion-family model from a plurality of different motion-family models based on the sensor information. The method further comprises providing the sensor information to the selected motion-family model and outputting an estimated pose of the device according to the selected motion-family model.

Claims

1. A method of assessing a pose of a device, the method comprising:

receiving, from one or more sensors of the device, sensor information including a change in location of the device along three orthogonal axes and a change in orientation of the device along the three orthogonal axes;

using a pre-configured motion behavior classifier to determine a physical state of a user of the device based on the change in location of the device and the change in orientation of the device;

selecting a motion-family model from a plurality of different pre-configured motion-family models based on the physical state of the user;

providing the sensor information to the selected motion-family model;

receiving an estimated pose of the device from the selected motion-family model, wherein the selected motion-family model interprets the sensor information in accordance with a family of motion associated with the device for the selected motion-family model, the family of motion including an acceleration profile for the device, and the selected motion-family model matching one or more measurements of the device to a selected trajectory of the device; and

displaying content on the device based on the estimated pose of the device.

2. The method of claim 1, wherein the device is a head-mounted display device configured to output the displayed content to a wearer of the device, the displayed content being mapped, based on the estimated pose, to changing locations of a real-world environment viewable through the head-mounted display device as the location and orientation of the head-mounted display device changes.

3. The method of claim 2, further comprising adjusting one or more display properties for the displayed content based on the estimated pose of the device.

4. The method of claim 1, wherein outputting the estimated pose of the device further comprises providing the sensor information and an output of the selected motion-family model to a pose filter and outputting the estimated pose of the device from the pose filter.

5. The method of claim 1, wherein the sensor information indicates a physical state of a wearer of the device, the physical state indicating one or more of a position, orientation, and movement characteristic of the wearer.

6. The method of claim 1, wherein selecting the motion-family model comprises classifying a family of motion exhibited by the device and comparing the classified family of motion to one or more families of motion associated with the plurality of different pre-configured motion-family models.

7. The method of claim 6, wherein classifying the family of motion exhibited by the device comprises determining the family of motion based on a previously-output estimated pose of the device.

8. A head-mounted display device comprising:

a sensor;

a logic machine; and

a storage machine, the storage machine holding instructions executable by the logic machine to;

receive, from the sensor, sensor information including a change in location of the head-mounted display device along three orthogonal axes and a change in orientation of the head-mounted display device along the three orthogonal axes;

use a pre-configured motion behavior classifier to determine a physical state of a user of the head-mounted display device based on the change in location of the head-mounted display device and the change in orientation of the head-mounted display device;

select a motion-family model from a plurality of pre-configured motion-family models based on the physical state of a wearer of the head-mounted display device, each of the plurality of pre-configured motion-family models associated with a respective family of motion for the device, each respective family of motion including a different acceleration profile, and each pre-configured motion-family model matching one or more measurements of the sensor to a different trajectory of head movement indicating predefined expected movement of the head-mounted display device for the associated physical state of the user;

provide the sensor information and an output of the selected motion-family model to a pose filter;

output, from the pose filter, an estimated position and orientation of the head-mounted display device based on the sensor information and the selected motion-family model; and

display content on the head-mounted display device based on the estimated pose of the head-mounted display device.

9. The head-mounted display device of claim 8, wherein the physical state of the wearer is determined based on the sensor information from the sensor.

10. The head-mounted display device of claim 8, wherein the sensor information includes one or more measurements of a position and/or orientation of a head of the wearer.

11. The head-mounted display device of claim 10, wherein the motion-family model matches the one or more measurements to a selected trajectory of head movement based on the physical state of the wearer.

12. The head-mounted display device of claim 8, wherein the pose filter includes a Kalman filter.

13. The head-mounted display device of claim 8, wherein selecting the motion-family model from the plurality of pre-configured motion-family models further comprises sending the sensor information and a previously-output estimated position and orientation to a pre-configured motion behavior classifier of the head-mounted display device.

14. The head-mounted display device of claim 13, wherein the sensor information is provided to the pose filter and the pre-configured motion behavior classifier.

15. The head-mounted display device of claim 13, wherein the pre-configured motion behavior classifier is used to analyze the sensor information and the previously-output estimated position and orientation to select the motion-family model.

16. The head-mounted display device of claim 15, further comprising changing the motion-family model if the selected motion-family model is different than a current motion-family model.

17. A hardware memory device holding instructions executable by a logic machine to:

collect sensor information from one or more sensors of a head-mounted display;

send the sensor information to a pre-configured motion behavior classifier for determining a selected motion-family model from a plurality of different pre-configured motion-family models based on the sensor information;

in response to determining that the selected motion-family model is different than a previously-used motion-family model, replace the previously-used motion-family model with the selected motion-family model;

wherein each of the previously-used motion-family model and the selected motion-family model is used to adjust the sensor information to match one or more measurements indicating head movement to a selected trajectory of head movement for that motion-family model;

receive a time-varying pose estimate that is output based on an output of both the previously-used motion-family model and the selected motion-family model; and

display content on the head-mounted display based on the time-varying pose estimate of the head-mounted display.

18. The hardware memory device of claim 17, wherein a first, earlier time-varying pose estimate is based on the previously-used motion-family model and a second, later time-varying pose estimate is based on the selected motion-family model.

19. The hardware memory device of claim 17, wherein a first, earlier time-varying pose estimate is based on the previously-used motion-family model and a second, later time-varying pose estimate is based on an average of an output of the previously-used motion-family model and the selected motion-family model.

20. The hardware memory device of claim 17, wherein one or more of the sensor information and the time-varying pose estimate is provided to a Kalman filter.