World-locked display quality feedback

• US 9,430,038 B2
• Filed: 05/01/2014
• Issued: 08/30/2016
• Est. Priority Date: 05/01/2014
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Ebstyne, Michael John, Schaffalitzky, Frederik, Steedly, Drew, Eade, Ethan, Shetter, Martin, Grabner, Michael
• Primary Examiner(s): Martello, Edward
• Application Number:
  US14/267,625
• Publication Number:
  US 20150317832A1
• Time in Patent Office: 852 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/0093 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06F 2200/1637 Sensing arrangement for detection of housing movement or orientation, e.g. for controlling scrolling or cursor movement on the display of an handheld computer
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00

Abstract

Embodiments that relate to communicating to a user of a head-mounted display device an estimated quality level of a world-lock display mode are disclosed. For example, in one disclosed embodiment a sensor data is received from one or more sensors of the device. Using the sensor data, an estimated pose of the device is determined. Using the estimated pose, one or more virtual objects are displayed via the device in either the world-lock display mode or in a body-lock display mode. One or more of input uncertainty values of the sensor data and pose uncertainty values of the estimated pose are determined. The input uncertainty values and/or pose uncertainty values are mapped to the estimated quality level of the world-lock display mode. Feedback of the estimated quality level is communicated to a user via device.

Claims

1. A method for communicating to a user of a head-mounted display device an estimated quality level of a world-lock display mode in which a position of one or more virtual objects appears to be fixed relative to real-world objects viewable through the head-mounted display device, the method comprising:

receiving sensor data from one or more sensors of the head-mounted display device;

using the sensor data, determining an estimated pose of the head-mounted display device;

using the estimated pose, displaying the one or more virtual objects via the head-mounted display device in either the world-lock display mode or in a body-lock display mode in which the position of each of the virtual objects appears to be fixed relative to the user of the head-mounted display device;

determining one or more input uncertainty values of the sensor data;

mapping the one or more input uncertainty values to the estimated quality level of the world-lock display mode; and

communicating feedback of the estimated quality level to the user via the head-mounted display device.

2. The method of claim 1, wherein communicating feedback of the estimated quality level further comprises displaying one or more graphical representations of the estimated quality level via the head-mounted computing device.

3. The method of claim 2, wherein displaying the one or more graphical representations comprises altering an appearance of a field of view viewable through the head-mounted display device.

4. The method of claim 2, wherein displaying the one or more graphical representations comprises altering an appearance of the one or more virtual objects.

5. The method of claim 1, wherein communicating feedback of the estimated quality level further comprises:

providing the estimated quality level to an application that utilizes the world-lock display mode and the body-lock display mode; and

conveying the feedback of the estimated quality level to the user via the application and the head-mounted display device.

6. The method of claim 1, wherein communicating feedback of the estimated quality level further comprises refraining from communicating the feedback of the estimated quality level until a user input is received that calls for the world-lock display mode.

7. The method of claim 1, further comprising disabling the world-lock display mode when the estimated quality level is below a quality level threshold.

8. The method of claim 1, wherein determining the input uncertainty values further comprises analyzing image data for one or more of illumination of the real-world objects, a scene depth, scene motion, feature density, hair of the user, a hand of the user, and a hat worn by the user.

9. The method of claim 1, further comprising:

determining pose uncertainty values of the estimated pose; and

mapping the pose uncertainty values to the estimated quality level.

10. A head-mounted display device operatively connected to a computing device, the head-mounted display device comprising:

a tracking system executed by a processor of the computing device, the tracking system configured to;

receive sensor data from one or more sensors of the head-mounted display device;

using the sensor data, determine an estimated pose of the head-mounted display device;

determine one or more input uncertainty values of the sensor data;

map the one or more input uncertainty values to an estimated quality level of a world-lock display mode in which a position of one or more virtual objects appears to be fixed relative to real-world objects viewable through the head-mounted display device; and

communicate feedback to a user via the head-mounted display device of the estimated quality level; and

a display program executed by a processor of the computing device, the display program configured to display the one or more virtual objects using the estimated pose in either the world-lock display mode or in a body-lock display mode in which the position of each of the virtual objects appears to be fixed relative to the user of the head-mounted display device.

11. The head-mounted display device of claim 10, wherein the tracking system is configured to communicate the feedback of the estimated quality level by causing the display program to display one or more graphical representations of the estimated quality level via the head-mounted computing device.

12. The head-mounted display device of claim 11, wherein causing the display program to display one or more graphical representations comprises causing the display program to display one or more graphical representations in the body-lock display mode in which the one or more graphical representations appear to be fixed relative to the user of the head-mounted display device.

13. The head-mounted display device of claim 11, wherein causing the display program to display one or more graphical representations comprises altering an appearance of the one or more virtual objects.

14. The head-mounted display device of claim 10, wherein the tracking system is configured to communicate the feedback of the estimated quality level by providing the estimated quality level to an application that utilizes the world-lock display mode and the body-lock display mode, wherein the application conveys the feedback of the estimated quality level to the user via the head-mounted display device.

15. The head-mounted display device of claim 10, wherein the tracking system is configured to communicate the feedback of the estimated quality level by refraining from communicating the feedback until a user input is received that calls for the world-lock display mode.

16. The head-mounted display device of claim 10, wherein the tracking system is further configured to disable the world-lock display mode when the estimated quality level is below a quality level threshold.

17. The head-mounted display device of claim 10, wherein the tracking system is configured to:

determine both the input uncertainty values and pose uncertainty values of the estimated pose; and

map both the input uncertainty values and the pose uncertainty values to the estimated quality level.

18. A method for communicating to a user of a head-mounted display device an estimated quality level of a world-lock display mode in which a position of one or more virtual objects appears to be fixed relative to real-world objects viewable through the head-mounted display device, the method comprising:

receiving sensor data from one or more sensors of the head-mounted display device;

using the sensor data, determining an estimated pose of the head-mounted display device;

using the estimated pose, displaying the one or more virtual objects via the head-mounted display device in either the world-lock display mode or in a body-lock display mode in which the position of each of the virtual objects appears to be fixed relative to the user of the head-mounted display device;

determining one or more input uncertainty values of the sensor data;

mapping the one or more input uncertainty values to the estimated quality level of the world-lock display mode; and

displaying one or more graphical representations of the estimated quality level to the user via the head-mounted computing device.