HEAD-MOUNTED DISPLAY RESOURCE MANAGEMENT

• US 20140145914A1
• Filed: 11/29/2012
• Published: 05/29/2014
• Est. Priority Date: 11/29/2012
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Latta, Stephen, Fleck, Rod G., Clevenger, Jack, Schaffalitzky, Frederik, Steedly, Drew, McCulloch, Daniel, McIntyre, Ian, Balan, Alexandru, Sugden, Ben, Hastings, Ryan, Mount, Brian, Perry, Jedd Anthony
• Granted Patent: US 9,851,787 B2
• Time in Patent Office: Days
• US Class Current: 345/8
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0174 holographic
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/0093 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06F 1/3215 Monitoring of peripheral devices
• CPC Class Codes: G06F 1/325 Power saving in peripheral device
• CPC Class Codes: G06F 1/3265 Power saving in display device
• CPC Class Codes: G06F 1/3287 by switching off individual functional units in the computer system
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: Y02D 10/00 Energy efficient computing, e.g. low power processors, power management or thermal management

Abstract

A system and related methods for a resource management in a head-mounted display device are provided. In one example, the head-mounted display device includes a plurality of sensors and a display system for presenting holographic objects. A resource management program is configured to operate a selected sensor in a default power mode to achieve a selected fidelity. The program receives user-related information from one or more of the sensors, and determines whether target information is detected. Where target information is detected, the program adjusts the selected sensor to operate in a reduced power mode that uses less power than the default power mode.

Claims

1. A resource management system, comprising:

a head-mounted display device configured to be worn by a user and operatively connected to a computing device, the head-mounted display device including a plurality of sensors and a display system for presenting holographic objects, and

a resource management program executed by a processor of the computing device, the resource management program configured to;

operate a selected sensor of the plurality of sensors in a default power mode to achieve a selected level of sensor fidelity;

receive user-related information from one or more of the plurality of sensors, the user-related information selected from the group consisting of audio information, user gaze information, user location information, user movement information, user image information, and user physiological information;

determine whether target information is detected in the user-related information; and

where the target information is detected, adjust the selected sensor to operate in a reduced power mode that uses less power than the default power mode, thereby achieving a reduced level of sensor fidelity.

2. The resource management system of claim 1, wherein the target information is selected from the group consisting of context-identifying audio information, a user gaze that is fixed on a holographic object or a physical object, a user location at a predetermined location, movement of the user below a movement threshold, image information indicating a user body part, and awareness-related user physiological information that indicates a user awareness below an awareness threshold.

3. The resource management system of claim 1, wherein the plurality of sensors are selected from the group consisting of an optical sensor, a position sensor, an eye-tracking sensor, and a microphone.

4. The resource management system of claim 1, wherein the reduced power mode comprises a reduced sensor resolution, a reduced sensor sampling rate, or a powered-off state.

5. The resource management system of claim 1, wherein the resource management program is further configured to:

receive physical environment information from one or more of the plurality of sensors; and

determine whether the target information is detected in the physical environment information.

6. The resource management system of claim 5, wherein the resource management program is further configured to:

receive the physical environment information via a network from an external source that is external to the head-mounted display device; and

based on receiving the physical environment information from the external source, adjust the selected sensor to operate in the reduced power mode.

7. The resource management system of claim 5, wherein the one or more of the plurality of sensors comprises an optical sensor, the physical environment information comprises physical object image data, and the resource management program is further configured to:

determine whether the optical sensor has captured a threshold amount of the physical object image data; and

when the threshold amount of the physical object image data has been captured, adjust the optical sensor to operate in the reduced power mode that uses less power than a default power mode of the optical sensor.

8. The resource management system of claim 1, wherein the plurality of sensors comprises a microphone, and the resource management program is further configured to:

operate the microphone at a slower polling frequency while target audio is not detected; and

adjust the microphone to operate at a faster polling frequency that is greater than the slower polling frequency when the target audio is detected.

9. The resource management system of claim 1, wherein the plurality of sensors comprises an eye-tracking sensor, the target information comprises a user'"'"'s gaze at one of the holographic objects or a physical object, and the resource management program is further configured to:

detect the user'"'"'s gaze at one of the holographic objects or the physical object using the eye-tracking sensor; and

based on detecting the user'"'"'s gaze, reduce an illumination output of the display system of the head-mounted display device.

10. The resource management system of claim 1, wherein the selected sensor comprises a position sensor, the target information comprises movement of the user below a movement threshold, and the resource management program is further configured to:

detect that the movement of the user is below the movement threshold using the position sensor; and

based on detecting the movement of the user below the movement threshold, adjust the position sensor to operate in the reduced power mode.

11. A method for managing resources in a head-mounted display device, comprising:

providing the head-mounted display device configured to be worn by a user and operatively connected to a computing device, the head-mounted display device including a plurality of sensors and a display system for presenting holographic objects;

operating a selected sensor of the plurality of sensors in a default power mode to achieve a selected level of sensor fidelity;

receiving user-related information from one or more of the plurality of sensors, the user-related information selected from the group consisting of audio information, user gaze information, user location information, user movement information, user image information, and user physiological information;

determining whether target information is detected in the user-related information; and

where the target information is detected, adjusting the selected sensor to operate in a reduced power mode that uses less power than the default power mode, thereby achieving a reduced level of sensor fidelity.

12. The method for managing resources of claim 11, wherein the target information is selected from the group consisting of context-identifying audio information, a user gaze that is fixed on a holographic object or a physical object, a user location at a predetermined location, movement of the user below a movement threshold, image information indicating a user body part, and awareness-related user physiological information that indicates a user awareness below an awareness threshold.

13. The method for managing resources of claim 11, further comprising:

receiving physical environment information from one or more of the plurality of sensors; and

determining whether the target information is detected in the physical environment information.

14. The method for managing resources of claim 13, further comprising:

receiving the physical environment information via a network from an external source that is external to the head-mounted display device; and

based on receiving the physical environment information from the external source, adjusting the selected sensor to operate in the reduced power mode.

15. The method for managing resources of claim 13, wherein the one or more of the plurality of sensors comprises an optical sensor, the physical environment information comprises physical object image data, and further comprising:

determining whether the optical sensor has captured a threshold amount of the physical object image data; and

when the threshold amount of the physical object image data has been captured, adjusting the optical sensor to operate in the reduced power mode that uses less power than a default power mode of the optical sensor.

16. The resource management system of claim 11, wherein the plurality of sensors comprises a microphone, and further comprising:

operating the microphone at a slower polling frequency when target audio is not detected; and

adjusting the microphone to operate at a faster polling frequency that is greater than the slower polling frequency when the target audio is detected.

17. The method for managing resources of claim 11, wherein the plurality of sensors comprises an eye-tracking sensor, the target information comprises a user'"'"'s gaze at one of the holographic objects or a physical object, and further comprising:

detecting the user'"'"'s gaze at one of the holographic objects or the physical object using the eye-tracking sensor; and

based on detecting the user'"'"'s gaze, reducing an illumination output of the display system of the head-mounted display device.

18. The resource management system of claim 11, wherein the selected sensor comprises a position sensor, the target information comprises movement of the user below a movement threshold, and further comprising:

detecting that the movement of the user is below the movement threshold using the position sensor; and

based on detecting the movement of the user below the movement threshold, adjusting the position sensor to operate in the reduced power mode.

19. A resource management system, comprising:

a head-mounted display device configured to be worn by a user and operatively connected to a computing device, the head-mounted display device comprising a display system for presenting holographic objects and a plurality of sensors selected from the group consisting of an optical sensor, a position sensor, an eye-tracking sensor, and a microphone; and

a resource management program executed by a processor of the computing device, the resource management program configured to;

operate a selected sensor of the plurality of sensors in a default power mode to achieve a selected level of sensor fidelity;

receive user-related information from one or more of the plurality of sensors, the user-related information selected from the group consisting of audio information, user gaze information, user location information, user movement information, user image information, and user physiological information;

receive physical environment information from one or more of the plurality of sensors;

determine whether target information is detected in the user-related information or the physical environment information; and

where the target information is detected, adjust the selected sensor to operate in a reduced power mode that uses less power than the default power mode, thereby achieving a reduced level of sensor fidelity.

20. The resource management system of claim 19, wherein the resource management program is further configured to:

receive the physical environment information via a network from an external source that is external to the head-mounted display device; and

based on receiving the physical environment information from the external source, adjust the selected sensor to operate in the reduced power mode.