AUGMENTED REALITY HELP

• US 20140139551A1
• Filed: 11/21/2012
• Published: 05/22/2014
• Est. Priority Date: 11/21/2012
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: McCulloch, Daniel, Hastings, Ryan, Scott, Jason, Tsunoda, Kudo, Lee, Abby Lin
• Granted Patent: US 9,030,495 B2
• Time in Patent Office: Days
• US Class Current: 345/633
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06F 9/453 Help systems
• CPC Class Codes: G09G 5/377 for mixing or overlaying two or more graphic patterns G09G5/02, G09G5/397 take precedence
• CPC Class Codes: G16B 99/00 Subject matter not provided for in other groups of this subclass
• CPC Class Codes: G16H 40/63 for local operation

Abstract

A system and related methods for an augmented reality help system in a head-mounted display device are provided. In one example, the head-mounted display device includes a plurality of sensors and a display system for presenting holographic objects. An augmented reality help program is configured to receive one or more user biometric parameters from the plurality of sensors. Based on the user biometric parameters, the program determines that the user is experiencing a stress response, and presents help content to the user via the head-mounted display device.

Claims

1. An augmented reality help system, comprising:

a head-mounted display device configured to be worn by a user and operatively connected to a computing device, the head-mounted display device including a plurality of sensors and a display system for presenting holographic objects, and

an augmented reality help program executed by a processor of the computing device, the augmented reality help program configured to;

receive user biometric parameters from one or more of the plurality of sensors;

based on one or more of the user biometric parameters, determine that the user is experiencing a stress response; and

based on determining that the user is experiencing the stress response, present help content to the user via the head-mounted display device.

2. The augmented reality help system of claim 1, wherein the augmented reality help program is further configured to determine that the user'"'"'s attention is focused on an object.

3. The augmented reality help system of claim 2, wherein the augmented reality help program is further configured to use one or more of eye-tracking information, head pose information, and user voice information to determine that the user'"'"'s attention is focused on the object.

4. The augmented reality help system of claim 2, wherein the augmented reality help program is further configured to attempt to identify the object, and if the augmented reality help program is unable to identify the object, then the help content presented to the user comprises default help content.

5. The augmented reality help system of claim 4, wherein if the augmented reality help program identifies the object, then the help content presented to the user comprises object-specific contextual help content that is tailored to the object.

6. The augmented reality help system of claim 5, wherein the object-specific contextual help content comprises either object-specific contextual help content provided by an authorized entity or crowd-sourced object-specific contextual help content that is provided by at least one or more third parties.

7. The augmented reality help system of claim 5, wherein the augmented reality help program is further configured to tailor the object-specific contextual help content to the user based on user-related information.

8. The augmented reality help system of claim 1, wherein the plurality of sensors are selected from the group consisting of an optical sensor, a position sensor, an eye-tracking sensor, a microphone, and a biometric parameter sensor.

9. The augmented reality help system of claim 1, wherein the user biometric parameters are selected from the group consisting of heart rate, pupillary response, hemoglobin saturation, skin conductivity, respiration, perspiration, and brainwave activity.

10. A method for presenting help content to a user via a head-mounted display device, comprising:

providing the head-mounted display device configured to be worn by the user and operatively connected to a computing device, the head-mounted display device including a plurality of sensors and a display system for presenting holographic objects;

receiving one or more user biometric parameters from one or more of the plurality of sensors;

based on one or more of the user biometric parameters, determining that the user is experiencing a stress response; and

based on determining that the user is experiencing the stress response, presenting help content to the user via the head-mounted display device.

11. The method of claim 10, further comprising determining that the user'"'"'s attention is focused on an object.

12. The method of claim 11, further comprising using one or more of eye-tracking information, head pose information, and user voice information to determine that the user'"'"'s attention is focused on the object.

13. The method of claim 11, further comprising attempting to identify the object, and if the object is not identified, then presenting default help content to the user.

14. The method of claim 13, wherein if the object is identified, then presenting to the user object-specific contextual help content that is tailored to the object.

15. The method of claim 14, wherein the object-specific contextual help content comprises either object-specific contextual help content provided by an authorized entity or crowd-sourced object-specific contextual help content that is provided by at least one or more third parties.

16. The method of claim 14, further comprising tailoring the object-specific contextual help content to the user based on user-related information.

17. The method of claim 10, wherein the plurality of sensors are selected from the group consisting of an optical sensor, a position sensor, an eye-tracking sensor, a microphone, and a biometric parameter sensor

18. The method of claim 10, wherein the user biometric parameters are selected from the group consisting of heart rate, pupillary response, hemoglobin saturation, skin conductivity, respiration, perspiration, and brainwave activity.

19. An augmented reality help system, comprising:

a head-mounted display device configured to be worn by a user and operatively connected to a computing device, the head-mounted display device including a plurality of sensors selected from the group consisting of an optical sensor, a position sensor, an eye-tracking sensor, and a microphone, the head-mounted display device also including a display system for presenting holographic objects; and

an augmented reality help program executed by a processor of the computing device, the augmented reality help program configured to;

determine that the user'"'"'s attention is focused on an object;

attempt to identify the object;

if the augmented reality help program is unable to identify the object, then present default help content to the user; and

if the augmented reality help program identifies the object, then present to the user either object-specific contextual help content provided by an authorized entity or crowd-sourced object-specific contextual help content that is provided by at least one or more third parties.

20. The augmented reality help system of claim 19, wherein the augmented reality help program is further configured to present the default help content, the object-specific contextual help content, or the crowd-sourced object-specific contextual help content to the user based on receiving a request for help from the user.