Augmented reality help

• US 9,030,495 B2
• Filed: 11/21/2012
• Issued: 05/12/2015
• Est. Priority Date: 11/21/2012
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: McCulloch, Daniel, Tsunoda, Kudo, Lee, Abby Lin, Hastings, Ryan, Scott, Jason
• Primary Examiner(s): Sams, Michelle L
• Application Number:
  US13/683,732
• Publication Number:
  US 20140139551A1
• Time in Patent Office: 902 Days
• Field of Search: 345/633, 345/636, 715705-715, 382/181
• US Class Current: 345/633
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06F 9/453 Help systems
• CPC Class Codes: G09G 5/377 for mixing or overlaying two or more graphic patterns G09G5/02, G09G5/397 take precedence
• CPC Class Codes: G16B 99/00 Subject matter not provided for in other groups of this subclass
• CPC Class Codes: G16H 40/63 for local operation

Abstract

A system and related methods for an augmented reality help system in a head-mounted display device are provided. In one example, the head-mounted display device includes a plurality of sensors and a display system for presenting holographic objects. An augmented reality help program is configured to receive one or more user biometric parameters from the plurality of sensors. Based on the user biometric parameters, the program determines that the user is experiencing a stress response, and presents help content to the user via the head-mounted display device.

Claims

1. An augmented reality help system, comprising:

a head-mounted display device configured to be worn by a user and operatively connected to a computing device, the head-mounted display device including a plurality of sensors and a display system for presenting holographic objects, and

an augmented reality help program executed by a processor of the computing device, the augmented reality help program configured to;

receive user biometric parameters from one or more of the plurality of sensors;

based on one or more of the user biometric parameters, determine that the user is experiencing a stress response;

determine that the user'"'"'s attention is focused on an object;

attempt to identify the object; and

when the augmented reality help program is unable to identify the object, and based on determining that the user is experiencing the stress response, present default help content to the user via the head-mounted display device.

2. The augmented reality help system of claim 1, wherein the augmented reality help program is further configured to use one or more of eye-tracking information, head pose information, and user voice information to determine that the user'"'"'s attention is focused on the object.

3. The augmented reality help system of claim 1, wherein when the augmented reality help program identifies the object, the augmented reality help program is further configured to present to the user object-specific contextual help content that is tailored to the object.

4. The augmented reality help system of claim 3, wherein the object-specific contextual help content comprises either object-specific contextual help content provided by an authorized entity or crowd-sourced object-specific contextual help content that is provided by at least one or more third parties.

5. The augmented reality help system of claim 3, wherein the augmented reality help program is further configured to tailor the object-specific contextual help content to the user based on user-related information.

6. The augmented reality help system of claim 1, wherein the plurality of sensors are selected from the group consisting of an optical sensor, a position sensor, an eye-tracking sensor, a microphone, and a biometric parameter sensor.

7. The augmented reality help system of claim 1, wherein the user biometric parameters are selected from the group consisting of heart rate, pupillary response, hemoglobin saturation, skin conductivity, respiration, perspiration, and brainwave activity.

8. A method for presenting help content to a user via a head-mounted display device, comprising:

providing the head-mounted display device configured to be worn by the user and operatively connected to a computing device, the head-mounted display device including a plurality of sensors and a display system for presenting holographic objects;

receiving one or more user biometric parameters from one or more of the plurality of sensors;

based on one or more of the user biometric parameters, determining that the user is experiencing a stress response;

determining that the user'"'"'s attention is focused on an object;

attempting to identify the object; and

when the augmented reality help program is unable to identify the object, and based on determining that the user is experiencing the stress response, presenting default help content to the user via the head-mounted display device.

9. The method of claim 8, further comprising using one or more of eye-tracking information, head pose information, and user voice information to determine that the user'"'"'s attention is focused on the object.

10. The method of claim 8, wherein if the object is identified, then presenting to the user object-specific contextual help content that is tailored to the object.

11. The method of claim 10, wherein the object-specific contextual help content comprises either object-specific contextual help content provided by an authorized entity or crowd-sourced object-specific contextual help content that is provided by at least one or more third parties.

12. The method of claim 10, further comprising tailoring the object-specific contextual help content to the user based on user-related information.

13. The method of claim 8, wherein the plurality of sensors are selected from the group consisting of an optical sensor, a position sensor, an eye-tracking sensor, a microphone, and a biometric parameter sensor.

14. The method of claim 8, wherein the user biometric parameters are selected from the group consisting of heart rate, pupillary response, hemoglobin saturation, skin conductivity, respiration, perspiration, and brainwave activity.

15. An augmented reality help system, comprising:

a head-mounted display device configured to be worn by a user and operatively connected to a computing device, the head-mounted display device including a plurality of sensors selected from the group consisting of an optical sensor, a position sensor, an eye-tracking sensor, and a microphone, the head-mounted display device also including a display system for presenting holographic objects; and

an augmented reality help program executed by a processor of the computing device, the augmented reality help program configured to;

determine that the user'"'"'s attention is focused on an object;

attempt to identify the object;

when the augmented reality help program is unable to identify the object, present default help content to the user; and

when the augmented reality help program identifies the object, present to the user either object-specific contextual help content provided by an authorized entity or crowd-sourced object-specific contextual help content that is provided by at least one or more third parties.

16. The augmented reality help system of claim 15, wherein the augmented reality help program is further configured to present the default help content, the object-specific contextual help content, or the crowd-sourced object-specific contextual help content to the user based on receiving a request for help from the user.