Symbiotic helper

• US 9,824,601 B2
• Filed: 06/12/2012
• Issued: 11/21/2017
• Est. Priority Date: 06/12/2012
• Status: Active Grant

• Current Assignee: Dassault Systemes SA
• Original Assignee: Dassault Systemes SA
• Inventors: Grimaud, Jean-Jacques, Coleman, Garth Edward
• Primary Examiner(s): Hong, Thomas
• Application Number:
  US13/494,223
• Publication Number:
  US 20130330705A1
• Time in Patent Office: 1,988 Days
• Field of Search: 434365, 351158
• CPC Class Codes: G09B 5/00 Electrically-operated educational appliances working with questions and answers G09B7/00; simulators G09B9/00; advertising or displaying in general G09F

Abstract

In one embodiment, a computer-based method includes detecting a state of a user and detecting a stimulus in an environment of the user. The computer-based method then provides a variable level of assistance to the user based on the detected state of the user and the detected stimulus in the environment. In another embodiment, detecting the stimulus in the environment of the user may include detecting a visual stimulus, an auditory stimulus, a physical stimulus, a vibratory stimulus, an electro-magnetic stimulus, an olfactory stimulus, a temperature stimulus, or a movement stimulus. Detecting the state of the user may include detecting natural language spoken by the user. The resulting variable level of assistance utilizes a range of immersive virtual reality, augmented reality and sparsely augmented reality. 3D models of the user'"'"'s situation (the environment) may be employed in the variable levels of assistance.

Claims

1. A computer-based method comprising:

by one or more computer processors;

detecting, by at least one sensor, a stimulus in an environment of a user, the stimulus including at least one of visual stimulus, auditory stimulus, olfactory stimulus, temperature stimulus, electromagnetic stimulus, biological stimulus, and location stimulus;

detecting one or more attributes of the user including vital signs of the user from a biological sensor, a level of training of the user from a database, a line of sight of the user to the stimulus in the environment from an electromagnetic sensor, or a location of the user within the environment relative to the stimulus from a global positioning (GPS) sensor; and

responsive to the detected stimulus, providing, on a head-mounted display housing the at least one sensor worn by the user, a variable level of assistance to the user based on the detected one or more attributes of the user and the detected stimulus in the environment by loading from a training database stored in a memory a training technique corresponding to the variable level of assistance based on the detected one or more attributes of the user, a determined level of skill of the user, and the detected stimulus in the environment, wherein providing the variable level of assistance comprises training the user to require a lower level of assistance, and further by capturing a log including controls selected by the user in a virtual environment presented on the head-mounted display, and based on the captured log, determining the variable level of assistance.

2. The computer-based method of claim 1, wherein detecting the stimulus in the environment of the user comprises detecting at least one of a visual stimulus, an auditory stimulus, a physical stimulus, a vibratory stimulus, an electro-magnetic stimulus, an olfactory stimulus, a temperature stimulus, and a movement stimulus.

3. The computer-based method of claim 1, wherein detecting the one or more attributes of the user comprises detecting natural language spoken by the user.

4. The computer-based method of claim 1, further comprising:

determining a level of skill of the user; and

adjusting the variable level of assistance based on the determined level of skill.

5. The computer-based method of claim 1, wherein providing a variable level of assistance comprises:

providing an immersive virtual reality at a display for a first level of assistance;

providing an augmented virtual reality for a second level of assistance; and

providing a sparsely augmented reality for a third level of assistance.

6. The computer-based method of claim 5, further comprising alerting the user to the detected stimulus in the environment in the immersive virtual reality, augmented virtual reality, or sparsely augmented reality.

7. The computer-based method of claim 5, further comprising authoring content within the immersive virtual reality, wherein providing variable levels of assistance further provides the authored content.

8. The computer-based method of claim 1, wherein providing a variable level of assistance to the user comprises providing personal safety training or team safety training.

9. The computer based method of claim 1, wherein the log are digital footprints, and wherein providing a variable level of assistance to the user comprises analyzing digital footprints of the user to determine a level of proficiency and modifying provided variable level of assistance based on the determined level of proficiency.

10. The computer-based method of claim 1, further comprising alerting the user to positive and negative inferences determined from the detected stimulus.

11. The computer-based method of claim 1, further comprising providing instructions to the user based on a risk assessment of the detected environment.

12. The computer-based method of claim 1, further comprising:

identifying an abnormality in the detected stimulus;

determining supplemental information about the abnormality; and

alerting the user of the presence of the abnormality and the determined supplemental information.

13. A system comprising:

a processor; and

a memory with computer instructions stored therein, the memory operatively coupled to said processor such that the computer instructions configure the processor to implement;

detecting, by at least one sensor, a stimulus in an environment of the user, the stimulus including at least one of visual stimulus, auditory stimulus, olfactory stimulus, temperature stimulus, electromagnetic stimulus, biological stimulus, and location stimulus; and

detecting attributes of the user including one or more vital signs of the user from a biological sensor, a level of training of the user from a database, a line of sight of the user to the stimulus in the environment from an electromagnetic sensor, or a location of the user within the environment relative to the stimulus from a global positioning (GPS) sensor; and

responsive to the detected stimulus, providing, on a head-mounted display housing the at least one sensor worn by the user, a variable level of assistance to the user based on the detected one or more attributes of the user and the detected stimulus in the environment by loading from a training database stored in a memory a training technique corresponding to the variable level of assistance based on the detected one or more attributes of the user, a determined level of skill of the user, and the detected stimulus in the environment, the assistance module further configured to train the user to require a lower level of assistance, and further by capturing a log including controls selected by the user in a virtual environment presented on the head-mounted display, and based on the captured log, determining the variable level of assistance.

14. The system of claim 13, wherein detecting the stimulus further includes detecting at least one of a visual stimulus, an auditory stimulus, a physical stimulus, a vibratory stimulus, an electro-magnetic stimulus, an olfactory stimulus, a temperature stimulus, and a movement stimulus.

15. The system of claim 13, wherein detecting attributes of the user further includes detecting natural language spoken by the user.

16. The system of claim 13, wherein the processor is further configured to:

determine a level of skill of the user; and

adjust the variable level of assistance based on the determined level of skill.

17. The system of claim 13, wherein the processor is further configured to:

provide an immersive virtual reality at a display for a first level of assistance;

provide an augmented virtual reality for a second level of assistance; and

provide a sparsely augmented reality for a third level of assistance.

18. The system of claim 17, further comprising alerting the user to the detected stimulus in the environment in the immersive virtual reality, augmented virtual reality, or sparsely augmented reality.

19. The system of claim 17, wherein the processor is further configured to author content within the immersive virtual reality, wherein the assistance module is further configured to provide the authored content.

20. The system of claim 13, wherein the processor is further configured to provide personal safety training or team safety training.

21. The computer based method of claim 13, wherein the captured log are digital footprints, and wherein providing a variable level of assistance to the user includes analyzing digital footprints of the user to determine a level of proficiency and modifying provided variable level of assistance based on the determined level of proficiency.

22. The system of claim 13, wherein the processor is further configured to alert the user to positive and negative inferences determined from the detected stimulus.

23. The system of claim 13, wherein the processor is further configured to provide instructions to the user based on a risk assessment of the detected environment.

24. The system of claim 13, wherein the processor is further configured to:

identify an abnormality in the detected stimulus;

determine supplemental information about the abnormality; and

alert the user of the presence of the abnormality and the determined supplemental information.

25. An apparatus comprising:

a memory area configured to store preset levels of assistance to a user;

a processor coupled to said memory area, said processor configured to;

detect, by at least one sensor, a stimulus in an environment of the user, the stimulus including at least one of visual stimulus, auditory stimulus, olfactory stimulus, temperature stimulus, electromagnetic stimulus, biological stimulus, and location stimulus;

detect attributes of the user including one or more vital signs of the user from a biological sensor, a level of training of the user from a database, a line of sight of the user to the stimulus in the environment from an electromagnetic sensor, or a location of the user within the environment relative to the stimulus from a global positioning (GPS) sensor, and defining the detected as a state of the user;

responsive to the detected stimulus, provide, at a head-mounted display worn by the user, at least one of the preset levels of assistance to the user based on the detected one or more attributes of the user and the detected stimulus in the environment by loading from a training database stored in a memory a training technique corresponding to the variable level of assistance based on the detected one or more attributes of the user, a determined level of skill of the user, and the detected stimulus in the environment, said providing by the processor including training the user to require a lower level of assistance, and further by capturing a log including controls selected by the user in a virtual environment presented on the head-mounted display, and based on the captured log, determining the variable level of assistance; and

a wearable module including at least one of a display and a speaker configured to provide the at least one of the preset levels of assistance to the user.