VIRTUAL REALITY TRAINING

• US 20170148214A1
• Filed: 07/17/2015
• Published: 05/25/2017
• Est. Priority Date: 07/17/2015
• Status: Abandoned Application

• Current Assignee: Exo Insights Corp.
• Original Assignee: IVD Mining, Co.
• Inventors: Muniz-Simas, Fernando Morera, Muniz-Simas, Silvia Regina Marega
• Application Number:
  US14/762,434
• Publication Number:
  US 20170148214A1
• Time in Patent Office: Days
• CPC Class Codes: A61B 2503/20 Workers
• CPC Class Codes: A61B 5/024 Detecting, measuring or recording pulse rate or heart rate A61B5/0205, A61B5/021 take precedence
• CPC Class Codes: A61B 5/165 Evaluating the state of mind, e.g. depression, anxiety
• CPC Class Codes: G05B 9/00 Safety arrangements G05B7/00 takes precedence; safety arrangements in programme-control systems G05B19/048, G05B19/406
• CPC Class Codes: G06F 1/163 Wearable computers, e.g. on a belt
• CPC Class Codes: G06F 2203/011 Emotion or mood input determined on the basis of sensed human body parameters such as pulse, heart rate or beat, temperature of skin, facial expressions, iris, voice pitch, brain activity patterns
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/015 Input arrangements based on nervous system activity detection, e.g. brain waves [EEG] detection, electromyograms [EMG] detection, electrodermal response detection
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/14 Digital output to display device ; Cooperation and interconnection of the display device with other functional units
• CPC Class Codes: G06T 13/40 of characters, e.g. humans, animals or virtual beings
• CPC Class Codes: G06T 19/003 Navigation within 3D models or images
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G09B 19/24 Use of tools
• CPC Class Codes: G09B 25/02 of industrial processes; of machinery
• CPC Class Codes: G09B 9/00 Simulators for teaching or training purposes for the use of weapons F41; computing aspects G06; protocols for games, networked simulations or virtual reality H04L67/131

Abstract

A virtual reality training system for industrial labor applications is disclosed. Users wear virtual reality equipment including a head mounted device and enter a virtual worksite replete with VR industrial equipment, VR hazards, and virtual tasks. Through the course of completing the tasks a plurality of sensors monitor the performance of the user or users and identify knowledge gaps and stresses of the user(s). The system generates an evaluation associated with the user(s) and then informs the user where there is room for improvement and informs an administrator of potential liabilities latent within evaluated employees.

Claims

1. A method for generating an immersive virtual reality(VR) platform for workers of dangerous mining, oil, and gas worksites to provide training or certification programs replete with a plurality of sensors to detect and correct knowledge gaps and prevent life threatening situations, all confined within the safety of a virtual reality worksite, comprising:

generating a VR resource extraction worksite including virtual dangers and massive virtual industrial machines;

displaying the VR resource extraction worksite to a user with a head mounted device including sensors;

tracking the user with the head mounted device and sensors as the user navigates the VR resource extraction worksite completing tasks and interacting with the virtual dangers and massive virtual industrial machines using a combination of eye contact detection, hand gestures, and heavy machinery remote controls;

identifying incorrect machine procedures and neglected virtual dangers as compared to a rubric of best practices;

collecting biometric data including stress response, heart rate, and fear of the user while the user performs tasks in the VR resource extraction worksite;

generating an evaluation of the user according to the best practices rubric, the evaluation concerning safety procedures, equipment operating procedures, and awareness of latent dangers such as electrocution, burns, downing, impact and crushing hazards; and

presenting the evaluation to the user to improve work performance and safety.

2. A method for virtual reality (VR) training, comprising:

generating, by a processor, a VR heavy industry worksite comprising VR industrial equipment and VR hazards;

displaying the VR heavy industry worksite to a user with a head mounted device including sensors;

tracking the user with the head mounted device as the user navigates the VR heavy industry worksite;

receiving, by the processor, sensor data collected by the sensors, the sensors comprising all of;

an eye tracking sensor;

peripheral controls simulating industrial equipment; and

a motion tracking sensor;

wherein, the sensor data comprises all of;

stress response data associated with the user to the VR resource extraction worksite;

active use procedure data associated with the user interacting with the VR industrial equipment; and

hazard awareness and resolution data associated with the user interacting with the VR hazards;

creating an evaluation associated with the sensor data by the processor according to a best practices rubric;

reporting the evaluation to either a physical display or digital display.

3. The method of claim 2, wherein the VR industrial equipment comprises any of:

virtual equipment associated with oil extraction;

virtual equipment associated with gas extraction;

virtual equipment associated with large scale construction;

or

virtual equipment associated with ore or mineral extraction.

4. The method of claim 2, wherein the VR hazards comprise any of:

virtual oil spills;

virtual oil leaks;

virtual misplaced tools;

virtual improperly balanced objects;

virtual lack of proper equipment;

virtual electrical systems;

virtual contact with electrical sources;

virtual contact with high pressures;

virtual contract with high temperatures sources;

virtual work at heights;

virtual contact with mobile equipment;

or

virtual contact with radiation.

5. The method of claim 2, wherein the head mounted device is configured to detect vertical motion of the user, and said VR hazards are situated at variable heights within the VR heavy industry worksite, and said best practices rubric includes identifying VR hazards at heights other than eye level.

6. The method of claim 5, wherein VR hazards are concealed behind virtual obstructions, and in order to view VR hazards, the user must circumvent the virtual obstructions.

7. The method of claim 2, wherein the stress response data comprises indicators for vertigo or fear of heights

8. The method of claim 2, wherein the motion tracking sensor is enabled to capture position and gesture data of a hand of the user, wherein the position and gesture data influence virtual conditions of the VR heavy industry worksite.

9. The method of claim 2, wherein the VR hazards are classified into sub categories including:

critical; and

non-critical;

wherein critical VR hazards are those which simulate significant danger to human health.

10. The method of claim 2, further comprising:

providing the user with one or more virtual tasks, the virtual tasks simulating work that takes place in a resource extraction worksite, wherein the evaluation is subdivided into each of the one or more virtual tasks.

11. The method of claim 2, wherein the user is a first user, and further comprising:

displaying a plurality of avatars of other users within the VR heavy industry worksite, the plurality of other users operative in the VR heavy industry worksite with the first user and the data collected associated with the first user further augmented by interaction with plurality of avatars of other users.

12. A method for identifying knowledge gaps associated with a user using virtual reality(VR), comprising:

generating, by a processor, a virtual reality resource extraction worksite comprising at least one important safety region, the at least one important safety region is a defined virtual location within the VR resource extraction worksite that is visually distinct to a user;

obtaining, by the processor, from a location aware head mounted device, position data associated with the location aware head mounted device, said position data comprising a location on a three dimensional coordinate plane and an orientation, said position data further corresponding to a location in the VR resource extraction worksite;

displaying the VR resource extraction worksite to the user with the location aware head mounted device according to the position data;

detecting, by an eye tracking sensor, eye contact data associated with the user and the VR resource extraction worksite, the eye tracking sensor affixed to the location aware head mounted device; and

evaluating the user with respect to the at least one important safety region, wherein said evaluating comprises;

detecting by the eye tracking sensor that the user makes eye contact with the at least one important safety region; and

receiving input from the user associated with a virtual condition of the at least one important safety region.

13. The method of claim 12, wherein the VR resource extraction worksite further comprises:

virtual obstructions, the virtual obstructions preventing line of sight between the user and the at least one important safety region, wherein the user is enabled to generate eye contact with the at least one important safety region only when the location aware head mounted device has predefined acceptable position data.

14. The method of claim 12, wherein input from the user identifies the virtual condition as either:

safe;

or

requires action; and

further comprising;

when the virtual condition is requires action, receiving input from the user directed towards the virtual condition.

15. The method of claim 12, wherein input from the user is any of:

auditory;

received through a peripheral device;

user hand gestures received by a motion sensor affixed to the location aware head mounted device; and

user selection through eye movement captured by the eye tracking sensor.

16. The method of claim 12, wherein the at least one important safety region comprises a virtual depiction of equipment, and the receiving input from the user associated with a virtual condition comprises the user virtually collecting the equipment.

17. The method of claim 12, further comprising:

classifying the at least one important safety region as critical or non-critical, wherein a critical important safety region simulates a real world condition that significantly endangers human safety.

18. The method of claim 12, wherein the at least one important safety region comprises at least two important safety regions, and further comprising:

providing the user with one or more virtual tasks, the virtual tasks simulating work that takes place in a resource extraction worksite, the virtual tasks including evaluation with respect to two or more important safety regions; and

generating a report of the user, the report associated with performance of the user on the one or more virtual tasks, wherein the report is based on the combination of said evaluation step with respect to two or more important safety regions.

19. The method of claim 12, wherein the user is a first user, and further comprising:

displaying a plurality of avatars of other users within the VR resource extraction worksite, the plurality of other users operative in the VR resource extraction worksite with the first user and wherein the plurality of avatars of other users each comprise an important safety region.

20. A virtual reality training apparatus, comprising:

a head mounted device including;

a motion tracker;

an eye tracker;

an immersive graphic display;

a processor communicatively coupled to the head mounted device;

peripheral controls simulating industrial equipment, the peripheral controls communicatively coupled to the processor; and

a memory communicatively coupled to the processor, the memory containing a best practices rubric and instructions, the instructions configured to cause the processor to generate a VR resource extraction worksite comprising VR industrial equipment and VR hazards, the immersive graphic display to display the VR resource extraction worksite to a user, and to receive data from the motion tracker, the eye tracker, and the peripheral controls simulating industrial equipment, wherein the data comprises all of;

stress response data associated with the user to the VR resource extraction worksite;

active use procedure data associated with the user interacting with the VR industrial equipment; and

hazard awareness and resolution data associated with the user interacting with the VR hazards;

and further causing the processor to create an evaluation associated with the data compared to the best practices rubric, then report the evaluation to either a physical display or digital display.

21. The apparatus of claim 20, wherein the peripheral controls simulating industrial equipment comprises repurposed remote controls for real industrial equipment.

22. The apparatus of claim 20, wherein the processor is body mounted on the user.

23. The apparatus of claim 20, wherein the processor communicates to the head mounted device wirelessly.