Cockpit flow training system

• US 9,583,019 B1
• Filed: 03/23/2012
• Issued: 02/28/2017
• Est. Priority Date: 03/23/2012
• Status: Active Grant

• Current Assignee: The Boeing Co.
• Original Assignee: The Boeing Co.
• Inventors: Korolev, Alexander V., Schwartz, Samantha Ann, Barraci, Nima
• Primary Examiner(s): Yao, Sam
• Assistant Examiner(s): Humphrey, Michael
• Application Number:
  US13/428,105
• Time in Patent Office: 1,803 Days
• Field of Search: 434/14, 434/15, 434 28- 59, 434/220, 434/241, 434/243, 434/372, 472/130, 446/7
• US Class Current: 1/1
• CPC Class Codes: G09B 9/16 Ambient or aircraft conditions simulated or indicated by instrument or alarm
• CPC Class Codes: G09B 9/206 for in-flight condition

Abstract

An aircraft simulation system comprising a user interface and a simulator in communication with the user interface. The user interface is configured to interact with an operator. The simulator is configured to generate a representation of controls in an aircraft that are visible from a current field of view identified by the user interface. The simulator is further configured to display the representation of the controls on the user interface. The simulator is further configured to identify interaction by the operator with the controls from user input received by the user interface. The simulator is further configured to record a sequence of performed operations from the interaction with the controls.

Claims

1. A method for reducing a time and a cost for performing a sequence in a full flight simulator, the method comprising, before entering the full flight simulator:

presenting virtual reality images, representing objects, on a flight deck that would be visible based on a position and an orientation of an operator relative to a display system, in a virtual reality of the full flight simulator, on a display system, such that as at least one of the display system and the position physically move, the virtual reality images presented on the display system change to recreate a new view of the flight deck that would be visible from a new position and orientation of the operator relative to the display system;

receiving, in a user input system comprising the display system, inputs interacting with objects;

connecting a processor to the display system and generating the new view for the virtual reality presented on the display system based upon;

the display system comprising a first sensor sending a coordinate of the display system sent to the processor;

a physical configuration of the full flight simulator represented in the virtual reality; and

a second sensor sending coordinates, representing the position of the operator relative to the display system, to the processor, such that a presentation of a current field of view changes in response to changing at least one of;

the coordinate of the display system, and the coordinates representing the position;

presenting, on the display system, images guiding interactions with the objects;

recognizing coordinates relative to the display system of an interaction with a presentation, of an object in the full flight simulator, in the current field of view for the virtual reality presented on the display system, via a third sensor, without requiring a physical representation, as presented in the full flight simulator, of the object;

changing the presentation of the object in response to the interaction; and

presenting a computation of a score, relative to briefing package that comprises a task and an identification of an aircraft simulated by the full flight simulator, for a sequence of interactions with the presentation of the object in the full flight simulator.

2. A machine that comprises:

i. a user interface system configured to receive inputs that interact with objects that exist in a virtual reality presented on a display system that comprises;

ii. a connection to a processor unit configured to;

iii. generate a current field of view images for the display system, wherein a field of view image recreates a view of a flight deck that would be visible based on a position and orientation of an operator relative to the display system, such that as at least one of the display system and the position physically move, images presented on the display system change to recreate a new view of the flight deck that would be visible from the new position and orientation, wherein a generation of the current field of view images is based upon;

iv. a first sensor of the display system that sends a coordinate of the display system sent to the processor unit;

iv. a physical configuration of the flight deck represented by the virtual reality presented; and

iv. a second sensor that sends coordinates that represent an operator position and orientation to the processor unit, such that a presentation of the new view for the display system changes in response to at least one of a change in;

the coordinate of the display system, and the coordinates that represent the operator position; and

iii. present, on the display system, a guide to interactions with the objects;

ii. a third sensor configured to recognize coordinates, relative to the display system, of an interaction with a presentation, in the view presented on the display system, of an object in the flight deck, without requiring a physical representation, as present in a full flight simulator, of the object, such that the presentation, in the view presented on the display system of the object in the flight deck, changes in response to the interaction;

ii. a computation of a score, relative to a briefing package that comprises a task and an identification of an aircraft that comprises the flight deck, for a sequence of interactions with the presentation of the object in the flight deck.

3. The machine of claim 2, wherein the processor unit is configured to compare the sequence of interactions with an expected sequence of the task and identify a set of differences between the sequence of interactions and the expected sequence of the task.

4. The machine of claim 3, wherein the processor unit is configured to generate the score from the set of differences.

5. The machine of claim 2, wherein the processor unit is configured to present a sequence of expected operations prior to recording the sequence of interactions.

6. The machine of claim 2, wherein the second sensor comprises:

a head motion sensor configured to identify movement of a head of the operator; and

a hand motion sensor configured to identify motion of a hand of the operator.

7. The machine of claim 2, wherein the processor unit comprises:

a learning processor configured to receive the briefing package and control a simulation generated by an aircraft simulation system;

a cockpit synthesizer configured for use with an aircraft configurator by the learning processor to generate a virtual representation, in the view, of controls in the flight deck; and

a flow configurator configured to detect a sequence of expected interactions with a user input system.

8. The machine of claim 2, wherein the display system and the processor unit are located in one of a tablet computer, and a mobile phone.

9. The machine of claim 2, wherein the display system and the processor unit are located in a computer system selected from one of a mobile computer, a tablet computer, and a mobile phone.

10. The machine of claim 2, further comprising the processor unit configured to compare a sequence of performed operations with a sequence of expected operations and identify a set of differences between the sequence of performed operations and the sequence of expected operations.

11. The machine of claim 10, the processor unit further configured to generate the score from the set of differences.

12. The machine of claim 2, wherein a user input system comprises the display system.

13. The machine of claim 12, wherein the display system is selected from one of stereoscopic glasses, a head worn display system, and a liquid crystal display monitor.

14. The machine of claim 12, wherein the user input system comprises at least one of a camera and a motion data capture glove.

15. The machine of claim 12, wherein the user input system further comprises an audio system.

16. A method for changing a position of an image of an object shown in a display system showing a field of view of a simulation of a flight deck, without requiring a physical representation, as present in a full flight simulator, of the object, the method comprising:

presenting virtual reality images representing objects, on a flight deck that would be visible based on a position and an orientation of an operator relative to a display system, in a virtual reality presented on the display system, such that as at least one of the display system and the position of the operator physically move, the virtual reality images presented on the display system change to recreate a new view of the flight deck that would be visible from a new position and orientation of the operator relative to the display system;

receiving, in the display system, inputs interacting with objects;

connecting a processor to the display system and;

generating a current field of view for the display system based upon;

a first sensor of the display system sending a coordinate of the display system sent to the processor;

a physical configuration of the full flight simulator represented in the virtual reality; and

a second sensor that sending coordinates that represent an operator position relative to the display system to the processor, such that a presentation of the current field of view for the display system changes in response to at least one of a change in;

the coordinate of the display system, and the coordinates that represent the operator position; and

presenting, on the display system, images guiding interactions with the objects;

recognizing coordinates, relative to the display system, of an interaction with a presentation of the object, in the flight deck, in the current field of view presented on the display system, via a third sensor, without requiring the physical representation, as present in the full flight simulator, of the object;

changing the presentation of the object in response to the interaction;

presenting a computation of a score, relative to a briefing package that comprises a task and an identification of an aircraft that comprises the flight deck, for a sequence of interactions with the presentation.

17. The method of claim 16, further comprising:

presenting a sequence of expected operations prior to recording the sequence of interactions with the presentation.

18. The method of claim 16, wherein recognizing coordinates of the interaction further comprises:

identifying motion of a hand of the operator with respect to flight deck controls represented in the display system.

19. The method of claim 16, further comprising:

comparing a sequence of performed operations with a sequence of expected operations and identifying a set of differences between the sequence of performed operations and the sequence of expected operations.

20. The method of claim 19, further comprising:

generating the score from the set of differences.