Aircrew automation system and method

• US 10,642,270 B2
• Filed: 07/08/2019
• Issued: 05/05/2020
• Est. Priority Date: 03/22/2016
• Status: Active Grant

• Current Assignee: Aurora Flight Sciences Corporation (The Boeing Co.)
• Original Assignee: Aurora Flight Sciences Corporation (The Boeing Co.)
• Inventors: Duda, Jessica E., Tylko, John, Mindell, David, Kunzi, Fabrice, Piedmonte, Michael, Allee, John, Torgerson, Joshua, Ryan, Jason, Paduano, James Donald, Wissler, John Brooke, Musto, Andrew, Feenstra, Wendy
• Primary Examiner(s): Nolan, Peter D
• Application Number:
  US16/504,988
• Publication Number:
  US 20200064840A1
• Time in Patent Office: 302 Days
• CPC Class Codes: B25J 11/00 Manipulators not otherwise provided for
• CPC Class Codes: B64C 13/18 using automatic pilot
• CPC Class Codes: B64D 2045/0075 Adaptations for use of electronic flight bags in aircraft; Supports therefor in the cockpit
• CPC Class Codes: B64D 2045/0085 Devices for aircraft health monitoring, e.g. monitoring flutter or vibration
• CPC Class Codes: B64D 43/00 Arrangements or adaptations of instruments arrangements of cameras B64D47/08; aeronautical measuring instruments per se G01C
• CPC Class Codes: B64D 45/00 Aircraft indicators or protectors not otherwise provided for camouflage F41H3/00
• CPC Class Codes: G05D 1/0088 characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours using knowledge based models G06N5/00
• CPC Class Codes: G05D 1/101 specially adapted for aircraft
• CPC Class Codes: G06N 5/02 Knowledge representation; Symbolic representation
• CPC Class Codes: Y02T 50/40 Weight reduction

Abstract

An aircrew automation system and method for use in an aircraft. The aircrew automation system comprises one or more processors, an optical perception system, an actuation system, and a human-machine interface. The optical perception system monitors, in real-time, one or more cockpit instruments of the aircraft visually to generate flight situation data. The actuation system mechanically engages at least one flight control of the aircraft in response to the one or more flight commands. The human-machine interface provides an interface between a human pilot and the aircrew automation system. The human-machine interface comprises a display device to display a status of the aircraft and the actuation system.

Claims

1. A method of operating an aircraft using an aircrew automation system having one or more processors and an optical perception system, the method comprising:

imaging, via the optical perception system, cockpit instruments of the aircraft to yield, in real-time, image data representing the cockpit instruments;

generating, via the one or more processors, real-time flight situation data based on the image data;

generating, via the one or more processors, one or more flight commands based at least in part on the real-time flight situation data;

communicating the one or more flight commands to a flight control system of the aircraft to control at least one flight control of the aircraft; and

displaying, via a display device operatively coupled with the one or more processors, a status of the aircraft and of the aircrew automation system.

2. The method of claim 1, further comprising the step of displaying, via the display device, at least one of a checklist, a health alert, a predicted aircraft state, and a failure prognosis.

3. The method of claim 1, further comprising the step of receiving, via a touch screen of a human-machine interface, one or more pilot commands from a human pilot to control the aircrew automation system.

4. The method of claim 1, wherein the real-time flight situation data includes airspeed, altitude, and vertical speed of the aircraft.

5. The method of claim 1, further comprising the step of determining, via the one or more processors, aircraft-specific information for the aircraft during a knowledge acquisition phase, wherein the aircraft-specific information includes a layout of cockpit instruments for the aircraft.

6. The method of claim 5, further comprising the step of, during the knowledge acquisition phase, receiving a flight plan from a flight control system of the aircraft, wherein the one or more flight commands are generated based at least in part on the flight plan and the real-time flight situation data.

7. The method of claim 5, wherein the aircraft-specific information further comprises one or more thresholds defined for the aircraft.

8. The method of claim 7, wherein the one or more thresholds is a maximum airspeed or maximum engine temperature.

9. The method of claim 1, further comprising the step of receiving flight data directly from the cockpit instruments via a data link, wherein the real-time flight situation data is based at least in part on the flight data.

10. An aircrew automation system for use in an aircraft, the aircrew automation system comprising:

one or more processors to operatively connect a plurality of systems or subsystems via one or more interfaces;

a perception system operatively coupled with the one or more processors to monitor, in real-time, one or more cockpit instruments of the aircraft visually to generate real-time flight situation data,

wherein the one or more processors are configured to generate one or more flight commands based at least in part on the real-time flight situation data, and

wherein at least one of the one or more processors is configured to control at least one flight control of the aircraft using the one or more flight commands; and

a human-machine interface operatively coupled with the one or more processors to provide an interface between a human pilot and the aircrew automation system, wherein the human-machine interface comprises a display device to display a status of the aircraft and of the aircrew automation system.

11. The aircrew automation system of claim 10, wherein the human-machine interface is configured to display a plurality of tasks on a touch-screen display in a form of a task list, wherein each of the plurality of tasks is marked as either completed or not completed based at least in part on an input from the human pilot via a touch-screen display.

12. The aircrew automation system of claim 10, wherein the one or more processors are configured to receive and aggregate the real-time flight situation data to determine a current aircraft state for the aircraft.

13. The aircrew automation system of claim 10, further comprising an actuation system configured to mechanically engage the at least one flight control, wherein the actuation system is configured to manipulate the at least one flight control in response to one or more actuation commands.

14. The aircrew automation system of claim 10, wherein the perception system comprises a vision system or an acoustic system, wherein the perception system is configured to read or comprehend flight situation information provided by the cockpit instruments using the vision system or the acoustic system.

15. The aircrew automation system of claim 10, wherein the human-machine interface comprises a microphone and employs speech-recognition techniques to recognize a verbal command from the human pilot.

16. The aircrew automation system of claim 15, wherein the human-machine interface is configured to generate a verbal reply in response to the verbal command from the human pilot.

17. The aircrew automation system of claim 10, further comprising a knowledge acquisition system operatively coupled with the one or more processors to determine aircraft-specific information for the aircraft during a knowledge acquisition phase.

18. The aircrew automation system of claim 10, wherein the one or more processors are configured to identify an anomaly in a behavior of a human pilot or an operation of the aircraft using one or more machine learning techniques, wherein, upon detection of the anomaly, the one or more processors executes a contingency operation to adjust control of the aircraft.

19. The aircrew automation system of claim 10, wherein the one or more processors are configured to identify an anomaly as a function of the real-time flight situation data and to generate an alert upon detection of the anomaly.