Aircrew automation system and method with integrated imaging and force sensing modalities

• US 10,509,415 B2
• Filed: 07/27/2017
• Issued: 12/17/2019
• Est. Priority Date: 07/27/2017
• Status: Active Grant

• Current Assignee: Aurora Flight Sciences Corporation (The Boeing Co.)
• Original Assignee: Aurora Flight Sciences Corporation (The Boeing Co.)
• Inventors: Bosworth, William, Jensen, Devin Richard, Reagan, Margaret
• Primary Examiner(s): Nguyen, Cuong H
• Application Number:
  US15/661,149
• Publication Number:
  US 20190033888A1
• Time in Patent Office: 873 Days
• Field of Search: 701 3, 701 36, 414735, 700245, 700248, 700258, 700260, 901 8
• CPC Class Codes: B25J 11/00 Manipulators not otherwise provided for
• CPC Class Codes: B64C 13/08 Trimming zero positions
• CPC Class Codes: B64C 13/16 actuated automatically, e.g. responsive to gust detectors
• CPC Class Codes: B64C 13/22 readily revertible to personal control
• CPC Class Codes: B64D 43/00 Arrangements or adaptations of instruments arrangements of cameras B64D47/08; aeronautical measuring instruments per se G01C
• CPC Class Codes: G01C 21/38 Electronic maps specially adapted for navigation; Updating thereof
• CPC Class Codes: G01L 5/00 Apparatus for, or methods of, measuring force, work, mechanical power, or torque, specially adapted for specific purposes
• CPC Class Codes: G01L 5/0028 Force sensors associated with force applying means G01L5/0052, G01L5/0057, G01L5/0061 take precedence
• CPC Class Codes: G01L 5/0061 Force sensors associated with industrial machines or actuators for the specific machine or actuator involved see relevant class, e.g. F01, F04, F16, B66, E21
• CPC Class Codes: G05D 1/0088 characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours using knowledge based models G06N5/00
• CPC Class Codes: G05D 1/0808 specially adapted for aircraft
• CPC Class Codes: G05D 1/101 specially adapted for aircraft
• CPC Class Codes: G05D 1/106 Change initiated in response to external conditions, e.g. avoidance of elevated terrain or of no-fly zones
• CPC Class Codes: G06F 3/02 Input arrangements using manually operated switches, e.g. using keyboards or dials
• CPC Class Codes: G06F 3/04817 using icons graphical or visual programming using iconic symbols G06F8/34
• CPC Class Codes: G06F 3/0483 Interaction with page-structured environments, e.g. book metaphor
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06V 20/59 inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions
• CPC Class Codes: G08G 5/065 Navigation or guidance aids, e.g. for taxiing or rolling
• CPC Class Codes: Y02T 50/40 Weight reduction

Abstract

An aircrew automation system relates to the field of flight control systems, methods, and apparatuses; even more particularly, to a system, method, and apparatus for providing aircraft state monitoring and/or an automated aircrew employing a robotic arm with integrated imaging and force sensing modalities.

Claims

1. An aircrew automation system for use in an aircraft, the aircrew automation system comprising:

a computer system having a processor and one or more interfaces to connect a plurality of systems or subsystems, wherein the computer system is communicatively coupled with a flight control system of the aircraft and is configured to generate commands based at least in part on flight situation data received from (1) one or more of the plurality of systems or subsystems or (2) the flight control system; and

an actuation system operatively coupled with the computer system via at least one of the one or more interfaces, wherein the actuation system comprises;

an optical sensor configured to visually identify one or more cockpit instruments corresponding to the flight control system;

an actuator configured to engage the one or more cockpit instruments;

a force sensor operably coupled to the actuator and configured to measure a force when the actuator makes contact with the one or more cockpit instruments; and

a controller operably coupled with the optical sensor, the actuator, and the force sensor, wherein the controller is configured (1) to map a location, type, and position of the one or more cockpit instruments as a function of data collected by the optical sensor and the force sensor, (2) to execute the commands received from the computer system via the actuator, and (3) to communicate data collected by the optical sensor or the force sensor to the computer system.

2. The aircrew automation system of claim 1, wherein the optical sensor is an infrared camera or a visual camera.

3. The aircrew automation system of claim 1, wherein the actuator comprises two or more digits that are individually manipulable.

4. The aircrew automation system of claim 3, wherein each of the two or more digits comprises a respective force sensor.

5. The aircrew automation system of claim 1, wherein the actuator is a robotic arm configured to maneuver in six degrees of freedom within a cockpit environment.

6. The aircrew automation system of claim 5, wherein the optical sensor and the force sensor are collocated at a distal end of the robotic arm, the distal end being configured to make contact with the one or more cockpit instruments.

7. The aircrew automation system of claim 1, further comprising a knowledge acquisition system operatively coupled with the computer system via at least one of the one or more interfaces to determine information specific to the aircraft.

8. The aircrew automation system of claim 7, wherein the knowledge acquisition system is configured to access a matrix of force values corresponding to a force profile of the one or more cockpit instruments.

9. The aircrew automation system of claim 8, wherein the controller is configured to compare measured data from the force sensor to the matrix of force values to identify a cockpit instrument of the one or more cockpit instruments based on a respective force profile.

10. The aircrew automation system of claim 7, wherein the knowledge acquisition system is configured to access a database containing a matrix of image data corresponding to a layout of the one or more cockpit instruments.

11. The aircrew automation system of claim 10, wherein the controller is configured to compare measured data from the optical sensor to the matrix of image data to identify a cockpit instrument of the one or more cockpit instruments based on a location of the cockpit instrument according to the layout.

12. The aircrew automation system of claim 1, further comprising a perception system operatively coupled with the computer system via at least one of the one or more interfaces to monitor the one or more cockpit instruments of the aircraft to generate flight situation data.

13. The aircrew automation system of claim 1, further comprising a human machine interface operatively coupled with the computer system via at least one of the one or more interfaces to provide an interface between a pilot and the aircrew automation system to display data to and accept commands from the pilot.

14. The aircrew automation system of claim 13, wherein the human machine interface includes a touch screen display.

15. The aircrew automation system of claim 1, wherein the actuator is a multi-tool configured to interact with a knob-style cockpit instrument and a switch-style cockpit instrument.

16. A method of implementing an aircrew automation system for use in an aircraft, the method comprising:

receiving a commanded heading from a flight control system of the aircraft to alter aircraft heading;

identifying, via an optical sensor, a knob amongst one or more cockpit instruments that is configured to alter aircraft heading;

activating a robotic arm configured to manipulate the one or more cockpit instruments;

controlling the robotic arm to engage the knob;

determining that a measured force profile for the knob matches a stored force profile value; and

turning the knob to reflect the commanded heading.

17. The method of claim 16, further comprising the step of reading a first heading before contacting the knob.

18. The method of claim 17, further comprising the step of calculating an amount of change required to achieve the commanded heading based on the reading of the first heading and the commanded heading.

19. The method of claim 16, further comprising the steps of:

reading a second heading after turning the knob; and

comparing the second heading to the commanded heading.

20. The method of claim 19, further comprising the steps of:

determining the second heading does not match the commanded heading; and

calculating a second amount of change required to achieve the commanded heading based on the reading of the second heading and the commanded heading.