AIRCRAFT SYSTEMS AND METHODS FOR RETROSPECTIVE PILOT INPUT ANALYSIS

• US 20200134940A1
• Filed: 10/31/2018
• Published: 04/30/2020
• Est. Priority Date: 10/31/2018
• Status: Active Grant

• Current Assignee: Honeywell International Inc.
• Original Assignee: Honeywell International Inc.
• Inventors: Gowda, Sushma
• Granted Patent: US 10,803,684 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 3/04817 using icons graphical or visual programming using iconic symbols G06F8/34
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/04847 Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 3/04886 by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus
• CPC Class Codes: G07C 5/0825 using optical means
• CPC Class Codes: G07C 5/085 using electronic data carriers

Abstract

Aircraft systems and methods are provided for capturing user inputs during operation. An aircraft system includes a user interface and a flight data recorder coupled to the user interface. The user interface includes a user input device to receive a user input and a control module coupled to the user input device to capture data associated with the user input and output the data. The flight data recorder receives the data associated with the user input and stores the data associated with the user input device in a time ordered sequence.

Claims

1. An aircraft system comprising:

a user interface comprising a user input device to receive a plurality of user inputs and a control module coupled to the user input device to capture data associated with each of the plurality of user inputs and output the data, wherein the data includes, for each user input of the plurality of user inputs, a respective coordinate location defining a position of the respective user input with respect to a display device and identification of a respective graphical user interface display presented on the display device contemporaneous to the respective user input; and

a flight data recorder coupled to the user interface to receive the data associated with the plurality of user inputs and persistently store, in a data storage element of the flight data recorder, the data associated with the user input device in a time ordered sequence for temporal reconstruction of the plurality of user inputs in the same order in which the plurality of user inputs were received, wherein the time ordered sequence comprises, for each user input of the plurality of user inputs, a respective association between the respective coordinate location of the respective user input, the respective graphical user interface display presented on the display device contemporaneous to the respective user input, and a respective timestamp assigned to the respective user input.

2. The aircraft system of claim 1, wherein:

the user input device comprises a touchscreen;

the plurality of user inputs comprise a plurality of tactile user inputs; and

the respective coordinate location comprises a respective coordinate location of a respective tactile user input of the plurality of tactile user inputs on the touchscreen.

4. The aircraft system of claim 1, wherein the respective coordinate location comprises the respective coordinate location of a pointer or a cursor on the display device.

8. The aircraft system of claim 1, wherein:

the user interface includes a display device having a checklist graphical user interface (GUI) display presented thereon; and

the data identifies user selections within the checklist GUI display.

23. The method of claim 1, further comprising:

capturing, by the data recording system, flight data from an avionics system; and

providing a timeline graphical user interface (GUI) display comprising first graphical indicia of the plurality of user inputs at their corresponding times with second graphical indicia at corresponding times where the flight data changed.

24. The method of claim 1, further comprising:

capturing, by the data recording system, flight data from an avionics system; and

providing a timeline graphical user interface (GUI) display depicting the plurality of user inputs temporally relative to changes in the flight data.

25. The method of claim 1, further comprising:

capturing, by the data recording system, flight data from an avionics system; and

providing a graphical user interface (GUI) display comprising first graphical indicia of the plurality of user inputs at their corresponding times with second graphical indicia of the flight data, wherein each respective user input is presented on the GUI display with respect to at least one of a contemporaneous speed, an altitude, and a distance-to-go temporally associated with the respective user input concurrently to presenting the respective user input.

3. (canceled)

5-7. -7. (canceled)

9-10. -10. (canceled)

11. A method of capturing user input data pertaining to operation of a vehicle for persistent storage, the method comprising:

capturing, by a data recording system onboard the vehicle, user input spatial data associated with a plurality of user inputs on a display device, wherein for each user input of the plurality of user inputs, the user input spatial data associated with the respective user input includes a coordinate location defining a position where the respective user input was received with respect to the display device;

capturing, by the data recording system, user input metadata associated with the plurality of user inputs, wherein for each user input of the plurality of user inputs, the user input metadata associated with the respective user input identifies a graphical user interface display presented on the display device contemporaneous to the respective user input; and

for each user input of the plurality of user inputs, maintaining, by the data recording system, an association between the user input spatial data associated with the respective user input, the user input metadata associated with the respective user input, and a respective timestamp assigned to the association in a data storage element in a time ordered sequence to allow temporal reconstruction of the plurality of user inputs in the same order in which the plurality of user inputs were received.

14. The method of claim 11, further comprising:

obtaining, by a control module coupled to a user input device associated with the respective user input of the plurality of user inputs, the user input spatial data from the user input device; and

identifying, by the control module coupled to the display device, the user input metadata characterizing a state of the display device contemporaneous to the respective user input of the plurality of user inputs, wherein the control module is coupled to the data recording system to provide the user input spatial data and the user input metadata to the data recording system.

15. The method of claim 14, further comprising identifying, by the control module, the respective timestamp corresponding to the respective user input of the plurality of user inputs.

16. The method of claim 14, wherein identifying the state of the display device comprises identifying a graphical user interface element selected by the user input.

17. The method of claim 14, further comprising identifying, by the control module, a type of input device associated with the user input device, wherein the user input metadata comprises identification of the type of input device associated with the respective user input.

21. The method of claim 11, wherein for one or more of the plurality of user inputs, the user input metadata includes indication of a graphical user interface (GUI) element presented on the display device that was selected by the respective user input of the one or more of the plurality of user inputs.

22. The method of claim 11, wherein for one or more of the plurality of user inputs, the user input metadata includes a numerical value associated with the respective user input of the one or more of the plurality of user inputs.

26. The method of claim 11, wherein for one or more of the plurality of user inputs, the user input metadata includes an attribute associated with a graphical user interface element selected by the respective user input of the one or more of the plurality of user inputs.

27. The method of claim 11, wherein for one or more of the plurality of user inputs, the user input metadata characterizes an actionable event resulting from selection of a graphical user interface element by the respective user input of the one or more of the plurality of user inputs.

28. The method of claim 27, wherein the actionable event comprises a command or request generated for an avionics system.

12-13. -13. (canceled)

18. An aircraft system comprising:

a touchscreen interface comprising a display device, a touch panel to receive a plurality of tactile user inputs, and a control module coupled to the touch panel and the display device to capture data associated with the plurality of tactile user inputs, wherein for each tactile user input of the plurality of tactile user inputs, the data includes a respective coordinate location defining a position where the respective tactile user input was received on the touch panel and identification of a respective graphical user interface display presented on the display device contemporaneous to the respective tactile user input; and

a flight data recorder coupled to the touchscreen interface and including a data storage element to persistently store the data associated with the plurality of tactile user inputs received from the touchscreen interface in a time ordered sequence to allow temporal reconstruction of the plurality of tactile user inputs in the same order in which the plurality of tactile user inputs were received, wherein the time ordered sequence comprises, for each tactile user input of the plurality of tactile user inputs, a respective association between the respective coordinate location of the respective tactile user input, the respective graphical user interface display presented on the display device contemporaneous to the respective tactile user input, and a respective timestamp assigned to the respective tactile user input.

19. The aircraft system of claim 18, wherein the data further comprises, for each tactile user input of the plurality of tactile user inputs, an identification of a respective graphical user interface element presented on the display device at the respective coordinate location where the respective tactile user input was received.

20. (canceled)