Systems and methods for contextual tagging of data on vehicle display

• US 10,275,427 B2
• Filed: 11/09/2016
• Issued: 04/30/2019
• Est. Priority Date: 11/09/2016
• Status: Active Grant

• Current Assignee: Honeywell (Honeywell International Inc.), Honeywell International Inc.
• Original Assignee: Honeywell International Inc.
• Inventors: Saptharishi, Hariharan, Srinivasan, Narayanan, Chandrashekarappa, Mohan Gowda
• Primary Examiner(s): Badawi, Angie M
• Application Number:
  US15/347,506
• Publication Number:
  US 20180129635A1
• Time in Patent Office: 902 Days
• CPC Class Codes: B64D 43/00 Arrangements or adaptations of instruments arrangements of cameras B64D47/08; aeronautical measuring instruments per se G01C
• CPC Class Codes: G01C 23/00 Combined instruments indicating more than one navigational value, e.g. for aircraft; Combined measuring devices for measuring two or more variables of movement, e.g. distance, speed or acceleration
• CPC Class Codes: G06F 3/04817 using icons graphical or visual programming using iconic symbols G06F8/34
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06F 40/117 Tagging; Marking up details of markup languages G06F40/143; Designating a block; Setting of attributes style sheets, e.g. eXtensible Stylesheet Language Transformation [XSLT], G06F40/154
• CPC Class Codes: G08G 5/0013 with a ground station
• CPC Class Codes: G08G 5/0021 located in the aircraft
• CPC Class Codes: G10L 15/1815 Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning
• CPC Class Codes: G10L 2015/088 Word spotting

Abstract

A system and method for contextual tagging of data on at least one display associated with a vehicle is provided. The method includes: receiving data regarding the vehicle from at least one of a source of verbal data and a source of non-verbal data; determining a context associated with the data and a variable parameter associated with the context; determining a display element associated with the at least one display that corresponds to the determined variable parameter; generating a symbol for the display to display adjacent to the determined display element based on at least one of the determined context and the determined variable parameter; generating a selectable icon for the display to render adjacent to the symbol based on received verbal data; associating the received verbal data to the selectable icon; and outputting the received verbal data to an audio device based on a selection of the selectable icon.

Claims

1. A method for contextual tagging of data on at least one display associated with a vehicle, comprising:

receiving data regarding the vehicle from a source of verbal data;

comparing, by a processor, the data to one or more instruction keywords to determine a context associated with the data and a variable parameter associated with the context, the variable parameter having a value;

querying, by the processor, a listing of display elements currently displayed on the at least one display and determining, by the processor, a display element from the listing of display elements that corresponds to the determined variable parameter;

generating, by the processor, a symbol for the display to display adjacent to the display element that is currently displayed on the at least one display based on at least one of the determined context and the determined variable parameter;

determining, by the processor, a position for the display of the symbol adjacent to the display element that is currently displayed on the at least one display based on the value of the variable parameter;

generating, by the processor, a selectable icon for the display to render adjacent to the symbol based on received verbal data;

associating, by the processor, the received verbal data to the selectable icon; and

outputting the received verbal data to an audio device based on a selection of the selectable icon.

2. The method of claim 1, wherein the source of verbal data is a radio communication system.

3. The method of claim 1, further comprising receiving data regarding the vehicle from a source of non-verbal data, wherein the source of non-verbal data is at least one of a user input device associated with the vehicle or a data link system associated with the vehicle.

4. The method of Claim 3, wherein the comparing, by the processor, the data to one or more instruction keywords further comprises:

identifying, by the processor, one or more keywords in the non-verbal data; and

determining, by the processor, the context and the variable parameter based on the identified keywords.

5. The method of claim 1, wherein the comparing, by the processor, the data to one or more instruction keywords further comprises:

transcribing, by the processor, the data received from the source of verbal data that includes a request for pilot action;

identifying, by the processor, one or more keywords from the transcribed data;

determining, by the processor, the context and the variable parameter based on the identified keywords.

6. The method of claim 1, wherein the vehicle is an aircraft, and the context is a requested pilot action.

7. A contextual tagging system for contextually tagging data on at least one display associated with a vehicle, comprising:

a source of verbal data associated with the vehicle;

a source of display element data that provides a plurality of display elements associated with the at least one display that are currently displayed on the at least one display; and

a processor associated with the vehicle, the processor in communication with the source of verbal data and the source of display element data, and the processor;

receives the verbal data;

compares the verbal data to one or more instruction keywords and determines a context associated with the verbal data;

determines a variable parameter associated with the context, the variable parameter having a value;

queries the source of display element data and determines whether at least one of the plurality of display elements currently displayed on the at least one display is associated with the variable parameter;

generates a symbol for the at least one display to display adjacent to the determined at least one of the plurality of display elements based on the variable parameter;

based on the determination that at least one of the plurality of display elements currently displayed on the at least one display is associated with the variable parameter, determines a position for the display of the symbol adjacent to the at least one of the plurality of display elements based on the value of the variable parameter;

generates a selectable icon for the at least one display to display adjacent to the symbol based on received verbal data;

associates the received verbal data to the selectable icon; and

outputs the received verbal data to an audio device based on a selection of the selectable icon.

8. The contextual tagging system of claim 7, wherein the at least one source of verbal data is a communication system associated with the vehicle.

9. The contextual tagging system of claim 7, further comprising a source of non-verbal data associated with the vehicle, the processor receives the non-verbal data and the processor determines the context associated with the non-verbal data by identifying one or more keywords in the non-verbal communication, and determines the context and the variable parameter based on the identified keywords.

10. The contextual tagging system of Claim 9, wherein the at least one source of non-verbal data is selected from the group comprising a data link system, a user input device and combinations thereof.

11. The contextual tagging system of claim 7, wherein the processor determines the context associated with the verbal data by transcribing the verbal data, the verbal data including a request for pilot action, identifying one or more keywords from the transcribed data and the processor determines the context and the variable parameter based on the identified keywords.

12. The contextual tagging system of claim 7, wherein based on a determination that at least one of the plurality of display elements is unassociated with the determined variable parameter, the processor determines a position for the display of the symbol on the at least one display based on the variable parameter.

13. The contextual tagging system of claim 7, wherein the vehicle is an aircraft, and the context is a requested pilot action.

14. An aircraft, comprising:

a communication system that receives at least a verbal communication;

a display onboard the aircraft having a plurality of display elements associated with the display;

a processor that contextually tags data on the display, the processor in communication with the communication system and the display, and the processor;

receives the verbal communication;

determines a pattern associated with the verbal communication;

compares the pattern to one or more instruction keywords to extract a context and a variable parameter associated with the context, the variable parameter having a value;

queries a listing of display elements currently displayed on the display and determines whether at least one of the plurality of display elements displayed on the display is associated with the variable parameter;

generates a symbol for the display to display adjacent to the determined at least one of the plurality of display elements currently displayed on the display based on at least one of the variable parameter and the context;

based on the determination that at least one of the plurality of display elements currently displayed on the display is associated with the variable parameter, determines a position for the display of the symbol adjacent to the at least one of the plurality of display elements based on the value of the variable parameter;

generates a selectable icon for the display to display adjacent to the symbol based on received verbal data;

associates the received verbal data to the selectable icon; and

outputs the

received verbal data to an audio device based on a selection of the selectable icon.

15. The aircraft of claim 14, wherein the processor determines the context associated with the verbal data by transcribing the verbal communication, the verbal communication including a request for pilot action, identifying one or more keywords from the transcribed data and the processor determines the context and the variable parameter based on the identified keywords.

16. The aircraft of claim 14, wherein based on a determination that at least one of the plurality of display elements is unassociated with the variable parameter, the processor determines a position for the display of the symbol based on the variable parameter.

17. The aircraft of claim 14, wherein the context is a pilot action.