Natrual voice speech recognition for flight deck applications

• US 9,830,910 B1
• Filed: 09/26/2013
• Issued: 11/28/2017
• Est. Priority Date: 09/26/2013
• Status: Active Grant

• Current Assignee: Rockwell Collins, Inc. (Raytheon Technologies Corporation d/b/a RTX)
• Original Assignee: Rockwell Collins, Inc. (Raytheon Technologies Corporation d/b/a RTX)
• Inventors: Shapiro, Geoffrey A., Leedom, David L., Lorch, Nicholas M.
• Primary Examiner(s): Zhu, Richard
• Application Number:
  US14/038,249
• Time in Patent Office: 1,524 Days
• CPC Class Codes: G08G 5/0013 with a ground station
• CPC Class Codes: G08G 5/0021 located in the aircraft
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command
• CPC Class Codes: G10L 2015/228 of application context

Abstract

A computer system receives a voice command and applies one or more contextual filters produce avionics commands based on the voice command. Elements of the voice command are displayed for acceptance by a pilot before being implemented by an avionics system. Individual elements are reevaluated as necessary. Contextual filters include flight paths, flight phase, aircraft location, current weather conditions or information associated with a particular airport.

Claims

1. A computer system comprising:

a processor;

memory connected to the processor;

a display connected to the processor;

a receiver connected to the processor;

a microphone connected to the processor; and

computer executable program code configured to execute on the processor,

wherein the computer executable program code is configured to;

continuously monitor air-traffic communications frequencies for an aircraft identification code;

receive air traffic controller data corresponding to a voice command received from an air traffic controller through the receiver, associated with the aircraft identification code;

parse the air traffic controller voice data into a plurality of words corresponding to individual command elements;

correlate individual voice command elements based on air traffic controller data and individual voice command elements based on pilot voice data;

receive pilot voice data corresponding to a pilot voice command from the microphone;

parse the pilot voice data into a plurality of words corresponding to individual command elements;

produce a contextual filter based on a pointer location in a graphical user interface, said pointer location associated with a weighted list of potential voice commands, a location of the aircraft, a flight plan, and at least one weather condition;

produce a dynamic contextual filter based on a flight phase selected from a list including taxiing, airborne, and final approach;

apply the contextual filter and dynamic contextual filter to one or more databases of command elements to find a first word of the plurality of words and define a first command element associated with the first word;

add the first command element to the dynamic contextual filter;

associate at least one command element corresponding to an airport identification with at least one predefined descriptor based on a distance from an airport identified by the processor, and available airport facilities;

produce a plurality of avionics commands based on the individual command elements; and

receive one or more confirmations, each of the one or more confirmations corresponding to an avionics command.

2. The computer system of claim 1, wherein the computer executable program code is further configured to:

determine that at least one of the plurality of avionics commands does not accurately correspond to an individual command element; and

receive a corrected avionics command.

3. The computer system of claim 1, wherein displaying the plurality of avionics commands comprises highlighting one or more commands in a predefined color indicating the computer system will take action on the highlighted commands when confirmed.

4. The computer system of claim 1,

wherein the computer executable program code is further configured to:

filter a voice command database based on the contextual filter; and

search for at least one word in the voice command database to produce the one or more avionics commands.

5. The computer system of claim 4,

wherein the contextual filter is based on the airport property.

6. An aircraft computer system comprising:

a processor;

memory connected to the processor;

an antenna connected to the processor;

a microphone connected to the processor; and

computer executable program code configured to execute on the processor,

wherein the computer executable program code is configured to;

continuously monitor air-traffic communications frequencies for an aircraft identification code;

receive air traffic controller data corresponding to a voice command received from an air traffic controller through the antenna, associated with the aircraft identification code;

parse the air traffic controller voice data into a plurality of words corresponding to individual command elements;

correlate individual voice command elements based on air traffic controller data and individual voice command elements based on pilot voice data;

receive pilot voice data corresponding to a pilot voice command through the microphone;

parse the pilot voice data into a plurality of words corresponding to individual command elements;

associate at least one command element corresponding to an airport identification with at least one predefined descriptor based on a distance from an airport identified by the processor, and available airport facilities;

produce a contextual filter based on a pointer location in a graphical user interface, said pointer location associated with a weighted list of potential voice commands, a location of the aircraft, a flight plan, and at least one weather condition;

produce a dynamic contextual filter based on a flight phase selected from a list including taxiing, airborne, and final approach;

filter a voice command database based on the contextual filter and dynamic contextual filter;

search for at least one word in the voice command database; and

add the at least one word to the dynamic contextual filter.

7. The aircraft computer system of claim 6, wherein the contextual filter is based on the airport property.

8. The aircraft computer system of claim 6, wherein the computer executable program code is further configured to:

produce a plurality of avionics commands based on the individual command elements;

display the plurality of avionics commands on the display;

mirror the plurality of avionics commands on a co-pilot display; and

receive one or more confirmations, each of the one or more confirmations corresponding to an avionics command.

9. The aircraft computer system of claim 8, wherein the computer executable program code is further configured to:

determine that at least one of the plurality of avionics commands does not accurately correspond to an individual command element; and

receive a corrected avionics command.

10. The aircraft computer system of claim 8, wherein displaying the plurality of avionics commands comprises highlighting one or more commands in a predefined color indicating the computer system will take action on the highlighted commands when confirmed.

11. A method for producing avionics commands through voice recognition comprising:

continuously monitoring air-traffic communications frequencies for an aircraft identification code;

receiving air traffic controller data corresponding to a voice command received from an air traffic controller;

parsing the air traffic controller voice data into a plurality of words corresponding to individual command elements;

correlating individual voice command elements based on air traffic controller data and individual voice command elements based on pilot voice data;

receiving pilot voice data corresponding to a pilot voice command;

parsing the pilot voice data into a plurality of words corresponding to individual command elements;

associating at least one command element corresponding to an airport identification with at least one predefined descriptor based on a distance from an airport identified by the processor, and available airport facilities;

producing a contextual filter based on a pointer location in a graphical user interface, said pointer location associated with a weighted list of potential voice commands, a location of the aircraft, a flight plan, and at least one weather condition;

producing a dynamic contextual filter based on a flight phase selected from a list including taxiing, airborne, and final approach;

filtering a voice command database based on the contextual filter and dynamic contextual filter;

searching for at least one word in the voice command database; and

adding the at least one word to the dynamic contextual filter.

12. The method of claim 11, wherein the contextual filter is based on at least one of an aircraft location, a weather event and an aircraft status.

13. The method of claim 11, further comprising:

producing a plurality of avionics commands based on the individual command elements;

displaying the plurality of avionics commands;

mirroring the plurality of avionics commands on a co-pilot display; and

receiving one or more confirmations, each of the one or more confirmations corresponding to an avionics command.

14. The method of claim 13, further comprising:

determining that at least one of the plurality of avionics commands does not accurately correspond to an individual command element; and

receiving a corrected avionics command.

15. The method of claim 13, wherein displaying the plurality of avionics commands comprises highlighting one or more commands in a predefined color indicating the computer system will take action on the highlighted commands when confirmed.