Voice activated cockpit management system for flight procedures and control of aircraft systems and flight management systems of single and multi-engine aircraft

• US 8,793,139 B1
• Filed: 11/06/2013
• Issued: 07/29/2014
• Est. Priority Date: 11/06/2013
• Status: Expired due to Fees

• Current Assignee: Nexgen Flight LLC
• Original Assignee: Doinita Diane Serban, Nexgen Flight LLC
• Inventors: Serban, Doinita, McBride, Justin
• Primary Examiner(s): Opsasnick, Michael N
• Application Number:
  US14/072,804
• Time in Patent Office: 265 Days
• Field of Search: 704/275
• US Class Current: 704/275
• CPC Class Codes: G01C 23/00 Combined instruments indicating more than one navigational value, e.g. for aircraft; Combined measuring devices for measuring two or more variables of movement, e.g. distance, speed or acceleration
• CPC Class Codes: G08G 5/0021 located in the aircraft
• CPC Class Codes: G08G 5/0052 for cruising combined instruments indicating more than one navigational value G01C23/00
• CPC Class Codes: G08G 5/0056 in an emergency situation, e.g. hijacking
• CPC Class Codes: G10L 15/20 Speech recognition techniques specially adapted for robustness in adverse environments, e.g. in noise, of stress induced speech G10L21/02 takes precedence
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command
• CPC Class Codes: G10L 2015/228 of application context

Abstract

A voice activated cockpit management system for flight procedures and control of aircraft systems and flight management systems of single and multi-engine aircraft, including a mean to recognize and communicate commands, and to deploy procedures utilizing a NEXT-GEN voice recognition system. A specific feature of this mode of initiation is the automatic communication of flight procedures (normal or emergency procedures) and control procedures for aircraft systems and flight management systems by the utterance of specific key words resulting in the automatic transference of these command words to executable procedure audio-files. The said cockpit management system for providing automated voice activated procedures enables cockpit specific audio procedures through wireless Bluetooth connection or wired communication, and generally includes A Voice Recognition and Audio-Display System, A Mini-PC, A Listening Device, A Microphone, A Power-Source, A Pairing System, Bluetooth Software and Hardware, A Method for Voice Recognition and Audio-Display of Procedures on to pilot.

Claims

1. A voice activated cockpit management system, comprising:

a voice recognition system configured to recognize continuous utterances of unique specific words and to engage a specific command to a specific aircraft operating procedure associated with said utterances;

a Key-to-Recognize utterance embedded into a Key-To-Recognize (KTR) application connected to said Voice Recognition (VR) system, to signal automatic speech recognition software of said Voice Recognition system to begin processing voice commands, wherein said Voice Recognition system comprises;

a) a mini-PC (MPC) operatively connected to a Bluetooth device for accepting commands from said Voice Recognition system initiated by said Key-To-Recognize utterance;

b) a Voice Recognition system operatively connected to said mini-PC comprising automatic speech recognition software for processing a digital signal received from said mini-PC;

c) said Voice Recognition system for sending out data audio files to a Listening Device (LD), and Network Data Objects (NDOs) to a Bluetooth device through Bluetooth or wired communication software and hardware;

d) said Voice Recognition system operatively connected to a decision management system and decision management system application (DMSA) software which interacts with the key-to-recognize (KTR) software application in the voice recognition system to operate a decision tree algorithm and signal automatic speech recognition software to begin processing voice commands and select procedures using the procedure system application software (PSA);

e) said Procedure System Application software (PSA) operatively connected to said Voice Recognition system for rendering and processing said procedure commands displayed by said Listening Device;

said cockpit management system further comprising said Listening Device (LD) for rendering said audio displays based on said data labels;

said cockpit management system comprising said voice recognition system enabling voice activation in the cockpit and rendering of audio displays through wireless Bluetooth connection communication software and hardware or wired connection communication software and hardware;

said cockpit management system comprising said Voice Recognition system configured to communicate commands and deploy automatic flight procedures in the cockpit utilizing voice.

2. The cockpit management system according to claim 1, wherein said Voice Recognition system is configured using performance parameters that communicate commands and deploy automatic flight procedures in the cockpit in an environment with elevated noise levels between 0-136 dB wherein said noise levels are compatible with aircraft operation and whereby said voice recognition system parameters satisfy high accuracy, precision, and robustness of voice recognition at said noise levels.

3. The cockpit management system according to claim 1, wherein said Voice Recognition system is configured to recognize a unique utterance and to encode said specific procedure steps into the use of procedure data files;

said unique utterance for said Voice Recognition system is “

PLAY . . . (procedure type, location and configuration in continuous speech)” and

said voice recognition system encodes this to a full command string of procedure steps in an interactive mode where said step execution by user is confirmed with command “

step complete”

.

4. The cockpit management system according to claim 3, said “

PLAY . . . (procedure type, location and configuration in continuous speech)”

for initiating a signal that an operator is issuing a voice command, wherein if said operator is a pilot, upon initiation of the Key-to-Recognize KTR, the procedure command string is relaying steps associated with each procedure with each step being interactive with operator.

5. The cockpit management system according to claim 1, wherein said Key-To-Recognize (KTR) utterance interacts with a Decision Management System Application (DMSA) of the voice recognition system to signal automatic speech recognition software to begin processing voice commands wherein said Key-To-Recognize KTR initiates automatic communication by the utterance of specific key words resulting in an automatic transference of the commands to executable audio procedures.

6. The cockpit management system according to claim 5, wherein said DMSA interacts with said KTR to operate a decision tree algorithm and signal automatic speech recognition to begin processing and parsing procedure voice commands whereby said DMSA encompasses a decision tree algorithm that enables continuous speech recognition of contiguity limits of up to 150 utterances.

7. Cockpit management system according to claim 6, further comprising a procedure system application (PSA) software and a procedure system data labels library which interacts with said DMSA system to provide search terms without a phonemic transcription.

8. The cockpit management system according to claim 5, wherein said VR system processes said digital signal by dividing a command string of said digital signal into its individual components, said individual components comprising commands, words and audio displays, each said component being routed by way of said NDO to an appropriate hardware device.

9. The cockpit management system according to claim 1, wherein said VR system processes speech recognition in a “

multi-gram”

mode, using partial speech recognition where the expression to recognize is placed at a beginning of a complete expression, a specific number of triphones are firstly captured with respect to time and are employed in recognition training where the remainder of the captured expression parts are trained using synthetic triphones.

10. The cockpit management system according to claim 1, wherein said VR system comprises cockpit grammar and cockpit syntax specific for audio flight commands and procedures of said VR system which are associated with operation of single and multi-engine aircraft.

11. The cockpit management system according to claim 1, wherein said VR system comprises a cockpit vocabulary specific for audio flight commands and procedures of said VR system, which are associated with operation of single and multi-engine aircraft wherein said VR system comprises cockpit synthetic triphones or linkers organized in a library in the vocabulary of said VR system.

12. The cockpit management system according to claim 5, wherein said Mini PC comprises a 700 MHz processor, a 500 MB of memory, a pairing switch, a battery source, a Bluetooth adapter, and a storage device of 4 GB.

13. A method for recognizing a unique continuous utterance in a speech recognition system of a cockpit management system for the purpose of encoding a specific flight procedure (emergency or normal) associated with that utterance, comprising the steps of:

utilizing a voice recognition (VR) system for translating a voice command utterance of unique specific command words to display a specific audio command to a specific step associated with said utterance;

utilizing a voice recognition (VR) subsystem operatively connected to said mini PC comprising automatic speech recognition software and hardware for processing a digital signal received from said mini PC;

utilizing a Key-to-Recognize (KTR) utterance to initiate said VR system, to signal automatic speech recognition software of said VR system to begin processing voice commands;

selecting a working speech recognition vocabulary for recognizing speech from said flight procedures based on a response by a user to an initial Key-to-Recognize utterance;

receiving a spoken word to be recognized;

responding to determining the confirmation of correctly recognizing the said spoken word using said procedure vocabulary by selecting another one of said voice commands of feedback speech recognition vocabularies, based on a response to a subsequent step in the procedure, for use as a said steps of the procedure recognition vocabulary

utilizing key-to-recognize (KTR) software application in the voice recognition subsystem;

utilizing KTR to interact with a decision management system application (DMSA) of the cockpit management system to operate a decision tree algorithm and signal automatic speech recognition software to begin processing voice commands

utilizing a Procedure System Application (PSA) operatively connected to said VR subsystem for rendering and processing said procedure commands displayed by said Listening Device from the mini PC (MPC) for rendering said audio displays based on said data labels;

utilizing said VR subsystem for sending out data labels to a listening device Listening Device, and Network Data Objects (NDOs) to a Bluetooth device;

utilizing a Bluetooth adapter wherein said Bluetooth adapter is operatively connected to said mini PC for managing placement of audio displays.

14. The method of claim 13, wherein said step of translating a voice command utterance of a key command word to a specific procedure further comprises the step of:

utilizing said VR system configured to recognize said unique key command word and to encode said specific procedure into the use of procedure (i.e. emergency) data files.

15. The method of claim 13, further comprising the step of encoding said voice recognition system a unique word “

PLAY . . . (procedure in continuous utterance for example ‘

engine fire during flight’

)”

to a full command string of steps comprising up to 14 steps, “

step complete”

feedback, and “

(emergency) procedure complete”

for procedure completion.

16. The method of claim 13, further comprising the step of:

a) utilizing said Key-To Recognize (KTR) utterance for interacting with a Decision Management System Application (DMSA) to signal an automatic speech recognition software to begin processing voice commands.

17. The method of claim 13, further comprising the step processing said digital signal by dividing a command string of said digital signal into its individual components, said individual components comprising commands, words and audio displays, each component being routed by way of a Network Data Object (NDO) to an appropriate hardware device.

18. The method of claim 13, further comprising the step of communicating digital and audio signal by said VR subsystem through a wireless Bluetooth connection hardware and software or through a wired connection hardware and software.

19. The method of claim 13, further comprising the step of a speech recognition mechanism with recognition for expressions having multiple expression parts, or “

multi-gram”

mode as a mean to communicate voice commands utilizing the said voice recognition system enabled by application software and hardware wherein said “

multi-gram”

mechanism further comprising the step of a novel speech recognition mechanism with the expression being subjected to a partial speech recognition using a modified novel Hidden-Markov-Model whereas the partial “

multi-gram”

expression to recognize is placed at a beginning of a complete expression, and a specific number of triphones are firstly captured with respect to time and are employed in recognition training, while the remainder of the captured expression parts are trained using synthetic triphones or ‘

synthetic linkers’

wherein said mechanism renders said VR speech recognition system a speaker independent speech recognition system optimized for recognition of speech having a range of dialect or accent.

20. A hardware apparatus and cockpit management system for voice activation of flight procedures in single-engine and multi-engine aircraft using voice recognition and audio display comprising:

a mini-PC, wherein said Mini PC comprises a minimum 700 MHz processor, and 500 MB of memory,

a pairing switch,

a battery source,

a Bluetooth adapter,

a storage device having a minimum storage capacity of 4 GB; and

a listening device wherein the hardware apparatus having stored thereon one or more software applications integrated into a voice recognition system wherein each software application comprises code sections executable by a machine for causing said machine to perform the steps of;

providing a single key-to-recognize utterance enabling a single voice command and an activating voice command;

recognizing the enabling voice command specifying a procedure and an action to be performed on said procedure;

storing said action and procedure of said enabling voice command in memory for subsequent use;

performing said procedure from data labels library stored onto the storage device and object of said enabling voice command

Said apparatus further having stored a voice recognition system wherein said speech recognition system enables unique and specific audio flight procedures through wireless Bluetooth communication software and hardware further comprising;

an automatic pairing software application which engenders automatic pairing of the listening device to the voice recognition system repeatedly; and

a pairing system which includes a pairing switch and a pairing software application which enables the one time initial pairing of the listening device to the voice recognition system.