Real-time reconfigurable adaptive speech recognition command and control apparatus and method

• US 5,774,841 A
• Filed: 09/20/1995
• Issued: 06/30/1998
• Est. Priority Date: 09/20/1995
• Status: Expired due to Fees

• Current Assignee: The Government of The US As Represented By The Secretary of The Navy
• Original Assignee: The United States of America As Represented By The Secretary of Agriculture
• Inventors: Salazar, George A., Sommers, Marc J., Haynes, Dena S.
• Primary Examiner(s): Hudspeth, David R.
• Assistant Examiner(s): Collins, Alphonso A.
• Application Number:
  US08/536,302
• Time in Patent Office: 1,014 Days
• Field of Search: 395/2.34, 395/2.42, 395/2.48, 395/2.49, 395/2.45, 395/2.53, 395/2.6, 395/2.63, 395/2.65
• US Class Current: 704/225
• CPC Class Codes: G10L 15/06 Creation of reference templates; Training of speech recognition systems, e.g. adaptation to the characteristics of the speaker's voice G10L15/14 takes precedence
• CPC Class Codes: G10L 15/063 Training
• CPC Class Codes: G10L 15/065 Adaptation
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 2015/0631 Creating reference templates; Clustering
• CPC Class Codes: G10L 2015/0635 updating or merging of old and new templates; Mean values; Weighting
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command

Abstract

An adaptive speech recognition and control system and method for controlling various mechanisms and systems in response to spoken instructions and in which spoken commands are effective to direct the system into appropriate memory nodes, and to respective appropriate memory templates corresponding to the voiced command. Spoken commands from any of a group of operators for which the system is trained may be identified, and voice templates are updated as required in response to changes in pronunciation and voice characteristics over time of any of the operators for which the system is trained. Provisions are made for both near-real-time retraining of the system with respect to individual terms which are determined not be positively identified, and for an overall system training and updating process in which recognition of each command and vocabulary term is checked, and in which the memory templates are retrained if necessary for respective commands or vocabulary terms with respect to an operator currently using the system. In one embodiment, the system includes input circuitry connected to a microphone and including signal processing and control sections for sensing the level of vocabulary recognition over a given period and, if recognition performance falls below a given level, processing audio-derived signals for enhancing recognition performance of the system.

Claims

1. An adaptive speech recognition and control system, comprising:

a speech recognition circuit section including memory means for storing a plurality of vocabulary subsets, each of the plurality of vocabulary subsets identified by a node, each subset comprising a plurality of voice templates;

a microphone;

analog circuitry associated with the microphone for processing voice commands spoken into the microphone and conducting the processed signals to the speech recognition circuitry;

processing means associated with the speech recognition circuitry for causing the speech recognition circuit section to user-selectively transition to a node which opens a memory section storing a selected vocabulary subset, corresponding to a spoken transitional command, in response to a predetermined transitional command being spoken into the microphone by an operator, and for subsequently causing the speech recognition circuitry to search the voice templates stored in the selected subset upon a vocabulary term subsequently being spoken while the speech recognition circuit is transitioned to the node corresponding to the selected vocabulary subset, and for determining, within a degree of certainty, whether the vocabulary term matches one of the voice templates in the selected subset.

2. The system of claim 1, wherein the analog circuitry comprises first and second, serially connected gain adjustment circuits, and wherein the processing means further comprises recognition feedback means for determining whether a plurality of successively processed vocabulary terms have been matched to respective voice templates with an acceptable level of probability or an unacceptable level of probability, and, if an unacceptable level of probability is sensed after a predetermined number of verbal commnands, for varying the operation of at least one of the gain adjustment circuits for modifying the signals fed to the speech recognition circuitry to increase the percentage of recognition within acceptable probability levels of spoken vocabulary terms.

3. The system of claim 2, wherein the first gain adjustment circuit comprises a voltage controlled analog switching circuit, and wherein the second gain adjustment circuit comprises a programmable, digitally controlled gain amplifier, and wherein the processing means comprises means for enabling the first gain adjustment circuit of the analog switching circuit upon an unacceptable level of speech recognition occurring and, if an unacceptable level of speech recognition continues, for digitally adjusting the programmable gain amplifier.

4. The system of claim 2, wherein the processing means comprising means for enabling and adjusting the gain of the first gain adjustment circuit in response to a series of spoken vocabulary terms of low mean volume level but within an acceptable, limited range of volume levels, and alternatively, for adjusting the gain of the second gain adjustment circuit in response to a series of spoken vocabulary terms of acceptable average volume level but of an undesirable wide range of volume levels.

5. The system of claim 1, further comprising means for indicating to the operator the vocabulary term of the voice template selected by the speech recognition circuit section as matching a spoken vocabulary term.

6. The system of claim 5, further comprising audible and visible means for indicating to the operator that the speech recognition circuit section has transitioned to a node, identified to the operator, in response to a transitional command.

7. The system of claim 1, further comprising means for storing selected vocabulary terms as voice templates and for subsequently retraining the voice templates in accordance with the current speech patterns of an operator.

8. The system of claim 7, wherein the means for retraining the voice templates comprises means for successively displaying vocabulary terms stored in the memory for prompting the operator to speak the respective terms and for comparing voice characteristics associated with respective vocabulary terms spoken by the operator, in response to the displayed terms, with the speech patterns stored in the respective sequentially activated voice templates, and for reprogramming respective voice templates upon determining that their stored speech patterns do not correspond, within acceptable limits, with the current voice pattern of the operator as a selected vocabulary term is spoken.

9. The system of claim 7, wherein the speech patterns stored within respective voice templates encompass a range of permissible voice patterns corresponding to voice patterns associated with speech characteristics of a plurality of operators.

10. The system of claim 7, wherein the system is operative, in accordance with the voice training and retraining process, in either of a training mode, or an operative mode, and wherein the system further compromises means for retraining the voice templates during the operative mode in response to a non-recognized vocabulary term spoken during the operative mode.

11. The system of claim 7, wherein the system includes a local RAM memory associated with the speech recognizer circuit and a vocabulary memory, and wherein updating changes are stored in voice templates stored in the local RAM memory, further comprising means for downloading the updated voiceprints to the vocabulary memory, and for downloading the voiceprints stored in the vocabulary memory into the local RAM memory during activation of the system.

12. The system of claim 7, further comprising a commanded system controlled by the voice command system and an output section connected to the system controlled by the voice command system, and wherein, in its operative mode, the speech recognition and control system comprises means for transmitting command signals, corresponding with verbal commands spoken by the operator, from the output section to the controlled system.

13. The system of claim 12, wherein the controlled system comprises a robotic system, and wherein the voice command system comprises means for controlling the robotic system in response to verbal commands of the operator.

14. The system of claim 7, wherein the system includes voiceprint training means for training the voice templates, within a user adjustable threshold of voice recognition, in accordance with spoken terms and, in the event, after an initial cycle of training, the voiceprint still does not match the spoken term, for forcing a retraining change in the voice template.

15. The system of claim 7, wherein the means for retraining the voiceprints comprises means for comparing the history of speech characteristics of an operator with respect to each vocabulary term and, during retraining of the voice template, integrating the newly entered voice characteristics with previously stored voice characteristics.

16. The apparatus of claim 1, further comprising input output circuitry including switching means operable by an operator for controlling the system independently of the voice activated circuit portions.

17. The apparatus of claim 16, wherein the switching means comprises means for node selection, and for permitting independent operator control of the processor node.

18. A speech recognition method comprising the steps of:

storing a plurality of vocabulary subsets in a memory of a speech recognition apparatus, each vocabulary subset comprising a plurality of voice templates, and each vocabulary subset identified by a node;

receiving a speech signal in an analog section of the speech recognition apparatus, the speech signal comprising a command term and a vocabulary term;

digitizing the received speech signal to a digital form;

processing the digitized speech signal, causing the speech recognition apparatus to transition to a node which opens a memory section storing a selected vocabulary subset, corresponding to the command term;

searching the voice templates stored in the selected subset for a match with the vocabulary term; and

determining, within a level of probability, whether the vocabulary term matches one of the voice templates in the selected subset.

19. The method of claim 18, further comprising the steps of:

determining whether a plurality of successively processed vocabulary terms have been matched to respective voice templates with an acceptable level of probability or an unacceptable level of probability, and,

if an unacceptable level of probability is determined after a predetermined number of command terms, modifying the speech signal fed to the speech recognition apparatus to increase the percentage of recognition within acceptable probability levels of spoken vocabulary terms.

20. The method of claim 19, wherein the step of modifying the speech signal fed to the speech recognition apparatus is performed by digitally adjusting a programmable gain amplifier.