Apparatuses and methods for training and operating speech recognition systems

• US 5,850,627 A
• Filed: 06/26/1997
• Issued: 12/15/1998
• Est. Priority Date: 11/13/1992
• Status: Expired due to Term

• Current Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Original Assignee: Dragon Systems, Inc. (Microsoft Corporation)
• Inventors: Baker, James K., Heitman, Peter S., Gould, Joel M., McGrath, Frank J., Sturtevant, Dean G., Steele, Elizabeth E., Parke, Joel W., Roberts, Jed M., Squires, Steven D.
• Primary Examiner(s): Hudspeth, David R.
• Assistant Examiner(s): ZINTEL, HAROLD ALBERT
• Application Number:
  US08/883,297
• Time in Patent Office: 537 Days
• Field of Search: 704/231, 704/251, 704/255, 704/256
• US Class Current: 704/231
• CPC Class Codes: G09B 19/04 Speaking with audible presentation of the material to be studied G09B5/04
• CPC Class Codes: G10L 15/063 Training
• CPC Class Codes: G10L 15/18 using natural language modelling
• CPC Class Codes: G10L 15/1815 Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/285 Memory allocation or algorithm optimisation to reduce hardware requirements
• CPC Class Codes: G10L 2015/0631 Creating reference templates; Clustering
• CPC Class Codes: G10L 2015/0638 Interactive procedures
• CPC Class Codes: G10L 2015/228 of application context

Abstract

A word recognition system can: respond to the input of a character string from a user by limiting the words it will recognize to words having a related, but not necessarily the same, string; score signals generated after a user has been prompted to generate a given word against words other than the prompted word to determine if the signal should be used to train the prompted word; vary the number of signals a user is prompted to generate to train a given word as a function of how well the training signals score against each other or prior models for the prompted word; create a new acoustic model of a phrase by concatenating prior acoustic models of the words in the phrase; obtain information from another program running on the same computer, such as its commands or the context of text being entered into it, and use that information to vary which words it can recognize; determine which program unit, such as an application program or dialog box, currently has input focus on its computer and create a vocabulary state associated with that program unit into which vocabulary words which will be made active when that program group has the focus can be put; detect the available computational resources and alter the instructions it executes in response; test if its ability to respond to voice input has been shut off without user confirmation, and, if so, turn that ability back on and prompt the user to confirm if that ability is to be turned off; store both a first and a second set of models for individual vocabulary words and enable a user to selectively cause the recognizer to disregard the second set of models for a selected word; and/or score a signal representing a given word against models for that word from different word model sets to select which model should be used for future recognition.

Claims

1. A computerized method of training pattern information used by a word recognition system to identify the word signals which represent individual vocabulary words, said method comprising the steps of:

storing a plurality of vocabulary words, including storing pattern information for identifying the word signals which represent each such vocabulary word;

prompting the user to generate a word signal representing a given vocabulary word;

receiving a post-prompt word signal generated after the user was prompted to generate the prompted word;

scoring the match of the post-prompt word signal against each word in an active vocabulary including one or more of said stored plurality of vocabulary words other than said prompted word;

responding, when said scoring of the post-prompt word signal against one or more of said active vocabulary words other than said prompted word is better than a certain measure, by reducing the use of the post-prompt word signal to train pattern information associated with the prompted word; and

responding, when said scoring of the post-prompt word signal against said active vocabulary words other than the prompted word which is worse than a certain measure, by increasing the use of the post-prompt word signal to train pattern information associated with the prompted word.

2. A computerized method as in claim 1 wherein:

said storing of a plurality of vocabulary words includes storing a plurality of command vocabulary words, each of which has a function associated with it;

said active vocabulary against which said word signal is scored includes one or more of said command vocabulary words; and

said responding to said scoring which is better than a certain level includes responding to a score of said post-prompt word signal against one of said command vocabulary words which is better than a certain measure by causing the function associated with that command vocabulary word to be executed.

3. A computerized method as in claim 2 wherein;

said receiving of a post-prompt word signal includes receiving an acoustic word signal representing the sound of a spoken utterance;

said pattern information stored for each vocabulary word includes information relating to the sounds of that word'"'"'s utterance; and

said scoring of the post-prompt word signal is performed upon the post-prompt acoustic word signal.

4. A computerized method as in claim 2 further including the step of responding to input from a user indicating that a command vocabulary word is to be added to or removed from the active vocabulary against which the post-prompt word signal is scored by adding or removing that command vocabulary word from that active vocabulary.

5. A computerized method as in claim 2 further including the step of responding, when input from a user indicating that no commands are to be active during training, by preventing said scoring of the post-prompt word signal against any command vocabulary word from

causing the function associated with the command vocabulary word from being executed;

and

preventing the use of the post-prompt word signal to train pattern information associated with the prompted word.

6. A computerized method as in claim 2:

wherein said prompting includes prompting the user to generate a word signal for a given command vocabulary word; and

further including the step of preventing said scoring of the post-prompt word signal against the prompted command vocabulary word from;

causing the function associated with the command vocabulary word from being executed; and

stopping the use of the post-prompt word signal to train pattern information associated with the prompted command vocabulary word.

7. A computerized method for training pattern information used by a word recognition system to identify which word signals generated by one or more users correspond to which vocabulary words, said method comprising the steps of:

prompting a user to generate a number of word signals representing a given word;

scoring the match of one or more post-prompt word signals, that is, word signals generated after the user is prompted to generate the prompted word, against pattern information associated with the prompted word to produce a score for each such match;

controlling the number of word signals the user is prompted to generate for the prompted word as a function of the scores produced by said scoring; and

incorporating information from the prompted word'"'"'s post-prompt word signals into pattern information associated with the prompted word.

8. A computerized method as in claim 7, wherein

said pattern information against which each post-prompt word signals is scored includes information derived from the other post-prompt word signals, so said scoring scores the closeness of the match of said post-prompt word signals against each other;

and

said controlling of the number of word signal the user is prompted to generate includes responding, when said scoring indicates that post-prompt word signals do not match well against each other, by increasing the number of such word signals the user is prompted to generate.

9. A computerized method as in claim 7, wherein

said scoring scores the match of said post-prompt word signals against pattern information, existing prior to the generation of said post-prompt word signals, for identifying which word signals correspond to the prompted word;

and

said controlling of the number of word signal the user is prompted to generate includes responding, when said scoring indicates that post-prompt word signals do not score well against said prior pattern information, by increasing the number of word signals the user is prompted to generate.

10. A computerized word recognition system comprising;

means for receiving user generated word signals representing words to be recognized;

means for storing a plurality of vocabulary words, including storing, for each such word, pattern information for identifying word signals which represent that word;

recognition means for performing pattern matching between word signals and said pattern information to produce scores indicating the relative probabilities that a given word signal corresponds to different ones of said vocabulary words; and

means for training the pattern information associated with vocabulary words to better represent the word signals generated for their respective words under given circumstances, including;

means for repeatedly prompting a user to generate a word signal representing a vocabulary word whose associated pattern information is to be trained;

means for causing the recognition means to pattern match each post-prompt word signal, that is, each word signal generated after a user has been prompted to generate a word signal for the prompted word, against prior pattern information associated with the prompted word and to produce a score for each such match;

repeat control means for causing the means for repeatedly prompting to stop repeatedly prompting the user to generate a word signal of the prompted word once a scoring requirement has been meet, that is, once one or more post-prompt word signals received for the prompted word have scored better than a certain measure against the pattern information for the prompted word; and

means for incorporating information from the post-prompt word for the prompted word into the pattern information associated with the prompted word.

11. A computerized word recognition system as in claim 10 wherein:

said means for receiving user generated word signals receives acoustic word signals representing the sounds of spoken words;

said pattern information stored for each vocabulary word includes information relating to the sounds of that word'"'"'s utterance;

said recognition means performs pattern matching upon such acoustic word signals; and

said means for repeatedly prompting prompts the user to say the prompted word.

12. A computerized word recognition system as in claim 10 wherein said scoring requirement is a requirement that a specified number of word signals must each receive better than a given score against the pattern information for the prompted word.

13. A computerized word recognition system as in claim 10 further including means for enabling a user to selectively change said scoring requirement.

14. A computerized word recognition system as in claim 10 further including limiting means for causing the means for repeatedly prompting to stop repeatedly prompting the user to generate a word signal of the prompted word at some point even if the scoring requirement has not been met.

15. A computerized word recognition system as in claim 14 wherein said limiting means includes:

means for causing the means for repeatedly prompting to stop repeatedly prompting the user after a given number of post-prompt word signals have been received in response to a given prompted word; and

means for enabling a user to select said given number.

16. A computerized word recognition system as in claim 10 wherein:

said means for storing a plurality of vocabulary words includes means for storing initial pattern information for each of a plurality of vocabulary words which can be used by said recognition means to recognize word signals generated by a given user without requiring that the user first train it; and

said means for causing the recognition means to pattern match each post-prompt word signal includes means for performing that pattern match against said initial pattern information for the prompted word.

17. A computerized word recognition system as in claim 16 wherein said means for incorporating information includes means for combining information from the post-prompt word signals generated for the prompted word with said initial pattern information of that word to form combined pattern information for the prompted word against which said recognition means can perform said pattern matching.