Speech recognition system which selects one of a plurality of vocabulary models

• US 6,073,097 A
• Filed: 06/26/1997
• Issued: 06/06/2000
• Est. Priority Date: 11/13/1992
• Status: Expired due to Term

• Current Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Original Assignee: Dragon Systems, Inc. (Microsoft Corporation)
• Inventors: Baker, James K., Gould, Joel M., Roberts, Jed M.
• Primary Examiner(s): Dorvil, Richemond
• Assistant Examiner(s): Chawan, Vijay B.
• Application Number:
  US08/882,811
• Time in Patent Office: 1,076 Days
• Field of Search: 704/251, 704/256, 704/255, 704/245, 704/236, 704/238, 704/257, 704/240
• US Class Current: 704/251
• CPC Class Codes: G09B 19/04 Speaking with audible presentation of the material to be studied G09B5/04
• CPC Class Codes: G10L 15/063 Training
• CPC Class Codes: G10L 15/18 using natural language modelling
• CPC Class Codes: G10L 15/1815 Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/285 Memory allocation or algorithm optimisation to reduce hardware requirements
• CPC Class Codes: G10L 2015/0631 Creating reference templates; Clustering
• CPC Class Codes: G10L 2015/0638 Interactive procedures
• CPC Class Codes: G10L 2015/228 of application context

Abstract

A word recognition system can: respond to the input of a character string from a user by limiting the words it will recognize to words having a related, but not necessarily the same, string; score signals generated after a user has been prompted to generate a given word against words other than the prompted word to determine if the signal should be used to train the prompted word; vary the number of signals a user is prompted to generate to train a given word as a function of how well the training signals score against each other or prior models for the prompted word; create a new acoustic model of a phrase by concatenating prior acoustic models of the words in the phrase; obtain information from another program running on the same computer, such as its commands or the context of text being entered into it, and use that information to vary which words it can recognize; determine which program unit, such as an application program or dialog box, currently has input focus on its computer and create a vocabulary state associated with that program unit into which vocabulary words which will be made active when that program group has the focus can be put; detect the available computational resources and alter the instructions it executes in response; test if its ability to respond to voice input has been shut off without user confirmation, and, if so, turn that ability back on and prompt the user to confirm if that ability is to be turned off; store both a first and a second set of models for individual vocabulary words and enable a user to selectively cause the recognizer to disregard the second set of models for a selected word; and/or score a signal representing a given word against models for that word from different word model sets to select which model should be used for future recognition.

Claims

1. A computerized word recognition system comprising:

means for receiving user generated word signals representing words;

means for defining a plurality of vocabulary words, including means for defining a plurality of word model sets, each of which defines a word model for each of said vocabulary words;

recognition means for pattern matching a word signal against a plurality of said word models to produce a recognition score for each such word model indicating the likelihood that said word model corresponds to said word signal; and

word model set selection means including;

set scoring means for causing said recognition means to pattern match each of one or more word signals against word models for one or more of said vocabulary words from each of said plurality of word model sets;

set selecting means for selecting a word model set whose one or more word models have the best recognition score against said one or more word signals; and

set favoring means for increasing the likelihood that said recognition means will expend more computation performing pattern matching against said selected word model set than against any other of said plurality of word model sets.

2. A computerized word recognition system as in claim 1 wherein said set scoring means includes means for producing a relative recognition score for the pattern matching of a given word signal against the word model representing a given vocabulary word from each of said plurality of word model sets, so each of said relative recognition scores indicates the relative likelihood that its associated word model corresponds to said given word signal.

3. A computerized word recognition system as in claim 2 wherein:

said recognition means performs a given pattern matching process by matching a given word signal against each of a group of active word models, and produces relative recognition scores for different word models in said active group indicating the relative likelihood that said different word models correspond to said word signal; and

said set scoring means includes means for causing word models from each of said plurality of model sets to be part of said active group during a given pattern matching process.

4. A computerized word recognition system as in claim 2 wherein:

said word signals are acoustic signals of a person uttering a spoken word;

each of said word models is an acoustic word model which represents the acoustic word signals associated with utterances of a given word; and

said recognition means is a speech recognition means.

5. A computerized word recognition system as in claim 4 wherein said different word model sets are designed to represent how different types of speakers speak said vocabulary words.

6. A computerized word recognition system as in claim 2 wherein:

said set scoring means includes means for causing said recognition means to pattern match each of a plurality of word signals against word models for each of a corresponding plurality of said vocabulary words from each of said plurality of word model sets; and

said set selecting means selects the word model set whose word models for a plurality of said vocabulary words have the best relative recognition score against said plurality of word signals.

7. A computerized word recognition system as in claim 6 wherein said set scoring means includes means for successively prompting the user to generate each of a plurality of vocabulary words and for pattern matching the word signals generated in response to each such prompt against word models from each word model set for the prompted vocabulary word.

8. A computerized word recognition system as in claim 2 wherein said set favoring means includes means for causing said recognition means to perform no pattern matching against word models from any of said plurality of word model sets other than said selected word model set.

9. A computerized word recognition system as in claim 2 wherein said means for defining a plurality of word model sets includes:

means for associating with each of said vocabulary words a spelling of shared word component symbols, each of which symbols represents a shared word component, that is, a signal pattern which occurs as part of the word signals of more than one word; and

means for associating with each of said word model sets a corresponding shared component model set, each of which shared component model sets includes a shared component model for each symbol used in said spellings, which represents the signal patterns associated with its associated shared word component.

10. A computerized word recognition system as in claim 9 wherein:

said recognition means includes means for defining all spelled word models, that is all word models defined by spellings of shared word component symbols, used by the recognizer in a match against a given word signal by interpreting the shared word component symbols of those word models by reference to a single active shared component model set;

said favoring means causes said shared component model set corresponding to the selected word model set to be made said active shared component model set after said selecting means has selected a said selected word model set; and

said set scoring means includes;

means for storing a plurality of non-spelled word models, one for each spelled word model for a given vocabulary word defined by said different shared word component model sets, with each such non-spelled word model having a sequence of non-shared component models corresponding to the shared component models of its corresponding shared word component model set defined by the spelling of said given vocabulary word; and

means for causing said recognition means to perform said pattern matching against said non-spelled models for purposes of producing said relative recognition scores.

11. A computerized word recognition system as in claim 1:

wherein said word model set selection means includes means for enabling a given user of said system to cause said word model set selection means to select a set of word models in response to word signals generated by said given user; and

further including means for performing adaptive training on word models of said selected word model set in response to information contained in word signals generated by said given user.

12. A computer program recorded in machine readable memory for performing word recognition comprising:

instructions for receiving user generated word signals representing words;

instructions for defining a plurality of vocabulary words, including instructions for defining a plurality of word model sets, each of which defines a word model for each of said vocabulary words;

recognition instructions for pattern matching a word signal against a plurality of said word models to produce a recognition score for each such word model indicating the likelihood that said word model corresponds to said word signal; and

word model set selection instructions including;

set scoring instructions for causing said recognition instructions to pattern match each of one or more word signals against word models for one or more of said vocabulary words from each of said plurality of word model sets;

set selecting instructions for selecting a word model set whose one or more word models have the best recognition score against said one or more word signals; and

set favoring instructions for increasing the likelihood that said recognition instructions will expend more computation performing pattern matching against said selected word model set than against any other of said plurality of word model sets.

13. A computer program as in claim 12 wherein said set scoring instructions include instructions for producing a relative recognition score for the pattern matching of a given word signal against the word model representing a given vocabulary word from each of said plurality of word model sets, so each of said relative recognition scores indicates the relative likelihood that its associated word model corresponds to said given word signal.

14. A computer program as in claim 13 wherein:

said word signals are acoustic signals of a person uttering a spoken word;

each of said word models is an acoustic word model which represents the acoustic word signals associated with utterances of a given word; and

said recognition instructions are speech recognition instructions.

15. A computer program as in claim 14 wherein said different word model sets are designed to represent how different types of speakers speak said vocabulary words.

16. A computer program as in claim 13 wherein:

said set scoring instructions include instructions for causing said recognition instructions to pattern match each of a plurality of word signals against word models for each of a corresponding plurality of said vocabulary words from each of said plurality of word model sets; and

said set selecting instructions select the word model set whose word models for a plurality of said vocabulary words have the best relative recognition score against said plurality of word signals.

17. A computer program as in claim 16 wherein said set scoring instructions include instructions for successively prompting the user to generate each of a plurality of vocabulary words and for pattern matching the word signals generated in response to each such prompt against word models from each word model set for the prompted vocabulary word.

18. A computer program as in claim 13 wherein said instructions for defining a plurality of word model sets include:

instructions for associating with each of said vocabulary words a spelling of shared word component symbols, each of which symbols represents a shared word component, that is, a signal pattern which occurs as part of the word signals of more than one word; and

instructions for associating with each of said word model sets a corresponding shared component model set, each of which shared component model sets includes a shared component model for each symbol used in said spellings, which represents the signal patterns associated with its associated shared word component.

19. A computer program as in claim 12:

wherein said word model set selection instructions include instructions for enabling a given user of said system to cause said word model set selection instructions to select a set of word models in response to word signals generated by said user; and

further including instructions for performing adaptive training on word models of said selected word model set in response to information contained in word signals generated by said given user.

20. A computerized method for performing word recognition comprising the steps of:

receiving user generated word signals representing words;

defining a plurality of vocabulary words, including defining a plurality of word model sets, each of which defines a word model for each of said vocabulary words;

pattern matching of a word signal against a plurality of said word models to produce a recognition score for each such word model indicating the likelihood that said word model corresponds to said word signal; and

word model set selection including the steps of;

causing said pattern matching to be performed on each of one or more word signals against word models for one or more of said vocabulary words from each of said plurality of word model sets;

selecting a word model set whose one or more word models have the best recognition score against said one or more word signals; and

increasing the likelihood that more computation will be expended performing pattern matching against said selected word model set than against any other of said plurality of word model sets.

21. A computerized method as in claim 20 wherein said step of causing pattern matching to be performed on word signals against word models from each of said word model sets include producing a relative recognition score for the pattern matching of a given word signal against the word model representing a given vocabulary word from each of said plurality of word model sets, so each of said relative recognition scores indicates the relative likelihood that its associated word model corresponds to said given word signal.

22. A computerized method as in claim 21 wherein:

said word signals are acoustic signals of a person uttering a spoken word;

each of said word models is an acoustic word model which represents the acoustic word signals associated with utterances of a given word; and

said pattern matching performs speech recognition.

23. A computerized method as in claim 22 wherein said different word model sets are designed to represent how different types of speakers speak said vocabulary words.

24. A computerized method as in claim 21 wherein:

said word model set selection includes performing said pattern matching for each of a plurality of word signals against word models for each of a corresponding plurality of said vocabulary words from each of said plurality of word model sets; and

said selecting of a word model set selects the word model set whose word models for a plurality of said vocabulary words have the best relative recognition score against said plurality of word signals.

25. A computerized method as in claim 24 wherein said word model set selection includes the step of successively prompting the user to generate each of a plurality of vocabulary words and for pattern matching the word signals generated in response to each such prompt against word models from each word model set for the prompted vocabulary word.

26. A computerized method as in claim 21 wherein said step of defining a plurality of word model sets includes:

the step of associating with each of said vocabulary words a spelling of shared word component symbols, each of which symbols represents a shared word component, that is, a signal pattern which occurs as part of the word signals of more than one word; and

the step of associating with each of said word model sets a corresponding shared component model set, each of which shared component model sets includes a shared component model for each symbol used in said spellings, which represents the signal patterns associated with its associated shared word component.

27. A computerized method as in claim 21:

wherein said step of word model set selection includes the step of enabling a given user of said system to cause said word model set selection to select a set of word models in response to word signals generated by said user; and

further including the step of performing adaptive training on word models of said selected word model set in response to information contained in word signals generated by said given user.