Method and apparatus for dynamic adaptation of a large vocabulary speech recognition system and for use of constraints from a database in a large vocabulary speech recognition system

• US 6,501,833 B2
• Filed: 10/03/1997
• Issued: 12/31/2002
• Est. Priority Date: 05/26/1995
• Status: Expired due to Term

• Current Assignee: SpeechWorks International, Inc. (Microsoft Corporation)
• Original Assignee: SpeechWorks International, Inc. (Microsoft Corporation)
• Inventors: Phillips, Michael S., Nguyen, John N.
• Primary Examiner(s): Hoosain, Allan
• Application Number:
  US08/943,557
• Publication Number:
  US 20020048350A1
• Time in Patent Office: 1,915 Days
• Field of Search: 704/244, 704/252, 704/253, 704/254, 704/255, 704/256, 704/257, 704/1, 704/231, 704/235, 704/243, 704/260, 704/240, 379/88.01, 379/88.02, 379/88.03, 379/88.04, 379/88.05, 379/88.07, 379/67.1, 379/88.08, 379/88.13, 379/218.01, 379/91.01, 379/93.03
• US Class Current: 379/88.07
• CPC Class Codes: G10L 15/06 Creation of reference templates; Training of speech recognition systems, e.g. adaptation to the characteristics of the speaker's voice G10L15/14 takes precedence
• CPC Class Codes: G10L 15/187 Phonemic context, e.g. pronunciation rules, phonotactical constraints or phoneme n-grams
• CPC Class Codes: G10L 2015/228 of application context

Abstract

The lexical network of a large-vocabulary speech recognition system is structured to effectuate the rapid and efficient addition of words to the system'"'"'s active vocabulary. The lexical network is structured to include Phonetic Constraint Nodes, which organize the inter-word phonetic information in the network, and Word Class Nodes which organize the syntactic semantic information in the network. Network fragments, corresponding to phoneme pronunciations and labeled to specify permitted interconnections to each other and to phonetic constraint nodes, are precompiled to facilitate the rapid generation of pronunciations for new words and thereby enhance the rapid addition of words to the vocabulary even during speech recognition. Functions defined in accordance with linguistic constraints may be utilized during recognition. Different language models and different vocabularies for different portions of a discourse may also be invoked depending, in part, on the discourse history.

Claims

1. A speech recognition system for dynamically adding words to an active portion of a total vocabulary, wherein interword relationships between pairs of words of the total vocabulary are defined by a lexical network, the speech recognition system comprising:

a plurality of nodes corresponding to interword relationships between adjacent words, the interword relationships being formed in accordance with a predefined congruence, wherein the nodes comprise;

A. a plurality of phonetic constraint nodes characterized by a phonetic constraint tuple (x, y, . . . ) of order two or greater, where x, y, . . . are phonetic constraints on adjacent words;

B. a plurality of connection nodes, wherein a set of phonetic constraint nodes corresponds to each connection node for identifying adjacent words from the active vocabulary satisfying the phonetic constraints of the set of phonetic constraint nodes; and

C. a plurality of word class nodes each associated with a word class indicative of syntactic or semantic information of at least one word associated with each word class node, and each word class node being associated with a phonetic constraint nodes; and

an apparatus configured to add a word to the active vocabulary portion of the total vocabulary, wherein the word to be added is selectively specified as a phonemic baseform pronunciation or a spelling and wherein the word has at least beginning and ending phonemes and a pre-associated word class, wherein the apparatus is configured to add the word by determining phonetic constraint nodes for the word corresponding to its phonemes and determining interword relationships for the word based on the connection nodes corresponding to the determined phonetic constraint nodes and based on the plurality of word class nodes and the pre-associated word class and by connecting a pronunciation network of the word to be added to at least one of the phonetic constraint nodes and the word class nodes.

2. The speech recognition system according to claim 1 in which said phonetic constraint tuples define at least a first phonetic constraint on a word and at least a second phonetic constraint on an adjacent word in a path through said lexical network.

3. The speech recognition system according to claim 1 wherein connection nodes corresponding to phonetic constraint nodes having corresponding word class nodes identify words from the active vocabulary corresponding to the phonetic constraints of the phonetic constraint nodes and the word classes of the corresponding word class nodes;

and

wherein, the apparatus configured to add the word configured to, if the word to be added to the active vocabulary has at least one word class, determine word class nodes corresponding to the word classes of the word to be added, and wherein the interword relationships for the word are determined by the connection nodes corresponding to the determined phonetic constraint nodes and word class nodes for word.

4. The speech recognition system according to claim 1 in which said phonetic constraint tuples define at least the last phoneme of a first word included in, or to be added to, the lexical network and the first phoneme of a second word included in, or to be added to, the lexical network in a path adjacent said first word.

5. The system of claim 1 wherein the apparatus configured to add the word is configured to associate the word with a word class node.

6. The system of claim 1 wherein the apparatus configured to add the word is configured to add a plurality of words to the active vocabulary each having at least beginning and ending phonemes and a pre-associated word class, wherein the apparatus is configured to add the words by determining phonetic constraint nodes for the words corresponding to their phonemes and determining interword relationships for the words based on the connection nodes corresponding to the determined phonetic constraint nodes and based on the plurality of word class nodes and the pre-associated word classes.

7. The system of claim 6 wherein the preassociated word classes are different.

8. A speech recognition system for dynamically augmenting an active vocabulary with a word from a total vocabulary, comprising:

A. a lexical network comprising phonemic sequences defining words of the active vocabulary;

B. constraint nodes defining constraints on permitted interconnection between selected phonemic sequences of words of the active vocabulary, the constraint nodes including phonetic constraint nodes defining phonetic constraints on adjacent words and word class nodes defining semantic or syntactic classes of words corresponding to the word class nodes, the word class nodes being associated with phonetic constraint nodes; and

C. means for establishing links between pronunciation networks of words in the active vocabulary through said constraint nodes and for establishing a link between a first word, having a pre-associated word class, and a second word based on said word class nodes and said pre-associated word class, said first word being selected from a baseform pronunciation or a spelling portion of the total vocabulary and being an addition to the active vocabulary, wherein the link between the first word and the second word is formed by forming a pronunciation network for the first word and connecting at least one of beginning and ending phonemes of the pronunciation network of the first word to one of the word class nodes.

9. The speech recognition system according to claim 8 in which said phonetic constraints comprise segmental or suprasegmental phonetic constraints.

10. The speech recognition system according to claim 8 in which the phonetic constraint nodes are characterized by at least one phoneme from each of a pair of words and defining a permissible word-pair sequence.

11. The speech recognition system according to claim 10, wherein the constraint nodes further include connection nodes for connecting congruent phonetic constraint nodes.

12. A speech recognition system according to claim 11 in which the connection between said phonetic constraint nodes and said connection nodes are formed prior to speech recognition.

13. The speech recognition system according to claim 11, further comprising means for dynamically adding words to the recognition vocabulary by associating said words to phonetic constraint nodes.

14. The speech recognition system according to claim 8 in which a new word has begin and end nodes for linking the new word to constraint nodes.

15. A method for producing a word pronunciation network for a given word to augment a pre-established active vocabulary of a speech recognition vocabulary, wherein the given word is selected from a baseform pronunciation or spelling vocabularies of the speech recognition vocabulary, comprising:

A. storing a set of phonemes for forming words of the speech recognition vocabulary;

B. for each phoneme, storing a set of phonetic fragments formed in accordance with predetermined rules that define alternative pronunciations of the phoneme dependent on phonemes with which it may be connected;

C. selecting and linking permissible phonetic fragments for each phoneme in the given word;

D. associating the linked phonetic fragments with a word class, from a plurality of word classes, of the pre-established active vocabulary of the speech recognition vocabulary, the word class being indicative of semantic of syntactic information of the given word; and

E. selectively associating the given word with at least one of the active vocabulary word classes based on the active vocabulary word classes and the associated word class.

16. The method of claim 15 in which phonetic fragments are indexed in storage by phoneme.

17. The method of claim 15 in which phonetic fragments are stored in the form of binary records that include fields defining permissible connection to other phonemes.

18. The method of claim 15 in which phonetic fragments are stored in the form of binary records that include fields defining permissible connections to preceding and succeeding phonemes.

19. The method of claim 15 in which a word pronunciation network is produced dynamically.

20. The method of claim 19 in which the phonetic fragments precompiled.

21. The method of claim 20 in which a phonetic fragment comprises one or more phones.

22. The method of claim 15 wherein each phonetic fragment has a begin node and an end node and each phoneme has a corresponding set of phonetic constraints, wherein the method further comprises precompiling associations between the set of phonetic constraints corresponding to a phoneme and the begin and the end nodes of the phonetic fragments for that phoneme.

23. The method of claim 22 in which at least one word class is associated with each vocabulary word and the method further comprises associating a set of word classes with each set of phonetic constraints.

24. The method of claim 22 further comprising joining the word pronunciation network for the given word to other words in a lexical network in accordance with the precompiled phonetic constraints associated with a first and a last phoneme in the given word.

25. The method of claim 23 further comprising joining the word pronunciation network for the given word to other words in a lexical network in accordance with the precompiled phonetic constraints associated with a first and a last phoneme in the given word and the at least one word class associated with the given word.

26. The method of claim 15 in which each phonetic fragment for a given phoneme includes labeled branches corresponding to permissible connections to adjacent phonemes, wherein a labeled branch connects only to a corresponding labeled branch of a phonetic fragment associated with an adjacent phoneme.