Lexical tree pre-filtering in speech recognition

• US 5,822,730 A
• Filed: 08/22/1996
• Issued: 10/13/1998
• Est. Priority Date: 08/22/1996
• Status: Expired due to Term

• Current Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Original Assignee: Dragon Systems, Inc. (Microsoft Corporation)
• Inventors: Walsh, Alan, Roth, Robert, Baker, James K., Gillick, Laurence S.
• Primary Examiner(s): Hudspeth, David R.
• Assistant Examiner(s): SOFOCLEOUS, MICHAEL D
• Application Number:
  US08/701,393
• Time in Patent Office: 782 Days
• Field of Search: 704/251-257
• US Class Current: 704/255
• CPC Class Codes: G10L 15/08 Speech classification or search
• CPC Class Codes: G10L 15/187 Phonemic context, e.g. pronunciation rules, phonotactical constraints or phoneme n-grams
• CPC Class Codes: G10L 15/197 Probabilistic grammars, e.g. word n-grams

Abstract

A speech recognition technique uses lexical tree pre-filtering to obtain lists of words for use in performing speech recognition. The lexical tree pre-filtering includes representing a vocabulary of words using a lexical tree and identifying a first subset of the vocabulary that may correspond to speech spoken beginning at a first time by propagating through the lexical tree information about the speech spoken beginning at the first time. A second subset of the vocabulary that may correspond to speech spoken beginning at a second time is identified by propagating through the lexical tree information about the speech spoken beginning at the second time. Words included in the speech are recognized by comparing speech spoken beginning at the first time with words from the first subset of the vocabulary and speech spoken beginning at the second time with words from the second subset of the vocabulary. The state of the lexical tree is not reset between identifying the first and second subsets.

Claims

1. A method of using lexical tree pre-filtering to identify sets of words for use in recognizing speech, the method comprising:

representing a vocabulary of words using a lexical tree;

identifying a first subset of the vocabulary that may correspond to speech spoken beginning at a first time by propagating through the lexical tree information about the speech spoken beginning at the first time;

identifying a second subset of the vocabulary that may correspond to speech spoken beginning at a second time by propagating through the lexical tree information about the speech spoken beginning at the second time; and

recognizing words included in the speech by comparing speech spoken beginning at the first time with words from the first subset of the vocabulary and speech spoken beginning at the second time with words from the second subset of the vocabulary;

wherein a state of the lexical tree is not reset between the first and second identifying steps so that the state of the lexical tree at initiation of the second identifying step corresponds to the state of the lexical tree at conclusion of the first identifying step.

2. The method of claim 1, wherein the step of identifying a first subset of the vocabulary includes performing a coarse comparison of the speech spoken beginning at the first time with words in the vocabulary.

3. The method of claim 2, wherein the coarse comparison includes propagating scores through the lexical tree.

4. The method of claim 3, wherein the coarse comparison includes associating a starting time with a score.

5. The method of claim 4, wherein the coarse comparison includes moving the starting time with the score as the score propagates through the lexical tree.

6. The method of claim 3, wherein a word is produced by the coarse comparison when a score corresponding to the word propagates out of a node of the lexical tree corresponding to the word.

7. The method of claim 6, wherein the first subset of the vocabulary comprises a first list of words, and wherein the first identifying step further comprises adding to the first list of words a word produced by the coarse comparison.

8. The method of claim 7, wherein:

a first time is associated with the first list of words, the coarse comparison includes associating a starting time with a score, and the first identifying step comprises adding a word to the first list of words when a starting time associated with a score corresponding to the word equals the first time.

9. The method of claim 8, wherein the first identifying step further comprises:

maintaining multiple lists of words, each of which is associated with a particular time, and adding the word to lists of words that are associated with times that precede or follow the first time by up to a certain time interval.

10. The method of claim 9, wherein the time interval is related to the length of the word.

11. The method of claim 9, wherein the step of adding the word to the first list of words further comprises adding a first score associated with the word to the first list of words.

12. The method of claim 10, wherein the step of adding the word to lists of words that are associated with times that precede or follow the first time by up to a certain time interval further comprises:

modifying the first score based on a difference between the first time and a time associated with a particular list of words to produce a modified score, and adding the modified score to the particular list of words in association with the word.

13. The method of claim 7, wherein the step of adding the word to the first list of words further comprises adding a first score associated with the word to the first list of words.

14. The method of claim 3, wherein the lexical tree includes nodes that each represent a phoneme so that each word of the vocabulary is represented by a sequence of nodes that together correspond to a sequence of phonemes that define the word, and wherein the step of propagating scores through the lexical tree comprises propagating scores between the nodes of the lexical tree.

15. The method of claim 14, wherein a language model score that represents a likelihood with which a word including the phoneme represented by the node occurs in speech is associated with each node, and wherein the step of propagating scores through the lexical tree comprises adding a language model score associated with a node to a score propagating from the node.

16. The method of claim 15, wherein a language model score associated with a node corresponds to an increase in a best language model score from a branch of the lexical tree that starts with a parent of the node to a branch of the lexical tree that starts with the node.

17. The method of claim 14, wherein, when a score is to be propagated from a node to several subnodes, the method further comprises, for each subnode:

performing a relatively crude acoustic match,

adding to the score for the node results of the crude match to produce a subnode score,

comparing the subnode score to a threshold value,

eliminating the subnode when the subnode score does not meet the threshold value, and

performing a more refined acoustic match for the subnode when the subnode score meets the threshold value.

18. The method of claim 17, wherein the relatively crude acoustic match is based on a phoneme corresponding to the subnode.

19. The method of claim 17, wherein the relatively crude acoustic match is based on speech spoken after speech corresponding to the node.

20. The method of claim 19, wherein the speech is divided into frames and the relatively crude acoustic match is based on an average of multiple frames.

21. The method of claim 3, wherein the step of propagating scores comprises normalizing the scores.

22. The method of claim 21, wherein the step of normalizing the scores comprises, for a first time increment, subtracting from the scores an updated version of a best score for a second time increment.

23. The method of claim 3, wherein the step of recognizing words included in the speech utilizes scores propagated for words in the first subset of the vocabulary during the coarse comparison.

24. The method of claim 2, wherein the coarse comparison of the speech spoken beginning at the first time with words in the vocabulary comprises:

comparing the speech with a first speech model associated with a node of the lexical tree to produce a first score associated with the node,

comparing the first score to a first threshold,

when the first score does not meet the threshold, saving the first score as a score for the node, and

when the first score meets the threshold, comparing the speech with a second, more accurate speech model associated with the node to produce a second score associated with the node and saving the second score as the score for the node.

25. The method of claim 24, wherein the first speech model is independent of the context of the node and the second speech model depends on the context of the node.

26. The method of claim 1, wherein the lexical tree includes active nodes and inactive nodes and wherein information for a particular increment of the speech is propagated through the lexical tree by:

selecting a node that has not been processed for the particular increment and that has no active subnodes that have not been processed for the particular increment,

processing the selected active node, and

repeating the selecting and processing steps until all active nodes have been processed.

27. An apparatus for using lexical tree prefiltering to identify sets of words for use in recognizing speech, the apparatus comprising:

means for representing a vocabulary of words using a lexical tree;

means for identifying a first subset of the vocabulary that may correspond to speech spoken beginning at a first time by propagating through the lexical tree information about the speech spoken beginning at the first time;

means for identifying a second subset of the vocabulary that may correspond to speech spoken beginning at a second time by propagating through the lexical tree information about the speech spoken beginning at the second time; and

means for recognizing words included in the speech by comparing speech spoken beginning at the first time with words from the first subset of the vocabulary and speech spoken beginning at the second time with words from the second subset of the vocabulary;

wherein a state of the lexical tree is not reset between identifying the first and second subsets of the vocabulary so that the state of the lexical tree at initiation of identifying the second subset corresponds to the state of the lexical tree at conclusion of identifying the first subset.

28. A speech recognition system that uses lexical tree pre-filtering to identify sets of words for use in recognizing speech, the system comprising:

an input device configured to convert speech into a digital signal, and

a processor configured to;

represent a vocabulary of words using a lexical tree;

identify a first subset of the vocabulary that may correspond to speech spoken beginning at a first time by propagating through the lexical tree information about the speech spoken beginning at the first time;

identify a second subset of the vocabulary that may correspond to speech spoken beginning at a second time by propagating through the lexical tree information about the speech spoken beginning at the second time; and

recognize words included in the speech by comparing a portion of the digital signal corresponding to speech spoken beginning at the first time with words from the first subset of the vocabulary and a portion of the digital signal corresponding to speech spoken beginning at the second time with words from the second subset of the vocabulary;

wherein the processor is further configured to maintain the state of the lexical tree between identifying the first and second subsets of the vocabulary so that the state of the lexical tree at initiation of identifying the second subset corresponds to the state of the lexical tree at conclusion of identifying the first subset.

29. Computer software, residing on a computer-readable storage medium, comprising instructions for causing a computer to:

represent a vocabulary of words using a lexical tree;

identify a first subset of the vocabulary that may correspond to speech spoken beginning at a first time by propagating through the lexical tree information about the speech spoken beginning at the first time;

identify a second subset of the vocabulary that may correspond to speech spoken beginning at a second time by propagating through the lexical tree information about the speech spoken beginning at the second time;

recognize words included in the speech by comparing speech spoken beginning at the first time with words from the first subset of the vocabulary and speech spoken beginning at the second time with words from the second subset of the vocabulary; and

maintain the lexical tree between identifying the first and second subsets of the vocabulary so that the state of the lexical tree at initiation of identifying the second subset corresponds to the state of the lexical tree at conclusion of identifying the first subset.

30. A method of using lexical tree pre-filtering to identify sets of words for use in recognizing speech, the method comprising:

representing a vocabulary of words using a lexical tree;

identifying a first subset of words from the vocabulary, the first subset including words that may correspond to speech spoken beginning at a first time, by propagating through the lexical tree information about the speech spoken beginning at the first time;

identifying a second subset of words from the vocabulary, the second subset including words that may correspond to speech spoken beginning at a second time, by propagating through the lexical tree information about the speech spoken beginning at the second time;

recognizing words included in the speech by comparing speech spoken beginning at the first time with words from the first subset of the vocabulary and speech spoken beginning at the second time with words from the second subset of the vocabulary; and

maintaining a state of the lexical tree between the first and second identifying steps so that the state of the lexical tree at initiation of the second identifying step corresponds to the state of the lexical tree at conclusion of the first identifying step.