Error corrective mechanisms for consensus decoding of speech

• US 6,859,774 B2
• Filed: 05/02/2001
• Issued: 02/22/2005
• Est. Priority Date: 05/02/2001
• Status: Expired due to Term

• Current Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Original Assignee: International Business Machines Corporation
• Inventors: Mangu, Lidia Luminita, Padmanabhan, Mukund
• Primary Examiner(s): MCFADDEN, SUSAN IRIS
• Application Number:
  US09/847,139
• Publication Number:
  US 20020165716A1
• Time in Patent Office: 1,392 Days
• Field of Search: 704/246, 704/250, 704236-240, 704/243, 704/251, 704/255, 704/270, 704/275, 329/88.01
• US Class Current: 704/255
• CPC Class Codes: G10L 15/08 Speech classification or search

Abstract

Techniques are described for decreasing the number of errors when consensus decoding is used during speech recognition. A number of corrective rules are applied to confusion sets that are extracted during real-time speech recognition. The corrective rules are determined during training of the speech recognition system, which entails using many training confusion sets. A learning process is used that generates a number of possible rules, called template rules, that can be applied to the training confusion sets. The learning process also determines the corrective rules from the template rules. The corrective rules operate on the real-time confusion sets to select hypothesis words from the confusion sets, where the hypothesis words are not necessarily the words having the highest score.

Claims

1. A method for correcting errors for consensus decoding of speech, the method comprising the steps of:

creating a confusion set comprising a plurality of candidate words, each of said candidate words having an associated score and corresponding to an acoustic event;

selecting a candidate word from the confusion set as a word recognized for the acoustic event, wherein a candidate word other than a candidate word having a highest associated score is selected when one or more criteria are met; and

outputting the selected candidate word as the word recognized for the acoustic event.

2. The method of claim 1, wherein the step of selecting a candidate word further comprises the step of applying a rule to the confusion set, the rule determining which of the candidate words is selected.

3. The method of claim 2, wherein the step of applying further comprises the steps of:

applying the rule to the confusion set when a candidate word in the confusion set has a posterior probability that is not greater than a predetermined threshold, wherein the step of applying selects one of the candidate words in the confusion set; and

selecting the candidate word having the posterior probability that is greater than a predetermined threshold when this candidate word has a posterior probability that is greater than a predetermined threshold.

4. The method of claim 1, further comprising the steps of:

determining a plurality of allowable transformations on a plurality of training confusion sets;

providing an objective function; and

learning a plurality of corrective rules for the training confusion sets, the step of learning using the allowable transformations and objective function.

5. The method of claim 1, further comprising the step of learning when to select a candidate word other than a candidate word having a highest associated score.

6. The method of claim 1, wherein said associated score is a posterior probability.

7. The method of claim 1, wherein the step of selecting a candidate word further comprises the step of employing a data classifier.

8. The method of claim 7, wherein the data classifier is a decision-tree.

14. The method of claim 2, wherein each of the allowable transformations comprises a template rule, wherein the step of learning further comprises the step of:

instantiating a plurality of template rules, each of the template rules having a form wherein one of the plurality of candidate words of a confusion set is selected if at least one predetermined criterion is met, each criterion comprising a selected feature, an operation and a threshold value for the selected feature.

15. The method of claim 14, further comprising the steps of:

providing a baseline predictor;

selecting a training confusion set;

determining which of the template rules are applicable to the selected training confusion set, whereby each applicable rule is a candidate rule;

determining values of the features in each of the at least one predetermined criteria, the values determined from the selected training confusion set;

scoring each of the candidate rules by using the objective function;

selecting a highest scoring candidate rule;

applying the highest scoring candidate rule to the baseline predictor to create a modified consensus hypothesis; and

selecting the highest scoring candidate rule as a corrective rule.

16. The method of claim 15, wherein the method further comprise the step of providing a truth, the truth indicating a correct word for each of the training confusion sets, wherein the step of scoring comprises the steps of:

selecting a candidate rule;

for each of the training confusion sets, performing the following steps;

determining if the selected candidate rule chooses or does not choose a correct word, as determined from the truth, from the candidate words in the corresponding one of the confusion sets;

providing a value determined from a number of incorrect words selected subtracted by a number of correct words selected.

9. A method for determining a plurality of corrective rules from a plurality of training confusion sets, the method comprising the steps of:

determining a plurality of allowable transformations on the training confusion sets, each training confusion set comprising a plurality of candidate words determined from utterances of one or more individuals, wherein each candidate word in a training confusion set has an associated score;

providing an objective function; and

learning a plurality of corrective rules for the training confusion sets, the step of learning using the allowable transformations and objective function, wherein at least one of the plurality of corrective rules selects, for a given training confusion set, a candidate word other than a candidate word having a highest associated score.

10. The method of claim 9 further comprising the steps of:

determining a real-time confusion set; and

applying at least one of the corrective rules to the real-time confusion set.

11. The method of claim 10, wherein the real-time confusion set comprises a plurality of second candidate words, and wherein the step of applying further comprises:

determining if a highest scoring second candidate word has a posterior probability that is greater than a predetermined threshold;

applying at least one of the corrective rules to the real-time confusion set when the highest scoring second candidate word has a posterior probability that is not greater than a predetermined threshold, wherein the step of applying selects one of the second candidate words in the real-time confusion set;

selecting the highest scoring second candidate word having the posterior probability that is greater than a predetermined threshold when the highest scoring second candidate word has a posterior probability that is greater than a predetermined threshold; and

outputting the selected second candidate word.

12. The method of claim 9, wherein the step of learning further comprises the steps of:

extracting a plurality of features from each of the training confusion sets, each of the features mathematically describing a characteristic of an associated one of the confusion sets.

13. The method of claim 12, wherein the features comprise one or more of the following:

word identity of at least one of the plurality of candidate words in a training confusion set;

duration of at least one of the plurality of candidate words in a training confusion set;

posterior probability of at least one of the plurality of candidate words in a training confusion set;

difference in posterior probabilities of two of the plurality of candidate words in a training confusion set;

temporal position of a training confusion set in a sentence comprising a plurality of training confusion sets; and

number of the plurality of candidate words in a training confusion set.

17. A system for correcting errors for consensus decoding of speech, comprising:

a memory that stores computer-readable code; and

a processor operatively coupled to said memory, said processor configured to implement said computer-readable code, said computer-readable code configured to;

create a confusion set comprising a plurality of candidate words, each of said candidate words having an associated score and corresponding to an acoustic event;

select a candidate word from the confusion set as a word recognized for the acoustic event, wherein a candidate word other than a candidate word having a highest associated score is selected when one or more criteria are met; and

output the selected candidate word as the word recognized for the acoustic event.

18. A system for determining a plurality of corrective rules from a plurality of training confusion sets, comprising:

a memory that stores computer-readable code; and

a processor operatively coupled to said memory, said processor configured to implement said computer-readable code, said computer-readable code configured to;

determine a plurality of allowable transformations on the training confusion sets, each training confusion set comprising a plurality of candidate words determined from utterances of one or more individuals, wherein each candidate word in a training confusion set has an associated score;

provide an objective function; and

learn a plurality of corrective rules for the training confusion sets, the step of learning using the allowable transformations and objective function, wherein at least one of the plurality of corrective rules selects, for a given training confusion set, a candidate word other than a candidate word having a highest associated score.

19. An article of manufacture for correcting errors for consensus decoding of speech, comprising:

a computer readable medium having computer readable code means embodied thereon, said computer readable program code means comprising;

a step to create a confusion set comprising a plurality of candidate words, each of said candidate words having an associated score and corresponding to an acoustic event;

a step to select a candidate word from the confusion set as a word recognized for the acoustic event, wherein a candidate word other than a candidate word having a highest associated score is selected when one or more criteria are met; and

a step to output the selected candidate word as the word recognized for the acoustic event.

20. An article of manufacture for determining a plurality of corrective rules from a plurality of training confusion sets, comprising:

a computer readable medium having computer readable code means embodied thereon, said computer readable program code means comprising;

a step to determine a plurality of allowable transformations on the training confusion sets, each training confusion set comprising a plurality of candidate words determined from utterances of one or more individuals, wherein each candidate word in a training confusion set has an associated score;

a step to provide an objective function; and

a step to learn a plurality of corrective rules for the training confusion sets, the step of learning using the allowable transformations and objective function, wherein at least one of the plurality of corrective rules selects, for a given training confusion set, a candidate word other than a candidate word having a highest associated score.

21. A system for correcting errors for consensus decoding of speech, comprising:

means for creating a confusion set comprising a plurality of candidate words, each of said candidate words having an associated score and corresponding to an acoustic event;

means for selecting a candidate word from the confusion set as a word recognized for the acoustic event, wherein a candidate word other than a candidate word having a highest associated score is selected when one or more criteria are met; and

means for outputting the selected candidate word as the word recognized for the acoustic event.

22. A system for determining a plurality of corrective rules from a plurality of training confusion sets, comprising:

means for determining a plurality of allowable transformations on the training confusion sets, each training confusion set comprising a plurality of candidate words determined from utterances of one or more individuals, wherein each candidate word in a training confusion set has an associated score;

means for providing an objective function; and

means for learning a plurality of corrective rules for the training confusion sets, the step of learning using the allowable transformations and objective function, wherein at least one of the plurality of corrective rules selects, for a given training confusion set, a candidate word other than a candidate word having a highest associated score.