Error corrective mechanisms for consensus decoding of speech

• US 20020165716A1
• Filed: 05/02/2001
• Published: 11/07/2002
• Est. Priority Date: 05/02/2001
• Status: Active Grant

• Current Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Original Assignee: International Business Machines Corporation
• Inventors: Padmanabhan, Mukund, Mangu, Lidia Luminita
• Granted Patent: US 6,859,774 B2
• Time in Patent Office: Days
• US Class Current: 704/255
• CPC Class Codes: G10L 15/08 Speech classification or search

Abstract

Techniques are described for decreasing the number of errors when consensus decoding is used during speech recognition. A number of corrective rules are applied to confusion sets that are extracted during real-time speech recognition. The corrective rules are determined during training of the speech recognition system, which entails using many training confusion sets. A learning process is used that generates a number of possible rules, called template rules, that can be applied to the training confusion sets. The learning process also determines the corrective rules from the template rules. The corrective rules operate on the real-time confusion sets to select hypothesis words from the confusion sets, where the hypothesis words are not necessarily the words having the highest score.

Claims

1. A method for correcting errors for consensus decoding of speech, the method comprising the steps of:

creating a confusion set comprising a plurality of candidate words, each of said candidate words having an associated score; and

selecting a candidate word from the confusion set, wherein a candidate word other than a candidate word having a highest associated score is selected when one or more criteria are met.

2. The method of claim 1, wherein the step of selecting a candidate word further comprises the step of applying a rule to the confusion set, the rule determining which of the candidate words is selected.

3. The method of claim 1, further comprising the step of outputting the selected word.

4. The method of claim 2, wherein the step of applying further comprises the steps of:

applying the rule to the confusion set when a candidate word in the confusion set has a posterior probability that is not greater than a predetermined threshold, wherein the step of applying selects one of the candidate words in the confusion set; and

selecting the candidate word having the posterior probability that is greater than a predetermined threshold when this candidate word has a posterior probability that is greater than a predetermined threshold.

5. The method of claim 1, further comprising the steps of:

determining a plurality of allowable transformations on a plurality of training confusion sets;

providing an objective function; and

learning a plurality of corrective rules for the training confusion sets, the step of learning using the allowable transformations and objective function.

6. The method of claim 1, further comprising the step of learning when to select a candidate other than a candidate having a highest associated score.

7. The method of claim 1, wherein said associated score is a posterior probability.

8. The method of claim 1, wherein the step of selecting a candidate word further comprises the step of employing a data classifier.

9. The method of claim 8, wherein the data classifier is a decision-tree.

10. A method for determining a plurality of corrective rules from a plurality of training confusion sets, the method comprising the steps of:

determining a plurality of allowable transformations on the training confusion sets;

providing an objective function; and

learning a plurality of corrective rules for the training confusion sets, the step of learning using the allowable transformations and objective function.

11. The method of claim 10, further comprising the steps of:

determining a real-time confusion set; and

applying at least one of the corrective rules to the real-time confusion set.

12. The method of claim 10, wherein the real-time confusion set comprises a plurality of candidate words, and wherein the step of applying further comprises:

determining a real-time confusion set;

determining if a candidate word has a posterior probability that is greater than a predetermined threshold;

applying at least one of the corrective rules to the real-time confusion set when the candidate word has a posterior probability that is not greater than a predetermined threshold, wherein the step of applying selects one of the candidate words in the confusion set;

selecting the candidate word having the posterior probability that is greater than a predetermined threshold when the candidate word has a posterior probability that is greater than a predetermined threshold; and

outputting the selected candidate word.

13. The method of claim 10, wherein the step of learning further comprises the steps of:

extracting a plurality of features from each of the training confusion sets, each of the features mathematically describing a characteristic of an associated one of the confusion sets.

14. The method of claim 13, wherein the features comprise one or more of the following:

word identity of at least one candidate word in a training confusion set;

duration of at least one candidate word in a training confusion set;

posterior probability of at least one candidate word in a training confusion set;

difference in posterior probabilities of two candidate words in a training confusion set;

temporal position of a training confusion set in a sentence comprising a plurality of training confusion sets; and

number of candidate words in a training confusion set.

15. The method of claim 13, wherein each of the allowable transformations comprises a template rule, wherein the step of learning further comprises the step of:

instantiating a plurality of template rules, each of the template rules having a form wherein one candidate word of a confusion set is selected if at least one predetermined criterion is met, each criterion comprising a selected feature, an operation and a threshold value for the selected feature.

16. The method of claim 15, further comprising the steps of:

providing a baseline predictor;

selecting a training confusion set;

determining which of the template rules are applicable to the selected training confusion set, whereby each applicable rule is a candidate rule;

determining values of the features in each of the at least one predetermined criteria, the values determined from the selected training confusion set;

scoring each of the candidate rules by using the objective function;

selecting a highest scoring candidate rule;

applying the highest scoring candidate rule to the baseline predictor to create a modified consensus hypothesis; and

selecting the highest scoring candidate rule as a corrective rule.

17. The method of claim 10, wherein the method further comprise the step of providing a truth, the truth indicating a correct word for each of the training confusion sets, wherein the step of scoring comprises the steps of:

selecting a candidate rule;

for each of the training confusion sets, performing the following steps;

determining if the selected candidate rule chooses or does not choose a correct word, as determined from the truth, from the candidate words in the corresponding one of the confusion sets;

providing a value determined from a number of incorrect words selected subtracted by a number of correct words selected.

18. A system for correcting errors for consensus decoding of speech, comprising:

a memory that stores computer-readable code; and

a processor operatively coupled to said memory, said processor configured to implement said computer-readable code, said computer-readable code configured to;

create a confusion set comprising a plurality of candidate words, each of said candidate words having an associated score; and

select a candidate word from the confusion set, wherein a candidate word other than a candidate word having a highest associated score is selected when one or more criteria are met.

19. A system for determining a plurality of corrective rules from a plurality of training confusion sets, comprising:

a memory that stores computer-readable code; and

a processor operatively coupled to said memory, said processor configured to implement said computer-readable code, said computer-readable code configured to;

determine a plurality of allowable transformations on the training confusion sets;

provide an objective function; and

learn a plurality of corrective rules for the training confusion sets, the step of learning using the allowable transformations and objective function.

20. An article of manufacture for correcting errors for consensus decoding of speech, comprising:

a computer readable medium having computer readable code means embodied thereon, said computer readable program code means comprising;

a step to create a confusion set comprising a plurality of candidate words, each of said candidate words having an associated score; and

a step to select a candidate word from the confusion set, wherein a candidate word other than a candidate word having a highest associated score is selected when one or more criteria are met.

21. An article of manufacture for determining a plurality of corrective rules from a plurality of training confusion sets, comprising:

a computer readable medium having computer readable code means embodied thereon, said computer readable program code means comprising;

a step to determine a plurality of allowable transformations on the training confusion sets;

a step to provide an objective function; and

a step to learn a plurality of corrective rules for the training confusion sets, the step of learning using the allowable transformations and objective function.

22. A system for correcting errors for consensus decoding of speech, comprising:

means for creating a confusion set comprising a plurality of candidate words, each of said candidate words having an associated score; and

means for selecting a candidate word from the confusion set, wherein a candidate word other than a candidate word having a highest associated score is selected when one or more criteria are met.

23. A system for determining a plurality of corrective rules from a plurality of training confusion sets, comprising:

means for determining a plurality of allowable transformations on the training confusion sets;

means for providing an objective function; and

means for learning a plurality of corrective rules for the training confusion sets, the step of learning using the allowable transformations and objective function.