SYSTEM AND METHOD FOR ESTIMATING THE RELIABILITY OF ALTERNATE SPEECH RECOGNITION HYPOTHESES IN REAL TIME

• US 20110099012A1
• Filed: 10/23/2009
• Published: 04/28/2011
• Est. Priority Date: 10/23/2009
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: AT&T Intellectual Property I LP (AT&T, Inc.)
• Inventors: BALAKRISHNAN, Suhrid, WILLIAMS, Jason
• Granted Patent: US 9,653,066 B2
• Time in Patent Office: Days
• US Class Current: 704/240
• CPC Class Codes: G10L 15/01 Assessment or evaluation of speech recognition systems
• CPC Class Codes: G10L 15/04 Segmentation; Word boundary detection
• CPC Class Codes: G10L 15/08 Speech classification or search
• CPC Class Codes: G10L 15/083 Recognition networks G10L15/142, G10L15/16 take precedence
• CPC Class Codes: G10L 15/14 using statistical models, e.g. Hidden Markov Models [HMMs] G10L15/18 takes precedence
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/28 Constructional details of speech recognition systems

Abstract

Disclosed herein are systems, methods, and computer-readable storage media for estimating reliability of alternate speech recognition hypotheses. A system configured to practice the method receives an N-best list of speech recognition hypotheses and features describing the N-best list, determines a first probability of correctness for each hypothesis in the N-best list based on the received features, determines a second probability that the N-best list does not contain a correct hypothesis, and uses the first probability and the second probability in a spoken dialog. The features can describe properties of at least one of a lattice, a word confusion network, and a garbage model. In one aspect, the N-best lists are not reordered according to reranking scores. The determination of the first probability of correctness can include a first stage of training a probabilistic model and a second stage of distributing mass over items in a tail of the N-best list.

Claims

1. A method for estimating reliability of alternate speech recognition hypotheses, the method causing a computing device to perform steps comprising:

receiving an N-best list of speech recognition hypotheses and features describing the N-best list;

determining a first probability of correctness for each hypothesis in the N-best list based on the received features;

determining a second probability that the N-best list does not contain a correct hypothesis; and

using the first probability and the second probability in a spoken dialog.

2. The method of claim 1, wherein the features describe properties of at least one of a lattice, a word confusion network, and a garbage model.

3. The method of claim 1, wherein the N-best lists are not reordered according to reranking scores.

4. The method of claim 1, wherein determining the first probability of correctness includes two stages.

5. The method of claim 4, wherein a first stage of the two stages comprises training a discriminative model P_a.

6. The method of claim 4, wherein a second stage of the two stages comprises distributing mass over items in a tail of the N-best list.

7. A system for estimating reliability of alternate speech recognition hypotheses, the system comprising:

a processor;

a module configured to control the processor to receive an N-best list of speech recognition hypotheses and features describing the N-best list;

a module configured to control the processor to determine a first probability of correctness for each hypothesis in the N-best list based on the received features;

a module configured to control the processor to determine a second probability that the N-best list does not contain a correct hypothesis; and

a module configured to control the processor to use the first probability and the second probability in a spoken dialog.

8. The system of claim 7, wherein the features describe properties of at least one of a lattice, a word confusion network, and a garbage model.

9. The system of claim 7, wherein the N-best lists are not reordered according to reranking scores.

10. The system of claim 7, wherein determining the first probability of correctness includes two stages.

11. The system of claim 10, wherein a first stage of the two stages comprises training a discriminative model P_a.

12. The system of claim 10, wherein a second stage of the two stages comprises distributing mass over items in a tail of the N-best list.

13. A computer-readable storage medium storing instructions which, when executed by a computing device, cause the computing device to estimate reliability of alternate speech recognition hypotheses, the instructions comprising:

receiving an N-best list of speech recognition hypotheses and features describing the N-best list;

determining a first probability of correctness for each hypothesis in the N-best list based on the received features;

determining a second probability that the N-best list does not contain a correct hypothesis; and

using the first probability and the second probability in a spoken dialog.

14. The computer-readable storage medium of claim 13, wherein the features describe properties of at least one of a lattice, a word confusion network, and a garbage model.

15. The computer-readable storage medium of claim 13, wherein the N-best lists are not reordered according to reranking scores.

16. The computer-readable storage medium of claim 13, wherein determining the first probability of correctness includes two stages.

17. The computer-readable storage medium of claim 16, wherein a first stage of the two stages comprises training a discriminative model P_a.

18. The computer-readable storage medium of claim 16, wherein a second stage of the two stages comprises distributing mass over items in a tail of the N-best list.