Confidence measure generation for speech related searching

• US 20090037175A1
• Filed: 08/03/2007
• Published: 02/05/2009
• Est. Priority Date: 08/03/2007
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Ju, Yun-Cheng, Yu, Dong, Wang, Ye-Yi
• Granted Patent: US 8,165,877 B2
• Time in Patent Office: Days
• US Class Current: 704/251
• CPC Class Codes: G10L 15/1822 Parsing for meaning understanding

Abstract

A voice search system has a speech recognizer, a search component, and a dialog manager. A confidence measure generator receives speech recognition features from the speech recognizer, search features from the search component, and dialog features from the dialog manager, and calculates an overall confidence measure for voice search results based upon the features received. The invention can be extended to include the generation of additional features, based on those received from the individual components of the voice search system.

Claims

1. A method of conducting a dialog with a user of a voice search system that includes a speech recognizer, a search component and a dialog manager, the method comprising:

generating a dialog feature, with the dialog manager, indicative of a performance of the voice search system in responding to the user;

recognizing a user speech input with the speech recognizer to generate a query and a speech recognition feature indicative of a measure of speech recognition accuracy;

searching a data store, with the search component, for search results based on the query, and generating a search feature indicative of a measure of search accuracy;

generating a voice search confidence measure based on the dialog feature, the speech recognition feature, and the search feature; and

conducting the dialog according to a dialog strategy chosen based, at least in part, on the voice search confidence measure.

2. The method of claim 1 and further comprising:

generating additional features based on at least some of the dialog feature, the speech recognition feature and the search feature.

3. The method of claim 1 wherein generating a voice search confidence measure comprises:

applying a maximum entropy model to the dialog feature, the speech recognition feature and the search feature to generate the voice search confidence measure.

4. The method of claim 3 wherein applying a maximum entropy model comprises:

classifying the search results a correct or incorrect, and calculating a probability associated with the classification.

5. The method of claim 1 wherein generating a dialog feature comprises:

generating a plurality of dialog features.

6. The method of claim 1 wherein recognizing a user speech input to generate a speech recognition feature comprises:

generating a plurality of speech recognition features.

7. The method of claim 1 wherein generating a search feature comprises:

generating a plurality of search features.

8. A method of generating a confidence measure generator for use in a voice search system, the voice search system including voice search components comprising a speech recognition system, a dialog manager and a search system, the method comprising:

selecting voice search features, from a plurality of the voice search components, to be considered by the confidence measure generator in generating a voice search confidence measure; and

training a model to generate the voice search confidence measure based on selected voice search features.

9. The method of claim 8 wherein selecting voce search features comprises:

selecting a set of features;

calculating an error rate for the model based on the set of features;

identifying insignificant features in the set of features; and

removing the insignificant features from the set of features to obtain a set of remaining features.

11. The method of claim 9 wherein identifying insignificant features comprises:

calculating how much each feature in the set of features affects the error rate of the model; and

identifying, as insignificant features, those features in the set of features that affect the error rate of the model less than a threshold amount.

12. The method of claim 11 wherein calculating how much each feature in the set of features affects the error rate of the model comprises:

selecting a feature from the set of features; and

calculating an error rate of the model without considering the selected feature.

13. The method of claim 9 wherein selecting voice search features further comprises:

determining whether each feature in the set of remaining features varies sufficiently linearly with respect to an error rate of the voice search system; and

for each given feature that does not vary sufficiently linearly, quantizing the given feature.

14. The method of claim 13 wherein determining whether each feature in the set of remaining features varies sufficiently linearly with respect to an error rate of the voice search system, comprises:

selecting a feature from the set of remaining features;

calculating voice search system error rate relative to a value of the selected feature; and

calculating a linearity measure indicative of a linearity of the selected feature.

15. The method of claim 8 wherein selecting voice features comprises:

selecting combined voice features based on combinations of the voice search features from the voice search components.

10. The method of claim 10 wherein selecting voice search features further comprises:

repeating the steps of selecting a set of features, calculating an error rate for the model based on the set of features, identifying insignificant features in the set of features, and removing the insignificant features from the initial set of features to obtain a set of remaining features, until a desired set of remaining features is obtained.

16. A voice search system, comprising:

a speech recognition system configured to receive a spoken input and to generate a query from the spoken input;

a search system configured to receive the query and search a data store for a search result based on the query;

a dialog component configured to conduct a user dialog; and

a confidence measure generator configured to receive features from a plurality of the speech recognizer, the search component and the dialog component and generate a voice search confidence measure corresponding to the search result, the dialog component being configured to conduct the user dialog according to a strategy based on the voice search confidence measure.

17. The voice search system of claim 16 wherein the speech recognition system is configured to generate at least one of the features as a speech recognition feature indicative of an accuracy of speech recognition of the spoken input.

18. The voice search system of claim 17 wherein the search system is configured to generate at least one of the features as a search feature indicative of an accuracy of the search result, given the query.

19. The voice search system of claim 18 wherein the dialog component is configured to generate at least one of the features as a dialog feature indicative of a performance of the voice search system in responding to the spoken input.

20. The voice search system of claim 19 wherein the search system is configured to receive a keyboard text input as a second query and search the data store for a second search result based on the second query and wherein the confidence measure generator is further configured to generate a text search confidence measure corresponding to the second search result.