Method and system of optimal selection strategy for statistical classifications

• US 8,024,188 B2
• Filed: 08/24/2007
• Issued: 09/20/2011
• Est. Priority Date: 08/24/2007
• Status: Active Grant

• Current Assignee: Robert Bosch GmbH
• Original Assignee: Robert Bosch GmbH
• Inventors: Hu, Junling, Liu, Xue, Morbini, Fabrizio, Weng, Fuliang
• Primary Examiner(s): CHAWAN, VIJAY B
• Application Number:
  US11/845,008
• Publication Number:
  US 20090055176A1
• Time in Patent Office: 1,488 Days
• Field of Search: 704/275, 704/270, 704/270.1, 704/240, 704/236, 704/238, 704/242, 704/9, 715/764, 715/804, 715/858, 715/256, 715/205, 715/854, 715/209
• US Class Current: 704/240
• CPC Class Codes: G10L 15/08 Speech classification or search
• CPC Class Codes: G10L 2015/0631 Creating reference templates; Clustering

Abstract

An optimal selection or decision strategy is described through an example that includes use in dialog systems. The selection strategy or method includes receiving multiple predictions and multiple probabilities. The received predictions predict the content of a received input and each of the probabilities corresponds to one of the predictions. In an example dialog system, the received input includes an utterance. The selection method includes dynamically selecting a set of predictions from the received predictions by generating ranked predictions. The ranked predictions are generated by ordering the plurality of predictions according to descending probability.

Claims

1. A computer-implemented method of processing an input utterance from a user in an adaptive dialog system (ADS), the method comprising:

a classifier component of the ADS receiving a plurality of predictions and a plurality of probabilities, wherein the predictions predict a probability of correctly recognizing the input utterance;

the classifier component defining a dynamic threshold value to select one or more predictions of the plurality of predictions;

a decision component of the ADS dynamically selecting a set of predictions from the plurality of predictions by generating ranked predictions by ordering the plurality of predictions according to descending probability, wherein the dynamic threshold value is selected for the input utterance based on the ordering of the plurality of predictions; and

a training component of the ADS using a result of the input utterance for retraining the ADS if the probability of correctly predicting the input utterance is below the dynamic threshold value.

2. The method of claim 1, wherein dynamically selecting the set of predictions by the decision component, comprises determining a sequential pair of predictions in the ranked predictions between which the difference in corresponding probabilities is greatest relative to any other sequential pair of predictions, wherein the sequential pair of predictions include a first prediction and a second prediction, the first prediction having a higher probability than the second prediction.

3. The method of claim 2, further comprising the classifier component classifying the received input according to an accumulated probability of the set of predictions, and wherein the dynamic threshold value is modified based on the accumulated probability of the set of predictions.

4. The method of claim 2, wherein dynamically selecting the set of predictions in the decision component, comprises identifying the set of predictions to include those predictions in a group bounded by a highest ranked prediction and the first prediction.

5. The method of claim 1, further comprising:

generating the plurality of predictions in the classifier component;

generating the plurality of probabilities each of which corresponds to a prediction; and

classifying the received input according to an accumulated probability of the set of predictions.

6. The method of claim 5, wherein the classifying in the classifier component, includes classifying the received input as a first input class when the accumulated probability of the set of predictions is at or above a threshold, and classifying the received input as a second input class when the accumulated probability of the set of predictions is below a threshold.

7. The method of claim 5, comprising generating a second plurality of predictions on a second received input, the generating of the second plurality of predictions dynamically using data of at least one input of the first input class.

8. An adaptive dialog system for processing an input utterance from a user, comprising:

a processor;

a classifier component coupled to the processor and configured to receive a plurality of predictions and a plurality of probabilities, wherein the predictions predict a probability of correctly recognizing the input utterance, the classifier component further configured to defining a dynamic threshold value to select one or more predictions of the plurality of predictions; and

a decision component coupled to the processor and configured to dynamically select a set of predictions from the plurality of predictions by generating ranked predictions by ordering the plurality of predictions according to descending probability, wherein the dynamic threshold value is selected for the input utterance based on the ordering of the plurality of predictions.

9. The system of claim 8, wherein the decision component is configured to dynamically select the set of predictions by determining a sequential pair of predictions in the ranked predictions between which the difference in corresponding probabilities is greatest relative to any other sequential pair of predictions, wherein the sequential pair of predictions include a first prediction and a second prediction, the first prediction having a higher probability than the second prediction.

10. The system of claim 9, wherein dynamically selecting the set of predictions by the decision component comprises identifying the set of predictions to include those predictions in a group bounded by a highest ranked prediction and the first prediction.

11. The system of claim 9, wherein the classifier component is configured to classify the received input according to an accumulated probability of the set of predictions, and wherein the dynamic threshold value is modified based on the accumulated probability of the set of predictions.

12. The system of claim 11, wherein the classifier component is configured to generate the plurality of predictions and to generate the plurality of probabilities, each of which corresponds to a prediction of the plurality of predictions, and classify the input according to an accumulated probability of the set of predictions.

13. The system of claim 12, wherein the decision component is configured to classify the input as a first input class when the accumulated probability of the set of predictions is at or above a threshold, and configured to classify the input as a second input class when the accumulated probability of the set of predictions is below a threshold.

14. The system of claim 12, wherein the decision component is configured to dynamically select the set of predictions including generating ranked predictions by ordering the plurality of predictions according to descending probability.

15. The system of claim 12, comprising a database coupled to the decision component, wherein the decision component is configured to transfer inputs classified as the second input class to the database and the database is configured to store the transferred inputs.

16. The system of claim 15, comprising a re-training module coupled to the classifier component and the database, wherein the re-training module is configured to dynamically re-train the classifier using inputs classified as the second input class prior to another plurality of predictions.

17. The system of claim 15, comprising a graphical user interface (GUI) coupled to the database, wherein the GUI is configured to generate a labeled input by applying a label to an input classified as the second input class.

18. A physical, non-transitory computer-readable media including executable instructions which, when executed in a processing system, control selection of output predictions by:

receiving in a classifier component of the processing system, a plurality of predictions and a plurality of probabilities, wherein the predictions predict a probability of correctly recognizing the input utterance;

defining, in the classifier component, a dynamic threshold value to select one or more predictions of the plurality of predictions;

dynamically selecting in a decision component of the processing system, a set of predictions from the plurality of predictions by generating ranked predictions by ordering the plurality of predictions according to descending probability, wherein the dynamic threshold value is selected for the input utterance based on the ordering of the plurality of predictions; and

using a result of the input utterance for retraining the ADS if the probability of correctly predicting the input utterance is below the dynamic threshold value.

19. The physical, non-transitory media of claim 18, wherein the instructions, when executed, dynamically select the set of predictions by determining a sequential pair of predictions in the ranked predictions between which the difference in corresponding probabilities is greatest relative to any other sequential pair of predictions, wherein the sequential pair of predictions include a first prediction and a second prediction, the first prediction having a higher probability than the second prediction.

20. The physical, non-transitory media of claim 19, wherein the instructions, when executed, classify the received input according to an accumulated probability of the set of predictions, and wherein the dynamic threshold value is modified based on the accumulated probability of the set of predictions.

21. The physical, non-transitory media of claim 19, wherein the instructions, when executed, dynamically select the set of predictions comprises identifying the set of predictions to include those predictions in a group bounded by a highest ranked prediction and the first prediction.