Method and System of Optimal Selection Strategy for Statistical Classifications in Dialog Systems

• US 20090055164A1
• Filed: 08/24/2007
• Published: 02/26/2009
• Est. Priority Date: 08/24/2007
• Status: Active Grant

• Current Assignee: Robert Bosch GmbH
• Original Assignee: Robert Bosch GmbH
• Inventors: Weng, Fuliang, Hu, Junling, Liu, Xue, Morbini, Fabrizio
• Granted Patent: US 8,050,929 B2
• Time in Patent Office: Days
• US Class Current: 704/9
• CPC Class Codes: G10L 15/1822 Parsing for meaning understanding

Abstract

An optimal selection or decision strategy is described through an example that includes use in dialog systems. The selection strategy or method includes receiving multiple predictions and multiple probabilities. The received predictions predict the content of a received input and each of the probabilities corresponds to one of the predictions. In an example dialog system, the received input includes an utterance. The selection method includes dynamically selecting a set of predictions from the received predictions by generating ranked predictions. The ranked predictions are generated by ordering the plurality of predictions according to descending probability.

Claims

1. A dialog system comprising:

a classifier coupled to a processor and configured to generate a plurality of predictions on a dialog act corresponding to an utterance received at an input; and

a decision component coupled to the processor and configured to generate a plurality of probabilities each of which corresponds to a prediction, dynamically select a set of predictions from the plurality of predictions, and classify the utterance according to an accumulated probability of the set of predictions.

2. The system of claim 1, wherein the decision component is configured to classify the utterance as understood or misunderstood when the accumulated probability of the set of predictions is at or above a threshold.

3. The system of claim 1, wherein the decision component is configured to classify the utterance as not-understood when the accumulated probability of the set of predictions is below a threshold.

4. The system of claim 1, wherein the decision component is configured to dynamically select the set of predictions including generating ranked predictions by ordering the plurality of predictions according to descending probability.

5. The system of claim 4, wherein the decision component is configured to dynamically select the set of predictions by determining a sequential pair of predictions in the ranked predictions between which the difference in corresponding probabilities is greatest relative to any other sequential pair of predictions, wherein the sequential pair of predictions include a first prediction and a second prediction, the first prediction having a higher probability than the second prediction.

6. The system of claim 5, wherein the decision component is configured to dynamically select the set of predictions by identifying the set of predictions to include those predictions in a group bounded by a highest ranked prediction and the first prediction.

7. The system of claim 1, comprising a database coupled to the decision component, wherein the decision component is configured to transfer utterances classified as not-understood to the database and the database is configured to store the utterances classified as not-understood.

8. The system of claim 7, comprising a re-training module coupled to the classifier and the database, wherein the re-training module is configured to dynamically re-train the classifier using a not-understood utterance prior to another plurality of predictions.

9. The system of claim 7, comprising a graphical user interface (GUI) coupled to the database, wherein the GUI is configured to generate a labeled utterance by applying a label to a not-understood utterance.

10. A method of classifying an utterance comprising:

receiving the utterance;

generating a plurality of predictions on a dialog act corresponding to the utterance;

generating a plurality of probabilities each of which corresponds to a prediction;

dynamically selecting a set of predictions from the plurality of predictions; and

classifying the utterance according to an accumulated probability of the set of predictions.

11. The method of claim 10, wherein the classifying includes classifying the utterance as understood or misunderstood when the accumulated probability of the set of predictions is at or above a threshold.

12. The method of claim 10, wherein the classifying includes classifying the utterance as not-understood when the accumulated probability of the set of predictions is below a threshold.

13. The method of claim 10, wherein a number of predictions in the set of predictions corresponds to the utterance.

14. The method of claim 10, wherein dynamically selecting the set of predictions comprises generating ranked predictions by ordering the plurality of predictions according to descending probability.

15. The method of claim 14, wherein dynamically selecting the set of predictions comprises determining a sequential pair of predictions in the ranked predictions between which the difference in corresponding probabilities is greatest relative to any other sequential pair of predictions, wherein the sequential pair of predictions include a first prediction and a second prediction, the first prediction having a higher probability than the second prediction.

16. The method of claim 15, wherein dynamically selecting the set of predictions comprises identifying the set of predictions to include those predictions in a group bounded by a highest ranked prediction and the first prediction.

17. The method of claim 10, comprising generating a second plurality of predictions on a second dialog act corresponding to a second utterance, the generating of the second plurality of predictions dynamically using data of at least one utterance classified as not-understood.

18. The method of claim 10, comprising:

storing any utterance classified as not-understood; and

generate labeled utterances by labeling a stored utterance.

19. The method of claim 18, comprising:

training a classifier using training data, wherein the classifier generates the plurality of predictions; and

dynamically re-training the classifier using the labeled utterances, the re-training occurring prior to generating a second plurality of predictions.

20. Computer readable media including executable instructions which, when executed in a processing system, classify an utterance by:

generating a plurality of predictions on a dialog act corresponding to the utterance;

generating a plurality of probabilities each of which corresponds to a prediction;

dynamically selecting a set of predictions from the plurality of predictions; and

classifying the utterance according to an accumulated probability of the set of predictions.

21. The media of claim 20, wherein execution of the instructions classifies the utterance as understood or misunderstood when the accumulated probability of the set of predictions is at or above a threshold, wherein execution of the instructions classifies the utterance as not-understood when the accumulated probability of the set of predictions is below a threshold.

22. The media of claim 20, wherein execution of the instructions dynamically selects the set of predictions including generating ranked predictions by ordering the plurality of predictions according to descending probability.

23. The media of claim 22, wherein execution of the instructions dynamically selects the set of predictions by determining a sequential pair of predictions in the ranked predictions between which the difference in corresponding probabilities is greatest relative to any other sequential pair of predictions, wherein the sequential pair of predictions include a first prediction and a second prediction, the first prediction having a higher probability than the second prediction.