DIALOGUE EVALUATION VIA MULTIPLE HYPOTHESIS RANKING
First Claim
1. A method of evaluating a dialogue with a user on a device having a processor, the method comprising:
- executing on the processor instructions causing the device to;
generate a dialogue hypothesis set comprising at least two dialogue hypotheses respectively having a hypothesis probability;
rank the dialogue hypothesis set according to the hypothesis probabilities;
after the ranking, upon identifying a low-ranking dialogue hypothesis having a hypothesis probability below a hypothesis retention threshold, discard the low-ranking dialogue hypothesis;
after the discarding, using a knowledge source, adjust the hypothesis probabilities of the respective dialogue hypotheses;
after the adjusting, re-rank the dialogue hypothesis set according to the hypothesis probabilities; and
for a high-ranking dialogue hypothesis having a hypothesis probability exceeding a hypothesis confidence threshold, execute an action fulfilling the high-ranking dialogue hypothesis.
3 Assignments
0 Petitions
Accused Products
Abstract
In language evaluation systems, user expressions are often evaluated by speech recognizers and language parsers, and among several possible translations, a highest-probability translation is selected and added to a dialogue sequence. However, such systems may exhibit inadequacies by discarding alternative translations that may initially exhibit a lower probability, but that may have a higher probability when evaluated in the full context of the dialogue, including subsequent expressions. Presented herein are techniques for communicating with a user by formulating a dialogue hypothesis set identifying hypothesis probabilities for a set of dialogue hypotheses, using generative and/or discriminative models, and repeatedly re-ranks the dialogue hypotheses based on subsequent expressions. Additionally, knowledge sources may inform a model-based with a pre-knowledge fetch that facilitates pruning of the hypothesis search space at an early stage, thereby enhancing the accuracy of language parsing while also reducing the latency of the expression evaluation and economizing computing resources.
-
Citations
20 Claims
-
1. A method of evaluating a dialogue with a user on a device having a processor, the method comprising:
executing on the processor instructions causing the device to; generate a dialogue hypothesis set comprising at least two dialogue hypotheses respectively having a hypothesis probability; rank the dialogue hypothesis set according to the hypothesis probabilities; after the ranking, upon identifying a low-ranking dialogue hypothesis having a hypothesis probability below a hypothesis retention threshold, discard the low-ranking dialogue hypothesis; after the discarding, using a knowledge source, adjust the hypothesis probabilities of the respective dialogue hypotheses; after the adjusting, re-rank the dialogue hypothesis set according to the hypothesis probabilities; and for a high-ranking dialogue hypothesis having a hypothesis probability exceeding a hypothesis confidence threshold, execute an action fulfilling the high-ranking dialogue hypothesis. - View Dependent Claims (2, 3, 4, 5, 6)
-
7. A computer-readable memory device storing instructions that, when executed on a processor of a device, cause the device to evaluate a dialogue with a user, by:
-
generating a dialogue hypothesis set comprising at least two dialogue hypotheses respectively having a hypothesis probability; ranking the dialogue hypothesis set according to the hypothesis probabilities; after the ranking, upon identifying a low-ranking dialogue hypothesis having a hypothesis probability below a hypothesis retention threshold, discarding the low-ranking dialogue hypothesis; after the discarding, using a knowledge source, adjust the hypothesis probabilities of the respective dialogue hypotheses; after the adjusting, re-rank the dialogue hypothesis set according to the hypothesis probabilities; and for a high-ranking dialogue hypothesis having a hypothesis probability exceeding a hypothesis confidence threshold, execute an action fulfilling the high-ranking dialogue hypothesis. - View Dependent Claims (8, 9, 10, 11)
-
-
12. A system for evaluating a dialogue with a user on a device having a processor and a memory, the system comprising:
-
a dialogue hypothesis set; an expression evaluator comprising instructions stored in the memory that, when executed on the processor, cause the device to, for respective expressions of the dialogue; apply an expression recognizer and a natural language processor to; store in the dialogue hypothesis set at least one dialogue hypothesis of the expression, the at least one dialogue hypothesis respectively comprising; at least one slot associated with a subject of the expression, and a hypothesis probability; and for respective previous dialogue hypotheses in the dialogue hypothesis set that were generated for a previous expression of the dialogue, update the subject of the at least one slot of the previous dialogue hypothesis; and using a knowledge source, adjust the hypothesis probabilities of the respective dialogue hypotheses; and a dialogue hypothesis comparator comprising instructions stored in the memory that, when executed on the processor, cause the device to; rank the dialogue hypothesis set according to the hypothesis probabilities; and for a high-ranking dialogue hypothesis having a hypothesis probability exceeding a hypothesis confidence threshold, execute an action fulfilling the high-ranking dialogue hypothesis. - View Dependent Claims (13, 14, 15, 16, 17, 18, 19, 20)
-
Specification