Reactive learning for efficient dialog tree expansion
First Claim
1. A method for generating dialogs and learning a dialog policy for a dialog system, comprising:
- for each of at least one scenario, in which annotators in a pool of annotators serve as virtual agents and users, generating a respective dialog tree in which each path through the tree corresponds to a dialog and nodes of the tree correspond to turn of a dialog, the generation comprising, with a processor;
a) computing a measure of uncertainty for nodes in the dialog tree, comprising;
for each of a plurality of nodes, computing a conflict coefficient Ci which quantifies the diversity of its child-node set, as a function of;
4 Assignments
0 Petitions
Accused Products
Abstract
A method for generating dialogs for learning a dialog policy includes, for each of at least one scenario, in which annotators in a pool of annotators serve as virtual agents and users, generating a respective dialog tree in which each path through the tree corresponds to a dialog and nodes of the tree correspond to dialog acts provided by the annotators. The generation includes computing a measure of uncertainty for nodes in the dialog tree, identifying a next node to be annotated, based on the measure of uncertainty, selecting an annotator from the pool to provide an annotation for the next node, receiving an annotation from the selected annotator for the next node, and generating a new node of the dialog tree based on the received annotation. A corpus of dialogs is generated from the dialog tree.
-
Citations
18 Claims
-
1. A method for generating dialogs and learning a dialog policy for a dialog system, comprising:
for each of at least one scenario, in which annotators in a pool of annotators serve as virtual agents and users, generating a respective dialog tree in which each path through the tree corresponds to a dialog and nodes of the tree correspond to turn of a dialog, the generation comprising, with a processor; a) computing a measure of uncertainty for nodes in the dialog tree, comprising; for each of a plurality of nodes, computing a conflict coefficient Ci which quantifies the diversity of its child-node set, as a function of; - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13)
-
14. In combination, a system for generating dialogs for learning a dialog policy and a computer-implemented dialog system, the system for generating dialogs comprising:
-
memory which stores a dialog tree for each of a plurality of scenarios, wherein paths through the tree correspond to respective dialogs and nodes of the tree each represent a turn of a dialog, whereby some of the nodes correspond to user annotations and others of the nodes correspond to agent annotations; a tree update component for updating the dialog trees based on annotations of annotators in a pool of annotators serving as virtual agents and users; a reactive tree expansion component which progressively expands the dialog trees by repeated selection of a next node to be annotated by one of the annotators in the pool, the next node being selected based on a respective computed measure of uncertainty for each of the current nodes in one of the dialog trees, whereby when the next node corresponds to a user annotation, a text annotation is provided by the selected annotator for the next node, and when the next node corresponds to an agent annotation, a dialog act is selected by the selected annotator for the node; a dialog corpus generator which generates a corpus of dialogs from the expanded dialog trees; a dialog policy learning component which learns a dialog policy based on the corpus of dialogs, the learning of the dialog policy including learning a classifier model which predicts a next action for a current state of a dialog; and a processor which implements the tree update component, reactive tree expansion component, and dialog corpus generator; the dialog system being configured for conducting a dialog between a virtual agent and a user, in which the learned dialog policy predicts, based on a state of the dialog, a next action to perform, the action being converted, by the dialog system, to a next utterance of the virtual agent. - View Dependent Claims (15, 16)
-
-
17. In a dialog system which conducts dialogs between a human user of the dialog system and a virtual agent, a dialog policy learned by a method comprising:
-
storing a respective dialog tree in memory for each of a plurality of scenarios, wherein paths through the tree correspond to respective dialogs and nodes of the tree each represent a turn of a dialog, some of the nodes corresponding to user annotations and others of the nodes corresponding to agent annotations; progressively expanding the dialog trees by repeated selection of a next node to be annotated by one of a pool of annotators, the next node being selected based on a respective computed measure of uncertainty for each of the nodes currently in the dialog trees, and updating the dialog trees based on the annotation of the one annotators in the pool of annotators, whereby when the next node corresponds to a user annotation, the selected annotator is requested to provide a text annotation for the next node, and when the next node corresponds to an agent annotation, the selected annotator is requested to select a dialog act for the node; generating a corpus of dialogs from the expanded dialog trees; and learning a dialog policy based on the corpus; the dialog system being configured for conducting a dialog between a virtual agent and a user, in which the learned dialog policy predicts, based on a state of the dialog, a next action to perform, the action being converted, by the dialog system, to a next utterance of the virtual agent.
-
-
18. A method for learning a dialog policy for a dialog system comprising:
-
for each of at least one scenario, in which annotators in a pool of annotators serve as both virtual agents and users, generating a respective dialog tree in which each path through the tree corresponds to a dialog and nodes of the tree correspond to turn of a dialog, the generation comprising; a) computing a measure of uncertainty for nodes in the dialog tree, b) identifying a next node to be annotated, based on the measure of uncertainty, c) selecting an annotator from the pool to provide an annotation for the next node, d) receiving an annotation from the selected annotator for the next node, wherein when the next node corresponds to a user annotation, the received annotation is a text annotation for the next node, and when the next node corresponds to an agent annotation, the received annotation is a dialog act for the next node, e) generating a new node of the dialog tree based on the received annotation, and f) repeating a)-e) a plurality of times with different annotators selected from the pool; generating a corpus of dialogs from the dialog tree; and based on the corpus of dialogs, learning a classifier model of a dialog policy that predicts a next action for the dialog system, based on a state of a dialog; and incorporating the learned dialog policy into a dialog system for conducting a dialog between a virtual agent and a user, in which the learned dialog policy predicts, based on a state of the dialog, a next action to perform, the action being converted, by the dialog system, to a next utterance of the virtual agent.
-
Specification