GENERATIVE DISCRIMINATIVE APPROACH FOR TRANSACTIONAL DIALOG STATE TRACKING VIA COLLECTIVE MATRIX FACTORIZATION

• US 20170091171A1
• Filed: 09/24/2015
• Published: 03/30/2017
• Est. Priority Date: 09/24/2015
• Status: Active Grant

• Current Assignee: Conduent Business Services, LLC (Conduent, Inc.)
• Original Assignee: Xerox Corporation (Xerox Holdings Corp.)
• Inventors: Perez, Julien
• Granted Patent: US 9,811,519 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 40/35 Discourse or dialogue representation
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue

Abstract

A computer-implemented method for dialog state tracking employs first and second latent variable models which have been learned by reconstructing a decompositional model generated from annotated training dialogues. The decompositional model includes, for each of a plurality of dialog state transitions corresponding to a respective turn of one of the training dialogues, state descriptors for initial and final states of the transition and a respective representation of the dialogue for that turn. The first latent variable model includes embeddings of the plurality of state transitions, and the second latent variable model includes embeddings of features of the state descriptors and embeddings of features of the dialogue representations. Data for a new dialog state transition is received, including a state descriptor for the initial time and a respective dialogue representation. A state descriptor for the final state of the new dialog state transition is predicted using the learned latent variable models.

Claims

1. A method for dialog state tracking comprising:

providing first and second latent variable models learned by reconstructing a decompositional model, the decompositional model having been generated from annotated training dialogues and including, for each of a plurality of dialog state transitions, state descriptors for initial and final states of the transition and a respective dialogue representation, the learned first latent variable model including embeddings of the plurality of state transitions, and the learned second latent variable model including embeddings of features of the state descriptors and embeddings of features of the dialogue representations;

receiving data for a new dialog state transition, the data including a state descriptor for the initial time and a respective dialogue representation;

predicting a state descriptor for the final state of the new dialog state transition using the learned first and second latent variable models,

wherein the predicting of the state descriptor is performed with a processor.

2. The method of claim 1, further comprising learning the first and second latent variable models.

3. The method of claim 2, wherein the first and second latent variable models are jointly learned.

4. The method of claim 2, wherein the learning is performed by collective matrix factorization.

5. The method of claim 2, wherein the learning of the first and second latent variable models is iteratively performed to optimize a loss function.

6. The method of claim 5, wherein the loss function is a function of a minimum of the difference between the decompositional model and a product of the latent variable models, conditioned by weights that increase the weight some of the state descriptor features, and optionally a function of regularization terms which are each a function of one of the latent variable models.

7. The method of claim 6, wherein the optimizing the loss function comprises:

repeating first and second steps,

wherein in the first step, with the embeddings in the second latent variable model fixed, computing the embeddings of the first latent variable model which minimize a function of the decompositional model, the weights, and the second latent variable model, and

wherein in the second step, with the embeddings in the first latent variable model fixed, computing the embeddings of the second latent variable model which minimize a function of the decompositional model and the first latent variable model, without the weights.

8. The method of claim 6, wherein the weights that increase the weight some of the state descriptor features increase the weight of some of the features of the final state descriptor more than the corresponding weights of features of the initial state descriptor.

9. The method of claim 1, wherein the predicting of the state descriptor for the final state of the new dialog state transition comprises computing an embedding of the state descriptor for the initial time and the respective dialogue representation and multiplying the embedding by the embeddings of the second latent variable model corresponding to the embeddings of the state descriptor for the final time.

10. The method of claim 1, wherein the state descriptors at the initial time and final time each represent a distribution over each of a plurality of variables of a transaction.

11. The method of claim 1, wherein the dialogue representation comprises a bag of words representation derived, at least in part, from a user utterance.

12. The method of claim 1, wherein the bag of words representation is also derived from an automatically-generated agent dialog act.

13. The method of claim 1, further comprising repeating the receiving of data for a new dialog state transition and the predicting of a state descriptor for the final state of the new dialog state transition for each of a plurality of turns of a dialogue.

14. The method of claim 1, further comprising generating an agent dialog act based on the predicted state descriptor for the final state.

15. The method of claim 14, when the agent dialog act is output in a form which is audible to a user on a client device.

16. A computer program product comprising a non-transitory recording medium storing instructions, which when executed on a computer, causes the computer to perform the method of claim 1.

17. A system comprising memory which stores instructions for performing the method of claim 1 and a processor in communication with the memory which executes the instructions.

18. A system for dialog state tracking comprising:

memory which stores first and second latent variable models that have been learned by reconstructing a decompositional model, the decompositional model having been generated from annotated training dialogues and including, for each of a plurality of dialog state transitions, state descriptors for initial and final states of the transition and a respective dialogue representation, the learned first latent variable model including embeddings of the plurality of state transitions, and the learned second latent variable model including embeddings of features of the state descriptors and embeddings of features of the dialogue representations;

an information gathering component which receives an utterance of a user for each of a plurality of new dialog state transitions;

a representation generation component which generates a dialog representation based on the user utterance;

a prediction component which predicts a state descriptor for a final state of each new dialog state transition using the learned first and second latent variable models, an initial state descriptor and the respective dialog representation; and

a processor which implements the information gathering component, representation generation component, and the prediction component.

19. The system of claim 18, further comprising a learning component for learning the first and second latent variable models.

20. A method for identifying a transaction comprising:

learning first and second latent variable models for reconstructing a decompositional model, the decompositional model having been generated from annotated training dialogues and including, for each of a plurality of dialog state transitions, state descriptors for initial and final states of the transition and a respective dialogue representation, the learned first latent variable model including embeddings of the plurality of state transitions, and the learned second latent variable model including embeddings of features of the state descriptors and embeddings of features of the dialogue representations;

for each of a plurality of turns of a dialogue;

receiving data for a new dialog state transition, the data including a state descriptor for the initial time and a respective dialogue representation;

predicting a state descriptor for the final state of the new dialog state transition using the learned first and second latent variable models; and

generating an agent dialog act based on the predicted state descriptor;

based on the predicted final state of at least one of the turns of the dialogue, identifying a transaction to be implemented.