END-TO-END LEARNING OF DIALOGUE AGENTS FOR INFORMATION ACCESS

• US 20180060301A1
• Filed: 01/13/2017
• Published: 03/01/2018
• Est. Priority Date: 08/31/2016
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Li, Lihong, Dhingra, Bhuwan, Gao, Jianfeng, Li, Xiujun, Chen, Yun-Nung, Deng, Li, Ahmed, Faisal
• Granted Patent: US 10,546,066 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 16/3329 Natural language query formulation or dialogue systems
• CPC Class Codes: G06F 40/289 Phrasal analysis, e.g. finite state techniques or chunking
• CPC Class Codes: G06F 40/35 Discourse or dialogue representation
• CPC Class Codes: G06N 3/02 Neural networks
• CPC Class Codes: G06N 3/044 Recurrent networks, e.g. Hopfield networks
• CPC Class Codes: G06N 3/08 Learning methods
• CPC Class Codes: G06N 5/022 Knowledge engineering; Knowledge acquisition
• CPC Class Codes: G06N 7/01 Probabilistic graphical models, e.g. probabilistic networks

Abstract

Described herein are systems, methods, and techniques by which a processing unit can build an end-to-end dialogue agent model for end-to-end learning of dialogue agents for information access and apply the end-to-end dialogue agent model with soft attention over knowledge base entries to make the dialogue system differentiable. In various examples the processing unit can apply the end-to-end dialogue agent model to a source of input, fill slots for output from the knowledge base entries, induce a posterior distribution over the entities in a knowledge base or induce a posterior distribution of a target of the requesting user over entities from a knowledge base, develop an end-to-end differentiable model of a dialogue agent, use supervised and/or imitation learning to initialize network parameters, calculate a modified version of an episodic algorithm, e.g., the REINFORCE algorithm, for training an end-to-end differentiable model based on user feedback.

Claims

1. A system comprising:

one or more processing unit(s);

one or more computer-readable media coupled to one or more of the processing unit(s), the one or more computer readable media having thereon one or more modules of computer-executable instructions to configure a computer to perform operations comprising;

building an end-to-end dialogue agent model for end-to-end learning of dialogue agents for information access;

applying the end-to-end dialogue agent model with soft attention over knowledge base entries to make the dialogue system differentiable; and

applying the end-to-end dialogue agent model to a source of input.

2. A system as claim 1 recites, further comprising a probabilistic framework for inducing a posterior distribution of a target of the requesting user over entities from a knowledge base.

3. A system as claim 1 recites, the one or more computer readable media having thereon one or more modules of computer-executable instructions to configure the computer to perform operations further comprising inducing a posterior distribution over the entities in a knowledge base.

4. A system as claim 1 recites, the one or more computer readable media having thereon one or more modules of computer-executable instructions to configure the computer to perform operations further comprising developing, an end-to-end differentiable model of a multi-turn information providing dialogue agent.

5. A system as claim 4 recites, wherein parameters of the end-to-end differentiable model of a multi-turn information providing dialogue agent can be tuned using gradient descent methods.

6. A system as claim 1 recites, the one or more computer readable media having thereon one or more modules of computer-executable instructions to configure the computer to perform operations further comprising using supervised learning to initialize network parameters.

7. A system as claim 1 recites, the one or more computer readable media having thereon one or more modules of computer-executable instructions to configure the computer to perform operations further comprising calculating a modified version of an episodic algorithm to update rules for training an end-to-end differentiable model based on user feedback.

8. A system as claim 7 recites, wherein the modified version of the episodic algorithm can allow the agent to explore at least one of:

a set of possible dialogue acts at each turn or a set of possible entity results from the knowledge base at the final turn.

9. A system as claim 1 recites, wherein a neural (NN) architecture is configured to read from the knowledge base entries for the contextual language understanding.

10. A method comprising:

building an end-to-end dialogue agent model for end-to-end learning of dialogue agents for information access;

applying the end-to-end dialogue agent with soft attention over knowledge base entries to make the dialogue system differentiable; and

filling slots for output from the knowledge base entries.

11. A method as claim 10 recites, wherein a neural (NN) architecture reads from the knowledge base entries.

12. A method as claim 10 recites, further comprising inducing a posterior distribution of a target of the requesting user over entities from a knowledge base.

13. A method as claim 10 recites, further comprising inducing a posterior distribution over the entities in a knowledge base.

14. A method as claim 10 recites, further comprising developing an end-to-end differentiable model of a multi-turn information providing dialogue agent.

15. A method as claim 10 recites, wherein parameters of the end-to-end differentiable model of a multi-turn information providing dialogue agent can be tuned using stochastic gradient descent methods.

16. A method as claim 10 recites, further comprising using supervised learning to initialize network parameters.

17. A method as claim 10 recites, further comprising calculating a modified version of an episodic algorithm to update rules for training an end-to-end differentiable model based on user feedback.

18. A method as claim 17 recites, wherein the modified version of the episodic algorithm can allow the agent to explore at least one of:

a set of possible dialogue acts at each turn or a set of possible entity results from the KB at the final turn

19. A system comprising:

one or more processing unit(s);

one or more computer-readable media coupled to one or more of the processing unit(s), the one or more computer readable media including;

a neural network (NN) architecture that reads from a knowledge base;

a training engine configured to generate a model, end-to-end trained using supervisory and/or reinforcement signals, by at least one of;

inducing a posterior distribution over the entities in the knowledge base;

developing an end-to-end differentiable model of a multi-turn information providing dialogue agent;

using supervised and/or imitation learning to initialize network parameters, e.g., to good values, reasonably good values, etc.;

or

calculating a modified version of an episodic algorithm to update rules for training an end-to-end differentiable model based on user feedback; and

an operation engine configured to operate the model including a soft attention vector to perform end-to-end semantic parsing of inputs.

20. A system as claim 19 recites, wherein parameters of the end-to-end differentiable model of a multi-turn information providing dialogue agent can be tuned using gradient descent methods.