Automated assistant for user interaction via speech

• US 10,276,160 B2
• Filed: 11/10/2016
• Issued: 04/30/2019
• Est. Priority Date: 11/12/2015
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Semantic Machines, Inc. (Microsoft Corporation)
• Inventors: Andreas, Jacob Daniel, Berg-Kirkpatrick, Taylor Darwin, Chen, Pengyu, Cohen, Jordan Rian, Gillick, Laurence Steven, Hall, David Leo Wright, Klein, Daniel, Newman, Michael, Pauls, Adam David, Roth, Daniel Lawrence, Rusak, Jesse Daniel Eskes, Volpe, Andrew Robert, Wegmann, Steven Andrew
• Primary Examiner(s): Pullias, Jesse S
• Application Number:
  US15/348,228
• Publication Number:
  US 20170140755A1
• Time in Patent Office: 901 Days
• Field of Search: 704231-257, 704270-275, 704 1, 704 9, 704 10
• CPC Class Codes: G06F 40/169 Annotation, e.g. comment data or footnotes
• CPC Class Codes: G06F 40/186 Templates
• CPC Class Codes: G06F 40/35 Discourse or dialogue representation
• CPC Class Codes: G06N 3/006 based on simulated virtual individual or collective life forms, e.g. social simulations or particle swarm optimisation [PSO]
• CPC Class Codes: G06N 3/08 Learning methods
• CPC Class Codes: G10L 15/063 Training
• CPC Class Codes: G10L 15/1822 Parsing for meaning understanding
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command
• CPC Class Codes: G10L 25/51 for comparison or discrimination

Abstract

An interaction assistant conducts multiple turn interaction dialogs with a user in which context is maintained between turns, and the system manages the dialog to achieve an inferred goal for the user. The system includes a linguistic interface to a user and a parser for processing linguistic events from the user. A dialog manager of the system is configured to receive alternative outputs from the parser, and selecting an action and causing the action to be performed based on the received alternative outputs. The system further includes a dialog state for an interaction with the user, and the alternative outputs represent alternative transitions from a current dialog state to a next dialog state. The system further includes a storage for a plurality of templates, and wherein each dialog state is defined in terms of an interrelationship of one or more instances of the templates.

Claims

1. An interaction system comprising:

an integration section including a first integration component for providing a linguistic interface to a user;

an event processing section including a parser for processing linguistic events from the first integration component; and

a dialog manager configured to receive alternative outputs from the event processing section, and selecting an action and causing the action to be performed based on the received alternative outputs;

wherein the system further includes a storage for a dialog state for an interaction with the user, and wherein the alternative outputs from the event processing section represent alternative transitions from a current dialog state to a next dialog state; and

wherein the system further includes a storage for a plurality of templates, and wherein each dialog state is defined in terms of an interrelationship of one or more instances of the templates.

2. The system of claim 1 wherein the parser is configured to process a linguistic input according to a current dialog state and the plurality of templates to determine the alternative outputs.

3. The system of claim 2 wherein each template defines a plurality of input fields and an output computable from the input fields.

4. The system of claim 3 wherein each template specifies a plurality of actions.

5. The system of claim 2 wherein the parser is further configured to determine each alternative output as a modification of the current dialog state.

6. The system of claim 5 wherein the modification of the current dialog state represents at least one of (a) a change in interrelationship of instances of templates of the current dialog state, (b) addition or removal of an instance of a template, and (c) setting of an input field value of an instance of a template.

7. The system of claim 2 wherein at least some of the alternative outputs are each associated with one or more actions, and the dialog manager is configured to select actions according to the alternative outputs.

8. The system of claim 1 wherein the first integration component includes an automatic speech recognizer configured to receive speech input from the user, and to determine text-based linguistic events for the parser.

9. The system of claim 1 wherein the integration section further includes an application integration component for providing an interface to external applications, and for providing application related events to the event processing section.

10. The system of claim 9 wherein the application related events include at least one of a user interface event, a communication event, and a data access event.

11. The system of claim 1 further includes a storage for salient information, the system being configured to update the salient information according to processing of event received by the event processing section, wherein the parser is configured to process a linguistic input using salience information determined from prior events processed by the event processing section.

12. The system of claim 1 wherein the dialog manager includes a machine learning component for selecting an action according to the alternative outputs.

13. An interaction system comprising:

an integration section including an integration component for providing a linguistic interface to a user;

an event processing section including a parser for processing linguistic events from the integration component; and

a dialog manager configured to receive alternative outputs from the event processing section, and selecting an action and causing the action to be performed based on the received alternative outputs;

wherein the system further includes a storage for a dialog state for an interaction with the user, and wherein the alternative outputs from the event processing section represent alternative transitions from a current dialog state to a next dialog state; and

wherein the system further includes a storage for a plurality of templates, and wherein each dialog state is defined by a graph of templates including;

a root template having one or more inputs, and one or more child templates, each child template having an output connected via a directed link to an input of the root template.

14. The system of claim 13, wherein the parser is configured to process a linguistic input according to a current dialog state and the plurality of templates to determine the alternative outputs.

15. The system of claim 14, wherein the parser is further configured to determine each alternative output as a modification of the current dialog state.

16. The system of claim 15 wherein the modification of the current dialog state represents at least one of (a) a change in interrelationship of instances of templates of the current dialog state, (b) addition or removal of an instance of a template, and (c) setting of an input field value of an instance of a template.

17. A method for interacting with a user via dialog, the method comprising:

operating an integration component to provide a linguistic interface to the user;

operating a parser to process linguistic events from the integration component;

operating a dialog manager configured to;

receive alternative outputs from the parser, and

select an action and cause the action to be performed based on the received alternative outputs;

maintaining a plurality of templates and a dialog state for an interaction with the user, wherein the dialog state is defined by a graph of templates including;

a root template having one or more inputs, and one or more child templates, each child template having an output connected via a directed link to an input of the root template; and

transforming a current dialog state to a next dialog state by selecting an alternative transition from the current dialog state to the next dialog state, based on the alternative outputs from the parser.

18. The system of claim 17, wherein the parser is configured to process a linguistic input according to a current dialog state and the plurality of templates to determine the alternative outputs.

19. The system of claim 18, wherein the parser is further configured to determine each alternative output as a modification of the current dialog state.

20. The system of claim 19 wherein the modification of the current dialog state represents at least one of (a) a change in interrelationship of instances of templates of the current dialog state, (b) addition or removal of an instance of a template, and (c) setting of an input field value of an instance of a template.