Contextual language understanding for multi-turn language tasks

• US 10,007,660 B2
• Filed: 05/26/2017
• Issued: 06/26/2018
• Est. Priority Date: 12/01/2014
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Sarikaya, Ruhi, Xu, Puyang, Rochette, Alexandre, Celikyilmaz, Asli
• Primary Examiner(s): Lerner, Martin
• Application Number:
  US15/606,652
• Publication Number:
  US 20170262432A1
• Time in Patent Office: 396 Days
• Field of Search: 704 9, 704257, 704275, 704255
• CPC Class Codes: G06F 16/90332 Natural language query formulation or dialogue systems
• CPC Class Codes: G06F 40/30 Semantic analysis
• CPC Class Codes: G06F 40/35 Discourse or dialogue representation

Abstract

Methods and systems are provided for contextual language understanding. A natural language expression may be received at a single-turn model and a multi-turn model for determining an intent of a user. For example, the single-turn model may determine a first prediction of at least one of a domain classification, intent classification, and slot type of the natural language expression. The multi-turn model may determine a second prediction of at least one of a domain classification, intent classification, and slot type of the natural language expression. The first prediction and the second prediction may be combined to produce a final prediction relative to the intent of the natural language expression. An action may be performed based on the final prediction of the natural language expression.

Claims

1. A system, comprising:

at least one processor; and

a memory encoding computer executable instructions that, when executed by the at least one processor, perform a method, comprising;

receiving a natural language expression;

determining, using a single-turn model, a first weighted prediction of at least one of a domain classification, intent classification, and slot type of the natural language expression;

determining, using a multi-turn model, a second weighted prediction of at least one of a domain classification, intent classification, and slot type of the natural language expression;

generating a final prediction based, at least in part, on the first weighted prediction and the second weighted prediction; and

performing an action based on the final prediction.

2. The system of claim 1, wherein the natural language expression is spoken language input.

3. The system of claim 1, wherein the natural langue expression is textual input.

4. The system of claim 1, wherein the natural language expression is a combination of textual input and spoken language input.

5. The system of claim 1, further comprising instructions for:

classifying the natural language expression into a supported domain of the single-turn model;

classifying the natural language expression into a supported intent of the single-turn model;

extracting at least one semantic word from the natural language expression; and

filling at least one supported slot type of the single-turn model with the at least one semantic word.

6. The system of claim 1, further comprising determining contextual information associated with the natural language expression.

7. The system of claim 1, further comprising determining contextual information associated with the natural language expression using knowledge content.

8. The system of claim 1, further comprising determining contextual information associated with the natural language expression using client context.

9. The system of claim 1, further comprising instructions for calculating a first score indicative of a probability that the first weighted prediction is correct.

10. The system of claim 1, further comprising instructions for calculating a second score indicative of a probability that the second weighted prediction is correct.

11. A method, comprising:

receiving a first natural language expression;

receiving a second natural language expression;

generating a first prediction by predicting at least one of a first domain classification, a first intent classification, and a first slot type of each of the first natural language expression and the second natural language expression using a single-turn model;

generating a second prediction by predicting at least one of a second domain classification, a second intent classification, and a second slot type of each of the first natural language expression and the second natural language expression using a multi-turn model;

combining the first prediction and the second prediction to produce a final prediction; and

performing an action based, at least in part, on the final prediction.

12. The method of claim 11, wherein the final prediction is based, at least in part, on a determined intent of the second natural language expression.

13. The method of claim 11, wherein performing an action comprises responding to the second natural language expression.

14. The method of claim 13, wherein responding to the second natural language expression comprises generating an answer to the second natural language expression.

15. The method of claim 13, wherein responding to the second natural language expression comprises generating a follow-up question to the second natural language expression.

16. The method of claim 13, wherein responding to the second natural language expression comprises performing a task based, at least in part, on the second natural language expression.

17. The method of claim 11, further comprising determining contextual information for one or more of the first natural language expression and the second natural language expression.

18. The method of claim 17, further comprising comprises evaluating the first natural language expression and the second natural language expression using the determined contextual information.

19. A method, comprising:

receiving a language input;

providing the language input to a single-turn model to determine a first weighted prediction of one or more of a domain classification and an intent classification associated with the language input;

providing the language input to a multi-turn model to determine a second weighted prediction of one or more of the domain classification and the intent classification;

generating a final prediction based on the first weighted prediction and the second weighted prediction; and

performing an action based on the final prediction.

20. The method of claim 19, wherein the action is a response to the language input.