Discriminating ambiguous expressions to enhance user experience

• US 9,836,452 B2
• Filed: 12/30/2014
• Issued: 12/05/2017
• Est. Priority Date: 12/30/2014
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Robichaud, Jean-Philippe, Sarikaya, Ruhi
• Primary Examiner(s): ZHU, RICHARD Z
• Application Number:
  US14/586,395
• Publication Number:
  US 20160188565A1
• Time in Patent Office: 1,071 Days
• CPC Class Codes: G06F 16/3325 Reformulation based on results of preceding query
• CPC Class Codes: G06F 16/3329 Natural language query formulation or dialogue systems
• CPC Class Codes: G06F 16/3331 Query processing
• CPC Class Codes: G06F 16/3334 Selection or weighting of terms from queries, including natural language queries
• CPC Class Codes: G06F 16/3335 Syntactic pre-processing, e.g. stopword elimination, stemming
• CPC Class Codes: G06F 16/3344 using natural language analysis
• CPC Class Codes: G06F 16/3349 Reuse of stored results of previous queries
• CPC Class Codes: G06F 40/295 Named entity recognition
• CPC Class Codes: G06F 40/30 Semantic analysis
• CPC Class Codes: G06F 40/35 Discourse or dialogue representation
• CPC Class Codes: G10L 15/1815 Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning
• CPC Class Codes: G10L 15/1822 Parsing for meaning understanding
• CPC Class Codes: G10L 15/183 using context dependencies, e.g. language models
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command

Abstract

Methods and systems are provided for discriminating ambiguous expressions to enhance user experience. For example, a natural language expression may be received by a speech recognition component. The natural language expression may include at least one of words, terms, and phrases of text. A dialog hypothesis set from the natural language expression may be created by using contextual information. In some cases, the dialog hypothesis set has at least two dialog hypotheses. A plurality of dialog responses may be generated for the dialog hypothesis set. The dialog hypothesis set may be ranked based on an analysis of the plurality of the dialog responses. An action may be performed based on ranking the dialog hypothesis set.

Claims

1. A system comprising:

at least one processor; and

memory encoding computer executable instructions that, when executed by at least one processor, perform a method for discriminating ambiguous requests comprising;

receiving a natural language expression, wherein the natural language expression includes at least one of words, terms, and phrases of text;

creating a dialog hypothesis set from the natural language expression by using contextual information, wherein the dialog hypothesis set has a first dialog hypothesis corresponding to a first domain and a second dialog hypothesis corresponding to a second domain;

generating, from a first domain engine component and a second domain engine component, a plurality of dialog responses for the dialog hypothesis set;

ranking by machine learning techniques the first domain engine component and the second domain engine component based on an analysis of the plurality of the dialog responses; and

performing an action with the highest ranked domain engine component.

2. The system of claim 1, wherein the natural language expression is at least one of a spoken language input and a textual input.

3. The system of claim 1, wherein the contextual information includes at least one of information extracted from a previously received natural language expression, a response to a previously received natural language expression, client context, and knowledge content.

4. The system of claim 3, wherein the information extracted from the previously received natural language expression includes at least a domain prediction, an intent prediction, and a slot type.

5. The system of claim 1, wherein creating the dialog hypothesis set comprises:

extracting at least one feature from the natural language expression; and

generating at least two dialog hypotheses, where each dialog hypothesis of the dialog hypothesis set includes a different natural language expression having at least one extracted feature.

6. The system of claim 1, wherein generating a plurality of dialog responses for the dialog hypothesis set comprises generating a plurality of responses for each dialog hypothesis of the dialog hypothesis set.

7. The system of claim 1, wherein generating a plurality of dialog responses for the dialog hypothesis set comprises at least one of sending the dialog hypotheses to a web backend engine and sending the dialog hypotheses to a domain specific component.

8. The system of claim 1, wherein the ranking further comprises:

extracting features from the at least two dialog hypotheses in the dialog hypothesis set; and

calculating a score for the extracted features, wherein the calculated score is indicative of the dialog hypothesis rank within the dialog hypothesis set.

9. The system of claim 1, wherein the ranking further comprises comparing the plurality of the dialog responses with a plurality of logged dialog responses.

10. The system of claim 1, wherein performing an action based on ranking the dialog hypothesis set comprises:

using a highest ranked dialog hypothesis to query a web backend engine for results; and

sending the results to a user of a client computing device.

11. One or more computer-readable storage media, having computer-executable instructions that, when executed by at least one processor, perform a method for training a dialog component to discriminate ambiguous requests, the method comprising:

receiving a natural language expression, wherein the natural language expression includes at least one of words, terms, and phrases of text;

creating a dialog hypothesis set from the natural language expression by using contextual information, wherein the dialog hypothesis set has a first dialog hypothesis corresponding to a first domain and a second hypothesis corresponding to a second domain;

generating, from a first domain engine component and a second domain engine component, a plurality of dialog responses for the dialog hypothesis set;

ranking by machine learning techniques the first domain engine component and the second domain engine component based on an analysis of the plurality of the dialog responses; and

performing an action with the highest ranked domain engine component.

12. The computer-readable storage media of claim 11, wherein the method further comprises comparing the plurality of dialog responses with a plurality of logged dialog responses, wherein plurality of logged dialog responses includes a plurality of responses generated from the natural language expression.

13. The computer-readable storage media of claim 11, wherein creating the dialog hypothesis set comprises:

extracting at least one feature from the natural language expression; and

generating at least two dialog hypotheses, where each dialog hypothesis of the dialog hypothesis set includes a different natural language expression having at least one extracted feature.

14. The computer-readable storage media of claim 12, wherein the method further comprises:

determining whether at least one of the plurality of dialog responses matches at least one of the logged dialog responses; and

labeling at least one of the two dialog hypotheses in the dialog hypothesis set corresponding to the at least one dialog response that matches the at least one logged dialog response.

15. A computer-implemented method comprising:

receiving a natural language expression, wherein the natural language expression includes at least one of words, terms, and phrases of text;

creating a dialog hypothesis set from the natural language expression by using contextual information, wherein the dialog hypothesis set has a first dialog hypothesis corresponding to a first domain and a second dialog hypothesis corresponding to a second domain;

generating, from a first domain engine component and a second domain engine component, a plurality of dialog responses for the dialog hypothesis set;

ranking, by machine learning techniques, the first domain engine component and the second domain engine component based on an analysis of the plurality of the dialog responses; and

performing an action with the highest ranked domain engine component.

16. The computer-implemented method of claim 15, wherein the natural language expression is at least one of a spoken language input and a textual input.

17. The computer-implemented method of claim 15, wherein the contextual information includes at least one of information extracted from a previously received natural language expression, a response to a previously received natural language expression, client context, and knowledge content.

18. The computer-implemented method of claim 17, wherein the information extracted from the previously received natural language expression includes at least a domain prediction, an intent prediction, and a slot type.

19. The computer-implemented method of claim 15, wherein creating the dialog hypothesis set comprises:

extracting at least one feature from the natural language expression; and

generating at least two dialog hypotheses, where each dialog hypothesis of the dialog hypothesis set includes a different natural language expression having at least one extracted feature.

20. The computer-implemented method of claim 15, wherein generating a plurality of dialog responses for the dialog hypothesis set comprises generating a plurality of responses for each dialog hypothesis of the dialog hypothesis set.