Ambiguity resolving conversational understanding system

• US 10,446,137 B2
• Filed: 10/19/2016
• Issued: 10/15/2019
• Est. Priority Date: 09/07/2016
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Khan, Omar Zia, Sarikaya, Ruhi, Jetley, Divya
• Primary Examiner(s): Ortiz-Sanchez, Michael
• Application Number:
  US15/297,790
• Publication Number:
  US 20180068657A1
• Time in Patent Office: 1,091 Days
• CPC Class Codes: G06F 16/3326 using relevance feedback from the user, e.g. relevance feedback on documents, documents sets, document terms or passages
• CPC Class Codes: G06F 40/30 Semantic analysis
• CPC Class Codes: G06F 9/451 Execution arrangements for user interfaces
• CPC Class Codes: G10L 15/063 Training
• CPC Class Codes: G10L 15/1815 Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 2015/0635 updating or merging of old and new templates; Mean values; Weighting
• CPC Class Codes: G10L 2015/221 Announcement of recognition results
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command

Abstract

Systems, components, devices, and methods for resolving ambiguity in a conversational understanding system are provided. A non-limiting example is a system or method for resolving ambiguity in a conversational understanding system. The method includes the steps of receiving a natural language input and identifying an agent action based on the natural language input. The method also includes the steps of determining an ambiguity value associated with the agent action and evaluating the ambiguity value against an ambiguity condition. The method includes the steps of when determined that the ambiguity value meets the ambiguity condition: selecting a prompting action based on the ambiguity associated with the identified agent action, performing the prompting action, receiving additional input in response to the prompting action, and updating the agent action to resolve the ambiguity based on the additional input. The method also includes the step of performing the agent action.

Claims

1. A network-based ambiguity resolving conversational understanding system, the system comprising:

at least one processor; and

memory, operatively connected to the at least one processor and storing instructions that, when executed by the at least one processor, cause the at least one processor to;

receive, via a network, a natural language input from a user computing device;

identify an agent action based, at least in part, on the natural language input;

determine an ambiguity in the agent action;

determine an ambiguity value associated with the ambiguity in the agent action;

evaluate the ambiguity value against a criticality level associated with the agent action, the criticality level corresponding to a consequential severity of performing an incorrect agent action, wherein the criticality level decreases when the agent action is reversible;

when it is determined that the ambiguity value meets the criticality level;

select a prompting action based, at least in part, on the ambiguity in the agent action, wherein the prompting action comprises one or more of an audio feedback output by an audio component of the user computing device and a visual feedback provided on a display of the user computing device;

initiate performance of the prompting action by the user computing device;

receive additional input in response to the prompting action; and

update the agent action to resolve the ambiguity based, at least in part, on the additional input; and

initiate performance of the agent action.

2. The system of claim 1, wherein the ambiguity value is based, at least in part, on at least two sources of ambiguity.

3. The system of claim 1, wherein the natural language input is a speech input.

4. The system of claim 3, further comprising instructions for determining an ambiguity value associated with an automatic speech recognition of the speech input.

5. The system of claim 1, further comprising instructions for:

performing language understanding based, at least in part, on the natural language input to attribute a meaning to the natural language input;

retrieving information from a data source based, at least in part, on the meaning attributed to the natural language input; and

determining the agent action based, at least in part, on the meaning attributed to the natural language input and the retrieved information.

6. The system of claim 5, further comprising instructions for:

accessing task state information associated with a current task; and

determining the agent action based, at least in part, on the task state information.

7. The system of claim 1, further comprising instructions for transmitting results of the performance of the agent action to the user computing device.

8. The system of claim 1, further comprising instructions for transmitting the agent action to the user computing device for performance by the user computing device.

9. The system of claim 1, further comprising instructions for transmitting the prompting action to the user computing device.

20. The system of claim 1, wherein the visual feedback is used to clarify the audio feedback when both the visual feedback and audio feedback are output by the user computing device.

10. A method for resolving an ambiguity in a conversational understanding system, comprising:

receiving, at a computing device, a speech input from a user;

performing automatic speech recognition on the speech input;

identifying an agent action based, at least in part, on the speech input;

determining whether there is an ambiguity in the agent action;

when it is determined that there is an ambiguity in the agent action, determining a criticality level of the agent action, the criticality level being based, at least in part, on a severity of one or more unintended consequences of taking an agent action that was not intended by the user, wherein the criticality level decreases when the agent action is reversible;

when it is determined that the criticality level is above a threshold;

providing a prompting action based, at least in part, on the ambiguity in the agent action, the prompting action being one or more of an audio prompt output by an audio component of the computing device and a visual prompt output on a display of the computing device;

receiving additional input in response to the prompting action; and

updating the agent action to resolve the ambiguity based, at least in part, on the additional input; and

performing the agent action.

11. The method of claim 10, wherein determining whether there is an ambiguity in the agent action comprise determining ambiguity values based on at least two sources of ambiguity.

12. The method of claim 11, wherein determining whether there is an ambiguity in the agent action comprises determining an ambiguity value associated with performing automatic speech recognition on the speech input.

13. The method of claim 10, wherein identifying the agent action comprises:

performing language understanding based, at least in part, on the speech input to attribute a meaning to the speech input;

retrieving information from a data source based, at least in part, on the meaning attributed to the speech input; and

determining the agent action based, at least in part, on the meaning attributed to the speech input and the retrieved information.

14. The method of claim 13, wherein identifying the agent action based, at least in part, on the natural language input further comprises accessing task state information associated with a current task and determining the agent action based, at least in part, on the task state information.

15. The method of claim 10, wherein receiving the natural language input comprises receiving a textual input.

16. A computing device, comprising:

at least one processor; and

a memory coupled to the at least one processor and storing instructions that, when executed by the at least one processor, perform a method, comprising;

receiving, by the computing device, an initial natural language input;

identifying an agent action based, at least in part, on the initial natural language input;

determining whether there is an ambiguity in the agent action;

when it is determined that there is an ambiguity in the agent action, determining a criticality level of the agent action, the criticality being based, at least in part, on a consequence of performing an unintended agent action, wherein the criticality level decreases when the agent action is reversible;

when it is determined that the criticality level is above a threshold;

providing a prompting action based, at least in part, on the ambiguity in the agent action, the prompting action being one or more of an audio prompt output by an audio component of the computing device and a visual prompt output on a display of the computing device;

receiving additional input; and

using the additional input to update the agent action to resolve the ambiguity; and

performing the agent action.

17. The computing device of claim 16, wherein determining whether there is an ambiguity in the agent action comprises determining ambiguity values based on at least two sources of ambiguity.

18. The computing device of claim 16, wherein receiving a natural language input comprises receiving a speech input.

19. The computing device of claim 16, wherein identifying the agent action comprises:

performing language understanding based, at least in part, on the natural language input to attribute a meaning to the natural language input;

retrieving information from a data source based, at least in part, on the meaning attributed to the natural language input; and

determining the agent action based, at least in part, on the meaning attributed to the natural language input and the retrieved information.