Building conversational understanding systems using a toolset

• US 10,572,602 B2
• Filed: 05/22/2017
• Issued: 02/25/2020
• Est. Priority Date: 06/21/2013
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Sarikaya, Ruhi, Boies, Daniel, Heck, Larry, Anastasakos, Tasos
• Primary Examiner(s): Abebe, Daniel
• Application Number:
  US15/600,995
• Publication Number:
  US 20170255612A1
• Time in Patent Office: 1,009 Days
• Field of Search: 704 9
• CPC Class Codes: G06F 40/35 Discourse or dialogue representation
• CPC Class Codes: G06F 40/40 Processing or translation of natural language natural language analysis G06F40/20; semantic analysis G06F40/30
• CPC Class Codes: G10L 15/18 using natural language modelling

Abstract

Tools are provided to allow developers to enable applications for Conversational Understanding (CU) using assets from a CU service. The tools may be used to select functionality from existing domains, extend the coverage of one or more domains, as well as to create new domains in the CU service. A developer may provide example Natural Language (NL) sentences that are analyzed by the tools to assist the developer in labeling data that is used to update the models in the CU service. For example, the tools may assist a developer in identifying domains, determining intent actions, determining intent objects and determining slots from example NL sentences. After the developer tags all or a portion of the example NL sentences, the models in the CU service are automatically updated and validated. For example, validation tools may be used to determine an accuracy of the model against test data.

Claims

1. A method, comprising:

receiving a language input by a computing device;

processing, by the computing device, the language input to determine a domain associated with the language input;

determining, by the computing device, an action associated with the domain;

determining, by the computing device, whether the action satisfies the language input;

when it is determined that an additional action is needed to satisfy the language input, generating, by the computing device, the additional action based, at least in part, on the language input; and

automatically updating the domain with the additional action.

2. The method of claim 1, further comprising providing the updated domain to a conversational understanding service.

3. The method of claim 1, wherein the domain is associated with a domain model.

4. The method of claim 1, further comprising automatically rephrasing the language input.

5. The method of claim 4, wherein rephrasing the language input generates additional language input that expresses a meaning of the language input in a different way.

6. The method of claim 1, further comprising determining an object from the language input.

7. The method of claim 6, wherein the object is associated with a confidence score.

8. The method of claim 7, further comprising displaying the confidence score.

9. A system, comprising:

at least one processor; and

a memory coupled to the at least one processor and storing instructions that, when executed by the processor, perform a method, comprising;

receiving examples of natural language input that provide instructions for an application;

determining a domain, an intent action, and an intent object for the natural language input;

providing the detected domain, the intent action and the intent object;

receiving input that corresponds to the one or more of the detected domain, the intent action and the intent object; and

automatically updating one or more models of a conversational understanding service based, at least in part, on the received input.

10. The system of claim 9, further comprising instructions for applying a domain detector to the natural language input.

11. The system of claim 9, further comprising instructions for displaying a confidence score for the intent action.

12. The system of claim 9, further comprising instructions for displaying a confidence score for the intent object.

13. The system of claim 9, further comprising instructions for performing slot tagging on the natural language input.

14. The system of claim 9, wherein automatically updating one or more models of the conversational understanding service comprises updating one or more of a domain model, an intent action model and an intent object model associated with the conversational understanding service.

15. A system for updating a conversation understanding service, comprising:

a processor; and

a memory for storing instructions that, when executed by the processor, perform a method comprising;

detecting a domain, an intent action, and an intent object from a natural language sentence used to interact with an application;

determining whether the detected domain should be updated based, at least in part, on the intent action and the intent object; and

when it is determined that the detected domain should be updated, automatically updating the detected domain using one or more of an additional intent action and an additional intent object.

16. The system of claim 15, further comprising instructions for displaying a confidence for the domain.

17. The system of claim 15, further comprising instructions for displaying a confidence for the detected intent action.

18. The system of claim 15, further comprising instructions for receiving input that indicates that a new intent action is to be added to the conversational understanding service.

19. The system of claim 15, further comprising instructions for displaying a confidence for the detected intent object.

20. The system of claim 15, further comprising instructions for receiving input that indicates that a new intent object is to be added to the conversational understanding service.

21. A computer-implemented method, comprising:

receiving a language input at a computing device;

processing, by the computing device, the language input to determine a domain associated with the language input;

determining, by the computing device, an action associated with the domain;

determining, by the computing device, whether the action satisfies the language input;

when it is determined that an additional action is needed to satisfy the language input, transmitting a request for additional information to a remote computing device; and

receiving the additional information from the remote computing device, the additional information including instructions on how to perform the additional action.

22. The computer-implemented method of claim 21, further comprising updating the domain with the additional action.

23. The computer-implemented method of claim 22, further comprising providing the updated domain to a conversational understanding service.

24. The computer-implemented method of claim 21, further comprising automatically rephrasing the language input.

25. The computer-implemented method of claim 24, wherein rephrasing the language input generates additional language input that expresses a meaning of the language input in a different way.

26. The computer-implemented method of claim 21, further comprising determining an object from the language input.

27. The computer-implemented method of claim 26, wherein the object is associated with a confidence score.

28. The computer-implemented method of claim 27, further comprising displaying the confidence score.