Building conversational understanding systems using a toolset

• US 9,697,200 B2
• Filed: 03/21/2016
• Issued: 07/04/2017
• Est. Priority Date: 06/21/2013
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Sarikaya, Ruhi, Boies, Daniel, Heck, Larry, Anastasakos, Tasos
• Primary Examiner(s): ABEBE, DANIEL DEMELASH
• Application Number:
  US15/076,125
• Publication Number:
  US 20160203125A1
• Time in Patent Office: 470 Days
• Field of Search: 704275
• CPC Class Codes: G06F 40/35 Discourse or dialogue representation
• CPC Class Codes: G06F 40/40 Processing or translation of natural language natural language analysis G06F40/20; semantic analysis G06F40/30
• CPC Class Codes: G10L 15/18 using natural language modelling

Abstract

Tools are provided to allow developers to enable applications for Conversational Understanding (CU) using assets from a CU service. The tools may be used to select functionality from existing domains, extend the coverage of one or more domains, as well as to create new domains in the CU service. A developer may provide example Natural Language (NL) sentences that are analyzed by the tools to assist the developer in labeling data that is used to update the models in the CU service. For example, the tools may assist a developer in identifying domains, determining intent actions, determining intent objects and determining slots from example NL sentences. After the developer tags all or a portion of the example NL sentences, the models in the CU service are automatically updated and validated. For example, validation tools may be used to determine an accuracy of the model against test data.

Claims

1. A method for enabling an application for Conversational Understanding (CU) using assets in a CU service, comprising:

receiving a natural language sentence;

processing the natural language sentence to determine one or more domains in a domain model that are associated with the natural language sentence;

determining one or more actions that are associated with each of the one or more domains;

determining, whether an additional domain should be added to the domain model based, at least in part, on the determined actions;

when it is determined that an additional domain should be added to the domain model, automatically updating the domain model with the additional domain and the determined actions; and

providing the domain model to the CU service.

2. The method of claim 1, further comprising automatically rephrasing the natural language sentence to generate additional sentences that contain different ways of expressing a meaning of the natural language sentence.

3. The method of claim 1, wherein determining one or more domains comprises:

utilizing a domain detector to determine domains that exist in the CU service; and

receiving a selection of one or more of the domains.

4. The method of claim 1, further comprising:

displaying the determined one or more actions; and

receiving a selection of at least one of the one or more actions.

5. The method of claim 1, further comprising:

automatically detecting intent objects associated with the one or more actions; and

displaying the detected intent objects.

6. The method of claim 5, further comprising performing slot tagging based on the one or more actions and the detected intent objects.

7. The method of claim 1, further comprising:

updating the domain model; and

validating the domain model.

8. The method of claim 1, further comprising incorporating the domain model into the CU service.

9. The method of claim 1, further comprising updating a slot model associated with the domain model.

10. The method of claim 1, wherein the additional domain is associated the one or more actions.

11. A computer-readable storage medium storing computer-executable instructions for updating a domain model in a Conversational Understanding (CU) service, comprising:

displaying tools used to create an application that interacts with the CU service;

receiving examples of natural language sentences used to interact with the application;

receiving one or more additional sentences associated with the natural language sentences, wherein the one or more additional sentences are generated by automatically rephrasing the natural language sentences;

detecting a domain, an intent action, and an intent object;

displaying the detected domain, the intent action and the intent object;

receiving input that labels the examples; and

automatically updating models for the CU service based on the received input.

12. The computer-readable medium of claim 11, wherein detecting the domain comprises applying a domain detector to the natural language sentences.

13. The computer-readable storage medium of claim 11, wherein detecting the intent action comprises:

detecting intent actions from the natural language sentences; and

displaying a confidence for each of the detected intent actions.

14. The computer-readable storage medium of claim 11, wherein detecting the intent objects comprises:

detecting intent objects from the natural language sentences; and

displaying a confidence for each of the detected intent objects.

15. The computer-readable storage medium of claim 11, further comprising performing slot tagging on the natural language sentences.

16. The computer-readable storage medium of claim 11, wherein automatically updating models for the CU service based on the received input comprises updating a domain model, an intent action model, an intent object model and a slot model.

17. A system for updating a domain in a Conversational Understanding (CU) service, comprising:

a processor; and

a memory for storing instructions which, when executed by the processor, performs a method comprising;

displaying a set of tools used to generate an application that utilizes the CU service;

detecting a domain, an intent action, and an intent object from examples of natural language sentences used to interact with the application;

displaying the detected domain, the intent action and the intent object;

determining whether the detected domain should be updated based, at least in part, on the intent action and the intent object;

when it is determined that the detected domain should be updated, automatically updating the detected domain using one or more of an additional intent action and an additional intent object.

18. The system of claim 17, wherein detecting the domain comprises:

utilizing a domain detector determine domains that currently exist in the CU service; and

displaying a confidence for each of the domains.

19. The system of claim 17, wherein detecting the intent action comprises:

detecting intent actions from the natural language sentences;

displaying a confidence for each of the detected intent actions; and

receiving a selection of one of the intent actions or receiving a selection to add a new intent action.

20. The system of claim 17, wherein detecting the intent objects comprises:

detecting intent objects from the natural language sentences;

displaying a confidence for each of the detected intent objects; and

receiving a selection of one of the intent objects or receiving a selection to add a new intent object.