Building conversational understanding systems using a toolset

• US 9,311,298 B2
• Filed: 06/21/2013
• Issued: 04/12/2016
• Est. Priority Date: 06/21/2013
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Sarikaya, Ruhi, Boies, Daniel, Heck, Larry, Anastasakos, Tasos
• Primary Examiner(s): ABEBE, DANIEL DEMELASH
• Application Number:
  US13/923,969
• Publication Number:
  US 20140379326A1
• Time in Patent Office: 1,026 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 40/35 Discourse or dialogue representation
• CPC Class Codes: G06F 40/40 Processing or translation of natural language natural language analysis G06F40/20; semantic analysis G06F40/30
• CPC Class Codes: G10L 15/18 using natural language modelling

Abstract

Tools are provided to allow developers to enable applications for Conversational Understanding (CU) using assets from a CU service. The tools may be used to select functionality from existing domains, extend the coverage of one or more domains, as well as to create new domains in the CU service. A developer may provide example Natural Language (NL) sentences that are analyzed by the tools to assist the developer in labeling data that is used to update the models in the CU service. For example, the tools may assist a developer in identifying domains, determining intent actions, determining intent objects and determining slots from example NL sentences. After the developer tags all or a portion of the example NL sentences, the models in the CU service are automatically updated and validated. For example, validation tools may be used to determine an accuracy of the model against test data.

Claims

1. A computer implemented method for enabling an application for Conversational Understanding (CU) using assets in a CU service, comprising:

receiving a selection of Application Programming Interfaces (APIs) that are associated with a domain to use in the application;

automatically updating models for the CU service based on the selection of the APIs and the determined domain;

receiving a Natural Language (“

NL”

) expression, wherein the NL expression may be used to interact with the application;

rephrasing the NL expressions to generate at least one additional expression, wherein the at least one additional expression contains a different, way of expressing a meaning of the NL expression; and

automatic updating the models for CU service based on the rephrased NL expressions;

making the models available to the CU service.

2. The method of claim 1, wherein determining the domain comprises receiving examples of natural language sentences used to interact with the application and automatically determining the domain using the examples.

3. The method of claim 1, further comprising determining the domain by applying a domain detector to examples to determine domains that currently exist in the CU service.

4. The method of claim 1, further comprising automatically detecting intent actions on example Natural Language sentences, displaying the detected intent actions and receiving a selection of one of the displayed intent actions or receiving a new intent action label.

5. The method of claim 1, further comprising automatically detecting intent objects on example Natural Language sentences and displaying the detected intent objects.

6. The method of claim 1, further comprising performing slot tagging based on detected intent actions and detected intent objects.

7. The method of claim 1, further comprising updating the models in the CU service for the application and validating the models.

8. The method of claim 1, further comprising incorporating the models into the CU service.

9. The method of claim 1, wherein the detected domain is sports.

10. A computer-readable storage medium storing computer-executable instructions for enabling an application for Conversational Understanding (CU) using assets in a CU service, comprising:

receiving examples of Natural Language (NL) sentences used to interact with the application;

detecting a domain, an intent action, and an intent object;

receiving input to label the examples;

automatically rephrasing the examples of the NL sentences to generate additional sentences that contain different ways of expressing a meaning of the NL sentences; and

automatically updating models for the CU service based on the received input, wherein the updated models include the additional sentences.

11. The computer-readable medium of claim 10, wherein detecting the domain comprises applying a domain detector to the examples.

12. The computer-readable storage medium of claim 10, wherein detecting the intent action comprises detecting intent actions from the examples and displaying a confidence for each of the detected intent actions.

13. The computer-readable storage medium of claim 10, wherein detecting the intent objects comprises detecting intent objects from the examples and displaying a confidence for each of the detected intent objects.

14. The computer-readable storage medium of claim 10, further comprising performing slot tagging on the examples.

15. The computer-readable storage medium of claim 10, wherein automatically updating models for the CU service based on the received input comprises updating a domain model, an intent action model, an intent object model and a slot model.

16. A system for using tools to enable an application for Conversational Understanding (CU) using assets from a CU service, comprising:

a processor and memory;

an operating environment executing using the processor; and

a development manager that is configured to perform actions comprising;

displaying a Graphical User Interface (GUI) that includes a display of tools used to enable the application;

receiving information to display a domain, an intent action, and an intent object, wherein the domain, the intent action, and the intent object has been detected from examples of Natural Language (NL) sentences used to interact with the application;

receiving information to display slot tagging, wherein the slot tagging has been performed on the examples;

displaying the detected domain, the intent action, the intent object and the slot tagging in the GUI;

receiving input from the GUI to label the examples; and

sending the received input for automatic updating of the models for the CU service based on the received input.

17. The system of claim 16, wherein detecting the domain comprises applying a domain detector to the examples to determine domains that currently exist in the CU service and displaying a confidence for each of the domains and receiving a selection of one of the domains or receiving a selection to add a new domain.

18. The system of claim 16, wherein detecting the intent action comprises detecting intent actions from the examples and displaying a confidence for each of the detected intent actions and receiving a selection of one of the intent actions or receiving a selection to add a new intent action.

19. The system of claim 16, wherein detecting the intent objects comprises detecting intent objects from the examples and displaying a confidence for each of the detected intent objects and receiving a selection of one of the intent objects or receiving a selection to add a new intent object.

20. The system of claim 16, wherein receiving the slot tagging comprises receiving a true intent of the examples of natural language sentences.