Rules-based grammar for slots and statistical model for preterminals in natural language understanding system

• US 20040220797A1
• Filed: 05/01/2003
• Published: 11/04/2004
• Est. Priority Date: 05/01/2003
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Wang, Ye Yi, Acero, Alejandro
• Granted Patent: US 7,603,267 B2
• Time in Patent Office: Days
• US Class Current: 704/9
• CPC Class Codes: G06F 40/205 Parsing
• CPC Class Codes: G06F 40/211 Syntactic parsing, e.g. based on context-free grammar [CFG] or unification grammars
• CPC Class Codes: G06F 40/216 using statistical methods
• CPC Class Codes: G10L 15/1822 Parsing for meaning understanding

Abstract

A NLU system includes a rules-based grammar for slots in a schema and a statistical model for preterminals. A training system is also provided.

Claims

1. A natural language understanding (NLU) system for mapping a natural language input into a schema, comprising:

a rules-based grammar component configured to map portions of the natural language input to slots derived from the schema;

a statistical model component configured to map portions of the natural language input to preterminals derived from the schema; and

a decoder coupled to the rules-based grammar component and the statistical model component.

2. The NLU system of claim 1 wherein the statistical model component comprises:

a statistical model corresponding to each of a plurality of different preterminals derived from the schema.

3. The NLU system of claim 1 wherein the statistical model component comprises:

a statistical slot transition model modeling transitions between slots.

4. The NLU system of claim 1 wherein the schema is indicative of tasks and wherein the statistical model component comprises:

a statistical task model modeling a prior probability of tasks.

5. The NLU system of claim 1 wherein the decoder is configured to receive the natural language input and map the natural language input into the schema by accessing the rules-based grammar component and the statistical model component.

6. The NLU system of claim 5 wherein the rules-based grammar component includes a chart parser, and wherein the decoder is configured to access the chart parser to identify one or more non-terminals that cover one or more spans of the natural language input.

7. The NLU system of claim 6 wherein the decoder is configured to perform dynamic programming decoding based on the one or more non-terminals and words in the natural language input, using the statistical model component.

8. An authoring component configured for generating components for use in mapping natural language inputs to slots and preterminals derived from a schema in a natural language understanding (NLU) system, comprising:

a model trainer configured to train a rules based grammar, based on training data, and to train a statistical model for mapping the natural language input to the preterminals derived from the schema.

9. The authoring component of claim 8 wherein the model trainer is configured to train a statistical model corresponding to each of a plurality of different preterminals.

10. The authoring component of claim 8 wherein the model trainer is configured to train a statistical slot transition model modeling transitions between slots.

11. The authoring component of claim 8 wherein the schema is indicative of tasks and wherein the model trainer is configured to train a statistical task model modeling a prior probability of tasks.

12. The authoring component of claim 8 wherein the model trainer is configured to enumerate segmentations associating slots and preterminals with training text.

13. The authoring component of claim 12 wherein the model trainer is configured to train the statistical model using the text associated with the preterminals as training data for the statistical model.

14. The authoring component of claim 13 wherein the model trainer is configured to train a statistical model for each preterminal derived from the schema using the text associated with each preterminal as training data for the statistical model for that preterminal.

15. The authoring component of claim 14 wherein the model trainer is configured to assign an expected count to each segmentation enumerated.

16. The authoring component of claim 15 wherein the model trainer is configured to select a preterminal and train the statistical model for the selected preterminal using the expected count assigned to a segmentation corresponding to the selected preterminal.

17. The authoring component of claim 15 wherein the model trainer is configured to assign the expected count to each segmentation generated based on application of an expectation maximization (EM) algorithm.

18. The authoring component of claim 8 and further comprising:

a probabilistic library grammar accessible by the model trainer.

19. The authoring component of claim 18 wherein the training data is semantically annotated training data and wherein the model trainer is configured to adapt probabilities in the probabilistic library grammar based on the semantically annotated training data.

20. A method of training a natural language understanding (NLU) model, comprising:

generating a plurality of segmentations of training data, for a plurality of slots and a plurality of preterminals; and

training a statistical model for a preterminal corresponding to at least one of the segmentations.

21. The method of claim 20 wherein generating a plurality of segmentations comprises:

associating slots and preterminals with portions of the training data.

22. The method of claim 21 wherein training a statistical model comprises:

generating a statistical model for each of the plurality of preterminals.

23. The method of claim 22 generating a statistical model for each of the plurality of preterminals comprises:

selecting a preterminal; and

generating a statistical model for the selected preterminal using the portion of the training data associated with the selected preterminal as training data for the statistical model.

24. The method of claim 20 and further comprising:

receiving the training data as a schema and semantically annotated training text.

25. The method of claim 24 wherein generating a plurality of segmentations comprises:

assigning a count to each segmentation based on occurrences of the segmentation supported by the training data.

26. The method of claim 22 wherein assigning a count to each segmentation comprises:

assigning an expected count to each segmentation by application of an expectation maximization (EM) algorithm during generation of the segmentation.

27. The method of claim 20 wherein training a statistical model comprises:

training a statistical slot transition model modeling transitions between slots.

28. The method of claim 20 wherein the NLU system is represented by a schema that includes tasks and wherein training the statistical model comprises:

training a statistical task model modeling a prior probability of the tasks.

29. The method of claim 20 wherein the training data comprises semantically annotated training data and further comprising:

accessing a probabilistic library grammar; and

adapting probabilities in the probabilistic library grammar based on the semantically annotated training data.