System for using statistical classifiers for spoken language understanding

• US 8,335,683 B2
• Filed: 01/23/2003
• Issued: 12/18/2012
• Est. Priority Date: 01/23/2003
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Acero, Alejandro, Chelba, Ciprian, Wang, Ye-Yi, Wong, Leon, Frey, Brendan
• Primary Examiner(s): Hudspeth, David R
• Assistant Examiner(s): SPOONER, LAMONT M
• Application Number:
  US10/350,199
• Publication Number:
  US 20040148154A1
• Time in Patent Office: 3,617 Days
• Field of Search: 704/9, 704/246, 704/255, 704/275, 707 1- 6
• US Class Current: 704/9
• CPC Class Codes: G06F 40/216 using statistical methods

Abstract

The present invention involves using one or more statistical classifiers in order to perform task classification on natural language inputs. In another embodiment, the statistical classifiers can be used in conjunction with a rule-based classifier to perform task classification.

Claims

1. A computer-readable storage medium including instructions readable by a computer which, when implemented, provide a text classifier in a natural language interface that receives a natural language user input, the text classifier comprising:

a feature extractor extracting a binary feature vector of features from a textual input indicative of the natural language user input;

a statistical classifier coupled to the feature extractor, receiving the feature vector of features, and outputting a selected class identifier, the selected class identifier identifying a task to be completed, associated with the textual input, based on the features in the feature vector, the statistical classifier comprising;

a plurality of statistical classification components each statistical classification component receiving the feature vector and outputting a respective class identifier based on the feature vector, each respective class identifier representing one of a plurality of tasks to be completed that can be identified using each statistical classification component; and

a class selector coupled to the plurality of statistical classification components and selecting one of the respective class identifiers as the selected class identifier identifying the task to be completed wherein the class selector comprises one of a group comprising;

a voting component that selects a class identifier that is output by more statistical classification components than other class identifiers; and

an additional statistical classifier that receives as an input the class identifiers output from the plurality of statistical classification components and that selects one of the class identifiers received in the input; and

a parser that receives the selected class identifier and the textual input and generates a semantic representation of the textual input based on the selected class identifier.

2. The text classifier of claim 1 wherein the parser further comprises:

a rule-based parser receiving the textual input and the selected class identifier and outputting the semantic representation of the textual input.

3. The text classifier of claim 2 wherein the semantic representation includes a class having slots, the rule-based parser filling the slots with semantic expressions.

4. The text classifier of claim 1 and further comprising:

a pre-processor identifying words in the textual input having semantic content.

5. The text classifier of claim 4 wherein the preprocessor is configured to remove words from the textual input that have insufficient semantic content.

6. The text classifier of claim 4 wherein the preprocessor is configured to insert tags for words in the textual input, the tags being semantic labels for the words.

7. The text classifier of claim 1 wherein the feature vector is based only on words in a vocabulary supported by the natural language interface.

8. The text classifier of claim 7 wherein the feature vector is based on n-grams of the words in the vocabulary.

9. The text classifier of claim 7 wherein the feature vector is based only on words in the vocabulary having semantic content.

10. The text classifier of claim 1 wherein the statistical classifier comprises a Naive Bayes Classifier.

11. The text classifier of claim 1 wherein the statistical classifier comprises a support vector machine.

12. The text classifier of claim 1 wherein the statistical classifier comprises a plurality of class-specific statistical language models.

13. The text classifier of claim 1 wherein a number c of classes are supported by the natural language interface and wherein the statistical classifier comprises c class-specific statistical language models.

14. The text classifier of claim 1 and further comprising:

a speech recognizer receiving a speech signal indicative of the natural language input and providing the textual input.

15. The text classifier of claim 1 wherein the statistical classifier identifies a plurality of n-best target classes.

16. The text classifier of claim 15 and further comprising:

an output displaying the n-best target classes for user selection.

17. The text classifier of claim 16 wherein each statistical classifier outputs a plurality of n-best target classes.

18. A computer-implemented method of processing a natural language input for use in completing a task represented by the natural language input, comprising:

generating a feature vector based on the natural language input;

performing statistical classification on the feature vector with a processor to obtain a class identifier identifying a target class associated with the natural language input, the statistical classification being performed by performing statistical classification on the natural language input using a plurality of different statistical classifiers, and selecting the class identifier output by a greatest number of the plurality of statistical classifiers as representing the target class;

selectively activating grammar rules in a grammar used in a rule-based analyzer, the activated grammar rules corresponding to the class identifier; and

analyzing the natural language input with the rule-based analyzer using the grammar with the activated grammar rules to generate semantic expressions from the natural language input to fill semantic slots in the target class.

19. The method of claim 18 and further comprising:

prior to performing statistical classification, identifying words in the natural language input that have semantic content.

20. The method of claim 18 wherein the natural language input is represented by a speech signal and further comprising:

performing speech recognition on the speech signal prior to performing statistical classification.

21. The method of claim 18 wherein selecting comprises:

performing statistical classification on the class identifiers output by the plurality of statistical classifiers to select the class identifier that represents the target class.

22. A computer-implemented method of processing a natural language input for use in completing a task represented by the natural language input, comprising:

generating a feature vector based on the natural language input;

performing statistical classification on the feature vector with a processor to obtain a class identifier identifying a target class associated with the natural language input;

selectively activating grammar rules in a grammar used in a rule-based analyzer, the activated grammar rules corresponding to the class identifier;

analyzing the natural language input with the rule-based analyzer using the grammar with the activated grammar rules to generate semantic expressions from the natural language input to fill semantic slots in the target class;

performing rule-based analysis on the natural language input to obtain another class identifier; and

identifying the target class based on the class identifier obtained from the statistical classification and the another class identifier obtained from the rule-based analysis.