Language usage classifier

• US 20070219776A1
• Filed: 03/14/2006
• Published: 09/20/2007
• Est. Priority Date: 03/14/2006
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Brockett, Christopher, Dolan, William, Gamon, Michael
• Granted Patent: US 8,170,868 B2
• Time in Patent Office: Days
• US Class Current: 704/9
• CPC Class Codes: G06F 40/20 Natural language analysis semantic analysis of natural language G06F40/30

Abstract

A corpus is provided of language usage by non-native users of the language. Characteristics of the corpus are measured and used to create a language usage classifier for indicating non-native usage of the language. Once the language usage classifier is created, a natural language input may be entered, and the characteristics thereof measured. These characteristics are then compared with the indicators of non-native usage, thereby detecting non-native usage. The evaluation of non-native usage may be used as a versatile foundation to enhance a wide variety of tools and applications dealing with user interaction in languages other than their native language.

Claims

1. A method, implementable at least in part by a computing device, comprising:

receiving a natural language input;

measuring characteristics of the natural language input; and

comparing the characteristics of the natural language input with indicators of non-native usage.

2. The method of claim 1, further comprising classifying one or more sentences in the natural language input as either:

non-native-like, if the characteristics of the natural language input meet a selected threshold of similarity to the indicators of non-native usage;

or

native-like, if the characteristics of the natural language input do not meet the selected threshold of similarity to the indicators of non-native usage.

3. The method of claim 2, further comprising applying a grammar checker to the natural language input, wherein the grammar checker provides output based at least in part on whether the natural language input is classified as non-native-like.

4. The method of claim 2, further comprising indicating an amount of non-native-like usage in the natural language input.

5. The method of claim 2, further comprising providing an option for using a different language, if the natural language input is classified as non-native-like.

6. The method of claim 1, further comprising:

comparing the natural language input with indicators of usage typical of a plurality of other native languages;

selecting an indicated native language that corresponds to the natural language input; and

providing an output in the indicated language.

7. The method of claim 1, wherein the natural language input is received via a non-text format of spoken language or written language.

8. A method, implementable at least in part by a computing device, comprising:

providing a corpus in a language by non-native users of the language;

measuring characteristics of the corpus; and

using the characteristics to create a classifier for indicating non-native usage of the language.

9. The method of claim 8, further comprising receiving an input, and using the classifier on the input for indicating non-native usage.

10. The method of claim 9, wherein the input received is gathered from a network, and further comprising:

adding the input indicated as non-native usage to the corpus;

re-measuring the characteristics of the corpus; and

using the re-measured characteristics to create a refined classifier for indicating non-native usage of the language.

11. The method of claim 8, wherein the characteristics measured comprise part-of-speech trigrams.

12. The method of claim 8, wherein the characteristics measured comprise parse tree segments.

13. The method of claim 8, wherein the characteristics measured comprise logical forms.

14. The method of claim 8, wherein the characteristics measured comprise lexical N-gram features.

15. The method of claim 8, wherein the characteristics measured comprise function word distances.

16. The method of claim 8, wherein the characteristics measured comprise spelling or grammar errors.

17. The method of claim 8, wherein training parameters of the classifier training process comprise the measured characteristics and parameter settings used for training the classifier, and the method further comprising:

using the classifier on a test sample for indicating non-native usage in the test sample;

evaluating which of the training parameters are effective for indicating non-native usage in the test sample; and

retraining the classifier based on which of the training parameters are effective for indicating non-native usage.

18. The method of claim 8, wherein the corpus comprises spoken usage of the language, and the characteristics measured include characteristics of spoken language.

19. The method of claim 8, further comprising:

providing a second corpus comprising revisions of at least part of the first corpus into native usage;

measuring characteristics of the second corpus; and

storing the characteristics of the second corpus as a further basis for indicating non-native usage of the language.

20. A medium, readable by a computing device and comprising executable instructions that are executable by the computing device, wherein the executable instructions enable a computing device to:

receiving a set of sentences comprising non-native usage of a language;

measuring features of the sentences;

using the measured features to define a classifier that models usage similar to the measured features.