Web-based collocation error proofing

• US 20080133444A1
• Filed: 12/05/2006
• Published: 06/05/2008
• Est. Priority Date: 12/05/2006
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Zhou, Ming, Gao, Jianfeng, Hon, Hsiao-Wuen, Dolan, William B.
• Granted Patent: US 7,774,193 B2
• Time in Patent Office: Days
• US Class Current: 706/52
• CPC Class Codes: G06F 16/30 of unstructured textual data document management systems G06F16/93
• CPC Class Codes: G06F 40/232 Orthographic correction, e.g. spell checking or vowelisation
• CPC Class Codes: G06F 40/284 Lexical analysis, e.g. tokenisation or collocates
• CPC Class Codes: G06F 40/49 using very large corpora, e.g. the web

Abstract

Collocation errors can be automatically proofed using local and network-based corpora, including the Web. For example, according to one illustrative method, one or more collocations from a text sample are compared with a corpus such as the content of the Web. The collocations are identified for whether they are disfavored in the corpus. Indications are provided via an output device of whether the collocations are disfavored in the corpus. Additional steps may then be taken such as searching for and providing potentially proper word collocations via a user output.

Claims

1. A method, implementable by a computing system, comprising:

comparing one or more collocations from a text sample with a corpus;

identifying whether the collocations are disfavored in the corpus; and

providing indications of whether the collocations are disfavored via an output device.

2. The method of claim 1, wherein the corpus comprises content on the Web.

3. The method of claim 1, wherein a collocation is disfavored if either it is not found in the corpus, or it does not score above a preselected threshold matching score indicating a significant presence of fuzzy matches of the collocation in the corpus.

4. The method of claim 1, wherein the corpus comprises content accessible on a network.

5. The method of claim 1, wherein comparing the collocations with the corpus comprises performing one or more searches of the Web using one or more query terms that comprise each of one or more of the collocations.

6. The method of claim 5, wherein for each of one or more of the collocations for which searches are performed, a search is performed for each of the one or more query terms that comprise the collocation until either one of the query terms provides search results that meet a preselected threshold for matching the collocation, or all the query terms that comprise the collocation are used without meeting the preselected threshold.

7. The method of claim 6, wherein for each of the collocations for which searches are performed, the collocation is disfavored if query terms that comprise the collocation do not score above a preselected threshold matching score indicating a significant presence of fuzzy matches of the query terms in the corpus.

8. The method of-claim 6, wherein at least one of the query terms comprises a sentence that comprises the collocation.

9. The method of claim 6, wherein at least one of the query terms comprises a reduced sentence that comprises the collocation, wherein the reduced sentence is formed by removing auxiliaries from a sentence that comprises the collocation.

10. The method of claim 6, wherein at least one of the query terms comprises a pair of chunks comprising the collocation.

11. The method of claim 6, wherein at least one of the query terms comprises a pair of words comprising the collocation.

12. The method of claim 1, further comprising part-of-speech tagging the text sample, and further wherein the collocations that are compared with the corpus are selected from the text sample for comprising at least one of a verb-noun collocation, a preposition-noun collocation, an adjective-noun collocation, and a verb-adverb collocation.

13. The method of claim 1, wherein the text sample is in a target language, and the method further comprises screening the corpus to compare the collocations only with content indicated to be in the target language.

14. The method of claim 1, further comprising screening the text sample to determine whether it is indicated to show non-native usage, and if the text sample is indicated to show non-native usage, then automatically initiating the step of comparing the collocations from the text sample with the corpus.

15. The method of claim 1, further comprising:

composing one or more query terms with a wild card replacing one of the words in one of the disfavored word collocations;

searching a word collocation reference for the query terms;

identifying results of the search having a relatively high proportion of a candidate word replacing the wild card; and

providing the results of the search having the candidate word via the output device as the potentially proper word collocations.

16. The method of claim 15, wherein the query terms comprise one or more of a sentence template, a chunk template, and a word template, and wherein the results of the search having a relatively high proportion of the candidate word replacing the wild card are evaluated by multiplying a proportion of the search results that comprise the candidate word by a query template weight that is highest for the sentence template, lower for the chunk template, and lowest for the word template.

17. The method of claim 15, wherein the query terms comprise one or more of a verb-noun collocation, a preposition-noun collocation, an adjective-noun collocation, and a verb-adverb collocation, and the wild card is selected as the verb in a verb-noun collocation, the noun in a verb-noun collocation, the preposition in a preposition-noun collocation, the adjective in an adjective-noun collocation, or the adverb in a verb-adverb collocation.

18. The method of claim 15, further comprising enabling a user to select one of the potentially proper word collocations to replace the disfavored word collocation to which it corresponds.

19. A medium comprising instructions executable by a computing system, wherein the instructions configure the computing system to:

receive an indication of a word collocation in a text;

perform a Web search for each of one or more query templates associated with the indicated word collocation, wherein one of the query templates comprises a sentence in which the word collocation was found, one of the query templates comprises a chunk pair comprising the word collocation, and one of the query templates comprises an individual word pair comprising the word collocation;

evaluating whether results of the Web search for each of the one or more query templates indicates that the word collocation corresponds to normal usage, as indicated by either an exact match of the query template comprising the sentence, or a matching score that is larger than a preselected threshold; and

indicating via a user-perceptible output device whether the word collocation corresponds to normal usage.

20. A computing system configured to:

identify word collocations in a text;

search the Web for a set of query templates based on each of the word collocations; and

i

indicate via a user output device whether results of the search indicate that the word collocations are relatively scarce on the Web.