Proofing of word collocation errors based on a comparison with collocations in a corpus

• US 7,774,193 B2
• Filed: 12/05/2006
• Issued: 08/10/2010
• Est. Priority Date: 12/05/2006
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Hon, Hsiao-Wuen, Zhou, Ming, Gao, Jianfeng, Dolan, William B.
• Primary Examiner(s): Dorvil; Richemond
• Assistant Examiner(s): He; Jialong
• Application Number:
  US11/633,788
• Publication Number:
  US 20080133444A1
• Time in Patent Office: 1,344 Days
• Field of Search: 704 1- 10
• US Class Current: 704/4
• CPC Class Codes: G06F 16/30 of unstructured textual data document management systems G06F16/93
• CPC Class Codes: G06F 40/232 Orthographic correction, e.g. spell checking or vowelisation
• CPC Class Codes: G06F 40/284 Lexical analysis, e.g. tokenisation or collocates
• CPC Class Codes: G06F 40/49 using very large corpora, e.g. the web

Abstract

Collocation errors can be automatically proofed using local and network-based corpora, including the Web. For example, according to one illustrative method, one or more collocations from a text sample are compared with a corpus such as the content of the Web. The collocations are identified for whether they are disfavored in the corpus. Indications are provided via an output device of whether the collocations are disfavored in the corpus. Additional steps may then be taken such as searching for and providing potentially proper word collocations via a user output.

Claims

1. A method, implemented by a computing system comprising one or more processors, the method comprising:

comparing, using one or more of the processors, one or more collocations from a text sample with a corpus;

identifying, using one or more of the processors, whether the collocations are disfavored in the corpus; and

providing indications of whether the collocations are disfavored via an output device;

in which comparing the collocations with the corpus comprises performing one or more searches of the World Wide Web using one or more query terms that comprise each of one or more of the collocations; and

in which for each of one or more of the collocations for which searches are performed, a search is performed for each of the one or more query terms that comprise the collocation until either one of the query terms provides search results that meet a preselected threshold for matching the collocation, or all the query terms that comprise the collocation are used without meeting the preselected threshold, and further comprising;

composing one or more query terms with a wild card replacing a word in one of the disfavored collocations;

searching a word collocation reference for the query terms;

identifying results of the search having a relatively high proportion of a candidate word replacing the wild card; and

providing the results of the search having the candidate word via the output device as potentially proper word collocations.

2. The method of claim 1, wherein the corpus comprises content on the World Wide Web.

3. The method of claim 1, wherein a collocation is disfavored if either it is not found in the corpus, or it does not score above a preselected threshold matching score indicating a significant presence of fuzzy matches of the collocation in the corpus.

4. The method of claim 1, wherein the corpus comprises content accessible on a network.

5. The method of claim 1, wherein for each of the collocations for which searches are performed, the collocation is disfavored if query terms that comprise the collocation do not score above a preselected threshold matching score indicating a significant presence of fuzzy matches of the query terms in the corpus.

6. The method of claim 1, wherein at least one of the query terms comprises a sentence that comprises the collocation.

7. The method of claim 1, wherein at least one of the query terms comprises a reduced sentence that comprises the collocation, wherein the reduced sentence is formed by removing auxiliaries from a sentence that comprises the collocation.

8. The method of claim 1, wherein at least one of the query terms comprises a pair of chunks comprising the collocation.

9. The method of claim 1, wherein at least one of the query terms comprises a pair of words comprising the collocation.

10. The method of claim 1, further comprising part-of-speech tagging the text sample, and further wherein the collocations that are compared with the corpus are selected from the text sample for comprising at least one of a verb-noun collocation, a preposition-noun collocation, an adjective-noun collocation, and a verb-adverb collocation.

11. The method of claim 1, wherein the text sample is in a target language, and the method further comprises screening the corpus to compare the collocations only with content indicated to be in the target language.

12. The method of claim 1, further comprising screening the text sample to determine whether it is indicated to show non-native usage, and if the text sample is indicated to show non-native usage, then automatically initiating the step of comparing the collocations from the text sample with the corpus.

13. The method of claim 1, wherein the query terms comprise one or more of a sentence template, a chunk template, and a word template, and wherein the results of the search having a relatively high proportion of the candidate word replacing the wild card are evaluated by multiplying a proportion of the search results that comprise the candidate word by a query template weight that is highest for the sentence template, lower for the chunk template, and lowest for the word template.

14. The method of claim 1, wherein the query terms comprise one or more of a verb-noun collocation, a preposition-noun collocation, an adjective-noun collocation, and a verb-adverb collocation, and the wild card is selected as the verb in a verb-noun collocation, the noun in a verb-noun collocation, the preposition in a preposition-noun collocation, the adjective in an adjective-noun collocation, or the adverb in a verb-adverb collocation.

15. The method of claim 1, further comprising enabling a user to select one of the potentially proper word collocations to replace the disfavored word collocation to which it corresponds.

16. A non-transitory computer readable storage medium comprising instructions executable by a computing system comprising one or more processors, wherein the instructions configure the computing system to:

receive an indication of a word collocation in a text;

using one or more of the processors, perform a Web search for each of one or more query templates associated with the indicated word collocation, wherein one of the query templates comprises a sentence in which the word collocation was found, one of the query templates comprises a reduced sentence based on the sentence in which the word collocation was found, one of the query templates comprises a chunk pair comprising the word collocation, and one of the query templates comprises an individual word pair comprising the word collocation;

using one or more of the processors, evaluate whether results of the Web search for each of the one or more query templates indicates that the word collocation corresponds to normal usage or is a disfavored collocation, as indicated by either an exact match of the query template comprising the sentence, or an exact match of the query template comprising the reduced sentence, or a matching score for the query template comprising the chunk pair that is larger than a preselected threshold for a chunk pair, or a matching score for the query template comprising the individual word pair that is larger than a preselected threshold for an individual word pair;

indicate via a user-perceptible output device whether the word collocation corresponds to normal usage or whether the word collocation is disfavored;

and further comprising;

composing one or more query terms with a wild card replacing a word in one of the disfavored collocations;

searching a word collocation reference for the query terms;

identifying results of the search having a relatively high proportion of a candidate word replacing the wild card; and

providing the results of the search having the candidate word via the output device as potentially proper word collocations.

17. A computing system comprising:

a computer processor and a data store accessed by the computer processor, the data store storing computer readable instructions and the computer processor accessing the computer readable instructions to;

identify word collocations in a text;

search the World Wide Web for a set of query templates based on each of one or more of the word collocations; and

indicate via a user output device whether results of the search indicate that the word collocations are relatively scarce on the World Wide Web;

compose one or more query terms with a wild card replacing a word in one of the relatively scarce word collocations;

search the World Wide Web for the query terms;

identify results of the search having a relatively high proportion of a candidate word replacing the wild card; and

provide the results of the search having the candidate word via the output device as possible replacements for the relatively scarce word collocations.