Method and Apparatus for Automatic Detection of Spelling Errors in One or More Documents

• US 20080195940A1
• Filed: 02/09/2007
• Published: 08/14/2008
• Est. Priority Date: 02/09/2007
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Sponsoring Entity: N/A
• Inventors: Lenchner, Jonathan, Milch, Daniel, Laker, Meir M., Hantler, Sidney L., Gail, H. Richard
• GRANTED PATENT: US 9,465,791 B2
• Time in USPTO: 0 Days
• US Class Current: 715/257
• CPC Class Codes: G06F 17/273 : Orthographic correction, e.g. spelling checkers, vowelisation
• CPC Class Codes: G06F 17/2735 : Dictionaries

Abstract

Methods and apparatus are provided for automatically detecting spelling errors in one or more documents, such as documents being processed for the creation of a lexicon According to one aspect of the invention, a spelling error is detected in one or more documents by determining if at least one given word in the one or more documents satisfies a predefined misspelling criteria, wherein the predefined misspelling criteria comprises the at least one given word having a frequency below a predefined low threshold and the at least one given word being within a predefined edit distance of one or mote other words in the one or more documents having a frequency above a predefined high threshold; and identifying a given word as a potentially misspelled word if the given word satisfies the predefined misspelling criteria

Claims

1. . A method for detecting a spelling error in one or more documents, comprising:

determining if at least one given word in said one or more documents satisfies a predefined misspelling criteria, wherein said predefined misspelling criteria comprises said at least one given word having a frequency below a predefined low threshold and said at least one given word being within a predefined edit distance of one or more other words in said one or more documents having a frequency above a predefined high threshold; and

identifying a given word as a potentially misspelled word if said given world satisfies said predefined misspelling criteria.

2. . The method of claim 1, further comprising the steps of adding all words in said one or more documents to a lexicon, determining a frequency of occurrence of each of said words in said one or more documents and removing said potentially misspelled word from said lexicon

3. . The method of claim 1, further comprising the steps of adding all words in said one or more documents to a lexicon, determining a frequency of occurrence of each of said words in said one or more documents and marking said potentially misspelled word within said lexicon in a manner that indicates that the word may be misspelled

4. . The method of claim 1, further comprising the step of presenting said potentially misspelled word to a user for validation.

5. . The method of claim 1, wherein said one or more documents comprise documents being processed by at least one user, and wherein said method further comprises the step of maintaining a word count for each word in said one or more documents and wherein said word frequencies in said predefined misspelling criteria are based on said word counts

6. . The method of claim 5, further comprising the step of suggesting a correction of said at least one given word using said word within said predefined edit distance and having a frequency above said predefined high threshold.

7. . The method of claim 6, further comprising the step of maintaining a count of the total words in said one or more documents and suggesting said correction only if said total word count satisfies a predefined criteria.

8. . The method of claim 1, wherein said edit distance comprises determining a distance N that said at least one given word is from a second word, whereby said at least one given word can be transformed into the second word via a sequence of N steps of one or more of transposing adjacent characters, deleting a character, inserting a character, and replacing a character with another character.

9. . The method of claim 1, wherein said predefined low threshold and said predefined high threshold are specified by a user.

10. . The method of claim 1, wherein said at least one given word comprises one or more words or phrases.

11. . The method of claim 1, further comprising the step of employing a reference lexicon of valid words and wherein said step of identifying a given word as a potentially misspelled word does not identity a given word as a potentially misspelled word if said given word appears in said reference lexicon

12. . A system for detecting a spelling error in one or more documents, said system comprising:

a memory; and

at least one processor, coupled to the memory, operative to;

determine if at least one given word in said one or more documents satisfies a predefined misspelling criteria, wherein said predefined misspelling criteria comprises said at least one given word having a frequency below a predefined low threshold and said at least one given word being within a predefined edit distance of one or more other words in said one or more documents having a frequency above a predefined high threshold; and

identify a given word as a potentially misspelled word if said given word satisfies said predefined misspelling criteria.

13. . The system of claim 12, wherein said processor is further configured to add all words in said one or more documents to a lexicon, determine a frequency of occurrence of each of said words in said one or more documents and remove said potentially misspelled word from said lexicon.

14. . The system of claim 12, wherein said processor is further configured to add all words in said one or more documents to a lexicon, determine a frequency of occurrence of each of said words in said one or more documents and mark said potentially misspelled word within said lexicon in a manner that indicates that the word may be misspelled

15. . The system of claim 12, wherein said processor is further configured to present said potentially misspelled word to a user for validation

16. . The system of claim 12, wherein said one or more documents comprise documents being processed by at least one user;

and wherein said processor is further configured to maintain a word count for each word in said one or more documents and wherein said word frequencies in said predefined misspelling criteria are based on said word counts.

17. . The system of claim 16, wherein said processor is further configured to suggest a correction of said at least one given word using said word within said predefined edit distance and having a frequency above said predefined high threshold

18. . The system of claim 17, wherein said processor is further configured to maintain a count of the total words in said one or more documents and suggesting said correction only if said total word count satisfies a predefined criteria.

19. . The system of claim 12, wherein said at least one given word comprises one or more words or phrases

20. . The system of claim 12, wherein said at least one given word comprises one or more words or phrases

21. . The system of claim 12, wherein said processor is further configured to employ a reference lexicon of valid words and wherein said step of identifying a given word as a potentially misspelled word does not identify a given word as a potentially misspelled word if said given word appears in said reference lexicon.

22. . An article of manufacture for detecting a spelling error in one or more documents, comprising a machine readable medium containing one or more programs which when executed implement the steps of;

determining if at least one given word in said one or more documents satisfies a predefined misspelling criteria, wherein said predefined misspelling criteria comprises said at least one given word having a frequency below a predefined low threshold and said at least one given word being within a predefined edit distance of one or more other words in said one or more documents having a frequency above a predefined high threshold; and

identifying a given word as a potentially misspelled word if said given word satisfies said predefined misspelling criteria