Method and apparatus for automatic detection of spelling errors in one or more documents

• US 9,465,791 B2
• Filed: 02/09/2007
• Issued: 10/11/2016
• Est. Priority Date: 02/09/2007
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Gail, H. Richard, Hantler, Sidney L., Laker, Meir M., Lenchner, Jonathan, Milch, Daniel
• Primary Examiner(s): Paula, Cesar
• Assistant Examiner(s): Schallhorn, Tyler J
• Application Number:
  US11/673,173
• Publication Number:
  US 20080195940A1
• Time in Patent Office: 3,532 Days
• Field of Search: 715/257, 704/10
• US Class Current: 1/1
• CPC Class Codes: G06F 40/232 Orthographic correction, e.g. spell checking or vowelisation
• CPC Class Codes: G06F 40/242 Dictionaries

Abstract

Methods and apparatus are provided for automatically detecting spelling errors in one or more documents, such as documents being processed for the creation of a lexicon According to one aspect of the invention, a spelling error is detected in one or more documents by determining if at least one given word in the one or more documents satisfies a predefined misspelling criteria, wherein the predefined misspelling criteria comprises the at least one given word having a frequency below a predefined low threshold and the at least one given word being within a predefined edit distance of one or more other words in the one or more documents having a frequency above a predefined high threshold; and identifying a given word as a potentially misspelled word if the given word satisfies the predefined misspelling criteria.

Claims

1. A method for detecting a spelling error in one or more documents, comprising:

obtaining a maximum edit distance at which a word, w, is to be considered a possible misspelling of another word, w′

;

determining if at least one given word in said one or more documents satisfies a predefined misspelling criteria, wherein said predefined misspelling criteria comprises said at least one given word having a frequency below a predefined low threshold and said at least one given word being within the obtained maximum edit distance of one or more other words in said one or more documents having a frequency above a predefined high threshold;

identifying a given word as a potentially misspelled word if said given word satisfies said predefined misspelling criteria; and

maintaining a lexicon such that said lexicon will include said given word if said given word does not satisfy said predefined misspelling criteria and will exclude said given word if said given word satisfies said predefined misspelling criteria, wherein one or more of said steps are performed by a processor.

2. The method of claim 1, further comprising the steps of adding all words in said one or more documents to a lexicon, determining a frequency of occurrence of each of said words in said one or more documents and removing said potentially misspelled word from said lexicon.

3. The method of claim 1, further comprising the steps of adding all words in said one or more documents to a lexicon, determining a frequency of occurrence of each of said words in said one or more documents and marking said potentially misspelled word within said lexicon in a manner that indicates that the word may be misspelled.

4. The method of claim 1, further comprising the step of presenting said potentially misspelled word to a user for validation.

5. The method of claim 1, wherein said one or more documents comprise documents being processed by at least one user, and wherein said method further comprises the step of maintaining a word count for each word in said one or more documents and wherein said word frequencies in said predefined misspelling criteria are based on said word counts.

6. The method of claim 5, further comprising the step of suggesting a correction of said at least one given word using said word within said predefined edit distance and having a frequency above said predefined high threshold.

7. The method of claim 6, further comprising the step of maintaining a count of the total words in said one or more documents and suggesting said correction only if said total word count satisfies a predefined criteria.

8. The method of claim 1, wherein said edit distance comprises determining a distance N that said at least one given word is from a second word, whereby said at least one given word can be transformed into the second word via a sequence of N steps of one or more of transposing adjacent characters, deleting a character, inserting a character, and replacing a character with another character.

9. The method of claim 1, wherein said predefined low threshold and said predefined high threshold are specified by a user.

10. The method of claim 1, wherein said at least one given word comprises one or more words or phrases.

11. The method of claim 1, further comprising the step of employing a reference lexicon of valid words and wherein said step of identifying a given word as a potentially misspelled word does not identify a given word as a potentially misspelled word if said given word appears in said reference lexicon.

12. The method of claim 1, wherein said maximum edit distance is a function of a length of the at least one given word.

13. A system for detecting a spelling error in one or more documents, said system comprising:

a memory; and

at least one processor, coupled to the memory, operative to;

obtain a maximum edit distance at which a word, w, is to be considered a possible misspelling of another word, w′

;

determine if at least one given word in said one or more documents satisfies a predefined misspelling criteria, wherein said predefined misspelling criteria comprises said at least one given word having a frequency below a predefined low threshold and said at least one given word being within the obtained maximum edit distance of one or more other words in said one or more documents having a frequency above a predefined high threshold;

identify a given word as a potentially misspelled word if said given word satisfies said predefined misspelling criteria; and

maintain a lexicon such that said lexicon will include said given word if said given word does not satisfy said predefined misspelling criteria and will exclude said given word if said given word satisfies said predefined misspelling criteria.

14. The system of claim 13, wherein said processor is further configured to add all words in said one or more documents to a lexicon, determine a frequency of occurrence of each of said words in said one or more documents and remove said potentially misspelled word from said lexicon.

15. The system of claim 13, wherein said processor is further configured to add all words in said one or more documents to a lexicon, determine a frequency of occurrence of each of said words in said one or more documents and mark said potentially misspelled word within said lexicon in a manner that indicates that the word may be misspelled.

16. The system of claim 13, wherein said processor is further configured to present said potentially misspelled word to a user for validation.

17. The system of claim 13, wherein said one or more documents comprise documents being processed by at least one user, and wherein said processor is further configured to maintain a word count for each word in said one or more documents and wherein said word frequencies in said predefined misspelling criteria are based on said word counts.

18. The system of claim 17, wherein said processor is further configured to suggest a correction of said at least one given word using said word within said predefined edit distance and having a frequency above said predefined high threshold.

19. The system of claim 18, wherein said processor is further configured to maintain a count of the total words in said one or more documents and suggesting said correction only if said total word count satisfies a predefined criteria.

20. The system of claim 13, wherein said at least one given word comprises one or more words or phrases.

21. The system of claim 13, wherein said at least one given word comprises one or more nucleotide sequences.

22. The system of claim 13, wherein said processor is further configured to employ a reference lexicon of valid words and wherein said step of identifying a given word as a potentially misspelled word does not identify a given word as a potentially misspelled word if said given word appears in said reference lexicon.

23. The system of claim 13, wherein said maximum edit distance is a function of a length of the at least one given word.

24. An article of manufacture for detecting a spelling error in one or more documents, comprising a tangible machine readable recordable medium containing one or more programs which when executed implement the steps of:

obtaining a maximum edit distance at which a word, w, is to be considered a possible misspelling of another word, w′

;

determining if at least one given word in said one or more documents satisfies a predefined misspelling criteria, wherein said predefined misspelling criteria comprises said at least one given word having a frequency below a predefined low threshold and said at least one given word being within the obtained maximum edit distance of one or more other words in said one or more documents having a frequency above a predefined high threshold;

identifying a given word as a potentially misspelled word if said given word satisfies said predefined misspelling criteria;

maintaining a lexicon such that said lexicon will include said given word if said given word does not satisfy said predefined misspelling criteria and will exclude said given word if said given word satisfies said predefined misspelling criteria.

25. The article of manufacture of claim 24, wherein said maximum edit distance is a function of a length of the at least one given word.