Systems and methods for processing text-based electronic documents

• US 7,106,905 B2
• Filed: 08/23/2002
• Issued: 09/12/2006
• Est. Priority Date: 08/23/2002
• Status: Active Grant

• Current Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Original Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Inventors: Simske, Steven J.
• Primary Examiner(s): CHAWAN, SHEELA C
• Application Number:
  US10/227,076
• Publication Number:
  US 20040037470A1
• Time in Patent Office: 1,481 Days
• Field of Search: 382/177, 382/181, 382/174, 382/176, 382/229, 382/310, 382/203, 382/289, 382/209, 382/218, 707/2, 707/3, 707/5, 707/6, 707/101
• US Class Current: 382/229
• CPC Class Codes: G06F 40/232 Orthographic correction, e.g. spell checking or vowelisation
• CPC Class Codes: G06F 40/242 Dictionaries
• CPC Class Codes: G06V 30/10 Character recognition
• CPC Class Codes: G06V 30/12 Detection or correction of errors, e.g. by rescanning the pattern
• CPC Class Codes: G06V 30/246 using linguistic properties, e.g. specific for English or German language
• CPC Class Codes: G06V 30/268 Lexical context
• CPC Class Codes: Y10S 707/99935 Query augmenting and refining, e.g. inexact access
• CPC Class Codes: Y10S 707/99942 Manipulating data structure, e.g. compression, compaction, compilation

Abstract

Systems and methods for processing text-based electronic documents are provided. Briefly described, one embodiment of a method for processing a text-based electronic document comprises the steps of: comparing at least one word in a text-based electronic document to a native language dictionary to determine whether the at least one word conforms to a predefined rule; for each of the at least one word that does not conform to the predefined rule, fragmenting the at least one word into word fragments; combining at least two consecutive word fragments; and comparing the combination of the word fragments to the native language dictionary.

Claims

1. A method for processing a text-based electronic document, the method comprising:

performing optical character recognition processing on the text-based electronic document by comparing at least one word in the text-based electronic document to a native language dictionary to determine whether the at least one word conforms to a predefined rule;

if the at least one word does not conform to the predefined rule;

fragmenting the at least one word into word fragments;

combining at least two consecutive word fragments of the at least one word to form a combination of the word fragments; and

comparing the combination of the word fragments to the native language dictionary such that, if the combination of the word fragments conforms to the predefined rule, the combination is used in the text-based electronic document; and

determining whether the at least one word matches a combination of;

a word entry in the native language dictionary; and

at least one of a common prefix and a common suffix such that, if the combination conforms to the predefined rule, the combination is used in the text-based electronic document.

2. The method of claim 1, further comprising generating the text-based electronic document with at least one optical character recognition system.

3. The method of claim 2, wherein the comparing the at least one word comprises:

modifying the at least one word based on predetermined statistics related to the character recognition accuracy of the optical character recognition system; and

determining whether the at least one word as modified matches a word entry in the native language dictionary.

4. The method of claim 3, wherein the modifying involves modifying the at least one word based on predetermined statistics related to at least one of the following:

a

glyph mistake, a glyph substitution, a glyph insertion, and a glyph deletion.

5. The method of claim 3, wherein the modifying involves modifying the at least one word based on predetermined statistics, which define two or more modified words.

6. The method of claim 1, further comprising modifying the text-based electronic document with the combination of the word fragments based on the comparison to the native language dictionary.

7. The method of claim 1, further comprising:

parsing the text-based electronic document into words.

8. The method of claim 1, wherein the comparing the at least one word comprises determining whether the at least one word matches a word entry in the native language dictionary.

9. The method of claim 1, further comprising:

scanning a document;

converting the document into a document image; and

generating the text-based electronic document with the document image.

10. The method of claim 9, wherein the generating the text-based electronic document comprises:

extracting a text portion from the document image; and

recognizing characters in the text portion.

11. A system for processing a text-based electronic document, the system comprising:

logic configured to perform optical character recognition processing, the logic being operative to;

compare at least one word in the text-based electronic document to a native language dictionary to determine whether the at least one word conforms to a predefined rule;

fragment the at least one word into word fragments if the at least one word does not conform to the predefined rule;

combine at least two consecutive word fragments of the at least one word to form a combination of the word fragments; and

compare the combination of the word fragments to the native language dictionary such that, if the combination of the word fragments conforms to the predefined rule, the combination is used in the text-based electronic document; and

determine whether the at least one word matches a combination of;

a word entry in the native language dictionary; and

at least one of a common prefix and a common suffix if the at least one word does not conform to the predefined rule such that, if the combination conforms to the predefined rule, the combination is used in the text-based electronic document.

12. The system of claim 11, further comprising a memory containing the native language dictionary.

13. The system of claim 11, wherein the text-based electronic document comprises a text file.

14. The system of claim 11, wherein the logic is further configured to modify the text-based electronic document with the combination of the word fragments based on the comparing of the at least one word to the native language dictionary.

15. The system of claim 11, wherein the logic is further configured to:

parse a text representation of the text-based electronic document into words.

16. The system of claim 11, wherein the logic configured to compare the at least one word is further configured to determine whether the at least one word matches a word entry in the native language dictionary.

17. The system of claim 11, wherein the logic is further configured to:

modify the at least one word based on predetermined statistics related to the character recognition accuracy of the optical character recognition system; and

determine whether the at least one word as modified matches a word entry in the native language dictionary.

18. The system of claim 17, wherein the predetermined statistics related to the character recognition accuracy of the optical character recognition system are related to at least one of the following:

a glyph mistake, a glyph substitution, a glyph insertion, and a glyph deletion.

19. The system of claim 17, wherein the predetermined statistics define at least two modified words and wherein the logic configured to determine whether the at least one word as modified matches a word entry in the native language dictionary is prioritized based on the predetermined statistics.

20. The system of claim 11, further comprising an optical character recognition system.

21. The system of claim 20, wherein the optical character recognition system comprises:

a scanning device configured to convert a document into a document image; and

an OCR engine configured to generate the text-based electronic document by extracting a text portion from the document image and recognizing characters in the text portion.

22. A system for processing a text-based electronic document, the system comprising:

a means for comparing, during optical character recognition processing, at least one word in the text-based electronic document to a native language dictionary to determine whether the at least one word conforms to a predefined rule;

a word fragmentation means for fragmenting the at least one word into word fragments if the at least one word does not conform to the predefined rule;

a word fragment integration means for combining at least two consecutive word fragments of the at least one word to form a combination of the word fragments;

a means for comparing the combination of the word fragments to the native language dictionary such that, if the combination of the word fragments conforms to the predefined rule, the combination is used in the text-based electronic document; and

a means for determining whether the at least one word matches a combination of;

a word entry in the native language dictionary; and

at least one of a common prefix and a common suffix such that, if the combination conforms to the predefined rule, the combination is used in the text-based electronic document.

23. The system of claim 22, further comprising a means for storing the native language dictionary.

24. The system of claim 22, further comprising:

a word parser means for parsing the text representation of the text-based electronic document into words.

25. The system of claim 22, further comprising:

a means for scanning a document into a document image; and

a character recognition means for generating the text-based electronic document by extracting a text portion from the document image and recognizing characters in the text portion.