Systems and methods for processing text-based electronic documents

• US 20040037470A1
• Filed: 08/23/2002
• Published: 02/26/2004
• Est. Priority Date: 08/23/2002
• Status: Active Grant

• Current Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Original Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Inventors: Simske, Steven J.
• Granted Patent: US 7,106,905 B2
• Time in Patent Office: Days
• US Class Current: 382/229
• CPC Class Codes: G06F 40/232 Orthographic correction, e.g. spell checking or vowelisation
• CPC Class Codes: G06F 40/242 Dictionaries
• CPC Class Codes: G06V 30/10 Character recognition
• CPC Class Codes: G06V 30/12 Detection or correction of errors, e.g. by rescanning the pattern
• CPC Class Codes: G06V 30/246 using linguistic properties, e.g. specific for English or German language
• CPC Class Codes: G06V 30/268 Lexical context
• CPC Class Codes: Y10S 707/99935 Query augmenting and refining, e.g. inexact access
• CPC Class Codes: Y10S 707/99942 Manipulating data structure, e.g. compression, compaction, compilation

Abstract

Systems and methods for processing text-based electronic documents are provided. Briefly described, one embodiment of a method for processing a text-based electronic document comprises the steps of: comparing at least one word in a text-based electronic document to a native language dictionary to determine whether the at least one word conforms to a predefined rule; for each of the at least one word that does not conform to the predefined rule, fragmenting the at least one word into word fragments; combining at least two consecutive word fragments; and comparing the combination of the word fragments to the native language dictionary.

Claims

1. A method for processing a text-based electronic document, the method comprising the steps of:

comparing at least one word in a text-based electronic document to a native language dictionary to determine whether the at least one word conforms to a predefined rule;

for each of the at least one word that does not conform to the predefined rule, fragmenting the at least one word into word fragments;

combining at least two consecutive word fragments; and

comparing the combination of the word fragments to the native language dictionary.

2. The method of claim 1, further comprising the step of generating the text-based electronic document by using at least one optical character recognition system.

3. The method of claim 2, wherein the comparing comprises:

modifying the at least one word based on predetermined statistics related to the character recognition accuracy of the optical character recognition system; and

determining whether the at least one word as modified matches one of the word entries in the native language dictionary.

4. The method of claim 3, wherein the modifying involves modifying the at least one word based on predetermined statistics related to at least one of the following:

a glyph mistake, a glyph substitution, a glyph insertion, and a glyph deletion.

5. The method of claim 3, wherein the modifying involves modifying the at least one word based on predetermined statistics, which define two or more modified words.

6. The method of claim 1, further comprising modifying the text-based electronic document with the combination of the word fragments based on the comparison to the native language dictionary.

7. The method of claim 1, further comprising:

receiving the text-based electronic document from an optical character recognition system; and

parsing the text-based electronic document into words.

8. The method of claim 1, wherein the comparing comprises determining whether the at least one word matches one of the word entries in the native language dictionary.

9. The method of claim 1, wherein the comparing comprises determining whether the at least one word matches a combination of one of the word entries in the native language dictionary and at least one of a common prefix and a common suffix.

10. The method of claim 1, further comprising:

scanning a document;

converting the document into a document image; and

generating the text-based electronic document.

11. The method of claim 10, wherein the generating the text-based electronic document comprises:

extracting a text portion from the document image; and

recognizing characters in the text portion.

12. A system for processing a text-based electronic document, the system comprising:

logic configured to;

compare at least one word in a text-based electronic document to a native language dictionary to determine whether the at least one word conforms to a predefined rule;

fragment each of the at least one word that does not conform to the predefined rule into word fragments;

combine at least two consecutive word fragments; and

compare the combination of the word fragments to the native language dictionary.

13. The system of claim 12, further comprising a memory containing the native language dictionary.

14. The system of claim 12, wherein the text-based electronic document comprises a document processed by at least one optical character recognition system.

15. The system of claim 12, wherein the text-based electronic document comprises a text file.

16. The system of claim 12, wherein the logic is further configured to modify the text-based electronic document with the combination of the word fragments based on the comparison to the native language dictionary.

17. The system of claim 12, wherein the logic is further configured to:

receive the text-based electronic document from an optical character recognition system; and

parse the text representation of the document into words.

18. The system of claim 12, wherein the logic configured to compare is further configured to determine whether the at least one word matches one of the word entries in the native language dictionary.

19. The system of claim 12, wherein the logic configured to is further configured to determine whether the at least one word matches a combination of one of the word entries in the native language dictionary and at least one of a common prefix and a common suffix.

20. The system of claim 12, wherein the logic configured to is further configured to:

modify the at least one word based on predetermined statistics related to the character recognition accuracy of the optical character recognition system; and

determine whether the at least one word as modified matches one of the word entries in the native language dictionary.

21. The system of claim 20, wherein the predetermined statistics related to the character recognition accuracy of the optical character recognition system are related to at least one of the following:

a glyph mistake, a glyph substitution, a glyph insertion, and a glyph deletion.

22. The system of claim 20, wherein the predetermined statistics define at least two modified words and wherein the logic configured to determine whether the at least one word as modified matches one of the word entries in the native language dictionary is prioritized based on the predetermined statistics.

23. The system of claim 12, further comprising an optical character recognition system.

24. The system of claim 23, wherein the optical character recognition system comprises:

a scanning device configured to convert a document into a document image; and

an OCR engine configured to generate the text-based electronic document by extracting a text portion from the document image and recognizing characters in the text portion.

25. A system for processing a text-based electronic document, the system comprising:

a means for comparing at least one word in a text-based electronic document to a native language dictionary to determine whether the at least one word conforms to a predefined rule;

a word fragmentation means for fragmenting each of the at least one word that does not conform to the predefined rule into word fragments;

a word fragment integration means for combining at least two consecutive word fragments; and

a means for comparing the combination of the word fragments to the native language dictionary.

26. The system of claim 25, further comprising a means for storing a native language dictionary.

27. The system of claim 25, further comprising:

a means for receiving the text-based electronic document from an optical character recognition system; and

a word parser means for parsing the text representation of the document into words.

28. The system of claim 25, further comprising:

a means for scanning a document into a document image; and

a character recognition means for generating the text-based electronic document by extracting a text portion from the document image and recognizing characters in the text portion.