System and method for automated reading of handwriting

• US 20070286486A1
• Filed: 05/30/2006
• Published: 12/13/2007
• Est. Priority Date: 05/30/2006
• Status: Active Grant

• Current Assignee: The Standard Register Company (Middleby Corporation)
• Original Assignee: Expedata, LLC
• Inventors: Goldstein, Ira P.
• Granted Patent: US 7,742,642 B2
• Time in Patent Office: Days
• US Class Current: 382/187
• CPC Class Codes: G06F 18/254 of classification results, e.g. of results related to same input data
• CPC Class Codes: G06V 30/10 Character recognition
• CPC Class Codes: G06V 30/1444 Selective acquisition, locating or processing of specific regions, e.g. highlighted text, fiducial marks or predetermined fields
• CPC Class Codes: G06V 30/1918 Fusion techniques, i.e. combining data from various sources, e.g. sensor fusion
• CPC Class Codes: G06V 30/2272 with lexical matching
• CPC Class Codes: G06V 30/36 Matching; Classification

Abstract

A system for automatically recognizing a handwriting image and converting such image to text data including a sequence of validated words, has an image input device, a number of handwriting recognition engines, and control unit. A first handwriting recognition engine is responsive to the image input device, for analyzing the data file and providing one or more possible text words for each successive word in the data file. The first handwriting recognition engine further provides a resemblance indication for each possible text word indicating a level of resemblance between its appearance and the appearance of the handwritten word in the data file. In the event that there is not a high level of confidence in the selection of the first handwriting recognition engine, a selection of a validated word is based on the selections of one or more of the other handwriting recognition engines.

Claims

1. A system for automatically recognizing a handwriting image and converting such image to text data including a sequence of validated words, comprising:

an image input device for inputting as a data file the handwriting image that is to be converted to text data,

a first handwriting recognition engine, responsive to said image input device, for analyzing said data file and providing one or more possible text words for each successive word in said data file, said first handwriting recognition engine further providing a resemblance indication for each possible text word indicating a level of resemblance between it and the handwritten word in the data file,

a second handwriting recognition engine for analyzing words in said data file for which there is not a high level of confidence regarding the analysis performed by the first handwriting recognition engine, said second handwriting recognition engine providing one or more possible text words and a resemblance indication for each possible text word, and

a control unit for selecting as a validated word the text word selected by the first handwriting recognition engine in the event that that text word has a high level of resemblance and a high level of confidence, and for selecting as a validated word the text word selected by the second handwriting recognition engine in the event that several text words selected by the first handwriting recognition engine have a high level of resemblance, and the text word selected by the second handwriting engine is one of the several text words selected by the first handwriting recognition engine as having a high resemblance indication.

2. The system according to claim 1 for automatically recognizing a handwriting image and converting such image to text data including a sequence of validated words, in which said image input device comprises a device for monitoring movement of the hand of a user.

3. The system according to claim 1 for automatically recognizing a handwriting image and converting such image to text data including a sequence of validated words, in which said first handwriting recognition engine includes a database comprising a plurality of words which may be selected as possible text words.

4. The system according to claim 1 for automatically recognizing a handwriting image and converting such image to text data including a sequence of validated words, in which said first and second handwriting recognition engines utilize differing criteria in analyzing said data file.

5. The system according to claim 1 for automatically recognizing a handwriting image and converting such image to text data including a sequence of validated words, in which a high level of confidence for a text word selected by a handwriting recognition engine is indicated by a high value resemblance indication for only the single selected text word.

6. The system according to claim 1 for automatically recognizing a handwriting image and converting such image to text data including a sequence of validated words, further comprising a preprocessing engine for deleting superfluous data from said data file before analysis of said data file by said first handwriting recognition engine.

7. The system according to claim 1 for automatically recognizing a handwriting image and converting such image to text data including a sequence of validated words, in which said control unit selects as a validated word the text word selected by the second handwriting recognition engine having a high resemblance indication in the event that the text words selected by the first handwriting recognition engine have a low level of resemblance.

8. A system for automatically recognizing a handwriting image and converting such image to text data including a sequence of validated words, comprising:

an image input device for inputting as a data file the handwriting image that is to be converted to text data,

a first handwriting recognition engine, responsive to said image input device, for analyzing said data file and providing one or more possible text words for each successive word in said data file, said first handwriting recognition engine further providing a resemblance indication for each possible text word indicating a level of resemblance between it and the handwritten word in the data file,

a plurality of additional handwriting recognition engines for analyzing words in said data file for which there is not a high level of confidence regarding the analysis performed by the first handwriting recognition engine, each of said plurality of additional handwriting recognition engines providing one or more possible text words and a resemblance indication for each possible text word, and

a control unit selecting as a validated word the text word selected by said first handwriting recognition engine in the event that that text word has a high level of resemblance and a high level of confidence, and, in the event that several text words selected by the first handwriting recognition engine have a high level of resemblance, selecting as a validated word the text word selected by said plurality of additional handwriting recognition engines as having the highest level of resemblance, provided that the text word selected by said plurality of additional handwriting engines is one of the several text words selected by the first handwriting recognition engine as having a high resemblance indication.

9. The system according to claim 8 for automatically recognizing a handwriting image and converting such image to text data including a sequence of validated words, in which said image input device comprises a device for monitoring movement of the hand of a user.

10. The system according to claim 8 for automatically recognizing a handwriting image and converting such image to text data including a sequence of validated words, in which said first handwriting recognition engine includes a database comprising a plurality of words which may be selected as possible text words.

11. The system according to claim 8 for automatically recognizing a handwriting image and converting such image to text data including a sequence of validated words, in which said first and said additional handwriting recognition engines utilize differing criteria in analyzing said data file.

12. The system according to claim 8 for automatically recognizing a handwriting image and converting such image to text data including a sequence of validated words, in which a high level of confidence for a text word selected by a handwriting recognition engine is indicated by a high value resemblance indication for only the single selected text word.

13. The system according to claim 8 for automatically recognizing a handwriting image and converting such image to text data including a sequence of validated words, further comprising a preprocessing engine for deleting superfluous data from said data file before analysis of said data file by said first handwriting recognition engine.

14. The system according to claim 8 for automatically recognizing a handwriting image and converting such image to text data including a sequence of validated words, in which said control unit selects as a validated word the text word selected by said additional handwriting recognition engines having a high resemblance indication in the event that the text words selected by the first handwriting recognition engine have a low level of resemblance.

15. A method of automatically recognizing a handwriting image and converting such image to text data including a sequence of validated words, comprising the steps of:

inputting a data file of a handwriting image that is to be converted to text data,

analyzing said data file utilizing a first handwriting recognition engine, and providing one or more possible text words for each successive word in said data file, and a resemblance indication for each of said possible text words indicating a level of resemblance between it and the handwritten word in the data file,

analyzing words in said data file utilizing a second handwriting recognition engine, where there is not a high level of confidence regarding the analysis performed by the first handwriting recognition engine, said second handwriting recognition engine providing one or more possible text words and a resemblance indication indicating a level of resemblance between it and the handwritten word in the data file, and

selecting as a validated word the text word selected by the first handwriting recognition engine in the event that that text word has a high level of resemblance and a high level of confidence, and

selecting as a validated word the text word selected by the second handwriting recognition engine in the event that several text words selected by the first handwriting recognition engine have a high level of resemblance, and the text word selected by the second handwriting engine is one of the several text words selected by the first handwriting recognition engine as having a high resemblance.

16. The method of automatically recognizing a handwriting image and converting such image to text data including a sequence of validated words according to claim 15, in which the step of inputting a data file of a handwriting image that is to be converted to text data includes the step of monitoring movement of the hand of a user.

17. The method of automatically recognizing a handwriting image and converting such image to text data including a sequence of validated words according to claim 15, further includes the step of preprocessing said data file before analysis of said data file by said first handwriting recognition engine to delete superfluous data from said data file.

18. The method of automatically recognizing a handwriting image and converting such image to text data including a sequence of validated words according to claim 15, in which said first and second handwriting recognition engines utilize differing criteria in analyzing said data file.

19. The method of automatically recognizing a handwriting image and converting such image to text data including a sequence of validated words according to claim 15, further comprising the step of selecting as a validated word the text word selected by the second handwriting recognition engine having a high resemblance indication in the event that the text words selected by the first handwriting recognition engine have a low level of resemblance.

20. The method of automatically recognizing a handwriting image and converting such image to text data including a sequence of validated words according to claim 15, further comprising the step of providing a database comprising a plurality of words which may be selected as possible text words.

21. A system for automatically recognizing a handwriting image and converting such image to text data including a sequence of validated words, comprising:

an image input device for inputting as a data file the handwriting image that is to be converted to text data,

one or more handwriting recognition engines, responsive to said image input device, for analyzing said data file and providing a text word for each successive word in said data file, and

a control unit for controlling operation of said one or more handwriting recognition engines, said control unit removing noise, identifying cross outs, removing overwrites and normalizing the sequence of strokes made in producing said handwriting image, said control unit selecting as a validated word the text word selected by said handwriting recognition engines.

22. A system for automatically recognizing handwriting images on a form having a plurality of fields and field areas associated with said fields, with each field having an associated data type, and converting such images to text data including a sequence of validated words, comprising:

an image input device for inputting as a data file the handwriting image that is to be converted to text data, said handwriting image including a plurality of images with each image associated with a respective one of a plurality of fields on the form being read,

one or more handwriting recognition engines, responsive to said image input device, for analyzing said data file, comparing words in said data file to text words in a data type lexicon, and providing a text word for each successive word in said data file based on said comparisons, and

a control unit for controlling operation of said one or more handwriting recognition engines, said control unit selecting the lexicon to be used in respect to each of said images based on the data type of the field in which the image appears, said control unit selecting as a validated word the text word selected by said one or more handwriting recognition engines.

23. A system for automatically recognizing handwriting images on a form having a plurality of fields according to claim 22 in which said control unit determines the portion of the images associated with each field.

24. A system for automatically recognizing handwriting images on a form having a plurality of fields according to claim 23 in which said control unit associates images with fields even if said images are not within one of said field areas on the form.