Hybrid on-line handwriting recognition and optical character recognition system

• US 6,011,865 A
• Filed: 05/12/1993
• Issued: 01/04/2000
• Est. Priority Date: 05/12/1993
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Modlin, William David, Mohiuddin, Kottappuram Mohammedali, Fujisaki, Tetsunosuke, Takahashi, Hiroyasu
• Primary Examiner(s): Kelley, Christopher S.
• Application Number:
  US08/061,495
• Time in Patent Office: 2,428 Days
• Field of Search: 382/3, 382/7, 382/9, 382/13, 382/14, 382/15, 382/20, 382/21, 382/25, 382/18, 382/19, 382/26, 382/27, 382/37, 382/41, 382/47, 382/187, 382/189, 382/156-159, 382/310
• US Class Current: 382/189
• CPC Class Codes: G06F 18/256 of results relating to different input data, e.g. multimodal recognition
• CPC Class Codes: G06V 30/1918 Fusion techniques, i.e. combining data from various sources, e.g. sensor fusion
• CPC Class Codes: G06V 30/347 Sampling; Contour coding; Stroke extraction
• CPC Class Codes: G06V 30/36 Matching; Classification

Abstract

The hybrid handwriting recognition method includes the steps of (a), in response to a handwriting input from a user, providing dynamic, time ordered stroke information; (b) determining a first list comprised of at least one probable character that the dynamic, time ordered stroke information is intended to represent; (c) converting the dynamic, time ordered stroke information to static stroke information; (d) determining a second list comprised of at least one probable character that the static stroke information represents; and (e) merging the first list and the second list to provide a third, unified list comprised of at least one element representing a most probable character that the dynamic, time ordered stroke information is intended to represent. The step of converting includes the steps of generating a static, bit-mapped representation of the dynamic stroke information, and generating one or more first stroke features based on contour directions of the bit-mapped stroke information. The first stroke features are applied as inputs to a plurality of neural network recognizers.

Claims

1. A handwriting recognition system, comprising:

handwriting transducer means, responsive to a handwriting input from a user, for outputting corresponding time ordered stroke information;

first means, having an input coupled to an output of said handwriting transducer means, for determining a first list comprised of at least one probable character that the corresponding time ordered stroke information is intended to represent;

means, having an input coupled to said output of said handwriting transducer means, for converting the corresponding time ordered stroke information to static stroke information;

second means, having an input coupled to an output of said converting means, for determining at least one second list comprised of at least one probable character that the static stroke information represents; and

means, having a first input coupled to an output of said first determining means and a second input coupled to an output of said second determining means, for merging said first list and said at least one second list to provide a third list comprised of at least one most probable character that the corresponding time ordered stroke information is intended to represent.

2. A handwriting recognition system as set forth in claim 1 wherein said converting means includes first means for generating one or more first stroke features based on contour directions of the static stroke information.

3. A handwriting recognition system as set forth in claim 2 wherein said converting means further includes second means for generating one or more second stroke features based on convexity and concavity of the static stroke information.

4. A handwriting recognition system as set forth in claim 2 wherein said second determining means is comprised of at least one neural network means having a plurality of input nodes coupled to said converting means for receiving the first stroke features, said at least one neural network means having at least one output node for outputting at least one probable character in response to the first stroke features.

5. A handwriting recognition system as set forth in claim 2 wherein said converting means includes means, responsive to the corresponding time ordered stroke information, for converting the corresponding time ordered stroke information to an N by M bit-mapped representation thereof, and wherein said first generating means is responsive to the bit-mapped representation.

6. A handwriting recognition system as set forth in claim 5 wherein said time ordered stroke converting means includes means for scaling the corresponding time-ordered stroke information such that a stroke is wholly contained within the N by M bit-mapped representation.

7. A method for operating a handwriting recognition system, comprising the steps of:

responsive to a handwriting input from a user, providing corresponding dynamic, time ordered stroke information;

converting the corresponding dynamic, time ordered stroke information to static stroke information;

determining a first list comprised of at least one probable character that the corresponding dynamic, time ordered stroke information is intended to represent;

determining a second list comprised of at least one probable character that the static stroke information represents; and

merging the first list and the second list to provide a third, unified list comprised of at least one most probable character that the corresponding dynamic, time ordered stroke information is intended to represent.

8. A method as set forth in claim 7 wherein the step of converting includes a step of generating one or more first stroke features based on contour directions of the static stroke information.

9. A method as set forth in claim 7 wherein the step of converting includes a further step of generating one or more second stroke features based on convexity and concavity of the static stroke information.

10. A method as set forth in claim 8 wherein the step of determining a second list is accomplished with at least one neural network means having a plurality of input nodes for receiving the first stroke features, the at least one neural network means having at least one output node for outputting at least one probable character in response to the first stroke features.

11. A method as set forth in claim 7 wherein the step of converting includes a step of mapping the corresponding dynamic, time ordered stroke information into an N by M bit-mapped representation thereof.

12. A method as set forth in claim 11 wherein the step of mapping includes a step of scaling the corresponding dynamic, time-ordered stroke information such that a stroke is wholly contained within the N by M bit-mapped representation.

13. A method for recognizing a handwriting, comprising the steps of:

inputting one or more feature vectors representing dynamic stroke information;

converting the one or more feature vectors into an N×

M representation of the one or more feature vectors;

counting occurrences of individual ones of a plurality of primitive features within the N×

M representation;

generating, in response to the counted occurrences, at least one first list comprised of one or more symbols that the N×

M bit-mapped image may represent;

generating, in response to the feature vectors, a second list comprised of one or more symbols that the feature vectors may represent; and

generating a third list from the at least one first list and the second list, the third list including at least one most probable symbol that the feature vectors may represent.

14. A method as set forth in claim 13 wherein the step of inputting includes a step of segmenting the feature vectors.

15. A method as set forth in claim 13 wherein the step of generating at least one first list includes a step of applying the counted occurrences to inputs of at least one neural network means, the at least one neural network means having been trained to recognize at least one predetermined symbol.

16. A method as set forth in claim 13 wherein the step of generating at least one first list includes a step of applying the counted occurrences to inputs of individual ones of a plurality of neural network means, each of the plurality of neural network means having been trained to recognize at least one predetermined symbol within a predetermined class of symbols.

17. A method as set forth in claim 13 wherein the step of counting occurrences includes the steps of:

partitioning the N×

M bit-mapped image into a plurality of slices;

applying an ixj mask over the N×

M bit-mapped image to locate instances of primitive features; and

counting the located instances within each of the plurality of slices.

18. A method as set forth in claim 17 wherein the step of applying includes a step of identifying primitive features based on contour direction.

19. A method as set forth in claim 17 wherein the step of applying includes a step of identifying primitive features based on convexity and concavity.

20. A method as set forth in claim 17 wherein the step of applying includes a step of identifying a first set of primitive features based on contour direction and a second set of feature vectors based on convexity and concavity.