Automatic handwriting recognition using both static and dynamic parameters

• US 5,539,839 A
• Filed: 05/25/1995
• Issued: 07/23/1996
• Est. Priority Date: 01/27/1993
• Status: Expired due to Term

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Bellegarda, Jerome R., Nahamoo, David, Nathan, Krishna S.
• Primary Examiner(s): Boudreau, Leo
• Assistant Examiner(s): Tran, Phuoc
• Application Number:
  US08/450,558
• Time in Patent Office: 425 Days
• Field of Search: 382/187, 382/188, 382/189, 382/186, 382/159, 382/160, 382/161, 382/120, 382/123, 382/190, 382/198, 382/203, 382/225, 382/227, 382/228
• US Class Current: 382/187
• CPC Class Codes: G06F 18/256 of results relating to different input data, e.g. multimodal recognition
• CPC Class Codes: G06V 30/1918 Fusion techniques, i.e. combining data from various sources, e.g. sensor fusion
• CPC Class Codes: G06V 30/347 Sampling; Contour coding; Stroke extraction
• CPC Class Codes: G06V 30/373 using a special pattern or subpattern alphabet

Abstract

Methods and apparatus are disclosed for recognizing handwritten characters in response to an input signal from a handwriting transducer. A feature extraction and reduction procedure is disclosed that relies on static or shape information, wherein the temporal order in which points are captured by an electronic tablet may be disregarded. A method of the invention generates and processes the tablet data with three independent sets of feature vectors which encode the shape information of the input character information. These feature vectors include horizontal (x-axis) and vertical (y-axis) slices of a bit-mapped image of the input character data, and an additional feature vector to encode an absolute y-axis displacement from a baseline of the bit-mapped image. It is shown that the recognition errors that result from the spatial or static processing are quite different from those resulting from temporal or dynamic processing. Furthermore, it is shown that these differences complement one another. As a result, a combination of these two sources of feature vector information provides a substantial reduction in an overall recognition error rate. Methods to combine probability scores from dynamic and the static character models are also disclosed.

Claims

1. A handwriting recognition system, comprising:

means for sampling handwriting inputs from at least one writer;

means for providing both static and dynamic parameter vector representations of said sampled handwriting inputs;

means for providing both static and dynamic spliced vector representations of said sampled handwriting inputs;

means for providing both static and dynamic feature vector representations of said sampled handwriting inputs; and

means for estimating a probability that a sampled handwriting input is one of a predetermined set of symbols in accordance with at least said dynamic feature vector representation and a first predetermined set of symbol prototypes derived from temporal characteristics of input handwritings, and in accordance with at least said static feature vector representation and a second predetermined set of symbol prototypes derived from spatial characteristics of input handwritings, said estimating means including means for outputting a most likely symbol that said sampled handwriting input represents.

2. A handwriting recognition system as set forth in claim 1, and further comprising:

means for determining a covariance matrix of each of said static and dynamic spliced vector representations;

means for determining eigenvalues and eigenvectors associated with each of the determined covariance matrices; and

means for applying a transformation to said determined eigenvectors for providing said static feature vector representations and said dynamic feature vector representations.

3. A handwriting recognition system as set forth in claim 2, and further comprising:

means for performing clustering in both a static feature vector space and in a dynamic feature vector space to provide both static and dynamic prototype distributions in said feature vector spaces;

means for performing Gaussian modelling in each of said feature vector spaces; and

means for determining both static and dynamic mixture coefficients for evaluating relative contributions of each prototype distribution to a current sample of handwriting inputs.

4. A handwriting recognition system, comprising:

means for sampling handwriting inputs from at least one writer;

means for providing both static and dynamic parameter vector representations of said sampled handwriting inputs;

means for providing both static and dynamic spliced vector representations of said sampled handwriting inputs;

means for providing both static and dynamic feature vector representations of said sampled handwriting inputs;

means for determining a covariance matrix of each of said static and dynamic spliced vector representations;

means for determining eigenvalues and eigenvectors associated with each of the determined covariance matrices;

means for applying a transformation to said determined eigenvectors for providing said static feature vector representations and said dynamic feature vector representations;

means for performing clustering in both a static feature vector space and in a dynamic feature vector space to provide both static and dynamic prototype distributions in said feature vector spaces;

means for performing Gaussian modelling in each of said feature vector spaces; and

means for determining both static and dynamic mixture coefficients for evaluating relative contributions of each prototype distribution to a current sample of handwriting inputs, said system further comprising;

means for recognizing a first candidate handwriting in accordance with a probabilistic comparison based at least on the static prototype distributions and on the static mixture coefficients;

means for recognizing a second candidate handwriting in accordance with a probabilistic comparison based at least on the dynamic prototype distributions and on the dynamic mixture coefficients; and

means for recognizing a most probable handwriting in accordance with a combination of the first and the second candidate handwritings.

5. A method for operating a handwriting recognition system, comprising the steps of:

sampling handwriting inputs from at least one writer;

providing both static and dynamic parameter vector representations of the sampled handwriting inputs;

providing both static and dynamic spliced vector representations of the sampled handwriting inputs;

providing both static and dynamic feature vector representations of the sampled handwriting inputs;

estimating a probability that a sampled handwriting input is one of a predetermined set of symbols in accordance with at least the dynamic feature vector representation and a first predetermined set of symbol prototypes derived from temporal characteristics of input handwritings, and in accordance with at least the static feature vector representation and a second predetermined set of symbol prototypes derived from spatial characteristics of input handwritings; and

outputting a most likely symbol that said sampled handwriting input represents.

6. A method as set forth in claim 5, and further comprising the steps of:

determining a covariance matrix of each of the static and dynamic spliced vector representations;

determining eigenvalues and eigenvectors associated with each of the determined covariance matrices; and

applying a transformation to the determined eigenvectors for providing the static feature vector representations and the dynamic feature vector representations.

7. A method as set forth in claim 6, and further comprising the steps of:

performing clustering in both a static feature vector space and in a dynamic feature vector space to provide both static and dynamic prototype distributions in the feature vector spaces;

performing Gaussian modelling in each of the feature vector spaces; and

determining both static and dynamic mixture coefficients for evaluating relative contributions of each prototype distribution to a current sample of handwriting inputs.

8. A method for operating a handwriting recognition system, comprising the steps of:

sampling handwriting inputs from at least one writer;

providing both static and dynamic parameter vector representations of the sampled handwriting inputs;

providing both static and dynamic spliced vector representations of the sampled handwriting inputs;

providing both static and dynamic feature vector representations of the sampled handwriting inputs;

determining a covariance matrix of each of the static and dynamic spliced vector representations;

determining eigenvalues and eigenvectors associated with each of the determined covariance matrices;

applying a transformation to the determined eigenvectors for providing the static feature vector representations and the dynamic feature vector representations;

performing clustering in both a static feature vector space and in a dynamic feature vector space to provide both static and dynamic prototype distributions in the feature vector spaces;

performing Gaussian modelling in each of the feature vector spaces;

determining both static and dynamic mixture coefficients for evaluating relative contributions of each prototype distribution to a current sample of handwriting inputs; and

further comprising the steps of;

recognizing a first candidate handwriting in accordance with a probabilistic comparison based at least on the static prototype distributions and on the static mixture coefficients;

recognizing a second candidate handwriting in accordance with a probabilistic comparison based at least on the dynamic prototype distributions and on the dynamic mixture coefficients; and

recognizing a most probable handwriting in accordance with a combination of the first and the second candidate handwritings.