Method and apparatus for on-line handwriting recognition based on feature vectors that use aggregated observations derived from time-sequential frames

• US 6,084,985 A
• Filed: 10/03/1997
• Issued: 07/04/2000
• Est. Priority Date: 10/04/1996
• Status: Expired due to Fees

• Current Assignee: US Philips Corporation (Koninklijke Philips N.V.)
• Original Assignee: US Philips Corporation (Koninklijke Philips N.V.)
• Inventors: Dolfing, Jannes G. A., Hab-Umbach, Reinhold
• Primary Examiner(s): Tran, Phuoc
• Assistant Examiner(s): Mariam, Daniel G.
• Application Number:
  US08/943,532
• Time in Patent Office: 1,005 Days
• Field of Search: 382/186, 382/187, 382/188, 382/203, 382/228, 382/229, 382/116
• US Class Current: 382/187
• CPC Class Codes: G06V 30/2276 with probabilistic networks, e.g. hidden Markov models
• CPC Class Codes: G06V 30/347 Sampling; Contour coding; Stroke extraction
• CPC Class Codes: G07C 9/35 by means of a handwritten signature

Abstract

A method for on-line handwriting recognition is based on a hidden Markov model and implies the following steps: sensing real-time at least an instantaneous write position of the handwriting, deriving from the handwriting a time-conforming string of segments each associated to a handwriting feature vector, matching the time-conforming string to various example strings from a data base pertaining to the handwriting, and selecting from the example strings a best-matching recognition string through hidden-Markov processing, or rejecting the handwriting as unrecognized. In particular, the feature vectors are based on local observations derived from a single segment, as well as on compacted observations derived from time-sequential segments.

Claims

1. A method for on-line handwriting recognition based on a hidden Markov model, said method comprising the steps of:

real-time sensing at least an instantaneous write position of said handwriting;

deriving from said handwriting a time-conforming string of samples, and from said string a series of handwriting intervals each associated by derivation to a handwriting feature vector, including calculating a center of gravity from the samples associated with each handwriting interval, and wherein the feature vectors contain one or more vector elements that are based on local observations derived from a single said handwriting interval, said handwriting interval comprising a handwriting frame, as well as being based on one or more vector elements that are based on aggregated observations derived from time-spaced said intervals defined over an associated delay, wherein the aggregated feature vector elements are representable as o'"'"'₁ and o'"'"'₂ as follows;

##EQU2## where o₁ and o₂ represent the vector elements that are based on local observations, d corresponds to a maximum value of the delay between successive frames, and m indicates a number of functions actually used;

matching the time-conforming sequence of feature vectors so derived to one or more example sequence modellings from a database pertaining to the handwriting in question;

selecting from said modellings a sufficiently-matching recognition string through hidden-Markov processing; and

outputting essentials of a result of said selecting, or alternatively, rejecting said handwriting as failing to be recognized.

2. The method as claimed in claim 1, wherein the aggregated feature vector elements comprise differential angles derived from the centers of gravities associated to successive handwriting intervals.

3. A method as claimed in claim 1, wherein said feature vector elements to be aggregated include spatial path lengths between principal points associated to successive handwriting intervals.

4. A method as claimed in claim 1, wherein said aggregated feature vector elements define spatially contiguous character segments generated in a temporally non-sequential manner.

5. A method as claimed in claim 1, wherein said aggregated feature vector elements include contour changes between successive character segments in a direction transverse to a general orientation of a line or writing.

6. A method as claimed in claim 1, applied for signature recognition.

7. A method as claimed in claim 1, wherein at least one aggregated feature vector element is based on sensing instantaneous writing pressure.

8. A method as claimed in claim 1, wherein at least one aggregated feature vector element is based on sensing instantaneous writing instrument tilt.

9. A data processing system for handwriting recognition comprising:

a graphical input means for real-time sensing at least an instantaneous write position of handwriting and converting said handwriting information into a handwriting signal;

a handwriting processing means for receiving and processing said handwriting signal, for deriving from said handwriting a time-conforming string of samples and from said string a series of handwriting intervals each associated by derivation to a handwriting feature vector, and for calculating a center of gravity from the samples associated with each handwriting interval, and wherein said feature vectors contain one or more vector elements that are based on local observations derived from a single said handwriting interval, said hand writing interval being selected from the group consisting of a handwriting frame and a handwriting segment, as well as being based on one or more vector elements that are based on aggregated observations derived from time-spaced said intervals defined over an associated delay, wherein the aggregated feature vector elements are representable as o'"'"'₁ and o'"'"'₂ as follows;

##EQU3## where o₁ and o₂ represent the vector elements that are based on local observations, d corresponds to a maximum value of the delay between successive frames or segments, and m indicates a number of functions actually used;

a handwriting model description database pertaining to the handwriting in question;

search and classification means for receiving a processed handwriting signal from the handwriting processing means, and for matching the time-conforming sequence of feature vectors so derived to one or more example sequence modeling from the handwriting model description database, and for selecting from said modeling a sufficiently-matching recognition string through hidden-Markov processing; and

results output means for outputting essentials of a result of said selecting, or alternatively, for rejecting said handwriting as failing to be recognized.

10. A system according to claim 9, wherein said graphical input means comprises an electromagnetically operating writing tablet.

11. A method for on-line handwriting recognition based on a hidden Markov model, said method comprising the steps of:

real-time sensing at least an instantaneous write position of said handwriting;

deriving from said handwriting a time-conforming string of samples, and from said string a series of handwriting intervals each associated by derivation to a handwriting feature vector, including calculating a center of gravity from the samples associated with each handwriting interval, and wherein the feature vectors contain one or more vector elements that are based on local observations derived from a single said handwriting interval, said handwriting interval comprising a handwriting segment, as well as being based on one or more vector elements that are based on aggregated observations derived from time-spaced said intervals defined over an associated delay, wherein the aggregated feature vector elements are representable as o'"'"'₁ and o'"'"'₂ as follows;

##EQU4## where o₁ and o₂ represent the vector elements that are based on local observations, d corresponds to a maximum value of the delay between successive segments, and m indicates a number of functions actually used;

matching the time-conforming sequence of feature vectors so derived to one or more example sequence modellings from a database pertaining to the handwriting in question;

selecting from said modellings a sufficiently-matching recognition string through hidden-Markov processing; and

outputting essentials of a result of said selecting, or alternatively, rejecting said handwriting as failing to be recognized.

12. The method as claimed in claim 11, wherein the aggregated feature vector elements comprise differential angles derived from the centers of gravities associated to successive handwriting intervals.

13. The method as claimed in claim 11, wherein said feature vector elements to be aggregated include spatial path lengths between principal points associated to successive handwriting intervals.

14. The method as claimed in claim 11, wherein said aggregated feature vector elements include signallings that spatially contiguous character segments are generated in a temporally non-sequential manner.

15. The method as claimed in claim 11, wherein said aggregated feature vector elements include contour changes between successive character segments in a direction transverse to a general orientation of a line or writing.

16. The method as claimed in claim 11, applied for signature recognition.

17. The method as claimed in claim 11, wherein at least one aggregated feature vector element is based on sensing instantaneous writing pressure.

18. The method as claimed in claim 11, wherein at least one aggregated feature vector element is based on sensing instantaneous writing instrument tilt.