Handwriting recognition training and synthesis

• US 20070154094A1
• Filed: 12/29/2005
• Published: 07/05/2007
• Est. Priority Date: 12/29/2005
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Wang, Jian, Lin, Zhouchen, Hu, Chun-Hui, Wan, Liang
• Granted Patent: US 7,657,094 B2
• Time in Patent Office: Days
• US Class Current: 382/187
• CPC Class Codes: G06V 30/10 Character recognition
• CPC Class Codes: G06V 30/166 Normalisation of pattern dimensions
• CPC Class Codes: G06V 30/347 Sampling; Contour coding; Stroke extraction

Abstract

Methods and systems for converting text into natural personal handwriting are provided. One aspect relates to the training of a computer to recognize a user'"'"'s handwriting style. In one embodiment, the computer receives handwriting samples of at least one character written by the user, such as the character being provided as the beginning, middle, or ending character among a plurality of other characters. Further embodiments allow for increased personalization of the handwriting. Another aspect relates to system and methods for displaying a representation of a computer user'"'"'s handwriting. In one embodiment, the handwriting comprises variant shapes of letters, personalized connection style between letters, and connection parts that look pressure-sensitive. In another embodiment, characters are adjusted, such as cutting portions of the character to create a more realistic recreation and synthesis of the handwriting.

Claims

1. A computer-implemented method of training a computer to recognize a computer user'"'"'s handwriting comprising the steps of:

(a) receiving handwriting samples of at least one character written by the user, the samples comprising at least one sample of the character being provided as the beginning character among a plurality of other characters, at least one sample of the character being provided in the middle of a plurality of other of characters, and at least one sample of the letter being provided as the ending character among a plurality of characters;

(b) extracting a plurality of control points from the samples, wherein the control points represent the shape of the character;

(c) determining the vertical position of the character in relation to a baseline comprising the step of determining the distance from the bottom of the character to a baseline; and

(d) normalizing the at least one sample of at least one character

2. The computer-implemented method of claim 1, wherein the at least one character is a plurality of characters arranged in a word-type arrangement, the method further comprising the steps of

(e) determining the average horizontal distance between the central axis of at least two characters arranged in the word-type arrangement.

3. The computer-implemented method of claim 1, wherein the at least one character is a plurality of characters arranged in a word-type arrangement, the method further comprising the steps of

(e) determining the connection inclination of at least two successive characters in the word-type arrangement.

4. The computer-implemented method of claim 3, wherein the connection inclination is calculated according to:

$P_{\mathrm{user}}=\frac{1}{m}<br><br>\underset{i=1}{\overset{m}{\sum <br><br>}}<br><br>p_{\mathrm{user}}^i.$

5. The computer-implemented method of claim 1, further comprising the steps of:

(e) determining the sample comprises a character that is a multi-stroke letter that requires a plurality of user strokes when written;

(f) presenting a graphical user interface that displays the multi-stroke character of step (e) with a different character that requires a single user stroke, wherein the graphical user interface is presented to the user to indicate the user should provide an input in the form of a handwriting sample that connects the multi-stroke letter with the single stroke character;

(g) receiving an input from the user in the form of a handwriting sample; and

(h) determining how the multi-stroke letter of step (e) is connected to other letters.

6. The computer-implemented method of claim 1, further comprising the step of:

(e) prior to normalization, categorizing at least one character into a group selected from the group consisting of;

ascenders, descenders, middle zone, multi-zone, and equivalents thereof.

7. The computer-implemented step of claim 1, further comprising the step of:

(e) determining the amount of pressure the user applies while inputting the handwriting sample in step (a).

8. The computer-implemented method of claim 1, wherein the character is an alphanumeric character.

9. The computer-implemented method of claim 8, wherein the character is provided in lower case format.

10. The computer-implemented method of claim 1, wherein step (c) comprises the step of determining the ratio of (1) the distance from the bottom of the character to a baseline to (2) the overall height of the character.

11. A computer-implemented method of displaying a representation of a computer user'"'"'s handwriting on a display device comprising the steps of:

(a) receiving a user input through a user input device comprising a plurality of characters;

(b) determining whether at least two characters received in the user input should be joined;

(c) for each character, selecting a sample among a plurality of samples stored on a computer-readable medium to represent the character, wherein the plurality of samples comprises at least one sample of the character being provided as the beginning character among a plurality of other characters, at least one sample of the character being provided in the middle of a plurality of other of characters, and at least one sample of the letter being provided as the ending character among a plurality of characters, wherein the selected sample comprises at least one segment selected from the group consisting of a heading segment, a tailing segment, and a central axis;

(d) determining a horizontal distance between the central axis of at least two of the samples, wherein the horizontal distance between one sample may be a different distance than the horizontal distance between another sample;

(e) for at least two characters determined to be connected in step (b), adjusting at least a portion of a tailing segment of at least one character and at least a portion of the heading segment of at least one character;

(f) joining the at least two characters of step (e); and

(g) displaying the at least two characters on a display device, the at least two characters being a representation of the characters received in step (a) in the form of the user'"'"'s handwriting.

12. The computer-implemented method of claim 11, wherein step (f) includes the use of a connection segment to connect the tailing segment of a first character to the heading segment of a second character.

13. The computer-implemented method of claim 12, wherein the connection segment is shaped according to a 3-order polynomial that considers the closeness of old and new connection parts, the bending energy, and the smoothness of the connection.

14. The computer-implemented method of claim 13, wherein the connection curve is parameterized by

$P$

( t ) = ∑

k = 0 P ⁢

P k ⁢

t k , t ∈

[ 0 , 1 ] .

15. The computer-implemented method of claim 11, further comprising the step of:

(h) applying a geometric variation of at least one of the samples for the characters selected in (c), wherein the geometric variation comprises a process selected from the group consisting of;

scaling and rotation of at least a portion of the sample.

16. The computer-implemented method of claim 11, further comprising the step of:

(h) calculating a value of pressure to be applied to at least a portion of a segment of a character.

17. The computer-implemented method of claim 16, where the value of pressure is calculated with the equation {tilde over (p)}_i=p_k, where k is the integer part of i/n*m, i=1, . . . , n.

18. The computer-implemented method of claim 11, wherein the representation of the user'"'"'s handwriting is displayed on more than one line, further comprising the step of:

(h) determining the interline spacing between the lines of handwriting with the equation d=H<sub>capital—

letters</sub>+H<sub>descendent—

letters</sub>−

H<sub>middle—

zone—

letters</sub>+Δ

h, where H<sub>capital—

letters</sub>, H<sub>descendent—

letters</sub>, and H<sub>middle—

zone—

letters</sub> are the average heights of all capital letters, all descendent letters, and middle-zone letters, respectively, and Δ

h is a predefined value.

19. In a computer system having a graphical user interface including a display and a user interface input device, a method of configuring a computer to recognize a user'"'"'s handwriting comprising the steps of:

(a) displaying on the display device at least one input field for receiving a user input in the form of handwritten sample characters, wherein the handwritten sample characters comprises at least one sample of the character being provided as the beginning character among a plurality of other characters, at least one sample of the character being provided as being the middle of a plurality of other of characters, and at least one sample of the letter being provided as the ending character among a plurality of characters;

(b) receiving an input signal indicative of the user interface input device providing at least one sample character comprising at least one segment selected from the group consisting of a heading segment, a tailing segment, and a central axis;

(c) based upon the signal, assigning the received sample character to be the representative character for a sub-category selected from the group consisting of;

the beginning character among a plurality of other characters, the character being provided as middle character among a plurality of other of characters, and the ending character among a plurality of characters, wherein the selected sample comprises at least one segment selected from the group consisting of a heading segment, a tailing segment, and a central axis; and

(d) normalizing the representative character; and

(e) storing the assigned, normalized character on a computer readable medium.

20. The method of claim 19, the system further comprising a second user interface input device, the method further comprising the steps of:

(f) receiving a input signal indicative of the second user interface input device providing at least two characters;

(g) determine whether to join the at least two characters received of step (e);

(h) selecting a representative character among a plurality of representative characters stored on the computer-readable medium to represent each of the characters received in step (f), wherein the plurality of characters comprise at least one sample of the character being provided as the beginning character among a plurality of other characters, at least one sample of the character being provided in the middle of a plurality of other of characters, and at least one sample of the letter being provided as the ending character among a plurality of characters;

(i) performing geometric variation on at least one of the selected characters; and

(j) displaying the at least two characters on a display device, the at least two characters being a representation of the characters received in step (f), wherein the characters are displayed in the form of the user'"'"'s handwriting.