Constraint-based correction of handwriting recognition errors

• US 7,720,316 B2
• Filed: 09/05/2006
• Issued: 05/18/2010
• Est. Priority Date: 09/05/2006
• Status: Expired due to Fees

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Shilman, Michael, Tan, Desney S., Simard, Patrice Y.
• Primary Examiner(s): Mehta; Bhavesh M
• Assistant Examiner(s): DRENNAN, BARRY T
• Application Number:
  US11/470,076
• Publication Number:
  US 20080056578A1
• Time in Patent Office: 1,351 Days
• Field of Search: 382/179, 382186-189, 382309-311
• US Class Current: 382/311
• CPC Class Codes: G06F 18/24765 Rule-based classification
• CPC Class Codes: G06F 18/28 Determining representative reference patterns, e.g. by averaging or distorting; Generating dictionaries
• CPC Class Codes: G06V 30/1914 Determining representative reference patterns, e.g. averaging or distorting patterns; Generating dictionaries, e.g. user dictionaries
• CPC Class Codes: G06V 30/387 using human interaction, e.g. selection of the best displayed recognition candidate

Abstract

A handwriting recognition system interprets handwritten text and produces a typed interpretation of that text. When the initial interpretation of the handwritten text is inaccurate, the handwriting recognition system alters the initial recognition by reinterpreting the handwritten text in view of a correction made by a user and constraints (e.g., derived by assumptions in user behavior). The handwriting recognition system intelligently reinterprets and renews its text recognition each time the user implements a correction. In effect, a single correction can trigger multiple adjustments to the text recognition. Therefore, with the use of a reinterpretation algorithm, the handwriting recognition system helps the user obtain the desired result in fewer correction steps.

Claims

1. A system that facilitates handwriting recognition, comprising:

a correction component that receives a correction to a typed text interpretation of a handwritten entry and creates correction constraints based on the received correction;

an iterative recognition component that receives the handwritten entry and the correction constraints and generates a revised typed text interpretation of the handwritten entry based on the handwritten entry and the correction constraints; and

a display device, wherein a segmentation correction is facilitated by solid bars displayed on the display appearing at least one of above and below the handwritten entry, and wherein the segmentation correction is at least one of an insert split operation and a join operation.

2. The system of claim 1, wherein the iterative recognition component resides on at least one of a server and a client.

3. The system of claim 1, wherein the constraint associated with a segmentation correction includes one of:

allowing re-segmentation of all text;

maintaining a segmentation configuration to the left of the segmentation correction and allowing segmentation and character changes to the right of the segmentation correction; and

maintaining all segmentation except a current change and only allowing character changes within immediate words formed by the correction.

4. The system of claim 1, wherein the constraint associated with a character correction includes one of:

allowing reinterpretation of all text;

maintaining a segmentation configuration in the typed text interpretation and a character configuration to the left of the character correction and allowing character changes to the right of the character correction; and

maintaining the character configuration to the left of the character correction and allowing character changes to the right of the character correction and re-segmentation.

5. The system of claim 1, wherein a character correction facilitates an insert character command and an overwrite character command as a single operation.

6. The system of claim 1, wherein the iterative recognition component is externally implemented on top of an existing recognizer.

7. The system of claim 1, wherein the iterative recognition component is implemented inside an integrated recognizer.

8. The system of claim 1, further comprising a correction data store that collects correction information from the correction component and the iterative recognition component, and wherein the iterative recognition component customizes recognition and correction results with the collected correction information.

9. The system of claim 1, wherein the insert split operation is implemented by approximating a location of the split according to a vertical line drawn across the solid bars.

10. The system of claim 1, wherein the join operation is activated by a horizontal line gesture that connects two solid bars.

11. A method that facilitates handwriting recognition, comprising the following acts:

receiving a handwritten entry input,

generating, by a handwriting recognition device, at least one constraint based on a correction to a typed text interpretation of the handwritten entry;

processing a split correction as an approximate break in segmentation in reference to a vertical line across a solid bar below the handwritten entry; and

reinterpreting the handwritten entry according to the correction to the typed text interpretation and the at least one constraint.

12. The method of claim 11, further comprising reinterpreting the handwritten entry each time an additional correction is made to the typed text reinterpretation of the handwritten entry.

13. The method of claim 12, further comprising:

storing the at least one constraint for the correction and each additional correction to the typed text interpretation of the handwritten entry; and

implementing the stored constraints for each reinterpretation of the handwritten entry.

14. The method of claim 11, further comprising presenting an interpretation of the handwritten entry as character options in a list, wherein the list displays multiple character options as suggestions for each character in order of likelihood of accuracy.

15. The method of claim 14, further comprising:

displaying one character option from the list at a time;

removing the displayed character option upon rejection of the displayed character option, wherein the rejection is communicated with a gesture by a user; and

displaying the next character option from the list.

16. The method of claim 11, further comprising loading customized settings relating to handwriting recognition from a server to at least one device.

17. The method of claim 11, further comprising:

aggregating customized settings on a server; and

implementing the customized settings for a group of affiliated devices.

18. The method of claim 11, further comprising:

processing a character insert correction and a character overwrite correction as a related and combined change;

overriding character constraints once a change in segmentation is submitted; and

precluding automatic changes to characters to the left of the submitted change in character.

19. A system that facilitates handwriting recognition, comprising:

means for enforcing at least one constraint related to a correction of a typed text interpretation of a handwritten entry;

means for processing a split correction as an approximate break in segmentation in reference to a vertical line across a solid bar below the handwritten entry;

means for reinterpreting the typed text interpretation of the handwritten entry in view of the correction and the at least one constraint; and

means for further reinterpreting the typed text interpretation of the handwritten entry every time a further correction and further constraints are implemented.