Method and system of matching ink processor and recognizer word breaks

• US 6,754,386 B1
• Filed: 08/22/2000
• Issued: 06/22/2004
• Est. Priority Date: 08/22/2000
• Status: Expired due to Fees

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Altman, Dan W., Williamson, Peter H., Lui, Charlton E.
• Primary Examiner(s): CHANG, JON CARLTON
• Application Number:
  US09/643,417
• Time in Patent Office: 1,400 Days
• Field of Search: 382/177, 382/178, 382/179, 382/186, 382/187, 382/188, 382/189, 382/229, 382/209, 382/217, 382/218
• US Class Current: 382/177
• CPC Class Codes: G06V 30/1423 the instrument generating sequences of position coordinates corresponding to handwriting preprocessing or recognising digital ink G06V30/32
• CPC Class Codes: G06V 30/2268 using stroke segmentation

Abstract

In a computing device that receives handwritten data, a method and system that corrects for parser segmentation errors by sending an entire line of ink to a recognizer, and then comparing, on a word-by-word basis, the initial segmentation guesses of the parser with the more-thoroughly recognized segmentation results of the handwriting recognition engine. In the correction process, the ink words are efficiently adjusted with relatively little data manipulation. As the recognizer is fed a series of strokes on a line, the recognizer returns segmentation information. For ink word breaks that are the same for any given set of data, the existing ink word is unchanged. For ink words that are recognized differently relative to their initial segmentation, one or more new ink words are created and the handwriting (including stroke) data of the parser'"'"'s ink word is manipulated to create a new ink processor word (or words) to match the recognizer output.

Claims

1. A computer-readable medium having computer-executable instructions, which when executed implement a method, the method comprising:

collecting a set of ink data;

parsing the set of ink data into a first segmentation of units of data;

providing an ink word data structure for each unit of data;

providing the set of ink data to a recognizer;

receiving information from the recognizer corresponding to a second segmentation of the set of ink data;

manipulating the segmented units of data in the first set to correspond to the information received from the recognizer, including moving stroke information from at least one ink word data structure into at least one new ink word data structure.

2. The computer-readable medium of claim 1 wherein providing the set of ink data to a recognizer comprises sending at least one line of ink stroke data thereto.

3. The computer-readable medium of claim 1 wherein manipulating the segmented units of data comprises determining whether the first segmentation corresponds to the second segmentation.

4. The computer-readable medium of claim 1, wherein manipulating the segmented units of data comprises moving stroke information from a plurality of ink word data structures into a new ink word data structure.

5. The computer-readable medium of claim 1, wherein manipulating the segmented units of data comprises moving stroke information from one ink word data structure into at least two new ink word data structures.

6. The computer-readable medium of claim 1, wherein manipulating the segmented units of data comprises moving sets of stroke data corresponding to a plurality of strokes from an ink word data structure into at least one new ink word data structure, and further comprising, maintaining information indicating whether each set of stroke data has been moved.

7. The computer-readable medium of claim 1, further comprising, receiving other data from the recognizer, and adding at least some of the other data to at least one ink word data structure.

8. The computer-readable medium of claim 1, wherein manipulating the segmented units of data comprises moving stroke information from one ink word data structure into two new ink word data structures, and further comprising, maintaining information indicative of a space between the two new ink word data structures.

9. The computer-readable medium of claim 1, wherein manipulating the segmented units of data comprises moving sets of stroke information from an existing ink word data structure into at least one new ink word data structure, and further comprising, checking whether each set of the stroke information from the existing ink word data has been moved, and if so, removing the existing ink word data structure.

10. In a computer device configured to receive handwritten data input from a user, a system comprising,

an ink processor;

a parsing mechanism that segments the handwritten data into distinct units of data, each unit having ink data maintained in an ink word data structure;

a handwriting recognizer connected to the ink processor and configured to receive the handwritten data and to translate the data into computer-readable codes, the handwriting recognizer further returning information corresponding to the recognizer'"'"'s interpretation of a segmentation of the handwritten data into distinct units of handwritten data; and

a matching process configured to receive information from the handwriting recognizer and to use the information to correct for differences between the distinct units of data as determined by the parsing mechanism and the distinct units of handwritten data based on the recognizer information, wherein the matching process evaluates the information returned by the recognizer against the ink word data structures, and if the matching process determines that the information returned by the recognizer does not correspond to the ink data in one of the ink word data structures, the matching process creates at least one new ink word data structure.

11. The system of claim 10, further comprising a text buffer configured to maintain information indicative of a relationship of each unit of data.

12. The system of claim 10, wherein the matching process moves data of at least one of the ink word data structures into the new ink word data structure.

13. The system of claim 10, further comprising a text buffer configured to maintain a relationship between each of the ink word data structures, and wherein the matching process inserts the new ink word data structure into the text buffer.

14. The system of claim 10, further comprising a text buffer configured to maintain a relationship between each of the ink word data structures, and wherein the matching process removes at least one ink word data structure from the text buffer.

15. In a computing environment, a method comprising:

receiving ink data corresponding to a set of strokes;

parsing the ink data into distinct parsed segments, each parsed segment corresponding to a subset of the set of strokes contained in a distinct ink word data structure;

providing the ink data to a recognizer;

displaying a representation of each distinct parsed segment, prior to receiving recognition information about the ink data from the recognizer; and

determining whether the parsed segments match with segmentation-related recognition information received from the recognizer, and if not, automatically correcting the parsed segments based on the segmentation-related recognition information and redisplaying a corrected representation of the ink data.

16. The method of claim 15, wherein automatically correcting the parsed segments comprises moving stroke information from an ink word data structure into a new ink word data structure.

17. The method of claim 15, wherein automatically correcting the parsed segments comprises moving stroke information from a plurality of ink word data structures into a new ink word data structure.

18. The method of claim 15, wherein automatically correcting the parsed segments comprises moving stroke information from one ink word data structure into at least two new ink word data structures.

19. The method of claim 15, wherein automatically correcting the parsed segments comprises moving sets of stroke data corresponding to a plurality of strokes from an ink word data structure into at least one new ink word data structure, and further comprising, maintaining information indicating whether each set of stroke data has been moved.

20. The method of claim 15, wherein automatically correcting the parsed segments comprises moving stroke information from one ink word data structure into two new ink word data structures, and further comprising, maintaining information indicative of a space between the two new ink word data structures.

21. The method of claim 15, automatically correcting the parsed segments comprises moving sets of stroke information from an existing ink word data structure into at least one new ink word data structure, and further comprising, checking whether each set of the stroke information from the existing ink word data has been moved, and if so, removing the existing ink word data structure.

22. The method of claim 15, wherein providing the ink data to a recognizer comprises sending at least one line of ink stroke data that includes the ink data to the recognizer.

23. A computer-readable medium having computer-executable instructions which when executed perform the method of claim 15.