Unifield digital ink recognition

• US 20090003703A1
• Filed: 06/26/2007
• Published: 01/01/2009
• Est. Priority Date: 06/26/2007
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Zhang, Dongmei, Hou, Xiaohui, Wang, Jian, Qiu, Yingjun
• Granted Patent: US 8,041,120 B2
• Time in Patent Office: Days
• US Class Current: 382/181
• CPC Class Codes: G06V 30/36 Matching; Classification

Abstract

Described is a unified digital ink recognizer that recognizes various different types of digital ink data, such as handwritten character data and custom data, e.g., sketched shapes, handwritten gestures, and/or drawn pictures, without further participation by a user such as recognition mode selection or parameter input. For a custom item, the output may be a Unicode value from a private use area of Unicode. Building the unified digital ink recognizer may include defining the data set to be recognized, extracting features of training samples corresponding to the dataset items to build a recognizer model, evaluating the recognizer model using testing data, and modifying the recognizer model using tuning data. The extracted features may be processed into feature data for a multi-dimensional nearest neighbor recognizer approach; the extracted features for the samples of each class is calculated and combined into the feature set for this class in the resulting recognizer model.

Claims

1. A computer-readable medium having computer-executable instructions, which when executed perform steps, comprising:

a) extracting features from a selected sample of a plurality of samples of digital ink training data, wherein the training data corresponds to digital ink representative of at least two different types of digital ink input, and the selected sample is associated with a recognition value as its label;

b) processing a feature dataset of the selected sample into a recognition model including by adjusting the combined feature data of the class to which the selected sample belongs and maintaining data representative of the features in association with the recognition value;

c) selecting another sample from the plurality and repeating steps a) and b) until each sample of the plurality has been processed; and

d) providing a unified recognizer that recognizes an input item including between the two different types of digital ink input without mode selection or recognition parameter input, including by extracting features of the input item and determining which data representative of the features of a class the features of the input item best match, and outputting the recognition value associated with that data.

2. The computer-readable medium of claim 1 wherein maintaining the data representative of the features comprises maintaining information representative of the features corresponding to multiple dimensions.

3. The computer-readable medium of claim 1 having further computer-executable instructions, comprising, processing the features for selecting useful features to effectively represent at least different types of digital ink.

4. The computer-readable medium of claim 1 wherein determining which data representative of the features of a sample the features of the input item best match comprises searching the information representative of the features corresponding to multiple dimensions to determine which information is nearest to a set of corresponding information processed from the input item.

5. The computer-readable medium of claim 1 having further computer-executable instructions, comprising, prior to step d), evaluating accuracy or efficiency, or both accuracy and efficiency of the recognizer model using a plurality of samples of digital ink testing data.

6. The computer-readable medium of claim 1 having further computer-executable instructions comprising, evaluating accuracy or efficiency, analyzing the evaluation of the accuracy or efficiency, or the evaluation of both the accuracy and efficiency, and using a plurality of samples of digital ink tuning data to improve recognition accuracy or efficiency, or both recognition accuracy and efficiency.

7. The computer-readable medium of claim 1 having further computer-executable instructions comprising, dividing a plurality of samples of digital ink into training data, testing data and tuning data.

8. The computer-readable medium of claim 1 having further computer-executable instructions comprising, defining a dataset of items for which the unified recognizer will recognize an input item, including customized data items.

9. The computer-readable medium of claim 8 having further computer-executable instructions comprising, mapping each item of the dataset to a Unicode value as its label, including mapping a customized data item to a private Unicode value as its label.

10. In a computing environment, a system comprising:

a feature extraction mechanism that featurizes digital ink data corresponding to training samples that represent at least two different types of digital ink data;

a recognition model builder mechanism coupled to the feature extraction mechanism that builds a recognition model including by persisting data representative of the features of each class of training sample in association with a recognition value of that class of training sample; and

an evaluation mechanism that evaluates the recognition model with respect to digital ink data corresponding to testing samples.

11. The system of claim 10 wherein the evaluation mechanism provides evaluation results corresponding to the accuracy or efficiency, or both the accuracy and efficiency of the recognition model, and further comprising an error analysis mechanism that tunes the recognition model with respect to digital ink data corresponding to tuning samples based at least in part on the evaluation results.

12. The system of claim 10 wherein the different types of digital ink data include handwritten characters, sketched shapes, handwritten gestures, or drawn pictures, or any combination of handwritten characters, sketched shapes, handwritten gestures, or drawn pictures.

13. The system of claim 10 further comprising a unified recognizer that uses the recognition model to recognize an input item.

14. The system of claim 10 wherein the data representative of the features comprises information corresponding to multiple dimensions for use in a nearest neighbor search.

15. The system of claim 10 including a mechanism that selects useful features from a set of features.

16. In a computing environment, a method comprising:

building a unified digital ink recognizer using two or more different types of digital ink data;

receiving an input item; and

recognizing the input item with the recognizer to output a value associated with one of the types of digital ink data.

17. The method of claim 16 wherein building the unified digital ink recognizer comprises, determining a dataset having items representative of the different types of digital ink data therein, and using features of training samples corresponding to the items to build a recognizer model.

18. The method of claim 16 wherein building the unified digital ink recognizer further comprises, evaluating the recognizer model using digital ink testing data, and modifying the recognizer model using digital ink tuning data.

19. The method of claim 16 wherein recognizing the input item with the recognizer to output a value associated with one of the types of digital ink data comprises outputting a Unicode value from a private use area of Unicode mapped to a custom item for a type of digital ink data.

20. The method of claim 16 further comprising, defining a dataset of items for which the unified recognizer will recognize an input item, the including defining at least one custom item, and mapping each custom item to a private use area of Unicode.