Unified digital ink recognition

• US 8,041,120 B2
• Filed: 06/26/2007
• Issued: 10/18/2011
• Est. Priority Date: 06/26/2007
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Qiu, Yingjun, Zhang, Dongmei, Hou, Xiaohui, Wang, Jian
• Primary Examiner(s): Desire; Gregory M
• Application Number:
  US11/821,858
• Publication Number:
  US 20090003703A1
• Time in Patent Office: 1,575 Days
• Field of Search: 382/188, 382/190, 707/3
• US Class Current: 382/190
• CPC Class Codes: G06V 30/36 Matching; Classification

Abstract

Described is a unified digital ink recognizer that recognizes various different types of digital ink data, such as handwritten character data and custom data, e.g., sketched shapes, handwritten gestures, and/or drawn pictures, without further participation by a user such as recognition mode selection or parameter input. For a custom item, the output may be a Unicode value from a private use area of Unicode. Building the unified digital ink recognizer may include defining the data set to be recognized, extracting features of training samples corresponding to the dataset items to build a recognizer model, evaluating the recognizer model using testing data, and modifying the recognizer model using tuning data. The extracted features may be processed into feature data for a multi-dimensional nearest neighbor recognizer approach; the extracted features for the samples of each class is calculated and combined into the feature set for this class in the resulting recognizer model.

Claims

1. A computer-readable storage medium having computer-executable instructions, which when executed perform steps, comprising:

a) extracting features from a selected sample of a plurality of samples of digital ink training data, wherein the digital ink training data corresponds to digital ink representative of at least two different types of digital ink input, each of the plurality of samples belong to one of a plurality of classes having combined feature data, and each of the plurality of samples is associated with a label comprising a recognition value;

b) processing a feature dataset of the selected sample into a recognition model by adjusting the combined feature data of the class to which the selected sample belongs and maintaining data representative of the features extracted from the selected sample in association with the recognition value associated with the selected sample;

c) selecting another sample from the plurality of samples and repeating steps a) and b) until each sample of the plurality of samples has been processed; and

d) providing a unified recognizer that recognizes an input item of one of the at least two different types of digital ink input without mode selection or recognition parameter input, the input item being recognized by extracting features of the input item and determining a matching class having combined feature data that best match features of the input item, and outputting a matching recognition value associated with the matching class.

2. The computer-readable storage medium of claim 1 wherein maintaining the data representative of the features comprises maintaining information representative of the features corresponding to multiple dimensions.

3. The computer-readable storage medium of claim 1 having further computer-executable instructions, which when executed perform steps, comprising:

processing the features from the plurality of samples to select useful features to effectively represent the at least two different types of digital ink.

4. The computer-readable storage medium of claim 1 wherein determining the matching class comprises a multi-dimensional nearest neighbor search.

5. The computer-readable storage medium of claim 1 having further computer-executable instructions, which when executed perform steps, comprising:

prior to step d), evaluating accuracy or efficiency, or both accuracy and efficiency of the recognizer model using a plurality of samples of digital ink testing data.

6. The computer-readable storage medium of claim 1 having further computer-executable instructions, which when executed perform steps, comprising:

evaluating accuracy or efficiency of the recognizer model, analyzing the evaluation of the accuracy or efficiency, or the evaluation of both the accuracy and efficiency, and using a plurality of samples of digital ink tuning data to improve recognition accuracy or efficiency of the recognizer model, or both recognition accuracy and efficiency.

7. The computer-readable storage medium of claim 1 having further computer-executable instructions, which when executed perform steps, comprising:

dividing samples of digital ink into the plurality of samples of digital ink training data, testing data and tuning data.

8. The computer-readable storage medium of claim 1 having further computer-executable instructions, which when executed perform steps, comprising:

defining a dataset of items the unified recognizer will recognize, including customized data items.

9. The computer-readable storage medium of claim 8 having further computer-executable instructions, which when executed perform steps, comprising:

mapping each item of the dataset to a label comprising a Unicode value, including mapping the customized data items to a private Unicode values.

10. In a computing environment, a system comprising:

a feature extraction mechanism that featurizes digital ink training data corresponding to training samples that represent at least two different types of digital ink data to be recognized by building a classifier according to the at least two different types of digital ink data to be recognized, the training samples belonging to a plurality of classes having features;

a recognition model builder mechanism coupled to the feature extraction mechanism that builds a recognition model including by persisting data representative of the features of each class of the training samples in association with a recognition value of each class of the training samples; and

an evaluation mechanism that evaluates the recognition model with respect to digital ink testing data corresponding to testing samples.

11. The system of claim 10 wherein the evaluation mechanism provides evaluation results corresponding to accuracy or efficiency, or both the accuracy and efficiency of the recognition model, and further comprising an error analysis mechanism that tunes the recognition model with respect to digital ink tuning data corresponding to tuning samples based at least in part on the evaluation results.

12. The system of claim 10 wherein the different types of digital ink data include handwritten characters, sketched shapes, handwritten gestures, or drawn pictures, or any combination of handwritten characters, sketched shapes, handwritten gestures, or drawn pictures.

13. The system of claim 10 further comprising a unified recognizer that uses the recognition model to recognize an input item.

14. The system of claim 10 wherein the data representative of the features comprises information corresponding to multiple dimensions for use in a nearest neighbor search.

15. The system of claim 10 wherein the features of each of the plurality of classes are extracted from a collection of samples for each of the plurality of classes, the system further comprising:

a unified recognizer that recognizes new digital ink items using a feature of the new digital ink item that is matched to an individual feature of the plurality of classes.

16. In a computing environment, a method comprising:

determining a dataset having digital ink items representative of a plurality of different classes of digital ink data to be recognized;

building a classifier according to the plurality of different classes of the digital ink data to be recognized by extracting features for the plurality of different classes of the digital ink data from samples of the plurality of different classes;

using features of the classifier corresponding to the digital ink items to build a unified digital ink recognizer for two or more different types of the digital ink data;

receiving an input item; and

recognizing the input item with the unified digital ink recognizer to output a value associated with one of the two or more different types of the digital ink data.

17. The method of claim 16 wherein building the classifier further comprises, evaluating the classifier using digital ink testing data, and modifying the classifier using digital ink tuning data.

18. The method of claim 16 wherein recognizing the input item with the unified digital ink recognizer to output the value associated with the one of the two or more different types of the digital ink data comprises outputting a Unicode value from a private use area of Unicode mapped to a custom item for an individual type of digital ink data.

19. The method of claim 16 further comprising, defining at least one custom item that the unified recognizer will recognize and mapping the custom item to a private use area of Unicode.

20. The method of claim 16, further comprising:

dividing the samples into a training set for building the classifier, a testing set for testing the classifier and a tuning set for tuning the classifier.