Combining online and offline recognizers in a handwriting recognition system

• US 8,363,950 B2
• Filed: 03/21/2012
• Issued: 01/29/2013
• Est. Priority Date: 06/28/2007
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Chen, Xinjian, Zhang, Dongmei, Zou, Yu, Chang, Ming, Han, Shi, Wang, Jian
• Primary Examiner(s): Seth, Manav
• Application Number:
  US13/426,427
• Publication Number:
  US 20120183223A1
• Time in Patent Office: 314 Days
• Field of Search: 382101-102, 382155-161, 382181-231, 706/20, 715/268
• US Class Current: 382/187
• CPC Class Codes: G06F 18/254 of classification results, e.g. of results related to same input data
• CPC Class Codes: G06F 18/29 Graphical models, e.g. Bayesian networks
• CPC Class Codes: G06V 10/94 Hardware or software architectures specially adapted for image or video understanding

Abstract

Described is a technology by which online recognition of handwritten input data is combined with offline recognition and processing to obtain a combined recognition result. In general, the combination improves overall recognition accuracy. In one aspect, online and offline recognition is separately performed to obtain online and offline character-level recognition scores for candidates (hypotheses). A statistical analysis-based combination algorithm, an AdaBoost algorithm, and/or a neural network-based combination may determine a combination function to combine the scores to produce a result set of one or more results. Online and offline radical-level recognition may be performed. For example, a HMM recognizer may generate online radical scores used to build a radical graph, which is then rescored using the offline radical recognition scores. Paths in the rescored graph are then searched to provide the combined recognition result, e.g., corresponding to the path with the highest score.

Claims

1. A method for recognizing handwritten input data, the method comprising combining, by a computer, a first score provided by an offline recognizer in response to recognizing the handwritten input data with a second score provided by an online recognizer in response to recognizing the handwritten input data, the combining based on a statistical analysis algorithm.

2. The method of claim 1 where first fuzzy features on a feature space represent a first normalized score of the first score provided by the online recognizer.

3. The method of claim 2 where second fuzzy features on the feature space represent a second normalized score of the second score provided by the offline recognizer.

4. The method of claim 3 where each of the first fuzzy features and the second fuzzy features is defined according to a membership function.

5. The method of claim 4 where the membership function is a cone function.

6. The method of claim 4 where the membership function is an exponential function.

7. The method of claim 4 where the membership function is a Cauchy function.

8. A method for recognizing handwritten input data, the method comprising combining, by a computer, a first score provided by an offline recognizer in response to recognizing the handwritten input data with a second score provided by an online recognizer in response to recognizing the handwritten input data, the combining based on a neural network back-propagation algorithm.

9. The method of claim 8 where a first node of an input layer of a multilayer network represents first score provided by the offline recognizer.

10. The method of claim 9 where a second node of the input layer of the multilayer network represents the second score provided by the online recognizer.

11. The method of claim 10 where a topology of the multilayer network is constrained to be feed-forward with connections allowed from the input layer to a hidden layer of at least one hidden layer, and with connections from a last hidden layer of the at least one hidden layers to an output layer of the multilayer network.

12. The method of claim 11 where a first sigmoid function is applied to the first score and a second sigmoid function is applied to the second score.

13. The method of claim 12 where an output of the output layer is based on a linear combination result of an output of the at least one hidden layer.

14. The method of claim 13 where the output of the output layer is based on a back propagation algorithm configured to employ gradient descent for learning various weights and parameters of the multilayer network.

15. A method for recognizing handwritten input data, the method comprising combining, by a computer, a first score provided by an offline recognizer in response to recognizing the handwritten input data with a second score provided by an online recognizer in response to recognizing the handwritten input data, the combining based on combined results of a statistical analysis algorithm, a repeated base learning algorithm, and a neural network back-propagation algorithm.

16. The method of claim 15, the method further comprising providing a final recognition of at least a character of the handwritten input data where the provided final recognition comprises a single best result.

17. The method of claim 15, the method further comprising providing a final recognition of at least a character of the handwritten input data where the provided final recognition comprises a set of score-ranked results.

18. The method of claim 15, the method further comprising providing a final recognition of at least a character of the handwritten input data where the provided final recognition comprises a result set based on a confidence level for each of the statistical analysis algorithm, the repeated base learning algorithm, and the neural network back-propagation algorithm.

19. The method of claim 15 where each confidence level is normalized.

20. The method of claim 15 where the offline recognizer rescores recognition data provided by the online recognizer.