Combining online and offline recognizers in a handwriting recognition system

• US 8,160,362 B2
• Filed: 04/19/2011
• Issued: 04/17/2012
• Est. Priority Date: 06/28/2007
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Chen, Xinjian, Zhang, Dongmei, Zou, Yu, Chang, Ming, Han, Shi, Wang, Jian
• Primary Examiner(s): Seth, Manav
• Application Number:
  US13/090,242
• Publication Number:
  US 20110194771A1
• Time in Patent Office: 364 Days
• Field of Search: 382101-102, 382155-161, 382181-231, 706/20, 715/268
• US Class Current: 382/187
• CPC Class Codes: G06F 18/254 of classification results, e.g. of results related to same input data
• CPC Class Codes: G06F 18/29 Graphical models, e.g. Bayesian networks
• CPC Class Codes: G06V 10/94 Hardware or software architectures specially adapted for image or video understanding

Abstract

Described is a technology by which online recognition of handwritten input data is combined with offline recognition and processing to obtain a combined recognition result. In general, the combination improves overall recognition accuracy. In one aspect, online and offline recognition is separately performed to obtain online and offline character-level recognition scores for candidates (hypotheses). A statistical analysis-based combination algorithm, an AdaBoost algorithm, and/or a neural network-based combination may determine a combination function to combine the scores to produce a result set of one or more results. Online and offline radical-level recognition may be performed. For example, a HMM recognizer may generate online radical scores used to build a radical graph, which is then rescored using the offline radical recognition scores. Paths in the rescored graph are then searched to provide the combined recognition result, e.g., corresponding to the path with the highest score.

Claims

1. A method for recognizing handwritten input data, the method comprising combining, by a computer in response to recognizing the handwritten input data, a first set of scores provided by an offline recognizer with a second set of scores provided by an online recognizer in response to recognizing the handwritten input data, the combining based on a repeated base learning algorithm.

2. The method of claim 1 wherein the base learning algorithm produces a combination function in a feature space composed of the first set of scores and the second set of scores.

3. The method of claim 2 wherein the combining is performed by the produced combination function.

4. The method of claim 2 wherein the first set of scores and the second set of scores comprise a feature pool.

5. The method of claim 4 further comprising applying an AdaBoost function to the feature pool.

6. The method of claim 1 further comprising returning, in response to the combining, a single result that represents the recognized handwritten input data.

7. The method of claim 1 further comprising returning, in response to the combining, a result set with each result in the set scored, each scored result representing the recognized handwritten input data.

8. At least one computer storage media storing computer-executable instructions that, when executed by a computer, cause the computer to perform a method for recognizing handwritten input data, the method comprising combining, in response to recognizing the handwritten input data, a first set of scores provided by an offline recognizer with a second set of scores provided by an online recognizer in response to recognizing the handwritten input data, the combining based on a repeated base learning algorithm.

9. The at least one computer storage media of claim 8 wherein the base learning algorithm produces a combination function in a feature space composed of the first set of scores and the second set of scores.

10. The at least one computer storage media of claim 9 wherein the combining is performed by the produced combination function.

11. The at least one computer storage media of claim 9 wherein the first set of scores and the second set of scores comprise a feature pool.

12. The at least one computer storage media of claim 11 further comprising applying an AdaBoost function to the feature pool.

13. The at least one computer storage media of claim 8 further comprising returning, in response to the combining, a single result that represents the recognized handwritten input data.

14. The at least one computer storage media of claim 8 further comprising returning, in response to the combining, a result set with each result in the set scored, each scored result representing the recognized handwritten input data.

15. A system configured for recognizing handwritten input data, the system comprising:

a computer,

a combiner mechanism implemented at least in part by the computer and configured for combining, in response to recognizing the handwritten input data, a first set of scores provided by an offline recognizer with a second set of scores provided by an online recognizer in response to recognizing the handwritten input data, the combining based on a repeated base learning algorithm.

16. The system of claim 15 wherein the base learning algorithm is configured to produce a combination function in a feature space composed of the first set of scores and the second set of scores.

17. The system of claim 16 wherein the combining is performed by the produced combination function.

18. The system of claim 16 wherein the first set of scores and the second set of scores comprise a feature pool.

19. The system of claim 18 further comprising the computer configured for applying an AdaBoost function to the feature pool.

20. The system of claim 15 further comprising the computer further configured for returning, in response to the combining, a result that represents the recognized handwritten input data.