Symbol grouping and recognition in expression recognition

• US 20060062467A1
• Filed: 06/20/2005
• Published: 03/23/2006
• Est. Priority Date: 09/22/2004
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Zhang, Dongmei, Hou, Xiaohui, Zou, Yu, Zhou, Jian-Lai, Huang, Lei, Li, Yue, Wang, Jian, Xu, Mingqing
• Granted Patent: US 7,561,738 B2
• Time in Patent Office: Days
• US Class Current: 382/186
• CPC Class Codes: G06V 30/1423 the instrument generating sequences of position coordinates corresponding to handwriting preprocessing or recognising digital ink G06V30/32

Abstract

A mechanism for recognizing and inputting handwritten mathematical expressions into a computer by providing a part of a multi-path framework is described. The part of the multi-path framework includes a symbol grouping and recognition component that is designed to group input strokes that correspond to a handwritten mathematical expression into a symbol and to recognize the symbol based upon information associated with the grouped input strokes. A method for grouping and recognizing symbols of a handwritten mathematical expression includes receiving a plurality of input strokes corresponding to a handwritten mathematical expression, grouping the plurality of input strokes into symbols, recognizing the symbols based upon information, such as shape and time series information, associated with the grouped input strokes. Intra-group and inter-group information associated with the plurality of input strokes may be utilized to group the input strokes.

Claims

1. A system for symbol grouping and recognition of a handwritten mathematical expression, the system comprising:

a symbol grouping component configured to group input strokes corresponding to a handwritten mathematical expression into symbols; and

a symbol recognition component configured to recognize the symbols based upon information associated with the grouped input strokes.

2. The system of claim 1, wherein the symbol grouping component is further configured to group the input strokes into the symbols based upon intra-group and inter-group information associated with the input strokes.

3. The system of claim 2, wherein the intra-group information associated with the input strokes includes a confidence representative of whether the input strokes are a certain symbol provided by the symbol recognition component, and a score representative of whether the input strokes are a valid symbol based upon intra-group spatial information.

4. The system of claim 3, wherein the symbol recognition component is further configured to recognize the input strokes as the certain symbol or a non-symbol and to classify the input strokes with the confidence.

5. The system of claim 3, wherein the symbol grouping component is further configured to receive the input strokes, to determine intra-group spatial information among the input strokes, and to determine whether to group the input strokes into a symbol based upon the intra-group spatial information, wherein the intra-group spatial information includes a size difference among the input strokes and an offset measurement among the input strokes.

6. The system of claim 3, wherein the symbol grouping component determines whether to group the input strokes into a symbol based upon dynamic programming, wherein multiple global optimization candidates are determined by combining local optimizations.

7. The system of claim 2, wherein the inter-group information associated with the input strokes includes spatial relationship information of neighboring symbols and syntax context.

8. The system of claim 1, wherein the information associated with the grouped input strokes includes shape and time series information.

9. The system of claim 8, wherein the symbol recognition component is further configured to extract off-line features of the input strokes, to classify the off-line features, to provide character candidates with a respective confidence for the input strokes, and to output the candidates with the respective confidence.

10. The system of claim 8, wherein the symbol recognition component is further configured to extract on-line features of the input strokes, to classify the on-line features, to provide character candidates with a respective confidence for the input strokes, and to output the candidates with the respective confidence.

11. A method for grouping and recognizing symbols of a handwritten mathematical expression, the method comprising:

receiving a plurality of input strokes corresponding to a handwritten mathematical expression;

grouping the plurality of input strokes into symbols; and

recognizing the symbols based upon information associated with the grouped input strokes.

12. The method of claim 11, wherein the step of grouping includes grouping based upon intra-group and inter-group information associated with the plurality of input strokes.

13. The method of claim 12, further comprising steps of:

determining a confidence representative of whether input strokes of the plurality are a certain symbol provided by a symbol recognition component;

determining a score representative of whether the input strokes of the plurality are a valid symbol based upon intra-group spatial information;

identifying a spatial relationship between two neighboring symbols; and

identifying a bi-gram probability between the two neighboring symbols.

14. The method of claim 13, further comprising steps of:

determining whether the input strokes of the plurality is the certain symbol or a non-symbol;

classifying the input strokes of the plurality with the confidence;

determining a distance among the input strokes of the plurality;

determining a size difference among the input strokes of the plurality;

determining an offset measurement among the input strokes of the plurality; and

determining whether to group the input strokes of the plurality into a symbol based upon the intra-group spatial information.

15. The method of claim 14, further comprising steps of:

extracting on-line features of the plurality of input strokes;

classifying the on-line features;

providing character candidates with a respective confidence for the plurality of input strokes; and

outputting the candidates with the respective confidence.

16. The method of claim 12, further comprising a step of determining whether to group input strokes of the plurality into a symbol based upon dynamic programming, wherein multiple global optimization candidates are determined.

17. The method of claim 11, wherein the information associated with the grouped input strokes includes shape and time series information.

18. The method of claim 11, further comprising steps of:

extracting off-line features of the plurality of input strokes;

classifying the off-line features;

providing character candidates with a respective confidence for the plurality of input strokes; and

outputting the candidates with the respective confidence.

19. A software architecture stored on one or more computer readable media for processing data representative of a handwritten mathematical expression, comprising:

at least one component configured to group and recognize symbols based upon a plurality of strokes corresponding to a handwritten mathematical expression; and

at least one application program interface to access the component.

20. The software architecture of claim 19, wherein the at least one component is further configured to recognize the symbols based upon shape and time series information associated with the plurality of strokes.