Path score calculating method for intelligent character recognition

• US 9,977,976 B2
• Filed: 06/29/2016
• Issued: 05/22/2018
• Est. Priority Date: 06/29/2016
• Status: Active Grant

• Current Assignee: Konica Minolta Laboratory U.S.A., Inc. (Konica Minolta Inc.)
• Original Assignee: Konica Minolta Laboratory U.S.A., Inc. (Konica Minolta Inc.)
• Inventors: Yang, Duanduan
• Primary Examiner(s): Wu, Jingge
• Application Number:
  US15/196,368
• Publication Number:
  US 20180005058A1
• Time in Patent Office: 692 Days
• CPC Class Codes: G06V 30/10 Character recognition
• CPC Class Codes: G06V 30/153 using recognition of characters or words
• CPC Class Codes: G06V 30/18076 by analysing connectivity, e.g. edge linking, connected component analysis or slices
• CPC Class Codes: G06V 30/2264 using word shape
• CPC Class Codes: G06V 30/333 Preprocessing; Feature extraction
• CPC Class Codes: G06V 30/36 Matching; Classification

Abstract

Disclosed herein is a method that improves the performance of handwriting recognition by calculating path scores so as to identify the path with the highest score as the basis for interpreting handwritten characters. Specifically, the method comprises the following steps: detecting connected regions in an input image comprising handwritten characters; determining a plurality of segmentation positions of the input image; obtaining a plurality of recognition results for each segment of each path in the input image, wherein each recognition result represents a character candidate for the segment and each path comprises one or more segments; obtaining a plurality of scores corresponding to the recognition results; calculating scores for each path in the input image based on segment lengths and the scores corresponding to the recognition results; and using the path with the highest score to interpret the handwritten characters in the input image.

Claims

1. A method for handwriting recognition, comprising:

detecting connected regions in an input image comprising handwritten characters;

determining a plurality of segmentation positions of the input image;

obtaining multiple alternative paths of the input image, each path containing one or more segments, each path covering all of the connected regions of the input image, the multiple alternative paths being different from each other;

for each path;

obtaining a plurality of recognition results for the segments of the path, wherein each recognition result represents a character candidate for a corresponding segment;

obtaining a plurality of scores corresponding to the recognition results of the segments; and

calculating a path score for the path based on segment lengths and the scores corresponding to the recognition results of the segments using the following equation;

S_path=a*(S₁₁*Len₁+S₁₂*Len₂+ . . . +S_1m*Len_m)+b*min(S₁₁, S₁₂, . . . S_1m), wherein Len₁, Len₂ . . . Len_m represent respective segment lengths for Seg₁, Seg₂, . . . , Seg_m of the path, (S₁₁, S₁₂, . . . , S_1m) represent scores corresponding to recognition results (R₁₁, R₁₂, . . . R_1m) of a decoding path, min means the minimization function, and a and b are pre-defined constants; and

identifying the path with the highest path score and using that path to interpret the handwritten characters in the input image.

2. The method of claim 1, wherein the recognition results (R₁₁, R₁₂, . . . R_1m) of the decoding path represent top character candidates for Seg₁, Seg₂ . . . Seg_m of the path.

3. The method of claim 1, further comprising:

estimating an upper baseline and a lower baseline for the input image based on the connected regions; and

estimating a character height based on the upper and lower baselines.

4. The method of claim 3, wherein the upper and lower baselines are estimated by:

detecting top and bottom positions of each connected region of the input image;

clustering the top positions into a first higher center and a first lower center, wherein the first lower center is selected as the upper baseline; and

clustering the bottom positions into a second higher center and a second lower center, wherein the second higher center is selected as the lower baseline.

5. The method of claim 1, wherein the segmentation positions are determined by:

obtaining a top contour and a bottom contour of the input image;

selecting a plurality of extremum points in the top and bottom contours as potential segmentation positions; and

for each of the plurality of potential segmentation positions, drawing a vertical line at the potential segmentation position, determining whether the vertical line crosses a foreground of the input image more than two times and if so, deleting the potential segmentation position, and determining whether the vertical line crosses a circle in the foreground and if so, deleting the potential segmentation position.

6. The method of claim 5, wherein the foreground of the input image comprises a connected region of black pixels.

7. The method of claim 5, further comprising:

for each of the plurality of potential segmentation positions, if the vertical line does not cross the foreground of the input image more than two times and does not crosses a circle in the foreground, keeping the potential segmentation position as a segmentation position.

8. The method of claim 1, wherein the segmentation positions define one or more segments in the input image.

9. The method of claim 1, wherein each segment comprises one or more non-overlapping black pixels in the input image.

10. The method of claim 1, wherein the recognition results and corresponding scores are obtained by a character classifier.

11. A computer program product comprising a computer usable non-transitory medium having a computer readable program code embedded therein for controlling a data processing apparatus, the computer readable program code configured to cause the data processing apparatus to execute a process for handwriting recognition, the process comprising:

detecting connected regions in an input image comprising handwritten characters;

determining a plurality of segmentation positions of the input image;

obtaining multiple alternative paths of the input image, each path containing one or more segments, each path covering all of the connected regions of the input image, the multiple alternative paths being different from each other;

for each path;

obtaining a plurality of recognition results for the segments of the path, wherein each recognition result represents a character candidate for a corresponding segment;

obtaining a plurality of scores corresponding to the recognition results of the segments; and

calculating a path score for the path based on segment lengths and the scores corresponding to the recognition results of the segments using the following equation;

S_path=a*(S₁₁*Len₁+S₁₂*Len₂+ . . . +S_1m*Len_m)+b*min(S₁₁, S₁₂, . . . S_1m), wherein Len₁, Len₂ . . . Len_m represent respective segment lengths for Seg₁, Seg₂, . . . , Seg_m of the path, (S₁₁, S₁₂, . . . , S_1m) represent scores corresponding to recognition results (R₁₁, R₁₂ . . . R_1m) of a decoding path, min means the minimization function, and a and b are pre-defined constants; and

identifying the path with the highest path score and using that path to interpret the handwritten characters in the input image.

12. The computer program product of claim 11, wherein the recognition results (R₁₁, R₁₂, . . . R_1m) of the decoding path represent top character candidates for Seg₁, Seg₂, . . . Seg_m of the path.

13. The computer program product of claim 11, wherein the segmentation positions define one or more segments in the input image.

14. The computer program product of claim 11, wherein each segment comprises one or more non-overlapping black pixels in the input image.

15. The computer program product of claim 11, wherein the recognition results and corresponding scores are obtained by a character classifier.

16. A computer program product comprising a computer usable non-transitory medium having a computer readable program code embedded therein for controlling a data processing apparatus, the computer readable program code configured to cause the data processing apparatus to execute a process for handwriting recognition, the process comprising:

detecting connected regions in an input image comprising handwritten characters;

estimating an upper baseline and a lower baseline for the input image based on the connected regions, wherein the upper and lower baselines are estimated by;

detecting top and bottom positions of each connected region of the input image;

clustering the top positions into a first higher center and a first lower center, wherein the first lower center is selected as the upper baseline; and

clustering the bottom positions into a second higher center and a second lower center, wherein the second higher center is selected as the lower baseline;

estimating a character height based on the upper and lower baselines;

determining a plurality of segmentation positions of the input image;

obtaining multiple alternative paths of the input image, each path containing one or more segments, each path covering all of the connected regions of the input image, the multiple alternative paths being different from each other;

for each path;

obtaining a plurality of recognition results for the segments of the path, wherein each recognition result represents a character candidate for a corresponding segment;

obtaining a plurality of scores corresponding to the recognition results of the segments; and

calculating a path score for the path based on segment lengths and the scores corresponding to the recognition results of the segments; and

identifying the path with the highest path score and using that path to interpret the handwritten characters in the input image.

17. A computer program product comprising a computer usable non-transitory medium having a computer readable program code embedded therein for controlling a data processing apparatus, the computer readable program code configured to cause the data processing apparatus to execute a process for handwriting recognition, the process comprising:

detecting connected regions in an input image comprising handwritten characters;

determining a plurality of segmentation positions of the input image, wherein the segmentation positions are determined by;

obtaining a top contour and a bottom contour of the input image;

selecting a plurality of extremum points in the top and bottom contours as potential segmentation positions; and

for each of the plurality of potential segmentation positions, drawing a vertical line at the potential segmentation position, determining whether the vertical line crosses a foreground of the input image more than two times and if so, deleting the potential segmentation position, and determining whether the vertical line crosses a circle in the foreground and if so, deleting the potential segmentation position;

obtaining multiple alternative paths of the input image, each path containing one or more segments, each path covering all of the connected regions of the input image, the multiple alternative paths being different from each other;

for each path;

obtaining a plurality of recognition results for the segments of the path, wherein each recognition result represents a character candidate for a corresponding segment;

obtaining a plurality of scores corresponding to the recognition results of the segments; and

calculating a path score for the path based on segment lengths and the scores corresponding to the recognition results of the segments; and

identifying the path with the highest path score and using that path to interpret the handwritten characters in the input image.

18. The computer program product of claim 17, wherein the foreground of the input image comprises a connected region of black pixels.

19. The computer program product of claim 17, wherein the process further comprises:

for each of the plurality of potential segmentation positions, if the vertical line does not cross the foreground of the input image more than two times and does not crosses a circle in the foreground, keeping the potential segmentation position as a segmentation position.