SEGMENTAL RESCORING IN TEXT RECOGNITION

• US 20100310172A1
• Filed: 06/03/2009
• Published: 12/09/2010
• Est. Priority Date: 06/03/2009
• Status: Active Grant

• Current Assignee: Raytheon BBN Technlogies Corp. (Rtx Corporation)
• Original Assignee: BBN Technologies (Rtx Corporation)
• Inventors: Schwartz, Richard, Subramanian, Krishnakumar, Natarajan, Premkumar, Prasad, Rohit
• Granted Patent: US 8,644,611 B2
• Time in Patent Office: Days
• US Class Current: 382/187
• CPC Class Codes: G06F 18/254 of classification results, e.g. of results related to same input data
• CPC Class Codes: G06V 30/10 Character recognition
• CPC Class Codes: G06V 30/1918 Fusion techniques, i.e. combining data from various sources, e.g. sensor fusion
• CPC Class Codes: G06V 30/2268 using stroke segmentation
• CPC Class Codes: G06V 30/246 using linguistic properties, e.g. specific for English or German language

Abstract

A method for text recognition includes generating a number of text hypotheses for an image, for example, using an HMM based approach using fixed-width analysis features. For each text hypothesis, one or more segmentations are generated and scored at the segmental level, for example, according to character or character group segments of the text hypothesis. In some embodiments, multiple alternative segmentations are considered for each text hypothesis. In some examples, scores determined in generating the text hypothesis and the segmental score are combined to select an overall text recognition of the image.

Claims

1. A method for text recognition comprising:

generating a plurality text hypotheses for an image that includes text, each text hypothesis being associated with a first score;

for each text hypothesis of the generated hypotheses, forming data representing one or more segmentations of the image associated with the hypothesis, each segmentation including a series of segments of the image, each segment corresponding to a part of the text hypothesis;

for each of the one or more segmentations, for each segment in the segmentation, forming data representing segmental features of the segment;

determining a segmental score for each segment according to the segmental features of the segment and the corresponding part of the text hypothesis associated with the segmentation including the segment;

for each text hypothesis, determining an overall segmental score according to the determined segmental score for the segments of the one or more segmentations associated with the text hypothesis, and determining an overall score by combining the overall segmental score and the first score associated with the hypotheses; and

providing data representing a text recognition of the image according to the determined overall score for each of the generated text hypotheses for the image.

2. The method of claim 1 wherein generating the plurality of text hypotheses includes forming a series of analysis features of the image, and generating the text hypothesis such that each character of the text hypothesis corresponds to a sequence of one or more of the analysis features, at least some characters corresponding to sequences of multiple analysis features.

3. The method of claim 2 wherein forming the series of analysis features includes forming a series of substantially regularly spaced analysis features of the image.

4. The method of claim 2 wherein forming the series of analysis features includes forming a series of substantially irregularly spaced analysis features of the image.

5. The method of claim 2 wherein generating the plurality of text hypotheses includes applying a statistical recognition approach that accepts the formed series of analysis features to determine the text hypotheses.

6. The method of claim 5 wherein applying the statistical recognition approach includes applying a Hidden Markov Model (HMM) recognition approach.

7. The method of claim 1 wherein generating the plurality text hypotheses for the image forming includes generating a first segmentation associated with each hypothesis, and wherein forming the data representing the one or more segmentations includes forming segmentations based on the first segmentation for the hypothesis.

8. The method of claim 7 wherein forming the segmentations based on the first segmentation includes iteratively forming successive segmentations.

9. The method of claim 8 wherein iteratively forming the successive segmentations includes using the overall segmental scores in determining successive segmentations.

10. The method of claim 7 wherein forming the segmentations based on the first segmentation includes searching for a set of best segmentations.

11. The method of claim 1 wherein forming the data representing segmental features of each segment includes forming features based on a distribution of pixels values in the segment of the image.

12. The method of claim 11 wherein forming the features includes determining quantitative features.

13. The method of claim 11 wherein forming the features includes determining stroke related features.

14. The method of claim 11 wherein forming the features includes determining categorical features.

15. The method of claim 1 wherein determining the segmental score for each segment includes determining a score that represents a degree to which segmental features for the segment are representative of the corresponding part of the text hypothesis that is associated with that segment.

16. The method of claim 15 wherein determining the score that represents the degree includes applying a classifier trained on examples of characters and associated segmental features of image segments for the examples of the characters.

17. The method of claim 16 wherein applying the classifier includes applying a Support Vector Machine (SVM) approach.

18. The method of claim 15 wherein applying the classifier includes a Neural Network approach.

19. A text recognition system comprising:

a first text recognition system configured to generating a plurality text hypotheses for an input image, each text hypothesis being associated with a first score, the first recognition system being further configured, for each text hypothesis of the generated hypotheses, to form data representing one or more segmentations of the image associated with the hypothesis, each segmentation including a series of segments of the image, each segment corresponding to a part of the text hypothesis;

a segment processor configured to accept the generated text hypotheses and associated segmentations from the first recognition system, and, for each text hypothesis, form one or more segmentations of the image associated with the hypothesis, each segmentation including a series of segments of the image, each segment corresponding to a part of the text hypothesis, and for each of the one or more segmentations, for each segment in the segmentation, forming data representing segmental features of the segment;

wherein the segment processor includes a segment scorer for determining a segmental score for each segment according to the segmental features of the segment and the corresponding part of the text hypothesis associated with the segmentation including the segment;

wherein the segment processor is further configured, for each text hypothesis, to determine an overall segmental score according to the determined segmental score for the segments of the one or more segmentations associated with the text hypothesis;

the system further comprising a scorer configured, for each text hypothesis, to determine an overall score by combining the overall segmental score and the first score generated by the first recognition system, and to output data representing a text recognition of the image according to the determined overall score for each of the generated text hypotheses for the image.

20. Software instructions embodied on a computer readable medium for causing a data processing system to:

generate a plurality text hypotheses for an image that includes text, each text hypothesis being associated with a first score;

for each text hypothesis of the generated hypotheses, form data representing one or more segmentations of the image associated with the hypothesis, each segmentation including a series of segments of the image, each segment corresponding to a part of the text hypothesis;

for each of the one or more segmentations, for each segment in the segmentation, form data representing segmental features of the segment;

determine a segmental score for each segment according to the segmental features of the segment and the corresponding part of the text hypothesis associated with the segmentation including the segment;

for each text hypothesis, determine an overall segmental score according to the determined segmental score for the segments of the one or more segmentations associated with the text hypothesis, and determine an overall score by combining the overall segmental score and the first score associated with the hypotheses; and

provide data representing a text recognition of the image according to the determined overall score for each of the generated text hypotheses for the image.