Holistic-analytical recognition of handwritten text

• US 6,950,555 B2
• Filed: 02/16/2001
• Issued: 09/27/2005
• Est. Priority Date: 02/16/2001
• Status: Expired due to Term

• Current Assignee: Parascript, LLC
• Original Assignee: Parascript, LLC
• Inventors: Filatov, Alexander, Kil, Igor, Seregin, Arseni
• Primary Examiner(s): Ahmed, Samir
• Assistant Examiner(s): LE, BRIAN Q
• Application Number:
  US09/788,032
• Publication Number:
  US 20020114523A1
• Time in Patent Office: 1,684 Days
• Field of Search: 382/101, 382/102, 382/113, 382173-180, 382/185, 382224-231, 382/159, 382/160, 382/165, 382/170, 382/171, 382/172, 358/462
• US Class Current: 382/229
• CPC Class Codes: G06F 18/254 of classification results, e.g. of results related to same input data
• CPC Class Codes: G06V 30/1918 Fusion techniques, i.e. combining data from various sources, e.g. sensor fusion
• CPC Class Codes: G06V 30/226 of cursive writing

Abstract

In a combined holistic and analytic recognition system, the holistic recognition module will recognize an input word or phrase image by matching an input string of character features for the whole word or phrase against a string of prototype features for a plurality of reference words in a lexicon. This will yield a holistic answer list of recognized word or phrase candidates for the input word or phrase along with a confidence value for each answer on the list. At the same time based on each answer in the answer list, the holistic recognition modules will generate a list of character features and segment the character features into sets for each character in an answer. The analytical recognition module uses segmentation hypotheses from the segmented character feature sets to cut the image of the input string of characters into individual character images. A plurality of character images for the various segmentation hypotheses will be recognized to produce an analytical answer list having a plurality of word or phrase answers for the input word or phrase. Each analytic word answer will have a confidence value based on the combined confidence of recognizing each character. The holistic answer list and the analytic answer list will be examined to find the best answer from the two lists as the recognition of the input handwritten text.

Claims

1. Apparatus for recognizing a string of characters of hand written text in an image loaded in a computing system, the apparatus comprising:

holistic recognition means for recognizing the string of characters as a whole and generating a first answer list and a segmentation list, the first answer list containing a plurality of recognition answers for the string of characters in the image each answer having a confidence value that the answer is correct, the segmentation list containing segmentation information separating each character in the answers;

analytical recognition means responsive to the segmentation list for recognizing a plurality of characters individually and generating a second answer list for the string of characters in the image each answer having a confidence value that the answer is correct; and

means responsive to the first answer list and the second answer list for finding the best recognition answer for the string of characters.

4. The apparatus of claim 1, wherein the string of characters is a series of alphanumeric characters and spaces that make up a word, a sequence of words, one or more numbers, or a mix of words, alphabetic characters and numbers.

5. The apparatus of claim 1, wherein means for finding comprises:

means for matching one or more recognition answers of the first answer list to one or more recognition answers of the second answer list to generate one or more matching answer pairs, each matching answer pair having an associated combined confidence value; and

means for evaluating the combined confidence value associated with each matching answer pair to designate a matching answer pair having a highest combined confidence value as the best recognition answer.

6. The apparatus of claim 5, wherein the combined confidence value associated with each matching answer pair is defined by an average of the confidence values of the recognition answer of the first answer list and the recognition answer of the second answer list of the matching answer pair.

7. The apparatus of claim 5, wherein the means for finding comprises:

means for testing the highest combined confidence value against a next to highest combined confidence value to define an answer separation value; and

means for rejecting the matching word pair associated with the highest combined confidence value as the best recognition answer if the answer separation value is less than a predetermined threshold value.

8. The apparatus of claim 1, wherein the means for finding comprises:

means for evaluating a highest confidence value of the first answer list and a highest confidence value of the second answer list against a probability algorithm to identify the best recognition answer for the string of characters.

2. In a computing system for processing information loaded as cursive text, a method for recognizing the cursive text to provide digital information corresponding to the cursive text, the method comprising:

loading into the computing system an image of an input phrase of cursive text;

identifying features of the input phrase, each feature representing at least a portion of a character in the input phrase;

matching features of the input phrase against features of a plurality of reference phrases and generating a holistic answer list containing as answers reference phrases that are most similar to the input phrase along with a confidence value, the confidence value for each answer being a measure of similarity between features of the input phrase and the features of the reference phrase;

constructing a character segmented features list from the features of the input phrase and from the holistic answer list, the character segmented features list being a list of character feature sets segmented by characters in each answer from the holistic answer list;

translating the image of the input phrase into images of characters segmented according to answers in the holistic answer list based upon the character segmented features list;

analytically recognizing characters in the input phrase from the images of characters segmented according to answers in the holistic answer list and generating an analytical answer list containing analytical answers for the input phrase, each analytical answer having a confidence value as a measure of the similarity between characters in the analytical answer and characters in a reference phrase; and

finding the best recognition answer from the answers on both the holistic answer list and the analytic answer list.

9. The method of claim 2, wherein the input phrase and each reference phrase is a series of alphanumeric characters and spaces that make up a word, a sequence of words, one or more numbers, or a mix of words, alphabetic characters and numbers.

3. In a handwritten character recognition system a method for recognizing an input word of handwritten text in an image provided to the recognition system, the method comprising:

identifying from the input word image an input string of metastrokes where each metastroke represents a portion of an alphanumeric character in the text;

storing the input string of metastrokes as character feature images;

comparing as a whole the input string of metastrokes to a prototype string of metastrokes for reference words to generate a first recognition answer list having a plurality of possible answers;

creating a plurality of character segmentation hypothesis based on character segmented metastrokes for answers in the first recognition answer list;

translating each character segmentation hypothesis into character cutout images of the input word;

recognizing characters from the character cutout images and generating a plurality of character variants for each character position in the input word based on each segmentation hypothesis;

interpreting the plurality of character variants of the input word for each segmentation hypothesis based on a vocabulary and generating a second recognition answer list having a plurality of possible answers; and

finding a best answer from the first and second answer lists as the recognition of the input word.

10. The method of claim 3 further comprising:

identifying each of the plurality of possible answers of the first recognition answer list with a metastroke confidence value corresponding to a degree of similarity between the metastrokes representing the input word and the prototype string of metastrokes associated with each possible answer in the first recognition answer list; and

identifying each of the plurality of possible answers of the second recognition answer list with a confidence value based on a character recognition confidence value of each character variant in each possible answer in the second recognition answer list, the character recognition confidence value corresponding to a degree of similarity between the character variant and the matched character cutout image.

11. The method of claim 10, wherein the operation of identifying each of the plurality of possible answers of the second recognition list comprises:

combining the character recognition confidence value of each character variant in each of the plurality of possible answers of the second recognition answer list to generate a resultant confidence value for each of the plurality of possible answers.

12. The method of claim 11, wherein the finding operation comprises:

matching one or more possible answers of the first recognition list to one or more possible answers of the second recognition list to produce one or more matching answer pairs;

combining the metastroke confidence value associated with the possible answer of the first recognition answer list and the resultant confidence value associated with the possible answer of the second recognition answer list in each matching pair to define a combined confidence value for each pair; and

designating the matching answer pair having a highest combined confidence value as the recognition of the input word.

13. The method of claim 12, wherein the finding operation further comprises:

testing the highest combined confidence value against a next to highest combined confidence value to define an answer separation value; and

rejecting the matching word pair associated with the highest combined confidence value as the recognition of the input word if the answer separation value is less than a predetermined threshold value.