Systems and methods for recognizing characters in digitized documents

• US 10,354,168 B2
• Filed: 04/07/2017
• Issued: 07/16/2019
• Est. Priority Date: 04/11/2016
• Status: Active Grant

• Current Assignee: A2iA SAS (Mitek Systems Incorporated)
• Original Assignee: A2iA SAS (Mitek Systems Incorporated)
• Inventors: Bluche, Theodore Damien Christian
• Primary Examiner(s): Yeh, Euengnan
• Application Number:
  US15/481,754
• Publication Number:
  US 20180005082A1
• Time in Patent Office: 830 Days
• CPC Class Codes: G06F 18/214 Generating training patterns; Bootstrap methods, e.g. bagging or boosting
• CPC Class Codes: G06N 3/044 Recurrent networks, e.g. Hopfield networks
• CPC Class Codes: G06N 3/045 Combinations of networks
• CPC Class Codes: G06N 3/082 modifying the architecture, e.g. adding, deleting or silencing nodes or connections
• CPC Class Codes: G06N 3/084 Backpropagation, e.g. using gradient descent
• CPC Class Codes: G06V 30/10 Character recognition
• CPC Class Codes: G06V 30/18057 Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]
• CPC Class Codes: G06V 30/333 Preprocessing; Feature extraction
• CPC Class Codes: G06V 30/373 using a special pattern or subpattern alphabet

Abstract

Methods and systems are provided for end-to-end text recognition in digitized documents of handwritten characters over multiple lines without explicit line segmentation. An image is received. Based on the image, one or more feature maps are determined. Each of the one or more feature maps include one or more feature vectors. Based at least in part on the one or more feature maps, one or more scalar scores are determined. Based on the one or more scalar scores, one or more attention weights are determined. By applying the one or more attention weights to each of the one or more feature vectors, one or more image summary vectors are determined. Based at least in part on the one or more image summary vectors, one or more handwritten characters are determined.

Claims

1. A method of recognizing a plurality of handwritten characters over multiple lines, the method comprising:

receiving an image of a document including a plurality of handwritten characters;

determining, based at least in part on the image, a plurality of feature maps, each of which is a feature vector at each grid point therein, corresponding to respective plurality of handwritten characters;

determining, based at least in part on the plurality of feature vectors, a scalar score at each grid point;

determining, based on the scalar score, an attention weight at each grid point;

determining, by applying the attention weight to the feature vector at each grid point, a plurality of image summary vectors; and

recognizing, based at least in part on the plurality of image summary vectors, the plurality of handwritten characters.

2. The method of claim 1, further comprising:

determining the size of the image,

wherein the number of the plurality of feature maps is based at least in part on the size of the image.

3. The method of claim 1, further comprising:

displaying the recognized plurality of handwritten characters on a display device.

4. The method of claim 1, wherein the recognized plurality of handwritten characters represent a line of handwritten characters within the image.

5. The method of claim 1, further comprising:

determining, based at least in part on the plurality of image summary vectors, a respective plurality of probability vectors comprising probability values,

wherein the one or more characters of the plurality of handwritten characters is recognized based at least in part on the probability values of the probability vectors.

6. The method of claim 1, further comprising:

determining, based on the number of the plurality of feature maps, a dimension of the plurality of image summary vectors.

7. The method of claim 1, further comprising:

encoding the image; and

extracting a plurality of features from the image,

wherein the plurality of feature maps are determined based at least in part on the plurality of features extracted from the image.

8. The method of claim 1, further comprising:

determining a dimension of the image; and

determining, based on the dimension of the image, a dimension of each of the plurality of feature maps.

9. The method of claim 1, further comprising:

determining, based on the number of the plurality of feature maps, a dimension for the feature vector at each grid point.

10. The method of claim 1, wherein the scalar score corresponds to a particular column coordinate of the image.

11. A system for recognizing a plurality of handwritten characters over multiple lines, the system comprising:

an image capturing device configured to capture an image of a document including a plurality of handwritten characters;

at least one processor and at least one memory coupled to the at least one processor, the at least one memory having instructions stored thereon, which, when executed by the at least one processor, cause the at least one processor to;

receive the image of the document including the plurality of handwritten characters;

determine, based at least in part on the image, a plurality of feature maps, each of which is a feature vector at each grid point therein, corresponding to respective plurality of handwritten characters;

determine, based at least in part on the plurality of feature vectors, a scalar score at each grid point;

determine, based on the scalar score, an attention weight at each grid point;

determine, by applying the attention weight to the feature vector at each grid point, a plurality of image summary vectors; and

recognize, based at least in part on the plurality of image summary vectors, the plurality of handwritten characters; and

a display device configured to display the plurality of recognized handwritten characters.

12. The system of claim 11, wherein the instructions, when executed by the at least one processor, further cause the at least one processor to:

determine the size of the image,

wherein the number of the plurality of feature maps is based at least in part on the size of the image.

13. The system of claim 11, wherein the recognized plurality of handwritten characters represent a line of handwritten characters within the image.

14. The system of claim 11, wherein the instructions, when executed by the at least one processor, further cause the at least one processor to:

determine, based at least in part on the plurality of image summary vectors, a respective plurality of probability vectors comprising probability values,

wherein the one or more characters of the plurality of handwritten characters is recognized based at least in part on the probability values of the probability vectors.

15. The system of claim 11, wherein the instructions, when executed by the at least one processor, further cause the at least one processor to:

determine, based on the number of the plurality of feature maps, a dimension of the plurality of image summary vectors.

16. The system of claim 11, wherein the instructions, when executed by the at least one processor, further cause the at least one processor to:

encode the image;

extract a plurality of features from the image; and

wherein the plurality of feature maps are determined based at least in part on the plurality of features extracted from the image.

17. The system of claim 11, wherein the instructions, when executed by the at least one processor, further cause the at least one processor to:

determine a dimension of the image; and

determine, based on the dimension of the image, a dimension of each of the plurality of feature maps.

18. The system of claim 11, wherein the instructions, when executed by the at least one processor, further cause the at least one processor to:

determine, based on the number of the plurality of feature maps, a dimension for the feature vector at each grid point.

19. The system of claim 11, wherein the scalar score corresponds to a particular column coordinate of the image.

20. The system of claim 11, wherein the image capturing device and the display device are part of a mobile computing device.