Image quality assessment and improvement for performing optical character recognition

• US 10,108,883 B2
• Filed: 10/28/2016
• Issued: 10/23/2018
• Est. Priority Date: 10/28/2016
• Status: Active Grant

• Current Assignee: Intuit, Inc.
• Original Assignee: Intuit, Inc.
• Inventors: Becker, Richard J., Kandpal, Rakesh, Kothari, Priya, Porcina, Sheldon, Malynin, Pavlo
• Primary Examiner(s): COLEMAN, STEPHEN P
• Application Number:
  US15/337,285
• Publication Number:
  US 20180121756A1
• Time in Patent Office: 725 Days
• Field of Search: None
• CPC Class Codes: G06F 18/28 Determining representative reference patterns, e.g. by averaging or distorting; Generating dictionaries
• CPC Class Codes: G06V 30/10 Character recognition
• CPC Class Codes: G06V 30/1914 Determining representative reference patterns, e.g. averaging or distorting patterns; Generating dictionaries, e.g. user dictionaries
• CPC Class Codes: G06V 30/224 of printed characters having additional code marks or containing code marks
• CPC Class Codes: G06V 30/40 Document-oriented image-based pattern recognition
• CPC Class Codes: G06V 30/416 Extracting the logical structure, e.g. chapters, sections or page numbers; Identifying elements of the document, e.g. authors

Abstract

Techniques are disclosed for performing optical character recognition (OCR) by assessing and improving quality of electronic documents to perform the OCR. For example a method for identifying information in an electronic document includes obtaining a reference image of the electronic document, distorting the reference image by adjusting different sets of one or more parameters associated with a quality of the reference image to generate a plurality of distorted images, analyzing each distorted image to detect the adjusted set of parameters and corresponding adjusted values, determining an accuracy of detection of the set of parameters and the adjusted values, and training a model based at least on the plurality of distorted images and the accuracy of the detection, wherein the trained model determines at least a first technique for adjusting a set of parameters in a second image to prepare the second image for optical character recognition.

Claims

1. A computer-implemented method for identifying information in an electronic document, comprising:

obtaining a reference image of the electronic document;

distorting the reference image by adjusting parameter values for a plurality of sets of parameters associated with a quality of the reference image to generate a plurality of distorted images;

for each distorted image;

analyzing the distorted image to attempt to detect a first set of parameters from the plurality of sets of parameters and corresponding parameter values used to generate the distorted image;

determining an accuracy of detection of the first set of parameters and the corresponding parameter values used to generate the distorted image, the determining including;

comparing each detected parameter determined as a result of the analyzing the distorted image with the first set of parameters used for generating the distorted image, and

determining the accuracy of the detection based on the comparison; and

training a model based at least on the plurality of distorted images and respective accuracies of the detection to generate a trained model;

obtaining a second image of the electronic document;

determining, based on the trained model, a second set of parameters to be adjusted in the second image and a value corresponding to each parameter in the second set by which the parameter is to be adjusted;

determining, based on the trained model, at least one technique for adjusting each parameter in the second set of parameters in the second image to prepare the second image for optical character recognition (OCR);

preparing the second image for the OCR by adjusting each determined parameter in the second set of parameters by a corresponding determined value based on a corresponding determined technique for the determined parameter to generate a prepared second image; and

performing OCR on the prepared second image.

2. The method of claim 1, wherein determining the accuracy of detection of the first set of parameters and the corresponding parameter values comprises:

comparing a detected parameter value corresponding to the detected parameter determined as a result of the analyzing the distorted image with a corresponding parameter value of a corresponding parameter from the first set of parameters used for the distorting the image; and

determining the accuracy of detection of the detected parameter value based on the comparison.

3. The method of claim 1, further comprising associating with the reference image information regarding a set of characteristics of the reference image, wherein the set of characteristics comprises at least one of a type of the reference image, a label associated with at least one region of the reference image or a format of a value associated with each label.

4. The method of claim 1, wherein the training comprises training the model based on the associated information regarding the set of characteristics.

5. The method of claim 1, further comprising determining, based on the trained model, whether a quality of the second image can be improved to an acceptable level for the OCR.

6. The method of claim 5, wherein determining whether the quality of the second image can be improved to an acceptable level comprises determining whether one or more of the second set of parameters can be adjusted by corresponding parameter values that are equal to or above threshold parameter values associated with the one or more of the second set of parameters.

7. The method of claim 1, further comprising:

determining accuracy of performing the OCR; and

feeding back the prepared second image into the model to improve accuracy of subsequent OCRs of the electronic document.

8. The method of claim 1, wherein distorting the reference image comprises modeling a distribution of a plurality of parameters using a random process.

9. The method of claim 1, wherein parameters in the plurality of sets of parameters comprise at least one of rotation, skew, shadow, luminosity, blur, or color density.

10. The method of claim 1, wherein each of the plurality of sets of parameters include a different combination of the parameters.

11. An apparatus for identifying information in an electronic document, comprising:

at least one processor configured to;

obtain a reference image of the electronic document;

distort the reference image by adjusting parameter values for a plurality of sets of parameters associated with a quality of the reference image to generate a plurality of distorted images;

for each distorted image;

analyze the distorted image to attempt to detect a first set of parameters from the plurality of sets or parameters and corresponding parameter values used to generate the distorted image;

determine an accuracy of detection of the first set of parameters and the corresponding parameter values used to generate the distorted image, wherein the at least one processor determines the accuracy of detection by;

comparing each detected parameter determined as a result of the analyzing the distorted image with the first set of parameters used for generating the distorted image; and

determining the accuracy of the detection based on the comparison; and

train a model based at least on the plurality of distorted images and respective accuracies of the detection to generate a trained model;

obtain a second image of the electronic document;

determine, based on the trained model, a second set of parameters to be adjusted in the second image and a value corresponding to each parameter in the second set by which the parameter is to be adjusted;

determine, based on the trained model, at least one technique for adjusting each parameter in the second set of parameters in the second image to prepare the second image for optical character recognition (OCR)

prepare the second image for the OCR by adjusting each determined parameter in the second set of parameters by a corresponding determined value based on a corresponding technique for the determined parameter to generate a prepared second image; and

perform OCR on the prepared second image; and

a memory coupled to the at least one processor.

12. The apparatus of claim 11, wherein the at least one processor determines the accuracy of detection of the first set of parameters and the corresponding parameter values by:

comparing a detected parameter value corresponding to the detected parameter determined as a result of the analyzing the distorted image with a corresponding parameter value of a corresponding parameter from the first set of parameters used for the distorting the image; and

determining the accuracy of detection of the detected parameter value based on the comparison.

13. The apparatus of claim 11, wherein the at least one processor is further configured to associate with the reference image information regarding a set of characteristics of the reference image, wherein the set of characteristics comprises at least one of a type of the reference image, a label associated with at least one region of the reference image or a format of a value associated with each label.

14. The apparatus of claim 11, wherein the training comprises training the model based on the associated information regarding the set of characteristics.

15. The apparatus of claim 11, wherein the at least one processor is further configured to determine, based on the trained model, whether a quality of the second image can be improved to an acceptable level for the OCR.

16. The apparatus of claim 15, wherein the at least one processor is configured to determine whether the quality of the second image can be improved to an acceptable level by determining whether one or more of the second set of parameters can be adjusted by corresponding values that are equal to or above threshold parameter values associated with the one or more of the second set of parameters.

17. The apparatus of claim 11, wherein the at least one processor is further configured to:

determine accuracy of performing the OCR; and

feed back the prepared second image into the model to improve accuracy of subsequent OCRs of the electronic document.

18. The apparatus of claim 11, wherein the at least one processor distorts the reference image by modeling a distribution of a plurality of parameters using a random process.

19. The apparatus of claim 11, wherein parameters in the plurality of sets of parameters comprise at least one of rotation, skew, shadow, luminosity, blur, or color density.

20. The apparatus of claim 11, wherein each of the plurality of sets of parameters includes a different combination of the parameters.