TABLE DATA RECOVERING IN CASE OF IMAGE DISTORTION

• US 20170372156A1
• Filed: 06/28/2016
• Published: 12/28/2017
• Est. Priority Date: 06/27/2016
• Status: Active Grant

• Current Assignee: ABBYY Development LLC
• Original Assignee: ABBYY Development LLC
• Inventors: Kalenkov, Maksim Petrovich, Chubanov, Dmitrij Yurievich
• Granted Patent: US 9,959,475 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 40/106 Display of layout of documents; Previewing
• CPC Class Codes: G06T 3/18 Image warping, e.g. rearranging pixels individually
• CPC Class Codes: G06T 7/13 Edge detection
• CPC Class Codes: G06V 10/44 Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
• CPC Class Codes: G06V 30/10 Character recognition
• CPC Class Codes: G06V 30/166 Normalisation of pattern dimensions
• CPC Class Codes: G06V 30/186 by deriving mathematical or geometrical properties from the whole image
• CPC Class Codes: G06V 30/224 of printed characters having additional code marks or containing code marks
• CPC Class Codes: G06V 30/412 Layout analysis of documents structured with printed lines or input boxes, e.g. business forms or tables
• CPC Class Codes: G06V 30/414 Extracting the geometrical structure, e.g. layout tree; Block segmentation, e.g. bounding boxes for graphics or text

Abstract

The subject matter of this specification can be implemented in, among other things, a method that includes identifying edges of a section of a document in a source image that includes at least one row of text. The method includes identifying characters in the document. The method includes identifying word portions. The method includes generating polynomials that approximate points of the characters within the word portions. The method includes generating a second polynomial that approximates the points of the characters of word portions. The method includes identifying a stretching coefficient of the row of text based on a length of the section between the edges relative to a length of the second polynomial. The method includes mapping portions of the source image along the row of text to new positions in a corrected image based on the second polynomial and the stretching coefficient.

Claims

1. A method comprising:

identifying edges of at least a section of a document in a source image, wherein the document comprises at least one first row of text;

identifying a plurality of characters in the document;

identifying ones of the characters as a plurality of word portions;

generating a plurality of first polynomials that approximate points of the characters within the word portions;

generating a second polynomial that approximates the points of the characters of at least a first word portion and a second word portion among the word portions and corresponds to the first row of text;

identifying at least one stretching coefficient of the first row of text based on a maximum length of the section between the edges and a length of the second polynomial; and

mapping, by at least one processing device, portions of the source image along the first row of text to new positions in a corrected image based on the second polynomial and the stretching coefficient.

2. The method of claim 1, wherein the points of the characters are along a plurality of baselines of the word portions.

3. The method of claim 1, wherein the second polynomial defines one or more second angles representing slopes along the second polynomial, wherein the method further comprises stepping through existing positions in the source image by a step size that is smaller than a pixel size of the source image, and wherein mapping the portions comprises mapping each of the existing positions to the new positions based on the second angles, the stretching coefficient, and the step size.

4. The method of claim 1, wherein the first polynomials define a plurality of first angles representing slopes along the first polynomials, wherein the second polynomial defines a plurality of second angles representing slopes along the second polynomial, and wherein the method further comprises verifying that ones of the first angles for the first word portion and the second word portion are within a threshold level of accuracy from corresponding ones of the second angles.

5. The method of claim 1, further comprising verifying that the first word portion and the second word portion are at least a threshold distance away from at least one second row of text in the document next to the first row of text, wherein the threshold distance is based on an average distance between polynomials corresponding to rows of text in the document.

6. The method of claim 1, further comprising verifying that the second polynomial is within a threshold level of similarity to at least one third polynomial corresponding to at least one second row of text in the document.

7. The method of claim 1, further comprising interpolating a color for each new position in the new positions of the corrected image based on colors from pixels of the source image in a vicinity of a portion among the portions of the source image that corresponds to the new position.

8. The method of claim 1, further comprising pre-processing the source image, wherein pre-processing comprises performing one or more of applying a gamma filter to the source image, normalizing a width and a height of the source image, adjusting a brightness of the source image, or binarizing a color depth of the source image.

9. The method of claim 1, wherein identifying the edges is based on a line detection algorithm or an edge detection algorithm.

10. The method of claim 1, wherein the new positions of the portions remove distortion from the first row of text in the corrected image.

11. The method of claim 1, further comprising:

generating at least one interpolated polynomial between the second polynomial and at least one third polynomial corresponding to at least one second row of text in the document; and

mapping portions of the source image between the first row of text and the second row of text to new positions in the corrected image based on the interpolated polynomial.

12. A non-transitory computer-readable medium having instructions stored thereon that, when executed by at least one processing device, cause the processing device to:

identify edges of at least a section of a document in a source image, wherein the document comprises at least one first row of text;

identify a plurality of characters in the document;

identify ones of the characters as a plurality of word portions;

generate a plurality of first polynomials that approximate points of the characters within the word portions;

generate a second polynomial that approximates the points of the characters of at least a first word portion and a second word portion among the word portions and corresponds to the first row of text;

identify at least one stretching coefficient of the first row of text based on a maximum length of the section between the edges and a length of the second polynomial; and

map, by the processing device, portions of the source image along the first row of text to new positions in a corrected image based on the second polynomial and the stretching coefficient.

13. A system comprising:

at least one memory that stores instructions; and

at least one processing device to execute the instructions to;

identify edges of at least a section of a document in a source image, wherein the document comprises at least one first row of text;

identify a plurality of characters in the document;

identify ones of the characters as a plurality of word portions;

generate a plurality of first polynomials that approximate points of the characters within the word portions;

generate a second polynomial that approximates the points of the characters of at least a first word portion and a second word portion among the word portions and corresponds to the first row of text;

identify at least one stretching coefficient of the first row of text based on a maximum length of the section between the edges and a length of the second polynomial; and

map portions of the source image along the first row of text to new positions in a corrected image based on the second polynomial and the stretching coefficient.

14. The system of claim 13, wherein the points of the characters are along a plurality of baselines of the word portions.

15. The system of claim 13, wherein the second polynomial defines one or more second angles representing slopes along the second polynomial, wherein the processing device is further to step through existing positions in the source image by a step size that is smaller than a pixel size of the source image, and wherein, to map the portions, the processing device is further to map each of the existing positions to the new positions based on the second angles, the stretching coefficient, and the step size.

16. The system of claim 13, wherein the first polynomials define a plurality of first angles representing slopes along the first polynomials, wherein the second polynomial defines a plurality of second angles representing slopes along the second polynomial, and wherein the processing device is further to verify that ones of the first angles for the first word portion and the second word portion are within a threshold level of accuracy from corresponding ones of the second angles.

17. The system of claim 13, wherein the processing device is further to verify that the first word portion and the second word portion are at least a threshold distance away from at least one second row of text in the document next to the first row of text, and wherein the threshold distance is based on an average distance between polynomials corresponding to rows of text in the document.

18. The system of claim 13, wherein the processing device is further to verify that the second polynomial is within a threshold level of similarity to at least one third polynomial corresponding to at least one second row of text in the document.

19. The system of claim 13, wherein the new positions of the portions remove distortion from the first row of text in the corrected image.

20. The system of claim 13, wherein the processing device is further to:

generate at least one interpolated polynomial between the second polynomial and at least one third polynomial corresponding to at least one second row of text in the document; and

map portions of the source image between the first row of text and the second row of text to new positions in the corrected image based on the interpolated polynomial.