Method and system that efficiently prepares text images for optical-character recognition

• US 10,366,469 B2
• Filed: 12/13/2016
• Issued: 07/30/2019
• Est. Priority Date: 06/28/2016
• Status: Active Grant

• Current Assignee: ABBYY Development LLC
• Original Assignee: ABBYY Production LLC (ABBYY Software)
• Inventors: Zagaynov, Ivan, Rybkin, Vladimir
• Primary Examiner(s): Mariam, Daniel G
• Application Number:
  US15/377,077
• Publication Number:
  US 20170372460A1
• Time in Patent Office: 959 Days
• Field of Search: 382176, 382199, 382200
• CPC Class Codes: G06T 3/06 Topological mapping of higher dimensional structures onto lower dimensional surfaces
• CPC Class Codes: G06V 10/247 by affine transforms, e.g. correction due to perspective effects; Quadrilaterals, e.g. trapezoids
• CPC Class Codes: G06V 10/40 Extraction of image or video features

Abstract

The current document is directed to methods and systems that straighten curvature in the text lines of text-containing digital images, including text-containing digital images generated from the two pages of an open book. Initial processing of a text-containing image identifies the outline of a text-containing page. Next, contours are generated to represent each text line. The midpoints and inclination angles of the links or vectors that comprise the contour lines are determined. A model is constructed for the perspective-induced curvature within the text image. In one implementation, the model, essentially an inclination-angle map, allows for assigning local displacements to pixels within the page image which are then used to straighten the text lines in the text image. In another implementation, the model is essentially a pixel-displacement map which is used to straighten the text lines in the text image.

Claims

1. An image-processing system that acquires an image containing text with curved text lines and generates a corresponding corrected image in which the text lines are straightened and have a rectilinear organization, the image-processing system comprising:

one or more processors, one or more memories; and

computer instructions stored in one or more physical data-storage devices that, when executed by one or more of the one or more processors, control the image-processing system to

receive the text-containing image as an input,

identify a page sub-image within the text-containing image,

correct a vertical perspective of the page sub-image,

generate contour lines to represent text lines of the page sub-image,

generate a text-line-curvature model for the page sub-image in view of the contour lines, wherein the text-line-curvature model includes an inclination-angle map that associates each pixel in the page sub-image with an inclination angle of a respective contour line corresponding to a respective pixel, and

straighten the text lines within the received text-containing image using the text-line-curvature model.

2. The image-processing system of claim 1 wherein the image-processing system identifies a page sub-image within the text-containing image by:

identifying intensity edges within the text-containing image; and

using the identified intensity edges along with known geometrical constraints associated with text-containing pages to identify the sub-images corresponding to one or more text-containing pages in the text-containing image.

3. The image-processing system of claim 1 wherein the image-processing system corrects the vertical perspective of the page sub-image by:

rotating the page sub-image so that the spine edge of the page adjacent to the spine of a book has a vertical orientation;

when the non-spine-adjacent edge is contained in the page sub-image,

adjusting the page sub-image so that the non-spine-adjacent edge also has a vertical orientation; and

when the non-spine-adjacent edge is not contained in the page sub-image,

correcting the vertical perspective of the page sub-image by another method.

4. The image-processing system of claim 1 wherein the image-processing system generates contour lines to represent text lines of the page sub-image by:

generating indications of the areas in the page sub-image corresponding to text lines; and

constructing contour lines within the indicated areas in the page sub-image corresponding to text lines.

5. The image-processing system of claim 4 wherein the image-processing system generates indications of the areas in the page sub-image corresponding to text lines by:

copying the page sub-image to a new image;

binarizing the new image to produce a binary image; and

applying one or more morphological operators to the binary image to generate bands of pixels corresponding to text lines with a first binary value on a background of the remaining pixels with a second binary value.

6. The image-processing system of claim 5 further including:

applying one or more vertical smoothing operators to the binary image following application of the one or more morphological operators.

7. The image-processing system of claim 4 wherein constructing contour lines within the indicated areas in the page sub-image corresponding to text lines further comprises:

for each indicated text-line area,

applying one or more non-maximum-suppression operator to the pixels of the text-line area to generate a set of seed pixels for each text line;

generating first links for the seed pixels, the head and tail of each link considered active endpoints; and

extending the active endpoints by adding additional links to the active endpoints.

8. The image-processing system of claim 7 wherein a next link is added to an active endpoint by:

considering a set of candidate links for the active endpoint within a range of angles with respect to the link including the active endpoint;

selecting the next link as the candidate link overlying a set of pixels with the greatest average intensity; and

replacing the active endpoint with the endpoint of the new link.

9. The image-processing system of claim 1 wherein the image-processing system generates a text-line-curvature model for the page sub-image by:

generating an inclination angle for each link of each contour line;

initializing values of a set of coefficients for the text-line-curvature model, wherein the text-line-curvature model is a polynomial text-line-curvature model; and

using a data-fitting method to determine coefficients for the polynomial text-line-curvature model from the inclination angles associated with the links.

10. The image-processing system of claim 9 wherein the polynomial text-line-curvature model returns a tangent of an inclination angle when called with arguments representing the location of a pixel within the page sub-image.

11. The image-processing system of claim 9 wherein the image-processing system straightens the text lines within the received text-containing image using the polynomial text-line-curvature model by:

generating local displacements, using the polynomial text-line-curvature model, for pixels in the page sub-image from the polynomial text-line-curvature model, and

transferring pixels from the page sub-image to a corrected page-sub-image using pixel displacements generated from the local displacements, wherein the corrected page sub-image comprises straitened text lines and text characters and symbols having a rectilinear arrangement.

12. The image-processing system of claim 11 wherein the image-processing system generates local displacements, using the polynomial text-line-curvature model, for pixels in the page sub-image from the polynomial text-line-curvature model by:

generating an inclination-angle map that includes an inclination angle for each pixel of the page sub-image produced by input of a position of the pixel within the page sub-image to the polynomial text-line-curvature model;

modifying the inclination angles in a vertical region of the angle map by multiplication by a coefficient that adjusts the inclination angles for increased curvature and compression in corresponding vertical region of the page sub-image; and

generating an offset map that includes a local displacement for each offset-map cell in the page sub-image.

13. The image-processing system of claim 12 wherein the image-processing system generates a local displacement for a cell in the offset map by:

determining a value for a vertical displacement as a compression coefficient multiplied by a cell width times the tangent of the inclination angle obtained from the polynomial text-line-curvature model;

setting a value for a horizontal displacement to the compression coefficient; and

associating the vertical displacement and the horizontal displacement with the cell.

14. The image-processing system of claim 13 wherein the compression coefficient for a cell is computed as a fraction equal to a cell width divided by a pixel width times one of the cosine of the difference between inclination angles for cells at the top and bottom of a vertical cell column spanning the page image and coincident with the cell and 0.4.

15. The image-processing system of claim 11 wherein the image-processing system transfers pixels from the page sub-image to a corrected page sub-image using pixel displacements generated from the local displacements by:

identifying pairs of corresponding anchor points in the page sub-image and the corrected page sub image;

transferring the intensity value of pixels corresponding to the anchor points in the page sub-image to pixels corresponding to pixels corresponding to the anchor points in the corrected page sub-image; and

for each non-anchor-point pixel in the page sub-image,

summing the local displacements coincident with a vector from the non-anchor-point pixel to a nearby anchor point to produce a pixel displacement, and

transferring the intensity value of the non-anchor-point pixel to a pixel displaced from a corresponding anchor point in the corrected page sub-image by the pixel displacement.

16. An image-processing system that acquires an image containing text with curved text lines and generates a corresponding corrected image in which the text lines are straightened and have a rectilinear organization, the image-processing system comprising:

one or more processors, one or more memories; and

computer instructions stored in one or more physical data-storage devices that, when executed by one or more of the one or more processors, control the image-processing system to

receive the text-containing image as an input,

identify a page sub-image within the text-containing image,

correct a vertical perspective of the page sub-image,

generate contour lines to represent text lines of the page sub-image,

generate a displacement map for the page sub-image in view of the contour lines, wherein;

the displacement map associates each pixel in the page sub-image with a displacement of a respective pixel from a median height of a corresponding contour line, and

the image-processing system generates the displacement map by;

generating a displacement for each link of each contour line;

initializing values of a set of coefficients for the displacement map, wherein the displacement map is a polynomial displacement model; and

using a data-fitting method to determine coefficients for the polynomial displacement model from the displacements associated with the links, and

straighten the text lines within the received text-containing image using the displacement map.

17. The image-processing system of claim 16 wherein the image-processing system straightens the text lines within the received text-containing image using the polynomial displacement model by:

transferring pixels from the page sub-image to a corrected page-sub-image using the pixel displacements generated by the polynomial displacement model.

18. A method that receives a text-containing image and generates a corresponding corrected image in which the text lines are straightened and have a rectilinear organization, the method comprising:

receiving the text-containing image as an input,

identifying a page sub-image within the text-containing image,

correcting a vertical perspective of the page sub-image,

generating contour lines to represent text lines of the page sub-image,

generating a text-line-curvature model for the page sub-image in view of the contour lines, wherein the text-line-curvature model includes an inclination-angle map that associates each pixel in the page sub-image with an inclination angle of a respective contour line corresponding to a respective pixel, and

straightening the text lines within the received text-containing image using the text-line-curvature model.

19. The method of claim 18 wherein generating a text-line-curvature model for the page sub-image further comprises:

generating an inclination angle for each link of each contour line;

initializing values of a set of coefficients for the text-line-curvature model, wherein the text-line-curvature model is a polynomial text-line-curvature model; and

using a data-fitting method to determine coefficients for the polynomial text-line-curvature model from the inclination angles associated with the links.

20. A method that receives a text-containing image and generates a corresponding corrected image in which the text lines are straightened and have a rectilinear organization, the method comprising:

receiving the text-containing image as an input,

identifying a page sub-image within the text-containing image,

correcting a vertical perspective of the page sub-image,

generating contour lines to represent text lines of the page sub-image,

generating a displacement map for the page sub-image in view of the contour lines, wherein;

the displacement map that associates each pixel in the page sub-image with a displacement of a respective pixel from a median height of a corresponding contour line; and

the generating of the displacement map further comprises;

generating a displacement for each link of each contour line;

initializing values of a set of coefficients for the displacement map, wherein the displacement map is a polynomial displacement model; and

using a data-fitting method to determine coefficients for the polynomial displacement model from the displacements associated with the links, and

straightening the text lines within the received text-containing image using the displacement map.

21. A non-transitory computer-readable medium having recorded thereon instructions, that when executed by one or more computer processors for image-processing system, perform operations comprising:

receiving a text-containing image as an input,

identifying a page sub-image within the text-containing image,

correcting a vertical perspective of the page sub-image,

generating contour lines to represent text lines of the page sub-image,

generating a text-line-curvature model for the page sub-image in view of the contour lines, wherein the text-line-curvature model includes an inclination-angle map that associates each pixel in the page sub-image with an inclination angle of a respective contour line corresponding to a respective pixel, and

straightening the text lines within the received text-containing image using the text-line-curvature model.