Method and system for determining vanishing point candidates for projective correction

• US 8,897,600 B1
• Filed: 12/20/2013
• Issued: 11/25/2014
• Est. Priority Date: 12/20/2013
• Status: Active Grant

• Current Assignee: I.R.I.S. (Fidelity National Information Services Incorporated)
• Original Assignee: I.R.I.S. (Fidelity National Information Services Incorporated)
• Inventors: Ma, Jianglin, Dauw, Michel, De Muelenaere, Pierre, Dupont, Olivier
• Primary Examiner(s): DRENNAN, BARRY T
• Application Number:
  US14/136,585
• Time in Patent Office: 340 Days
• Field of Search: None
• US Class Current: 382/290
• CPC Class Codes: G06T 2207/10008 from scanner, fax or copier
• CPC Class Codes: G06T 2207/20076 Probabilistic image processing
• CPC Class Codes: G06T 2207/30176 Document
• CPC Class Codes: G06T 3/02 Affine transformations for image registration G06T3/147; for image mosaicing G06T3/4038
• CPC Class Codes: G06T 5/80 Geometric correction
• CPC Class Codes: G06V 10/247 by affine transforms, e.g. correction due to perspective effects; Quadrilaterals, e.g. trapezoids
• CPC Class Codes: G06V 30/1478 of characters or characters lines

Abstract

Method, system, device and computer program product for determining vanishing point candidates of a text portion in an image document distorted by perspective. The method includes the steps of image binarization, connected component analysis, estimating a number of text lines in a Cartesian coordinate system, transforming the text lines to data points in a homogenous coordinate system, assigning a confidence level to the data points, grouping a number of data points into a priority sample array, clustering the data points in the priority sample array into a number of sample groups and assigning a group confidence value to each sample group. A RANSAC algorithm is applied to determine among the data points a set of inliers, initiated with the sample group having the highest group confidence value. A vanishing point candidate is determined from the text lines corresponding to the set of inliers.

Claims

1. A method for determining vanishing point candidates of a text portion in an image document that is distorted by perspective, comprising the steps of:

image binarization, wherein said image is binarized;

performing connected component analysis, wherein pixel blobs are detected in said at least one text portion of said binarized image and wherein for each of said pixel blobs a position determining pixel is selected on a pixel blob baseline of the pixel blob, said position determining pixel defining the position of the pixel blob in the binarized image;

estimating a number of text lines in a Cartesian coordinate system, each text line representing an approximation of a horizontal or vertical text direction of said text portion, on the basis of the position determining pixels;

transforming each of said text lines to a data point in a homogenous coordinate system;

assigning a confidence level to each of the data points;

grouping a number of data points having a confidence level above a predetermined threshold into a priority sample array;

clustering the data points in the priority sample array into a number of sample groups, wherein each sample group comprises at least two data points;

assigning a group confidence value to each sample group on the basis of at least the confidence level assigned to each data point in the sample group;

applying a RANSAC algorithm to determine among said data points a set of inliers with respect to a first fitted line, wherein said RANSAC algorithm is initiated with the sample group having the highest group confidence value in the priority sample array;

estimating at least one vanishing point candidate from the text lines corresponding to said set of inliers.

2. Method according to claim 1, wherein said confidence level that is assigned to said data points is based on at least the length of the respective text line and the proximity of the position determining pixels to the respective text line.

3. Method according to claim 1, wherein said RANSAC algorithm comprises the following steps:

iteratively selecting sample groups of data points from the priority sample array for line fitting, wherein said iteration starts with the sample group having the highest group confidence value in the priority sample array;

performing line fitting for the first sample group resulting in a first fitted line and subsequently performing line fitting for each further sample group resulting in further fitted lines; and

determining on the basis of the first and further fitted lines a set of data points which are positioned below a predetermined distance threshold from the first fitted line, said set of data points forming said set of inliers.

4. Method according to claim 3, wherein the predetermined distance threshold from the first fitted line is a fixed parameter.

5. Method according to claim 3, wherein the predetermined distance threshold from the first fitted line is an adaptive parameter which is adapted based on content of the image document.

6. Method according to claim 1, wherein at least a first and a second vanishing point candidate are estimated from the text lines corresponding to said set of inliers.

7. Method according to claim 6, wherein the first and second vanishing point candidates are estimated using different approximation methods chosen from the group consisting of:

a least squares method, a weighted least squares method and an adaptive least squares method.

8. Method according to claim 1, further comprising the step of selecting a vanishing point from the estimated vanishing point candidates, said selection comprising the steps of:

performing projective correction on the image document on the basis of each estimated vanishing point candidate;

comparing the proximity of each vanishing point candidate to the resulting horizontal or vertical text direction after projective correction; and

selecting the vanishing point candidate which is closest to the horizontal or vertical text direction of the image document after projective correction.

9. Method according to claim 1, wherein the group confidence value of each sample group is further based on the distances between the respective estimated text lines corresponding to the data points in the sample group.

10. Method according to claim 1, wherein the confidence level of each of the data points is further based on a dominant direction of the pixel blobs used for estimating each respective text line, wherein said dominant direction is defined by the eccentricity of the shape of each pixel blob.

11. Method according to claim 1, wherein the maximum number of data points grouped into the priority sample array is between 2 and 20.

12. Method according to claim 1, wherein each of the at least one vanishing point candidates is a horizontal vanishing point candidate and the position determining pixels are eigenpoints of the pixel blobs.

13. Method according to claim 1, wherein each of the at least one vanishing point candidates is a vertical vanishing point candidate and wherein the estimated text lines are vertical text blob lines which each correspond to the direction of a selected one of said pixel blobs, selected by a blob filtering algorithm on the text portion of the image.

14. The method according to claim 1, wherein a step of text and picture separation is performed after said image binarization and before said connected component analyses, and only textual information is kept in said binarized image.

15. Method according to claim 1, wherein the maximum number of data points grouped into the priority sample array is between 5 and 10.

16. A method for projective correction of an image containing at least one text portion that is distorted by perspective, the method comprising the steps of:

image binarization, wherein said image is binarized;

performing connected component analysis, wherein pixel blobs are detected in said at least one text portion of said binarized image and wherein for each of said pixel blobs a position determining pixel is selected on a pixel blob baseline of the pixel blob, said position determining pixel defining the position of the pixel blob in the binarized image;

horizontal vanishing point determination, comprising estimating text baselines by means of said position determining pixels of said pixel blobs and determining at least one horizontal vanishing point candidate of said at least one text portion by means of said text baselines;

vertical vanishing point determination, comprising estimating vertical text blob lines which each correspond to the direction of a selected one of said pixel blobs, selected by a blob filtering algorithm on the text portion of the image, and determining at least one vertical vanishing point candidate of said at least one text portion by means of said vertical text blob lines;

wherein at least one of said horizontal and vertical vanishing point determination comprises the steps of;

transforming each of said estimated text lines to a data point in a homogenous coordinate system;

assigning a confidence level to each of the data points;

grouping a number of data points having a confidence level above a predetermined threshold into a priority sample array;

clustering the data points in the priority sample array into a number of sample groups, wherein each sample group comprises at least two data points;

assigning a group confidence value to each sample group on the basis of at least the confidence level assigned to each data point in the sample group;

applying a RANSAC algorithm to determine among said data points a set of inliers with respect to a first fitted line, wherein said RANSAC algorithm is initiated with the sample group having the highest group confidence value in the priority sample array;

and estimating said at least one vanishing point candidate from the text lines corresponding to said set of inliers;

and projective correction, wherein said perspective in said image is corrected on the basis of a horizontal vanishing point selected among said at least one horizontal vanishing point candidate and a vertical vanishing point selected among said at least one vertical vanishing point candidate.

17. A system for projective correction of an image containing at least one text portion that is distorted by perspective, the system comprising at least one processor and an associated storage containing a program executable by means of said at least one processor and comprising:

first software code portions configured for image binarization, which when executed binarize said image;

second software code portions configured for connected component analysis, which when executed detect pixel blobs in said at least one text portion of said binarized image and for each of said pixel blobs select a position determining pixel on a pixel blob baseline of the pixel blob, said position determining pixel defining the position of the pixel blob in the binarized image;

third software code portions configured for horizontal vanishing point determination, which when executed estimate text baselines by means of said position determining pixels of said pixel blobs and determine at least one horizontal vanishing point candidate of said at least one text portion by means of said text baselines;

fourth software code portions configured for vertical vanishing point determination, which when executed estimate vertical text blob lines which each correspond to the direction of a selected one of said pixel blobs, selected by a blob filtering algorithm on the text portion of the image, and determine at least one vertical vanishing point candidate of said at least one text portion by means of said vertical text blob lines;

wherein at least one of said third and fourth software code portions is configured for performing the steps of;

transforming each of said estimated text lines to a data point in a homogenous coordinate system;

assigning a confidence level to each of the data points;

grouping a number of data points having a confidence level above a predetermined threshold into a priority sample array;

clustering the data points in the priority sample array into a number of sample groups, wherein each sample group comprises at least two data points;

assigning a group confidence value to each sample group on the basis of at least the confidence level assigned to each data point in the sample group;

applying a RANSAC algorithm to determine among said data points a set of inliers with respect to a first fitted line, wherein said RANSAC algorithm is initiated with the sample group having the highest group confidence value in the priority sample array;

and estimating said at least one vanishing point candidate from the text lines corresponding to said set of inliers;

and fifth software code portions configured for performing projective correction, which when executed correct said perspective in said image on the basis of a horizontal vanishing point selected among said at least one horizontal vanishing point candidate and a vertical vanishing point selected among said at least one vertical vanishing point candidate.

18. The system of claim 17, comprising one of the following:

a personal computer, a portable computer, a laptop computer, a netbook computer, a tablet computer, a smartphone, a digital still camera, a video camera, a mobile communication device, a personal digital assistant, a scanner, a multi-function device.

19. A non-transient storage medium on which a computer program product is stored comprising software code portions in a format executable on a computer device and configured for performing the following steps when executed on said computer device:

image binarization, wherein said image is binarized;

performing connected component analysis, wherein pixel blobs are detected in said at least one text portion of said binarized image and wherein for each of said pixel blobs a position determining pixel is selected on a pixel blob baseline of the pixel blob, said position determining pixel defining the position of the pixel blob in the binarized image;

estimating a number of text lines in a Cartesian coordinate system, each text line representing an approximation of a horizontal or vertical text direction of said text portion, on the basis of the position determining pixels;

transforming each of said text lines to a data point in a homogenous coordinate system;

assigning a confidence level to each of the data points;

grouping a number of data points having a confidence level above a predetermined threshold into a priority sample array;

clustering the data points in the priority sample array into a number of sample groups, wherein each sample group comprises at least two data points;

assigning a group confidence value to each sample group on the basis of at least the confidence level assigned to each data point in the sample group;

applying a RANSAC algorithm to determine among said data points a set of inliers with respect to a first fitted line, wherein said RANSAC algorithm is initiated with the sample group having the highest group confidence value in the priority sample array;

estimating at least one vanishing point candidate from the text lines corresponding to said set of inliers.