Method and system for correcting projective distortions with elimination steps on multiple levels

• US 8,811,751 B1
• Filed: 12/20/2013
• Issued: 08/19/2014
• Est. Priority Date: 12/20/2013
• Status: Active Grant

• Current Assignee: I.R.I.S. (Fidelity National Information Services Incorporated)
• Original Assignee: I.R.I.S. (Fidelity National Information Services Incorporated)
• Inventors: De Muelenaere, Pierre, Dupont, Olivier, Ma, Jianglin, Dauw, Michel
• Primary Examiner(s): Wu, Jingge
• Application Number:
  US14/136,695
• Time in Patent Office: 242 Days
• Field of Search: None
• US Class Current: 382/199
• CPC Class Codes: G06T 2207/20164 Salient point detection; Corner detection
• CPC Class Codes: G06T 3/00 Geometric image transformations in the plane of the image
• CPC Class Codes: G06T 5/80 Geometric correction
• CPC Class Codes: G06V 10/243 by compensating for image skew or non-uniform image deformations

Abstract

Method, system, device and computer program product for projective correction of an image containing at least one text portion that is distorted by perspective. The method includes the steps of image binarization, connected component analysis, horizontal vanishing point determination, vertical vanishing point determination and projective correction. The horizontal vanishing point is determined by estimating text baselines by means of position determining pixels of pixel blobs, identifying horizontal vanishing point candidates from the baselines, and determining a horizontal vanishing point from the candidates. The vertical vanishing point is determined on the basis of vertical features of the text portion. The method includes a first elimination step on the level of position determining pixels, a second elimination step on the level of text baselines and a third elimination step on the level of horizontal vanishing point candidates.

Claims

1. A method for projective correction of an image containing at least one text portion that is distorted by perspective, the method comprising the steps of:

image binarization, wherein said image is binarized;

connected component analysis, wherein pixel blobs are detected in said at least one text portion of said binarized image and wherein for each of said pixel blobs a position determining pixel is selected on a pixel blob baseline of the pixel blob, said position determining pixel defining the position of the pixel blob in the binarized image;

horizontal vanishing point determination, comprising the steps of;

estimating text baselines by means of said position determining pixels of said pixel blobs, identifying horizontal vanishing point candidates from said estimated text baselines, and determining a horizontal vanishing point of said at least one text portion by means of said horizontal vanishing point candidates;

vertical vanishing point determination, wherein a vertical vanishing point is determined for said at least one text portion on the basis of vertical features thereof;

and projective correction, wherein said perspective in said image is corrected on the basis of said horizontal and vertical vanishing points;

wherein said horizontal vanishing point determination comprises a first elimination step on the level of said position determining pixels, a second elimination step on the level of text baselines and a third elimination step on the level of horizontal vanishing point candidates.

2. The method according to claim 1, wherein said position determining pixels are eigenpoints of said pixel blobs.

3. The method according to claim 2, wherein the first elimination step comprises the step of detecting confusing eigenpoints which are out of line with respect to eigenpoints in the vicinity of the eigenpoint in consideration and wherein said confusing eigenpoints are disregarded for said text baseline estimation.

4. Method according to claim 3, wherein said confusing eigenpoints are detected by means of the following steps:

determining the width and height of the pixel blobs;

determining mean values for width and height of the pixel blobs;

and detecting said confusing eigenpoints as eigenpoints belonging to pixel blobs of which at least one of the width and height of the pixel blob in consideration differs by a predetermined extent from said calculated mean values.

5. Method according to claim 2, wherein said step of estimating text baselines comprises a step of clustering eigenpoints into eigenpoint groups, wherein said eigenpoint groups fulfil at least one of the following conditions:

a point to point distance between the eigenpoints of the group is below a first distance threshold,

a point to line distance between each eigenpoint of the group and a line formed by the eigenpoints of the group is below a second distance threshold,

an off horizontal angle of the line formed by the eigenpoints of the group is below a maximum angle, and

the eigenpoint group contains a minimum number of eigenpoints;

and wherein said text baselines are estimated based on said eigenpoint groups.

6. Method according to claim 5, wherein said first distance threshold, said second distance threshold, said maximum angle and said minimum number of eigenpoints are set adaptively based on content of the image.

7. Method according to claim 5, wherein said step of estimating text baselines further comprises a step of eigenpoint group merging wherein eigenpoint groups on both sides of a disregarded eigenpoint are merged into a larger eigenpoint group.

8. Method according to claim 1, wherein the second elimination step comprises the steps of:

assigning confidence levels to said text baselines, and

eliminating text baselines on the basis of said confidence levels.

9. Method according to claim 8, wherein said confidence levels are determined on the basis of at least the length of the respective text baseline and the proximity of the group of eigenpoints used for estimating the text baseline and the resulting text baseline.

10. Method according to claim 8, wherein said elimination of text baselines is performed by means of a RANSAC algorithm in which said confidence levels are taken into account.

11. Method according to claim 1, wherein the third elimination step comprises:

performing projective correction on the basis of each identified horizontal vanishing point candidate;

comparing the proximity of each horizontal vanishing point candidate to the resulting horizontal text direction after projective correction; and

selecting the horizontal vanishing point candidate which is closest to the horizontal text direction of the image document after projective correction.

12. Method according to claim 1, wherein a first and a second horizontal vanishing point candidate are estimated from said text baselines after said second elimination step and wherein for said estimation of said first and second horizontal vanishing point candidates different approximation methods are used, chosen from the group consisting of:

a least squares method, a weighted least squares method and an adaptive least squares method.

13. The method according to claim 1, wherein a step of text and picture separation is performed after said image binarization and before said connected component analyses, and only textual information is kept in said binarized image.

14. A system for projective correction of an image containing at least one text portion that is distorted by perspective, the system comprising at least one processor and an associated storage containing a program executable by means of said at least one processor and comprising:

first software code portions configured for image binarization, which when executed binarize said image;

second software code portions configured for connected component analysis, which when executed detect pixel blobs in said at least one text portion of said binarized image and wherein for each of said pixel blobs a position determining pixel is selected on a pixel blob baseline of the pixel blob, said position determining pixel defining the position of the pixel blob in the binarized image;

third software code portions configured for horizontal vanishing point determination, which when executed perform the steps of;

estimating text baselines by means of said position determining pixels of said pixel blobs, identifying horizontal vanishing point candidates from said estimated text baselines, and determining a horizontal vanishing point of said at least one text portion by means of said horizontal vanishing point candidates;

fourth software code portions configured for vertical vanishing point determination, which when executed determine a vertical vanishing point for said at least one text portion on the basis of vertical features thereof;

and fifth software code portions for projective correction, which when executed correct said perspective in said image on the basis of said horizontal and vertical vanishing points;

wherein said third software code portions when executed perform a first elimination step on the level of said position determining pixels, a second elimination step on the level of text baselines and a third elimination step on the level of horizontal vanishing point candidates.

15. The system of claim 14, comprising one of the following:

a personal computer, a portable computer, a laptop computer, a netbook computer, a tablet computer, a smartphone, a digital still camera, a video camera, a mobile communication device, a personal digital assistant, a scanner, a multi-function device.

16. A non-transient storage medium on which a computer program product is stored comprising software code portions in a format executable on a computer device and configured for performing the following steps when executed on said computer device:

image binarization, wherein said image is binarized;

connected component analysis, wherein pixel blobs are detected in said at least one text portion of said binarized image and wherein for each of said pixel blobs a position determining pixel is selected on a pixel blob baseline of the pixel blob, said position determining pixel defining the position of the pixel blob in the binarized image;

horizontal vanishing point determination, comprising the steps of;

estimating text baselines by means of said position determining pixels of said pixel blobs, identifying horizontal vanishing point candidates from said estimated text baselines, and determining a horizontal vanishing point of said at least one text portion by means of said horizontal vanishing point candidates;

vertical vanishing point determination, wherein a vertical vanishing point is determined for said at least one text portion on the basis of vertical features thereof;

and projective correction, wherein said perspective in said image is corrected on the basis of said horizontal and vertical vanishing points;

wherein said horizontal vanishing point determination comprises a first elimination step on the level of said position determining pixels, a second elimination step on the level of text baselines and a third elimination step on the level of horizontal vanishing point candidates.