Correcting device and method for perspective transformed document images

• US 8,170,368 B2
• Filed: 03/13/2008
• Issued: 05/01/2012
• Est. Priority Date: 03/16/2007
• Status: Expired due to Fees

• Current Assignee: Fujitsu Limited
• Original Assignee: Fujitsu Limited
• Inventors: Yin, Xu-Cheng, Sun, Jun, Fujimoto, Katsuhito, Takebe, Hiroaki, Kurokawa, Koji, Fuji, Yusaku, Naoi, Satoshi
• Primary Examiner(s): Le, Brian Q
• Assistant Examiner(s): TORRES, JOSE
• Application Number:
  US12/076,122
• Publication Number:
  US 20080226171A1
• Time in Patent Office: 1,510 Days
• Field of Search: 382/113, 382/174, 382/176, 382/177, 382/190, 382/195, 382/202, 382/225, 382/276, 382/289
• US Class Current: 382/276
• CPC Class Codes: G06V 10/247 by affine transforms, e.g. correction due to perspective effects; Quadrilaterals, e.g. trapezoids
• CPC Class Codes: G06V 30/1478 of characters or characters lines

Abstract

This invention provides a correcting device and a correcting method for perspective transformation of document images. The correcting device comprises a horizontal vanishing point determining unit, for detecting a horizontal vanishing point of the perspective transformed document image; a vertical vanishing point determining unit, for detecting a vertical vanishing point of the perspective transformed document image; and a perspective transformation correcting and converting unit, for correcting the perspective transformed document image; wherein the horizontal vanishing point determining unit comprises a direct horizontal line segment detecting unit, an indirect horizontal line segment detecting unit and a horizontal vanishing point detecting unit, and wherein the horizontal vanishing point detecting unit detects a horizontal vanishing point in accordance with a direct horizontal line segment detected by the direct horizontal line segment detecting unit and an indirect horizontal line segment detected by the indirect horizontal line segment detecting unit.

Claims

1. A correcting device for a perspective transformed document image, said correcting device comprising:

a horizontal vanishing point determining unit, for detecting a horizontal vanishing point of the perspective transformed document image;

a vertical vanishing point determining unit, for detecting a vertical vanishing point of the perspective transformed document image; and

a perspective transformation correcting and converting unit, for correcting the perspective transformed document image;

wherein the horizontal vanishing point determining unit comprises a direct horizontal line segment detecting unit, an indirect horizontal line segment detecting unit and a horizontal vanishing point detecting unit, and wherein the horizontal vanishing point detecting unit detects a horizontal vanishing point in accordance with direct horizontal line segments detected by the direct horizontal line segment detecting unit and indirect horizontal line segments detected by the indirect horizontal line segment detecting unit; and

wherein the horizontal vanishing point detecting unit comprises;

a horizontal line segment intersection point clustering unit, for clustering point collections formed by intersection points of two horizontal line segments in a collection formed by direct horizontal line segments and indirect horizontal line segments to obtain a plurality of clusters;

a candidate horizontal vanishing point selecting unit, for selecting each cluster as a candidate horizontal vanishing point and obtaining a weight coefficient of each candidate horizontal vanishing point;

a horizontal direction perspective projection analyzing unit, for performing perspective projection analysis in a horizontal direction on the document image with regard to the candidate horizontal vanishing points and obtaining another weight coefficient of each candidate horizontal vanishing point; and

a horizontal vanishing point synthesis analyzing unit, for selecting a final horizontal vanishing point in accordance with the weight coefficient and the another weight coefficient of each of the candidate horizontal vanishing points.

2. The correcting device for a perspective transformed document image according to claim 1, characterized in that the horizontal line segment intersection point clustering unit performs clustering by means of a K-Means method.

3. The correcting device for a perspective transformed document image according to claim 1, characterized in that the horizontal vanishing point synthesis analyzing unit combines the another weight coefficient obtained by the horizontal direction perspective projection analyzing unit and the weight coefficient obtained by the candidate horizontal vanishing point selecting unit.

4. The correcting device for a perspective transformed document image according to claim 1, characterized in that the vertical vanishing point determining unit comprises a direct vertical line segment detecting unit, an indirect vertical line segment detecting unit and a vertical vanishing point detecting unit, and that the vertical vanishing point detecting unit detects a vertical vanishing point in accordance with a direct vertical line segment detected by the direct vertical line segment detecting unit and an indirect vertical line segment detected by the indirect vertical line segment detecting unit.

5. The correcting device for a perspective transformed document image according to claim 4, characterized in that the indirect vertical line segment detecting unit comprises:

a connected component calculating unit, for calculating connected components of the document image; and

a character vertical stroke detecting unit, for analyzing a shape and size of the connected component, determining a candidate vertical stroke, and analyzing shapes of the connected components of the candidate vertical stroke to obtain a vertical line segment.

6. The correcting device for a perspective transformed document image according to claim 5, characterized in that the character vertical stroke detecting unit selects a connected component whose height is close to the height of the character and whose direction is close to the vertical direction as the candidate vertical stroke.

7. The correcting device for a perspective transformed document image according to claim 4, characterized in that the vertical vanishing point detecting unit comprises:

a vertical line segment intersection point clustering unit, for clustering point collections formed by intersection points of two vertical line segments in a collection formed by the direct vertical line segment and the indirect vertical line segment to obtain a plurality of clusters;

a vertical vanishing point selecting unit, for selecting of each cluster as a candidate vertical vanishing point and obtaining a weight coefficient of each candidate vertical vanishing point;

a vertical direction perspective projection analyzing unit, for performing perspective projection analysis in a vertical direction on the document image with regard to each candidate vertical vanishing point and obtaining another weight coefficient of each candidate vertical vanishing point; and

a vertical vanishing point synthesis analyzing unit, for analyzing the weight coefficient obtained by the vertical vanishing point selecting unit and the another weight coefficient obtained by the vertical direction perspective projection analyzing unit with regard to each candidate vertical vanishing point to obtain a final vertical vanishing point.

8. The correcting device for a perspective transformed document image according to claim 7, characterized in that the vertical line segment intersection point clustering unit performs clustering by means of a K-Means method.

9. The correcting device for a perspective transformed document image according to claim 7, characterized in that the vertical direction perspective projection analyzing unit projects each row of horizontal text line and synthesizes a projection analysis result of text lines.

10. The correcting device for a perspective transformed document image according to claim 7, characterized in that the vertical vanishing point selecting unit takes the ratio of the number of intersection points contained in the clusters to the number of all points of intersection as the weight coefficient of the candidate vertical vanishing point.

11. A correcting device for a perspective transformed document image, said correcting device comprising:

a horizontal vanishing point determining unit, for detecting a horizontal vanishing point of the perspective transformed document image;

a vertical vanishing point determining unit, for detecting a vertical vanishing point of the perspective transformed document image; and

a perspective transformation correcting and converting unit, for correcting the perspective transformed document image;

wherein the vertical vanishing point determining unit comprises a direct vertical line segment detecting unit, an indirect vertical line segment detecting unit and a vertical vanishing point detecting unit, and wherein the vertical vanishing point detecting unit detects a vertical vanishing point in accordance with direct vertical line segments detected by the direct vertical line segment detecting unit and indirect vertical line segments detected by the indirect vertical line segment detecting unit; and

wherein the vertical vanishing point detecting unit comprises;

a vertical line segment intersection point clustering unit, for clustering point collections formed by intersection points of every two vertical line segments in a collection formed by the direct vertical line segment and the indirect vertical line segment to obtain a plurality of clusters;

a vertical vanishing point selecting unit, for selecting the central point of each cluster as a candidate vertical vanishing point and obtaining a weight coefficient of each candidate vertical vanishing point;

a vertical direction perspective projection analyzing unit, for performing perspective projection analysis in a vertical direction on the document image with regard to each candidate vertical vanishing point and obtaining another weight coefficient of each candidate vertical vanishing point; and

a vertical vanishing point synthesis analyzing unit, for analyzing the weight coefficient obtained by the vertical vanishing point selecting unit and the another weight coefficient obtained by the vertical direction perspective projection analyzing unit with regard to each candidate vertical vanishing point to obtain a final vertical vanishing point.

12. The correcting device for a perspective transformed document image according to claim 11, characterized in that the indirect vertical line segment detecting unit comprises:

a connected component calculating unit, for calculating connected components of the document image; and

a character vertical stroke detecting unit, for analyzing a shape and size of the connected component, determining a candidate vertical stroke, and analyzing shapes of the connected components of the candidate vertical stroke to obtain a vertical line segment.

13. The correcting device for a perspective transformed document image according to claim 12, characterized in that the character vertical stroke detecting unit selects a connected component whose height is close to the height of the character and whose direction is close to the vertical direction as the candidate vertical stroke.

14. The correcting device for a perspective transformed document image according to claim 11, characterized in that the vertical line segment intersection point clustering unit performs clustering by means of a K-Means method.

15. The correcting device for a perspective transformed document image according to claim 11, characterized in that the vertical direction perspective projection analyzing unit projects each row of horizontal text line and synthesizes a projection analysis result of text lines.

16. The correcting device for a perspective transformed document image according to claim 11, characterized in that the vertical vanishing point selecting unit takes the ratio of the number of intersection points contained in the clusters to the number of all points of intersection as the weight coefficient of the candidate vertical vanishing point.

17. A correcting method for a perspective transformed document image, carried out by a correcting device for a perspective transformed document image, said correcting method comprising:

a horizontal vanishing point determining step, for detecting a horizontal vanishing point of the perspective transformed document image;

a vertical vanishing point determining step, for detecting a vertical vanishing point of the perspective transformed document image; and

a perspective transformation correcting and converting step, for correcting the perspective transformed document image;

wherein the horizontal vanishing point determining step comprises a direct horizontal line segment detecting step, an indirect horizontal line segment detecting step and a horizontal vanishing point detecting step, and wherein the horizontal vanishing point detecting step detects a horizontal vanishing point in accordance with a direct horizontal line segment detected by the direct horizontal line segment detecting step and an indirect horizontal line segment detected by the indirect horizontal line segment detecting step, and

wherein the horizontal vanishing point detecting step comprises;

clustering point collections formed by intersection points of every two horizontal line segments in a collection formed by all direct horizontal line segments and indirect horizontal line segments to obtain a plurality of clusters;

selecting the central point of each cluster as a candidate horizontal vanishing point and obtaining a weight coefficient of each candidate horizontal vanishing point;

performing perspective projection analysis in a horizontal direction on the document image with regard to the candidate horizontal vanishing point and obtaining another weight coefficient of each candidate horizontal vanishing point; and

selecting a final horizontal vanishing point in accordance with the weight coefficient and the another weight coefficient of each of the candidate horizontal vanishing point.

18. A correcting method for a perspective transformed document image, carried out by a correcting device for a perspective transformed document image, said correcting method comprising:

a horizontal vanishing point determining step, for detecting a horizontal vanishing point of the perspective transformed document image;

a vertical vanishing point determining step, for detecting a vertical vanishing point of the perspective transformed document image; and

a perspective transformation correcting and converting step, for correcting the perspective transformed document image;

wherein the vertical vanishing point determining step comprises a direct vertical line segment detecting step, an indirect vertical line segment detecting step and a vertical vanishing point detecting step, and wherein the vertical vanishing point detecting step detects a vertical vanishing point in accordance with a direct vertical line segment detected by the direct vertical line segment detecting step and an indirect vertical line segment detected by the indirect vertical line segment detecting step, and

wherein the vertical vanishing point detecting step comprises;

clustering point collections formed by intersection points of every two vertical line segments in a collection formed by the direct vertical line segment and the indirect vertical line segment to obtain a plurality of clusters;

selecting the central point of each cluster as a candidate vertical vanishing point and obtaining a weight coefficient of each candidate vertical vanishing point;

performing perspective projection analysis in a vertical direction on the document image with regard to each candidate vertical vanishing point and obtaining another weight coefficient of each candidate vertical vanishing point; and

analyzing the weight coefficient obtained by the selecting and the another weight coefficient obtained by the performing and obtaining with regard to each candidate vertical vanishing point to obtain a final vertical vanishing point.