Method and apparatus for correction of perspective distortion

• US 7,737,967 B2
• Filed: 07/02/2007
• Issued: 06/15/2010
• Est. Priority Date: 02/17/2004
• Status: Active Grant

• Current Assignee: Corel Corporation (Corel Holdings LP)
• Original Assignee: Dmitry Alexandrovich Chernichenko, Krzysztof Antoni Zaklika
• Inventors: Chernichenko, Dmitry Alexandrovich, Zaklika, Krzysztof Antoni
• Primary Examiner(s): Chauhan; Ulka
• Assistant Examiner(s): BROOME, SAID A
• Application Number:
  US11/772,775
• Publication Number:
  US 20070280554A1
• Time in Patent Office: 1,079 Days
• Field of Search: None
• US Class Current: 345/419
• CPC Class Codes: G06T 2207/20104 Interactive definition of region of interest [ROI]
• CPC Class Codes: G06T 5/80 Geometric correction
• CPC Class Codes: G06T 7/536 from perspective effects, e.g. by using vanishing points

Abstract

Perspective distortion in an imaged scene is corrected by identifying a group of points in the imaged scene such that the group of points define a shape. Two vanishing points are determined based on the group of points in the imaged scene. The process determines a first set of reference points within the shape defined by the group of points and transforms the first set of reference points into a second set of reference points. An aspect ratio associated with the first set of reference points is adjusted and the imaged scene is transformed to reduce perspective distortion in the imaged scene.

Claims

1. A method of correcting perspective distortion in an original image, the method comprising:

determining two vanishing points associated with a plurality of points identified in the original image;

defining a reference rectangle centered about a singular reference point, the singular reference point being bounded within the plurality of points;

determining a set of reference points defining a reference quadrilateral within the reference rectangle, each reference point being located at an intersection of two lines, each line connecting a point on the reference rectangle to one of the vanishing points, the reference quadrilateral having an aspect ratio;

transforming the reference rectangle to have the aspect ratio of the reference quadrilateral using a processor, so as to generate a modified reference rectangle;

mapping the set of reference points onto the modified reference rectangle to generate a new image using a processor.

2. The method of claim 1, further comprising rotating the reference rectangle to ensure that lines extended from the two vanishing points through the singular reference point intersect adjacent sides of the reference rectangle.

3. The method of claim 1, further comprising receiving user input identifying the plurality of points.

4. The method of claim 1, wherein the plurality of points defines a quadrilateral and wherein determining the two vanishing points comprises extending pairs of lines on opposite sides of the quadrilateral until each pair of lines intersect one another.

5. The method of claim 1, wherein the singular reference point is located at a center of gravity of the plurality of points.

6. The method of claim 1, further comprising determining a metric that measures a change in scale of the new image relative to the original image.

7. The method of claim 6, wherein determining the metric comprises calculating an inverse of a minimum eigenvalue of a Jacobian of a transformation matrix used in mapping the set of reference points onto the modified reference rectangle.

8. The method of claim 1, wherein one of the two vanishing points is closer to the singular reference point and the other of the two vanishing points is farther from the singular reference point, wherein determining the set of reference points within the reference rectangle comprises:

determining a side of the reference rectangle intersected by a line extended from the farther vanishing point through the singular reference point;

identifying points on the reference rectangle that are intersected by lines extended from ends of the side opposite the determined side of the reference rectangle to the farther vanishing point, wherein the identified points and the ends of the side opposite the determined side of the reference rectangle define an interim quadrilateral.

9. The method of claim 8, wherein determining the set of reference points within the reference rectangle further comprises:

determining a side of the interim quadrilateral intersected by a line extended from the closer vanishing point through the singular reference point;

identifying points on another side of the interim quadrilateral that are intersected by lines extending from ends of the side opposite the determined side of the interim quadrilateral to the closer vanishing point, wherein the identified points on the another side of the interim quadrilateral and the ends of the side opposite the determined side of the interim quadrilateral define the reference quadrilateral.

10. The method of claim 1, wherein mapping the set of reference points onto the modified reference rectangle comprises applying a transformation matrix to the reference points, wherein the transformation matrix is an inverse of a perspective projection matrix.

11. The method of claim 1, wherein transforming the reference rectangle comprises modifying lengths of sides of the reference rectangle such that a ratio of the sides of the reference rectangle equals a ratio of sides of a quadrilateral defined by the plurality of points.

12. A computing system for correcting perspective distortion in an original image, the computing system comprising:

a memory storing a reference point generator, a reference point transformation module, an aspect ratio adjustment module, and an image mapping module; and

an image processor that implements

the reference point generator to define a reference rectangle centered about a singular reference point bounded within a plurality of points identified in the original image;

the reference point transformation module to determine a set of reference points defining a reference quadrilateral from the reference rectangle, each reference point being located at an intersection of two lines, each line connecting a point on the reference rectangle to one of two determined vanishing points, the reference quadrilateral having an aspect ratio;

the aspect ratio adjustment module to modify the reference rectangle to have an aspect ratio equal to the aspect ratio of the reference quadrilateral, so as to generate a modified reference rectangle; and

the image mapping module to map the set of reference points onto the modified reference rectangle to create a new image.

13. A computer readable medium storing computer executable instructions, which cause a computer to carry out a process, the process comprising:

determining two vanishing points associated with a plurality of points identified in an original image;

defining a reference rectangle centered about a singular reference point, the singular reference point being bounded within the plurality of points;

determining a set of reference points defining a reference quadrilateral within the reference rectangle, each reference point being located at an intersection of two lines, each line connecting a point on the reference rectangle to one of the vanishing points, reference quadrilateral having an aspect ratio;

transforming the reference rectangle to have the aspect ratio of reference quadrilateral, so as to generate a modified reference rectangle;

mapping the set of reference points onto the modified reference rectangle to generate a new image.

14. The computer readable medium of claim 13, the process further comprising receiving user input identifying the plurality of points.

15. The computer readable medium of claim 13, wherein one of the two determined vanishing points is closer to the singular reference point and the other of the two vanishing points is farther from the singular reference point, wherein determining the set of reference points comprises:

determining a side of the reference rectangle intersected by a line extended from the farther vanishing point through the singular reference point;

identifying points on the reference rectangle that are intersected by lines extended from ends of the side opposite the determined side of the reference rectangle to the farther vanishing point, wherein the identified points and the ends of the side opposite the determined side of the reference rectangle define an interim quadrilateral.

16. The computer readable medium of claim 15, wherein determining the set of reference points further comprises:

determining a side of the interim quadrilateral intersected by a line extended from the closer vanishing point through the singular reference point;

identifying points on another side of the interim quadrilateral that are intersected by lines extending from ends of the side opposite the determined side of the interim quadrilateral to the closer vanishing point, wherein the identified points on the another side of the interim quadrilateral and the ends of the side opposite the determined side of the interim quadrilateral define a reference quadrilateral.

17. The computer readable medium of claim 13, further comprising estimating the aspect ratio of the set of reference points.

18. The computer readable medium of claim 13, wherein mapping the set of reference points onto the modified reference rectangle comprises applying a transformation matrix to the reference points, wherein the transformation matrix is an inverse of a perspective projection matrix.

19. The computer readable medium of claim 13, wherein the transforming the reference rectangle to have the aspect ratio of the set of reference points comprises modifying lengths of sides of the reference rectangle such that a ratio of the sides of the reference rectangle equals a ratio of sides of a quadrilateral defined by the plurality of points.

20. The computer readable medium of claim 13, wherein the singular reference point is located at a center of gravity of the plurality of points.

21. The computer readable medium of claim 13, the process further comprising rotating the reference rectangle to ensure that lines extended from the two vanishing points through the singular reference point intersect adjacent sides of the reference rectangle.