Cylindrical rectification to minimize epipolar distortion

• US 6,011,863 A
• Filed: 03/31/1998
• Issued: 01/04/2000
• Est. Priority Date: 06/12/1997
• Status: Expired due to Term

• Current Assignee: NEC Corporation
• Original Assignee: NEC Research Institute, Inc. (NEC Corporation)
• Inventors: Roy, Sebastien
• Primary Examiner(s): Mancuso, Joseph
• Assistant Examiner(s): Bali, Vikkram
• Application Number:
  US09/052,672
• Time in Patent Office: 644 Days
• Field of Search: 382/285, 382/154, 345/418, 345/139, 345/419, 345/425, 345/427, 356/12, 356/13, 356/14, 348/42, 348/47
• US Class Current: 382/154
• CPC Class Codes: H04N 13/111 Transformation of image signals corresponding to virtual viewpoints, e.g. spatial image interpolation
• CPC Class Codes: H04N 13/122 Improving the 3D impression of stereoscopic images by modifying image signal contents, e.g. by filtering or adding monoscopic depth cues H04N13/128 takes precedence
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 13/246 Calibration of cameras

Abstract

A method for rectifying first and second images of a single scene for stereoscopic analysis is provided. The method has first and second steps of providing the first and second images from first and second cameras. The first camera having a first optical center and a first geometry relative to the scene, the second camera having a second optical center and a second geometry relative to the scene. The first and second images each comprise a plurality of epipolar lines wherein each point on each of the epipolar lines is expressed in a first and second image coordinate system, respectively. Third and fourth steps are provided wherein each point on each epipolar line of the first and second images are rotated to be parallel to a line connecting the first and second optical centers of the first and second cameras. In fifth and sixth steps, the first and second image coordinate systems of each point on each of the epipolar lines of the first and second images are transformed to a first and second cylindrical coordinate system, respectively. In seventh and eighth steps, each rotated epipolar line of the first and second images expressed in the first and second cylindrical coordinate systems are projected onto the surface of first and second cylinders having first and second diameters, wherein each projected epipolar line of the first and second images have a position around the first and second cylinder surfaces such that rectified first and second images are formed thereon.

Claims

1. A method for rectifying first and second images of a single scene for stereoscopic analysis, the method comprising the steps of:

providing the first image from a first camera having a first optical center and a first geometry relative to the scene, the first image comprising a plurality of epipolar lines wherein each point on each of the epipolar lines is expressed in a first image coordinate system;

providing the second image from a second camera having a second optical center and a second geometry relative to the scene, the second image comprising a plurality of epipolar lines wherein each point on each of the epipolar lines is expressed in a second image coordinate system;

rotating each point on each epipolar line of the first image to be parallel to a line connecting the first and second optical centers of the first and second cameras;

rotating each point on each epipolar line of the second image to be parallel to the line connecting the first and second optical centers of the first and second cameras;

transforming the first image coordinate system of each point on each of the epipolar lines of the first image to a first cylindrical coordinate system;

transforming the second image coordinate system of each point on each of the epipolar lines of the second image to a second cylindrical coordinate system;

projecting each rotated epipolar line of the first image expressed in the first cylindrical coordinate system onto the surface of a first cylinder having a first diameter, wherein each projected epipolar line of the first image has a position around the first cylinder surface such that a rectified first image is formed thereon; and

projecting each rotated epipolar line of the second image expressed in the second cylindrical coordinate system onto the surface of a second cylinder having a second diameter, wherein each projected epipolar line of the second image has a position around the second cylinder surface such that a rectified second image is formed thereon.

2. The method of claim 1, wherein the first and second cameras are still cameras oriented arbitrarily to each other and in which the first and second images are simultaneous pictures of the scene.

3. The method of claim 1, wherein the first and second camera comprise a single still camera which moves relative to the scene and takes two still pictures of the scene, one being the first image at the first geometry relative to the scene, and the other being the second image relative to the scene.

4. The method of claim 1, wherein the first and second camera comprise a single video camera which moves relative to the scene and takes two consecutive pictures of the scene, one being the first image at the first geometry relative to the scene, and the other being the second image relative to the scene.

5. The method of claim 1, wherein the first and second image coordinate system is an x,y,z coordinate system with their origin at the respective optical center of the first and second camera.

6. The method of claim 5, wherein the transformation steps for the epipolar lines of one of the first and second images, are done according to the following equations, where the first image is denoted as a, and the second image as b:

##EQU19## where n(v)=v/∥

v∥

is a normalizing function,

space="preserve" listing-type="equation">FOE.sub.a =r.sub.aw ·

(pos.sub.b -pos.sub.a)

space="preserve" listing-type="equation">r.sub.ab =r.sub.aw r.sub.bw.sup.T

space="preserve" listing-type="equation">r.sub.aw =rot(axis.sub.a1 -φ

.sub.a)

space="preserve" listing-type="equation">r.sub.bw =rot(axis.sub.b1 -φ

.sub.b)

and where pos_a, pos_b, axis_a, φ

_a, axis_b, φ

_b are camera positions and rot(A, θ

) is a 3×

3 rotation matrix of angle θ

around axis A.

7. The method of claim 6, wherein the rotation steps for the epipolar lines of one of the first and second images are done according to the following equations, where the first image is denoted as a, and the second image as b:

space="preserve" listing-type="equation">R.sub.FOE =rot.sub.p'"'"', z'"'"'

where ##EQU20## and where p_xyz is a point on the epipolar line to rotate and FOE is the focus of expansion of the camera.

8. The method of claim 7, wherein the projection steps for the epipolar lines of one of the first and second images are done according to the following equations;

##EQU21## where ##EQU22## where c₃ is the third column of rotation matrix R_FOE and q_FOE is a rectified point.

9. The method of claim 1, further comprising the step of outputting the rectified first and second images to an output device for further analysis.

10. The method of claim 9, wherein the output device is selected from a group consisting of a printer and display device.

11. The method of claim 9, further comprising the step of computing depth information from the first and second rectified images for stereo matching analysis.

12. The method of claim 9, further comprising the step of accumulating groups of first and second rectified images for each scene of a motion sequence comprising a plurality of scenes for producing a mosaic of the motion sequence.