METHOD AND SYSTEM FOR THE DYNAMIC CALIBRATION OF STEREOVISION CAMERAS

• US 20100208034A1
• Filed: 02/17/2009
• Published: 08/19/2010
• Est. Priority Date: 02/17/2009
• Status: Active Grant

• Current Assignee: Arriver Software LLC
• Original Assignee: Autoliv ASP Incorporated (Autoliv Incorporated)
• Inventors: Chen, Tie-Qi
• Granted Patent: US 8,120,644 B2
• Time in Patent Office: Days
• US Class Current: 348/46
• CPC Class Codes: G06T 2207/10021 Stereoscopic video; Stereoscopic image sequence
• CPC Class Codes: G06T 2207/20016 Hierarchical, coarse-to-fine, multiscale or multiresolution image processing; Pyramid transform
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 7/85 Stereo camera calibration
• CPC Class Codes: G06T 7/97 Determining parameters from multiple pictures depth or shape recovery from multiple images G06T7/55; stereo camera calibration G06T7/85
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence

Abstract

The present invention generally provides a method of performing dynamic calibration of a stereo vision system using a specific stereo disparity algorithm adapted to provide for the determination of disparity in two dimensions, X and Y. In one embodiment of the present invention, an X/Y disparity map may be calculated using this algorithm without having to perform pre-warping or first finding the epipolar directions. Thus information related to camera misalignment and/or distortion can be preserved in the resulting X/Y disparity map and later extracted.

Claims

1. A method of analyzing the stereovision image of a natural scene in order to produce an X/Y disparity map of the scene for use in the control of an adaptive or active system, the method comprising the steps of:

capturing first image data of the scene with a first sensor;

capturing second image data of the scene with a second sensor;

applying pyramidal decomposition to the first image data and second image data to produce a plurality of decomposition levels with each level having a set of image pairs derived from the first image data and second image data and wherein the image resolution at each subsequent decomposition level is lower than the image resolution at the preceding decomposition level;

executing a stereo disparity algorithm at each decomposition level;

the algorithm assigning a predetermined disparity shift value to each decomposition level;

determining a disparity map at each decomposition level;

combining the disparity maps from all decomposition levels to form an initial X/Y disparity map of the scene where the disparity shift values compensate for resolution differences that exist between the decomposition levels;

estimating the pitch, roll, and yaw errors associated with angular misalignment of the first and second sensors on the captured image data;

correcting the image data for the pitch, roll, and yaw errors; and

determining a final X/Y disparity map;

wherein the stereo dispersity algorithm applies interpolation onto the image data instead of onto derived disparity data;

wherein the pitch, roll, and yaw errors are dynamically determined each time image data is captured.

2. The method according to claim 1, wherein the first and second image data are represented by a continuous function equal to the summation of a gradient G(x,y) for the first image and gradient F(x,y) for the second image over the entire decomposition level.

3. The method according to claim 2, wherein the gradient for the first and second images include factors describing the pixel location, disparity shift, X/Y disparity, and bias intensity at each decomposition level.

4. The method according to claim 1, wherein pyramidal decomposition is accomplished using one selected from the group of pixel resize, pixel average, Gaussian, and a wavelet decomposition processing.

5. The method according to claim 1, wherein the disparity map determined for a decomposition level is used as the initial disparity shift value for the next lower decomposition level.

6. The method according to claim 1, wherein the X/Y disparity map is determined without pre-warping or finding the epipolar directions.

7. A stereovision system for producing an X/Y disparity map of a natural scene that can be used to control an adaptive or active system, the stereovision system comprising:

a first sensor adapted to capture a first image data of the scene;

a second sensor adapted to capture a second image data of the scene;

a processing unit capable of applying a predetermined algorithm to the first and second image data to determine an X/Y disparity map of the scene without pre-warping or finding the epipolar directions; and

a calibration module in contact with the first sensor, second sensor, and processing unit that is capable of dynamic estimation of pitch, range, and yaw errors;

wherein the processing unit corrects the X/Y disparity map for the errors estimated by the calibration module.

8. The stereovision system according to claim 7, wherein the predetermined algorithm applies pyramidal decomposition to the first image data and second image data to produce a plurality of decomposition levels with each level having a set of image pairs derived from the first image data and second image data and wherein the image resolution at each subsequent decomposition level is lower than the image resolution at the preceding decomposition level.

9. The stereovision system according to claim 7, wherein the predetermined algorithm assigns fixed disparity shift value to each decomposition level and determines a disparity map at each decomposition level.

10. The stereovision system according to claim 9, wherein the processing unit combines the disparity maps from all decomposition levels to form an X/Y disparity map of the scene where the disparity shift values compensate for resolution differences that exist between the decomposition levels.