METHOD AND A SYSTEM FOR CALIBRATING AN IMAGE CAPTURE DEVICE

• US 20110216201A1
• Filed: 10/01/2008
• Published: 09/08/2011
• Est. Priority Date: 10/01/2008
• Status: Active Grant

• Current Assignee: BMW AG (Bayerische Motoren Werke AG), Application Solutions (Electronics And Vision) Limited (Continental AG), Hi-Key Ltd. (Valeo SE)
• Original Assignee: BMW AG (Bayerische Motoren Werke AG), Hi-Key Ltd. (Valeo SE)
• Inventors: McAndrew, Fintan, Bone, Peter, Zeller, Armin, Kellaway, Trevor Ian, Wilhelm, Sascha
• Granted Patent: US 9,105,080 B2
• Time in Patent Office: Days
• US Class Current: 348/148
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 7/80 Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration

Abstract

A system (1) for calibrating an image capture device (2) mounted on a motor vehicle for offset from an ideal position and an ideal angular orientation while the vehicle is moving comprises selecting an image (12) in a captured image frame (10) of a stationary object relative to the motor vehicle, which is capable of being tracked through a plurality of successively captured image frames (14a,14d), predicting the locations at which the target image (12b) should appear in the respective successively captured image frames (14a,14d), comparing the actual location of the target image (12a) in the respective successively captured image frames (14a,14d) with the respective predicted locations (12b) and determining calibration values for the camera (2) from the results of the comparison, in the event that the actual (12a) and predicted locations (12b) of the target image in the respective image frames (14a,14d) do not coincide.

Claims

1. A method for calibrating an image capture device mounted on a vehicle for offset from at least one of an ideal position and an ideal angular orientation about a focal axis of the image capture device while the vehicle is moving, the method comprising:

selecting an image frame captured by the image capture device as a reference image frame when image data thereof in a time domain in at least one portion of the image frame is indicative of a high spatial frequency content greater than a predefined high spatial frequency content;

selecting, as a target image, an image of a stationary object in an environment external of the vehicle in the reference image frame suitable as a target object;

predicting a location at which an image of the target object should appear in a subsequently captured image frame in response to at least one parameter of the moving vehicle;

comparing the actual location of the image of the target object in the subsequently captured image frame with the predicted location thereof;

determining from the comparison of the actual and the predicted locations of the images of the target object whether the image capture device is offset from the at least one of the ideal position and the ideal angular orientation; and

determining calibration data from the difference between the actual and the predicted locations of the image of the target object for applying to image frames later captured by the image capture device for correcting the later captured image frames for the offset of the image capture device from the at least one of the ideal position and the ideal angular orientation when the image capture device is determined to be offset therefrom.

2. A method as claimed in claim 1, wherein the locations at which the images of the target object should appear in a plurality of respective subsequently captured image frames are predicted, and the actual locations of the images of the target object in the subsequently captured image frames are compared with the corresponding predicted locations thereof.

3. A method as claimed in claim 2, wherein the differences between the actual locations of the images of the target object and the predicted locations thereof in the respective subsequently captured image frames are averaged for determining the calibration data.

4. A method as claimed in claim 2, wherein the subsequently captured image frames are sequentially captured image frames.

6. A method as claimed in claim 4, wherein at least some of the image data in the time domain representative of the captured image frames is passed through a high pass filter for determining if the image frame contains a target image.

7. A method as claimed in claim 1, wherein the target image is selected from an area of predefined size forming a target image area in the reference image frame, and wherein the location at which the image area corresponding to the target image area should appear in each of the subsequently captured image frames is predicted, and the actual image of the area corresponding to the target image area at the predicted location in each of the subsequently captured image frames is compared with the target image area for determining the difference between the actual location of the image of the target object with the predicted location thereof.

8. A method as claimed in claim 7, wherein the location of the target image area in the reference image frame is at a predefined location therein.

9. A method as claimed in claim 7, wherein the actual image of the area corresponding to the target image area at the predicted locations in each of the subsequently captured image frames is compared with the target image area by a template matching process.

10. A method as claimed in any of claim 9, wherein the actual image of the area corresponding to the target image area at the predicted location in each of the subsequently captured image frames is compared with the target image area by comparing image data in a frequency domain representative of the actual image of the area corresponding to the target image area at the predicted locations with image data in a frequency domain representative of the target image area.

11. A method as claimed in claim 10, wherein the image data in the frequency domain representative of the actual image of the area corresponding to the target image area at the predicted location in each of the subsequently captured image frames is compared with the image data in the frequency domain representative of the target image area by a cross-correlation process to produce a joint power spectrum of the actual image of the area at the corresponding predicted location and the target image area.

12. A method as claimed in claim 11, wherein the location of a peak of maximum peak value in a correlation plane of each joint power spectrum is used for determining the difference between the actual location of the image of the target object and the predicted location thereof for the corresponding subsequently captured image plane.

13. A method as claimed in claim 1, wherein the location of the image of the target object in each of the subsequent image frames is predicted in response to at least one of the following parameters of the moving vehicle:

the forward/rearward direction of movement of the vehicle,

the speed of movement of the vehicle, and

the steering angle of the steerable wheels of the vehicle.

14. A method as claimed in claim 1, wherein the captured image frames are initially corrected for at least one selected from a group consisting of perspective distortion and fisheye distortion, and wherein the captured image frames are subsequently converted to top plan view images prior to being processed by the calibration method.

5. (canceled)

15. A system for calibrating an image capture device mounted on a vehicle for offset from at least one of an ideal position and an ideal angular orientation while the vehicle is moving, the system comprising:

means for selecting an image frame captured by the image capture device as a reference image frame;

means for selecting as a target image an image of a stationary object in an environment external of the vehicle in the reference image frame suitable as a target object;

means for predicting a location at which an image of the target object should appear in a subsequently captured image frame in response to at least one parameter of the moving vehicle;

means for comparing the actual location of the image of the target object in the subsequently captured image frame with the predicted location thereof;

means for determining from the comparison of the actual and the predicted locations of the images of the target object whether the image capture device is offset from the at least one of the ideal position and the ideal angular orientation; and

means for determining calibration data from the difference between the actual and the predicted locations of the images of the target object for applying to image frames later captured by the image capture device for correcting the later captured image frames for the offset of the image capture device from the at least one of the ideal position and the ideal angular orientation when the image capture device is determined to be offset therefrom.