Method and apparatus for calibrating an image capturing device, and a method and apparatus for outputting image frames from sequentially captured image frames with compensation for image capture device offset

• US 7,949,486 B2
• Filed: 10/31/2006
• Issued: 05/24/2011
• Est. Priority Date: 10/28/2005
• Status: Active Grant

• Current Assignee: Connaught Electronics Limited (Valeo SE)
• Original Assignee: Hi-Key Ltd. (Valeo SE)
• Inventors: Sharman, Lloyd Anthony, Raab, Tycho Lorenz Roland, Denny, Patrick Eoghan
• Primary Examiner(s): Bui; Bryan
• Application Number:
  US12/091,636
• Publication Number:
  US 20090179773A1
• Time in Patent Office: 1,666 Days
• Field of Search: 702/94, 348/144, 348/187, 348/219.1, 348/208.5, 382/276, 382/295, 382/309, 382/291
• US Class Current: 702/94
• CPC Class Codes: G06T 2207/30204 Marker
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 7/80 Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration

Abstract

A camera (17) mounted on a rear view minor housing (5) of a vehicle (1), which captures a plan view image of the ground on one side of the vehicle for display on an in-vehicle visual display unit (13) is calibrated to correct for offset of the camera (17) from an ideal position. During calibration an image of a reference point (34) on the vehicle (1) is captured as the mirror housing (5) is swiveled from a rest position to an operative position. The offset of the actual position of the image of the reference point in the captured image from its ideal position is computed, and a look-up table is prepared, which indicates the position at which pixels of subsequently captured image frames should be located to produce image frames with offset correction.

Claims

1. A method for calibrating an output of an image capture device mounted on a vehicle to compensate for offset of the image capture device from an ideal position, the method comprising:

reading digital data representative of an image frame captured by the image capture device,

determining the actual position in the image frame of a reference element captured in the image frame,

determining an offset error value of the actual position of the reference element in the captured image frame from an ideal position of the reference element in a corresponding ideal image frame, and

producing an offset correction transformation function in response to the determined offset error value for correcting image frames subsequently captured by the image capture device to produce image frames with compensation for the offset of the image capture device.

2. A method as claimed in claim 1 in which the offset error value is determined by comparing the actual position of the reference element in the captured image frame with the ideal position of the reference element in the corresponding ideal image frame.

3. A method as claimed in claim 1 in which the offset correction transformation function is expressed as an N parameter function and as an H order polynomial expression, where N is an integer value equal to one or greater, and H is an integer value equal to one or greater.

4. A method as claimed in claim 3 in which the determined actual position of the reference element in the captured image frame is expressed as a function of Cartesian co-ordinates of the actual position of the reference element in the captured image frame, and the ideal position of the reference element in the corresponding ideal image frame is expressed as a function of Cartesian co-ordinates of the ideal position of the reference element in the corresponding ideal image frame, and preferably the offset correction transformation function is expressed as two polynomial expressions, one for each of the Cartesian co-ordinates.

5. A method as claimed in claim 3 in which an offset correction look-up table is produced from the offset correction transformation function for use in the reconstruction of image frames with compensation for the offset of the image capture device from captured image frames subsequently captured by the image capture device, and the offset correction look-up table is stored.

6. A method as claimed in claim 1 in which M offset error values are determined of the actual positions of the reference element in M captured image frames from the ideal positions of the reference element in M respective corresponding ideal image frames, where M is an integer value equal to one or greater, and preferably the offset correction transformation function is produced from a predetermined one of the M offset error values.

7. A method as claimed in claim 6 in which the respective M captured image frames from which the corresponding offset error values are determined of the positions of the reference element from the ideal positions thereof in the corresponding ideal image frames are captured from M respective different angles.

8. A method as claimed in claim 6 in which the image capture device is adapted for mounting on a moveable carrier means mounted on the vehicle, and the image capture device is moved through the M discrete positions by moving the carrier means.

9. A method as claimed in claim 6 in which one of the image capture device and the reference element is moved relative to the other for capturing the M image frames.

10. A method as claimed in claim 1 in which the reference element is defined by a portion of the vehicle.

11. A method as claimed in claim 1 in which the reference element is defined by a reference point, which is defined by a pixel in the captured image frame.

12. A vehicle comprising at least one image capture device mounted thereon, the output of the image capture device being calibrated according to the method as claimed in claim 1.

13. A calibration system for calibrating an output of an image capture device mounted on a vehicle to compensate for offset of the image capture device from an ideal position, the system comprising:

a reading means for reading digital data representative of an image frame captured by the image capture device,

a means for determining the actual position in the captured image frame of a reference element in the captured image frame,

a means for determining an offset error value of the actual position of the reference element in the captured image frame from an ideal position of the reference element in a corresponding ideal image frame, and

a means for producing an offset correction transformation function from the determined offset error value for correcting image frames subsequently captured by the image capture device to produce image frames with compensation for the offset of the image capture device.

14. A calibration system as claimed in claim 13 in which the means for producing the offset correction transformation function produces the offset correction transformation function as an N parameter function and as an H order polynomial expression, where N is an integer value equal to one or greater, and H is an integer value equal to one or greater.

15. A calibration system as claimed in claim 14 in which the means for determining the actual position of the reference element in the captured image frame expresses the actual position of the reference element as a function of Cartesian co-ordinates of the actual position of the reference element in the captured image frame, and the ideal position of the reference element in the corresponding ideal image frame is expressed as a function of Cartesian co-ordinates of the ideal position of the reference element in the corresponding ideal image frame, and the means for producing the offset correction transformation function produces the offset correction transformation function as two polynomial expressions, one for each of the Cartesian co-ordinates.

16. A calibration system as claimed in claim 13 in which the means for determining the offset error value of the actual positions of the reference element from the ideal position thereof determines M offset error values of the actual position of the reference element from respective ideal positions thereof in M captured image frames, where M is an integer value equal to one or greater, and preferably the means for producing the offset correction transformation function produces the offset correction transformation function from a predetermined one of the M offset error values.

17. A calibration system as claimed in claim 16 in which one of the image capture device and the reference element is moveably mounted relative to the other for capturing the M image frames.

18. A calibration system as claimed in claim 16 in which the image capture device is adapted for mounting on a moveable carrier means mounted on the vehicle, and the image capture device is moved through the M discrete positions by moving the carrier means.

19. A calibration system as claimed in claim 13 in which

the reference element is defined by a reference point,

which is defined by a pixel in the captured image frame.

20. A method for sequentially producing a plurality of output image frames derived from respective corresponding captured image frames captured by an image capture device mounted on a vehicle with the output image frames including compensation for offset of the image capture device from an ideal position, the method comprising:

sequentially reading digital data representative of the respective captured image frames,

determining appropriate locations at which at least some of the pixels in each captured image frame should be located in the corresponding output image frame based on calibration data to compensate for the offset of the image capture device, and

constructing each output image frame with the pixels therein corresponding to the at least some of the pixels of the corresponding captured image frame located in the determined appropriate locations,

the calibration data being derived from a method for calibrating the image capture device comprising;

reading digital data representative of an image frame captured by the image capture device,

determining the actual position in the image frame of a reference element captured in the image frame,

determining an offset error value of the actual position of the reference element in the captured image frame from an ideal position of the reference element in a corresponding ideal image frame, and

producing an offset correction transformation function in response to the determined offset error value for correcting image frames subsequently captured by the image capture device to produce image frames with compensation for the offset of the image capture device.

21. A method as claimed in claim 20 in which the calibration data comprises an offset correction look-up table derived from the offset correction transformation function, and the offset correction look-up table identifies the appropriate locations at which the at least some of the pixels of the captured image frames are to be located in the respective corresponding output image frames.

22. A method as claimed in claim 20 in which the location at which the at least some of the pixels of the respective captured image frames are to be located in the respective corresponding output image frames is determined on the fly as the digital data representative of the respective pixels is being received.

23. An image vision system for sequentially producing a plurality of output image frames derived from corresponding captured image frames captured by an image capture device mounted on a vehicle with the output image frames including compensation for offset of the image capture device from an ideal position, the image vision system comprising a reading means for reading digital data representative of the respective captured image frames, a means for determining appropriate locations at which at least some of the pixels in each captured image frame should be located in the corresponding output image frame based on calibration data to compensate for the offset of the image capture device, and a means for constructing each output image frame with the pixels therein corresponding to at least some of the pixels of the corresponding captured image frame located in the determined appropriate locations, the calibration data being derived from a method for calibrating the image capture device, the method comprising:

reading digital data representative of an image frame captured by the image capture device,

determining the actual position in the image frame of a reference element captured in the image frame,

determining an offset error value of the actual position of the reference element in the captured image frame from an ideal position of the reference element in a corresponding ideal image frame, and

producing an offset correction transformation function in response to the determined offset error value for correcting image frames subsequently captured by the image capture device to produce image frames with compensation for the offset of the image capture device.

24. An image vision system as claimed in claim 23 in which the predetermined calibration data is stored in the form of an offset correction look-up table, and the offset correction look-up table identifies the appropriate locations at which the at least some of the pixels of each captured image frame are to be located in the corresponding output image frame.

25. An image vision system as claimed in claim 23 in which a visual display screen responsive to the respective output image frames displays images derived from the output image frames, the visual display screen being adapted for mounting in the field of view of a driver, and the image capture device is adapted for mounting on a carrier means of the vehicle.