CALIBRATION METHOD AND APPARATUS FOR AUTOMOTIVE CAMERA SYSTEM, AND METHOD AND ECU FOR DETERMINING ANGULAR MISALIGNMENTS OF AUTOMOTIVE CAMERA SYSTEM

• US 20100253784A1
• Filed: 06/18/2009
• Published: 10/07/2010
• Est. Priority Date: 04/06/2009
• Status: Abandoned Application

• Current Assignee: Samsung Electro-Mechanics Co., Ltd. (Samsung Group)
• Original Assignee: Samsung Electro-Mechanics Co., Ltd. (Samsung Group)
• Inventors: Oleg, Konevsky
• Application Number:
  US12/487,103
• Publication Number:
  US 20100253784A1
• Time in Patent Office: Days
• US Class Current: 348/187
• CPC Class Codes: G06T 2207/30204 Marker
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 7/80 Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
• CPC Class Codes: H04N 17/002 for television cameras
• CPC Class Codes: H04N 21/41422 located in transportation means, e.g. personal vehicle arrangements specially adapted for transportation systems in broadcast systems H04H20/62
• CPC Class Codes: H04N 21/4223 Cameras H04N23/00 takes precedence
• CPC Class Codes: H04N 21/44008 involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 7/163 by receiver means only

Abstract

A method of automotive camera system calibration is disclosed in the present invention. At least two targets are placed within the field of view of each camera included in the system. Each camera captures the targets. Then, the vehicle is moved, and another image is captured. The images are used to derive the camera orientation parameters. Based on these parameters, misalignment of the cameras can be compensated by the image processing unit.

Claims

1. Method of calibrating a camera equipped on vehicle, comprising the steps:

a) capturing a first image featuring at least two targets with a camera attached to each side of the vehicle, the targets being placed in the peripheral area of the vehicle;

b) displacing the vehicle in the longitudinal direction without moving the targets;

c) capturing a second image featuring the targets at the displaced position;

d) detecting the targets on the captured first and second images;

e) calculating world coordinates of the targets based on the captured images; and

f) searching for misalignment angles while optimizing a metric function based on the calculated world coordinates of the targets.

2. The method according to claim 1, wherein the targets are placed or painted on the ground in the peripheral area of the vehicle.

3. The method according to claim 1, wherein a distance between the two targets adjacent to each other in the lateral direction is maximized but still kept within a field of view of the camera.

4. The method according to claim 1, wherein the vehicle is configured to be displaced on its own wheels or on a conveyor or other equivalent means.

5. The method according to claim 1, wherein step e) comprises calculating spherical coordinates of the chief ray entering an optical unit of the camera and world coordinates of its cross point with the ground plane.

6. The method according to claim 5, further comprising adjusting chief ray spherical coordinates according to the lens optical distortion.

7. The method according to claim 1, wherein step f) comprises starting the search with nominal rotation angles of a camera.

8. Apparatus for calibrating a camera equipped on vehicle, comprising:

a plurality of camera units attached to four sides of vehicle, each camera unit being configured to capture a image featuring at least two targets; and

a electronic control unit configured to form a single top view by synthesizing the images captured by each of the plurality of camera units, wherein the electronic control unit determines an angular misalignment of vehicle with a method comprising the steps;

a) capturing a first image featuring at least two targets per camera unit attached to a side of the vehicle, the targets being placed in the peripheral area of the vehicle;

b) displacing the vehicle in the longitudinal direction without moving the targets;

c) capturing a second image featuring the targets at the displaced position;

d) detecting the targets on the captured first and second images;

e) calculating world coordinates of the targets based on the captured images; and

f) searching for misalignment angles while optimizing a metric function based on the calculated world coordinates of the targets.

9. The apparatus according to claim 8, wherein the electronic control unit is incorporated into an electronic control unit already implemented in the vehicle.

10. The apparatus according to claim 8, wherein the electronic control unit is a stand-alone device separated from an electronic control unit already implemented in the vehicle.

11. Method for determining an angular misalignment of a camera equipped on vehicle, comprising:

a) capturing a first image featuring at least two targets placed at one side in the peripheral area of the vehicle at a first position of the vehicle;

b) capturing a second image featuring the targets at a second position away from the first position of the vehicle in the longitudinal direction;

c) detecting the targets on the first and second images and obtaining their coordinates on an imager;

d) initiating rotation angles α

, β

, γ

with nominal angle values, respectively;

e) calculating angles φ and

θ

for each of the targets at the first and second positions, the angles φ and

θ

defining chief ray in the spherical coordinate system whose origin coincides with world coordinate system'"'"'s origin;

f) calculating an affine transformation rotation matrix with current values of α

, β

, γ

;

g) calculating world coordinates for the targets at the first and second positions using the angles θ and

φ

, and elevation of the camera over the ground;

h) calculating metric used for rotation angles optimization using the calculated world coordinates;

i) updating the angles α

, β

, γ

with optimization method until a termination criterion specific for the optimization method is met; and

j) calculating misalignment angles from the differences between the undated angles α

, β

, γ

meeting the termination criterion specific and the nominal rotation angles.

12. Electronic control unit for determining an angular misalignment of a camera equipped on vehicle, the electronic control unit being configured to control and/or perform the steps comprising:

a) at a first position of the vehicle and at a second position away from the first position of the vehicle in the longitudinal direction, capturing first and second images each featuring at least two targets placed at one side in the peripheral area of the vehicle, and in turn detecting the targets on the first and second images and then obtaining their coordinates on an imager;

b) initiating rotation angles α

, β

, γ

with nominal angle values, respectively;

c) calculating angles φ and

θ

for each of the targets at the first and second positions, the angles φ and

θ

defining chief ray in the spherical coordinate system whose origin coincides with world coordinate system'"'"'s origin;

d) calculating an affine transformation rotation matrix with current values of α

, β

, γ

;

e) calculating world coordinates for the targets at the first and second positions using the angles θ

0 and φ

, and elevation of the camera over the ground;

f) calculating metric used for rotation angles optimization using the calculated world coordinates;

g) updating the angles α

, β

, γ

with optimization method until a termination criterion specific for the optimization method is met; and

h) calculating misalignment angles from the differences between the undated angles α

, β

, γ

meeting the termination criterion specific and the nominal rotation angles.

13. The electronic control unit according to claim 12, wherein the electronic control unit is incorporated into an electronic control unit already implemented in the vehicle.

14. The electronic control unit according to claim 12, wherein the electronic control unit is a stand-alone device separated from an electronic control unit already implemented in the vehicle.

15. The electronic control unit according to claim 12, wherein the targets are placed or painted on the ground in the peripheral area of the vehicle.

16. The electronic control unit according to claim 15, wherein a distance between the two targets adjacent to each other in the lateral direction is maximized but still kept within a field of view of the camera.