Method and system for dynamically calibrating vehicular cameras
First Claim
1. A method for dynamically ascertaining the position of a vehicular camera relative to a vehicle to which the camera is attached, the method comprising:
- (a) establishing a plurality of vehicular steering angle ranges;
(b) acquiring a set of image frames in a video stream provided by the camera whilst the vehicle is in motion, the image frames defining an image plane;
(c) measuring the steering angle of the vehicle and, for each steering angle range;
(i) selecting a plurality of feature points in the image frames;
(ii) tracking a motion trajectory of each selected feature point in the set of image frames, and(iii) determining a vanishing point in the image plane for the plurality of tracked motion trajectories;
(d) determining a vanishing line in the image plane provided by the camera based on a locus of said vanishing points;
(e) determining the position or alignment of the camera based on the position of the vanishing line and the position of a central vanishing point, the central vanishing point being the vanishing point that is determined for the steering angle range that encompasses zero degrees;
removing distortion in the motion trajectories caused by use of a wide angle camera lens;
linearly fitting the undistorted motion trajectories to generate straight lines;
wherein determining a vanishing point comprises determining the vanishing point in a given steering range by finding a substantial intersection point for the distortion free, straight line tracked motion trajectories in the given steering range;
wherein the camera is a front facing camera and rotational angles α and
β
of the camera are determined as a function of d1 and d2, where;
α
is the rotation of the camera around an X-axis in a camera coordinate system,β
is the rotation of the camera around a Y-axis in the camera coordinate system,d1 is a distance in the image plane from a principal point of the camera to a right angled projection point on the vanishing line, andd2 is a distance in the image plane along the vanishing line from the projection point to the central vanishing point; and
wherein a rotational angle γ
of the camera is determined by solving for γ
as follows;
A cos γ
+B sin γ
=C
where,
A−
f sin α
sin β
B=f cos α
C−
(cvpX−
axisX) sin α
cos β
where γ
is the rotation angle around the Z-axis of a vehicle coordinate system, f is the focal length of the camera, cvpX is an image row coordinate of the central vanishing point in the image plane, and axisX is an image column location of the principal point.
1 Assignment
0 Petitions
Accused Products
Abstract
A method of dynamically ascertaining the alignment of a vehicular camera. The method involves acquiring a sequence of images provided by the camera whilst the vehicle is in motion. For each range of steering angles, the method (i) selects a plurality of feature points in the images, (ii) tracks a motion trajectory for each selected feature point, and (iii) determines a vanishing point in the image plane based on the tracked motion trajectories. The method determines a vanishing line in the image plane based on a locus of these vanishing points and determines the alignment of the camera based on the position of a central vanishing point (corresponding to the zero degree angle) and the vanishing line.
302 Citations
8 Claims
-
1. A method for dynamically ascertaining the position of a vehicular camera relative to a vehicle to which the camera is attached, the method comprising:
-
(a) establishing a plurality of vehicular steering angle ranges; (b) acquiring a set of image frames in a video stream provided by the camera whilst the vehicle is in motion, the image frames defining an image plane; (c) measuring the steering angle of the vehicle and, for each steering angle range; (i) selecting a plurality of feature points in the image frames; (ii) tracking a motion trajectory of each selected feature point in the set of image frames, and (iii) determining a vanishing point in the image plane for the plurality of tracked motion trajectories; (d) determining a vanishing line in the image plane provided by the camera based on a locus of said vanishing points; (e) determining the position or alignment of the camera based on the position of the vanishing line and the position of a central vanishing point, the central vanishing point being the vanishing point that is determined for the steering angle range that encompasses zero degrees; removing distortion in the motion trajectories caused by use of a wide angle camera lens; linearly fitting the undistorted motion trajectories to generate straight lines; wherein determining a vanishing point comprises determining the vanishing point in a given steering range by finding a substantial intersection point for the distortion free, straight line tracked motion trajectories in the given steering range; wherein the camera is a front facing camera and rotational angles α and
β
of the camera are determined as a function of d1 and d2, where;α
is the rotation of the camera around an X-axis in a camera coordinate system,β
is the rotation of the camera around a Y-axis in the camera coordinate system,d1 is a distance in the image plane from a principal point of the camera to a right angled projection point on the vanishing line, and d2 is a distance in the image plane along the vanishing line from the projection point to the central vanishing point; and wherein a rotational angle γ
of the camera is determined by solving for γ
as follows;
A cos γ
+B sin γ
=C
where,
A−
f sin α
sin β
B=f cos α
C−
(cvpX−
axisX) sin α
cos βwhere γ
is the rotation angle around the Z-axis of a vehicle coordinate system, f is the focal length of the camera, cvpX is an image row coordinate of the central vanishing point in the image plane, and axisX is an image column location of the principal point. - View Dependent Claims (2, 3)
-
-
4. A method for dynamically ascertaining the position of a vehicular camera relative to a vehicle to which the camera is attached, the method comprising:
-
(a) establishing a plurality of vehicular steering angle ranges; (b) acquiring a set of image frames in a video stream provided by the camera whilst the vehicle is in motion, the image frames defining an image plane; (c) measuring the steering angle of the vehicle and, for each steering angle range; (i) selecting a plurality of feature points in the image frames; (ii) tracking a motion trajectory of each selected feature point in the set of image frames, and (iii) determining a vanishing point in the image plane for the plurality of tracked motion trajectories; (d) determining a vanishing line in the image plane provided by the camera based on a locus of said vanishing points; and (e) determining the position or alignment of the camera based on the position of the vanishing line and the position of a central vanishing point, the central vanishing point being the vanishing point that is determined for the steering angle range that encompasses zero degrees; removing distortion in the motion trajectories caused by use of a wide angle camera lens; linearly fitting the undistorted motion trajectories to generate straight lines; wherein determining a vanishing point comprises determining the vanishing point in a given steering range by finding a substantial intersection point for the distortion free, straight line tracked motion trajectories in the given steering range; wherein the camera is a side facing camera and rotational angles α and
β
of the side facing camera are determined as a function of d1 and d2, whereα
is a rotation of the side facing camera around an X-axis of a side facing camera coordinate system,β
is a rotation of the side facing camera around a Y-axis of the side camera coordinate system,d1 is a distance in the image plane from a principal point of the camera to a right angled projection point on the vanishing line, and d2 is a distance in the image plane along the vanishing line from the projection point to the central vanishing point; wherein a rotational angle v is determined by measuring an angle projected by the central vanishing point on a horizontal axis of the image plane; and wherein rotations Rx, Ry and Rz, for the side camera around the X-, Y- and Z-axes, respectively, of the vehicle coordinate system are determined from the rotations α
, β and
γ
around the X-, Y- and Z-axes of the camera coordinate system. - View Dependent Claims (5)
-
-
6. A system for dynamically ascertaining the position of a vehicular camera relative to a vehicle to which the camera is attached, the system comprising a controller which:
-
(a) establishes a plurality of vehicular steering angle ranges; (b) acquires a set of image frames in a video stream provided by the camera whilst the vehicle is in motion, wherein the image frames define an image plane; (c) measures the steering angle of the vehicle and, for each steering angle range; (i) selects a plurality of feature points in the image frames; (ii) tracks a motion trajectory of each selected feature point in the set of image frames, and (iii) determines a vanishing point in the image plane for the plurality of tracked motion trajectories; (d) determines a vanishing line in the image plane provided by the camera based on a locus of said vanishing points; and (e) determines the position or alignment of the camera based on the position of the vanishing line and the position of a central vanishing point, wherein the central vanishing point is the vanishing point that is determined for the steering angle range that encompasses zero degrees; wherein the controller removes distortion in the motion trajectories caused by use of a wide angle camera lens, linearly fits the undistorted motion trajectories to generate straight lines, and determines the vanishing point in a given steering range by finding a substantial intersection point for the distortion free, straight line tracked motion trajectories in the given steering range; wherein the camera is a front facing camera and rotational angles α and
β
of the camera are determined as a function of d1 and d2, whereα
is the rotation of the camera around an X-axis in a camera coordinate system,β
is the rotation of the camera around a Y-axis in the camera coordinate system,d1 is a distance in the image plane from a principal point of the camera to a right angled projection point on the vanishing line, and d2 is a distance in the image plane along the vanishing line from the projection point to the central vanishing point; wherein α and
β
are determined from a table correlating unique values of d1 and d2 with unique values of α and
β
, said table created by setting the α and
β
positions of the front camera to known quantities and recording the resultant d1 and d2 values; andwherein the controller determines a rotational angle γ
of the camera by solving for γ
as follows;
A cos γ
+B sin γ
=C
where,
A−
f sin α
sin β
B=f cos α
C−
(cvpX−
axisX) sin α
cos βwhere γ
is the rotation angle around the Z-axis of a vehicle coordinate system, f is the focal length of the camera, cvpX is an image row coordinate of the central vanishing point in the image plane, and axisX is an image column location of the principal point.
-
-
7. A system for dynamically ascertaining the position of a vehicular camera relative to a vehicle, the system comprising:
-
a camera disposed at a front portion of a vehicle equipped with the system, the camera having a forward field of view, wherein the camera is operable to capture image data; a controller, wherein the controller is operable to establish a plurality of vehicular steering angle ranges of the equipped vehicle; wherein the controller is operable to receive image data captured by the camera and to acquire a set of image frames in a video stream of image data captured by the camera while the equipped vehicle is in motion, and wherein the image frames define an image plane; wherein the controller is operable to measure the steering angle of the equipped vehicle and, for each steering angle range, (i) select a plurality of feature points in the image frames, (ii) track a motion trajectory of each selected feature point in the set of image frames, and (iii) determine a vanishing point in the image plane for the plurality of tracked motion trajectories; wherein the controller determines a vanishing line in the image plane provided by the camera based on a locus of said vanishing points; wherein the controller determines the position or alignment of the camera based on the position of the vanishing line and the position of a central vanishing point, wherein the central vanishing point is the vanishing point that is determined for the steering angle range that encompasses zero degrees; and wherein rotational angles α and
β
of the camera are determined as a function of d1 and d2, where;α
is the rotation of the camera around an X-axis in a camera coordinate system,β
is the rotation of the camera around a Y-axis in the camera coordinate system,d1 is a distance in the image plane from a principal point of the camera to a right angled projection point on the vanishing line, and d2 is a distance in the image plane along the vanishing line from the projection point to the central vanishing point; and wherein the controller determines a rotational angle y of the camera by solving for y as follows;
A cos γ
+B sin γ
=C
where,
A−
f sin α
sin β
B=f cos α
C−
(cvpX−
axisX) sin α
cos βwhere γ
is the rotation angle around the Z-axis of a vehicle coordinate system, f is the focal length of the camera, cvpX is an image row coordinate of the central vanishing point in the image plane, and axisX is an image column location of the principal point. - View Dependent Claims (8)
-
Specification