Method and apparatus for calibrating a camera system of a motor vehicle

• US 10,424,081 B2
• Filed: 11/14/2016
• Issued: 09/24/2019
• Est. Priority Date: 05/14/2014
• Status: Active Grant

• Current Assignee: Volkswagen AG
• Original Assignee: Volkswagen AG
• Inventors: Jaehnisch, Marco, Siebeneicher, Manuel, Fietz, Alexander, Haja, Andreas, Liebehenschel, Daniel, Soehnel, Christoph
• Primary Examiner(s): Jiang, Zaihan
• Application Number:
  US15/350,406
• Publication Number:
  US 20170061623A1
• Time in Patent Office: 1,044 Days
• Field of Search: 348148
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30244 Camera pose
• CPC Class Codes: G06T 2207/30248 Vehicle exterior or interior
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06T 7/80 Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration

Abstract

In a method and an apparatus for calibrating a camera system of a motor vehicle, the calibration parameters comprising the rotation angle, pitch angle, yaw angle and roll angle as well as the height of the camera above the road, the rotation angle is determined from the ascertainment of the vanishing point from a first optical flow between a first and a second successive camera image, and the height of the camera is determined from a second optical flow between a first and a second, successive camera image. To determine the first optical flow, a regular grid is placed over the first camera image, correspondences of the regular grid are searched for in the second camera image, and the first optical flow is determined from the movement of the grid over the camera images.

Claims

1. A method for calibrating a camera system of a motor vehicle, the calibration parameters comprising a rotation angle, a pitch angle, a yaw angle, a roll angle, and a height of a camera above a road, the method comprising:

determining the rotation angle from an ascertainment of a vanishing point from a first optical flow between a first and a second camera image;

determining the height of the camera from a second optical flow between a first and a second camera image;

placing a regular grid over the first camera image;

searching for correspondences of the regular grid in the second camera image; and

determining the first optical flow from a movement of the regular grid over the first and second camera images,

wherein epipoles in both camera images are calculated based on the ascertained first optical flow, and wherein the epipoles are used to estimate the vanishing point.

2. The method according to claim 1, wherein distances between the grid points are parameterizable.

3. The method according to claim 1, wherein, to determine the pitch angle and the yaw angle, the vanishing point is determined from the two epipoles of the two camera images when a distance between the epipoles is situated within a first predefined interval.

4. The method according to claim 3, wherein the ascertained vanishing points are stored for a predefined time interval, and wherein a time-stable vanishing point is ascertained from the quantity of the ascertained vanishing points for determining the pitch angle and the yaw angle.

5. The method according to claim 1, wherein the two epipoles of two camera images are used to determine the roll angle when the distance between the epipoles is situated within a second, predefined interval, and wherein the roll angle is estimated from a position of the epipoles in the particular camera image.

6. The method according to claim 5, wherein the ascertained roll angles are collected over time and divided into predefined groups as a function of the yaw rate of the motor vehicle.

7. The method according to claim 6, wherein a mean group roll angle is formed for each group of stored roll angles when the number of measurements in each group reaches a minimum quantity, the final roll angle being formed from the mean group roll angles of the groups.

8. A method for calibrating a camera system of a motor vehicle, the calibration parameters comprising a rotation angle, a pitch angle, a yaw angle, a roll angle, and a height of a camera above a road, the method comprising:

determining the rotation angle from an ascertainment of a vanishing point from a first optical flow between a first and a second camera image;

determining the height of the camera from a second optical flow between a first and a second camera image;

placing a regular grid over the first camera image;

searching for correspondences of the regular grid in the second camera image; and

determining the first optical flow from a movement of the regular grid over the first and second camera images,

wherein, to determine the second optical flow in a section of a first image of the road, a rectangle ascertained in the second image is placed in front of the vehicle in a top view, wherein the second optical flow is formed by a movement of the rectangle in the two images, and wherein the height of the camera is calculated from the movement of the rectangle.

9. The method according to claim 8, wherein a time histogram of the measured heights of the camera is generated for a large number of first and second camera images, and wherein a height estimate is derived from the histogram upon reaching a minimum number of measurements.

10. An apparatus for calibrating a camera system of a motor vehicle with calibration parameters comprising a rotation angle, a pitch angle, a yaw angle, a roll angle and a height of a camera above a road, the apparatus comprising:

a module for determining a first optical flow between a first and a second successive camera image;

a module for determining vanishing points from the first optical flow;

a module for determining the rotation angle from the ascertained vanishing points;

a module for determining a second optical flow between a first and a second successive camera image; and

a module for determining the height of the camera from a second optical flow,

wherein, to determine the first optical flow, a regular grid is placed over the first camera image, correspondences of the regular grid are searched for in the second camera image, and the first optical flow is determined from the movement of the grid over the camera images, and

wherein the module for determining the vanishing point calculates epipoles in the two camera images on the basis of the ascertained first optical flow and uses the epipoles to estimate the vanishing point.

11. An apparatus for calibrating a camera system of a motor vehicle with calibration parameters comprising a rotation angle, a pitch angle, a yaw angle, a roll angle and a height of a camera above a road, the apparatus comprising:

a module for determining a first optical flow between a first and a second successive camera image;

a module for determining vanishing points from the first optical flow;

a module for determining the rotation angle from the ascertained vanishing points;

a module for determining a second optical flow between a first and a second successive camera image; and

a module for determining the height of the camera from a second optical flow,

wherein, to determine the first optical flow, a regular grid is placed over the first camera image, correspondences of the regular grid are searched for in the second camera image, and the first optical flow is determined from the movement of the grid over the camera images, and

wherein to determine the second optical flow in a section of a first image of the road, a rectangle ascertained in the second image is placed in front of the vehicle in a top view, the second optical flow being formed by the movement of the rectangle in the two images, and the height of the camera being calculated from the movement of the rectangle.

12. The apparatus as according to claim 11, wherein the module for determining the camera height generates a time histogram of the ascertained heights of the camera for a large number of first and second camera images and derives a height estimate from the histogram upon reaching a minimum number of measurements.