Method and device for determining a change in the pitch angle of a camera of a vehicle

• US 9,117,272 B2
• Filed: 12/29/2009
• Issued: 08/25/2015
• Est. Priority Date: 12/30/2008
• Status: Active Grant

• Current Assignee: CARIAD SE (Volkswagen AG)
• Original Assignee: Hella KGaA Hueck & Company
• Inventors: Klebanov, Boris, Hoffmann, Ingo
• Primary Examiner(s): Czekaj, Dave
• Assistant Examiner(s): Owens, Tsion B
• Application Number:
  US12/648,676
• Publication Number:
  US 20100165102A1
• Time in Patent Office: 2,065 Days
• US Class Current: 1/1
• CPC Class Codes: B60Q 1/115 by electric means details of the electric drive components, e.g. motors, shafts, ball joints B60Q1/076
• CPC Class Codes: B60Q 2300/3321 Detection of streetlights
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30244 Camera pose
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80

Abstract

A method and device determine the change in pitch angle of a camera of a vehicle. At least three images, each reproducing a detection area in front of the vehicle, are sequentially recorded and corresponding image data are generated. A position of a first reproduction of a stationary object in the first image is determined as a first image position. A position of a second reproduction of the stationary object in the second image is determined as a second image position. On the basis of at least the first image position and the second image position, the course of a straight line is determined. A position of a third reproduction of the stationary object in the third image is determined as a third image position. On the basis of the distance between the third image position and the straight line, the change in the pitch angle is determined.

Claims

1. A method for determining a change in a pitch angle of a camera of a vehicle, comprising:

sequentially recording at least three images, each of which includes a reproduction of a detection area in front of the vehicle, and the at least three images including at least a first image, a second image and a third image;

wherein the camera is mounted in a fixed position at the vehicle and the vehicle performs a driving motion during the sequential recording of the at least three images;

generating image data corresponding to each of the images; and

processing the image data by;

determining a first image position as a position of a first reproduction of a stationary object in the first image;

determining a second image position as a position of a second reproduction of the stationary object in the second image;

determining a course of a straight line based on at least the first image position and the second image position, the course of the straight line representing a current pitch angle of the camera;

determining a third image position as a position of a third reproduction of the stationary object in the third image, the third image position spaced apart from the course of the straight line; and

determining the change in the pitch angle of the camera from the current pitch angle of the camera based on a length of a line that extends between the third image position and a point on the straight line that is an expected position of the third reproduction of the stationary object in the third image were there no change in the pitch angle of the camera, wherein the line extends between the third image position and the point only along a Y-axis of an X- and Y-image point matrix of an image recording sensor of the camera.

2. The method according to claim 1, further comprising:

recording at least a fourth image of the at least three images after recording the second image and before recording the third image; and

determining a fourth image position as a position of a fourth reproduction of the stationary object in the fourth image; and

wherein

determining the course of the straight line includes determining the course of the straight line based on the first image position, the second image position and the fourth image position.

3. The method according to claim 1, further comprising:

determining a course of a second straight line based on at least the first image position, the second image position and the third image position, the course of the second straight line representing a new pitch angle of the camera;

recording a fifth image of the at least three images after recording the third image;

determining a fifth image position as a position of a fifth reproduction of the stationary object in the fifth image; and

determining a further change in the pitch angle of the camera based on a length of a line that extends between the fifth image position and a point on the second straight line that is an expected position of the fifth reproduction of the stationary object in the fifth image were there no further change in the pitch angle of the camera, wherein the line extends between the fifth image position and the point on the second straight line only along the Y-axis of the X- and Y-image point matrix of the image recording sensor of the camera.

4. The method according to claim 2 wherein at least one of the course of the straight line is approximated to the first, second and fourth image positions and the course of the straight line runs through at least one of the first, second and fourth image positions.

5. The method according to claim 1 wherein the course of the straight line is determined with the aid of a displacement vector of image positions between at least two of the at least three images.

6. The method according to claim 5, further comprising:

determining one displacement vector of the image positions each time between at least two image pairs of two sequentially recorded images of the at least three images; and

determining an average displacement vector using each of the displacement vectors;

wherein the average displacement vector is used to determine the course of the straight line, a first image pair comprises the first image and the second image, and a second image pair comprises the second image and a fourth image recorded between the second image and the third image.

7. The method according to claim 1 wherein at least the first image, the second image and the third image include a reproduction of a second stationary object, the method further comprising:

determining image positions of the reproduction of the second stationary object in each of the first image, the second image and the third image;

determining a first course of the image positions of the reproduction of the second stationary object between the first image and the second image;

determining a second course of the image positions of the reproduction of the second stationary object between the second image and the third image;

determining a third course of the first image position and the second image position between the first image and the second image;

determining a fourth course of the second image position and the third image position between the second image and the third image; and

determining at least one displacement vector between the first and second courses and between the third and fourth courses; and

wherein the course of the straight line is determined based on the at least one displacement vector.

8. The method according to claim 1 wherein an X-axis of the X- and Y-image point matrix runs parallel to a Y-axis of a vehicle coordinate system or the X-axis is at a preset angle with respect to the Y-axis of the vehicle coordinate system.

9. The method according to claim 1 wherein the change in the pitch angle Δ

α

is determined according to the following equation;

tan Δ

α

=a/b;

wherein

a is the length of the line between the third image position and the point; and

b is the distance between an image plane and an image-side principal point of a camera optical system of the camera.

10. The method according to claim 1, further comprising:

determining a steer angle of the vehicle between sequential image recording of at least two images of the at least three images; and

correcting an image position in at least one of the at least two images using the steer angle of the vehicle.

11. The method according to claim 1, further comprising:

determining at least one of light distribution to be set for at least one headlight of the vehicle based on the change in the pitch angle of the camera and a correction value for correction of the light distribution of the at least one headlight of the vehicle.

12. The method according to claim 1 wherein the camera is a monocular camera.

13. The method according to claim 1 wherein the stationary object comprises at least one of a light source of a street light, a light source of a traffic signal facility, a light source of a traffic sign illumination, an object illuminated by a light source and reflecting at least part of a light of the light source, a traffic sign and a traffic management system.

14. The method according to claim 13, further comprising:

determining whether the light source has brightness fluctuations as a result of an energy supply by a power grid with a power frequency of 50 Hz or 60 Hz; and

classifying the light source as the stationary object.

19. The method according to claim 1, further comprising:

determining the expected position of the third reproduction of the stationary object in the third image, based on the course of the straight line, were there no change in the pitch angle of the camera from the current pitch angle of the camera.

20. The method according to claim 1 wherein the at least three images are represented on an X- and Y-coordinate plane, the method further comprising:

predicting the change in the pitch angle based on a plurality of previously-determined changes in the pitch angle;

using the predicted change to control headlights of the vehicle so that a light and dark boundary on a driving surface in the at least three images changes position along an Y-axis of the X- and Y-coordinate plane while determining the change in the pitch angle of the camera.

15. A device for determining a change in a pitch angle of a camera of a vehicle, the camera sequentially recording at least three images, each of which includes a reproduction of a detection area in front of the vehicle and generates image data corresponding to each of the images, the at least three images including at least a first image, a second image and a third image, the device comprising:

a processing unit for processing the image data and configured to;

determine a first image position as a position of a first reproduction of a stationary object in the first image wherein the at least three images are sequentially recorded while the vehicle performs a driving motion and the camera is mounted in a fixed position at the vehicle;

determine a second image position as a position of a second reproduction of the stationary object in the second image;

determine a course of a straight line based on at least the first image position and the second image position, the course of the straight line representing a current pitch angle of the camera;

determine a third image position as a position of a third reproduction of the stationary object in the third image, the third image position spaced apart from the course of the straight line; and

determine the change in the pitch angle of the camera from the current pitch angle of the camera based on a length of a line that extends between the third image position and a point on the straight line that is an expected position of the third reproduction of the stationary object in the third image were there no change in the pitch angle of the camera, wherein the line extends between the third image position and the point only along a Y-axis of an X- and Y-image point matrix of an image recording sensor of the camera.

16. The device according to claim 15 wherein the change in the pitch angle Δ

α

is determined according to the following equation;

tan Δ

α

=a/b;

wherein

a is the length of the line between the third image position and the point; and

b is a distance between an image plane and an image-side principal point of a camera optical system of the camera.

17. The device according to claim 15 wherein the at least three images are represented on an X- and Y-coordinate plane;

and wherein the processing unit is configured to;

control headlights of the vehicle so that a light and dark boundary on a driving surface in the at least three images changes position along an Y-axis of the X- and Y-coordinate plane based on the change in the pitch angle.

18. The device according to claim 17 wherein the processing unit is configured to:

predict the change in the pitch angle based on a plurality of previously-determined changes in the pitch angle; and

use the predicted change to control the headlights while determining the change in the pitch angle of the camera.