METHOD AND DEVICE FOR ONLINE CALIBRATION OF VEHICLE CAMERAS

• US 20150036885A1
• Filed: 01/17/2013
• Published: 02/05/2015
• Est. Priority Date: 01/20/2012
• Status: Active Grant

• Current Assignee: ESG Mobility GmbH (ESG Elektroniksystem-und Logistik-GmbH)
• Original Assignee: ESG Elektroniksystem-und Logistik-GmbH
• Inventors: Platonov, Juri, Pflug, Christopher, Gebauer, Thomas, Kaczmarczyk, Pawel
• Granted Patent: US 9,262,828 B2
• Time in Patent Office: Days
• US Class Current: 382/104
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30244 Camera pose
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06T 7/80 Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
• CPC Class Codes: H04N 17/002 for television cameras

Abstract

A method for determining the orientation of a video camera attached to a vehicle relative to the vehicle coordinate system, comprising:

• • Measuring an incremental vehicle motion in the coordinate system of the video camera based on the optical flow of the video stream of the video camera;
  • representing the incremental vehicle motion by a motion consisting of a linear motion component and a rotational motion, wherein the linear motion component is represented by a directional vector and the rotational motion is represented by a rotation axis and a rotational angle;
  • classifying the measured incremental vehicle motion as a linear motion or as a rotational motion based on the determined rotational angle;
  • measuring at least one incremental motion which has been classified as a linear motion;
  • measuring at least one incremental motion which has been classified as a rotational motion; and
  • using the directional vector of the at least one measured linear motion for estimating the longitudinal axis of the vehicle;
  • using the rotational vector of the at least one measured rotational motion for estimating the normal to the vehicle plane so that the orientation of the camera relative to the vehicle coordinate system follows from the vehicle longitudinal axis and the normal to the vehicle plane, which have been determined in the coordinate system of the camera, respectively.

Claims

1. A method for determining the orientation of a video camera attached to a vehicle relative to the vehicle coordinate system, comprising:

Measuring an incremental vehicle motion in the coordinate system of the video camera based on the optical flow of the video stream of the video camera;

representing the incremental vehicle motion by a motion consisting of a linear motion component and a rotational motion, wherein the linear motion component is represented by a directional vector and the rotational motion is represented by a rotation axis and a rotational angle;

classifying the measured incremental vehicle motion as a linear motion or as a rotational motion based on the determined rotational angle;

measuring at least one incremental motion which has been classified as a linear motion;

measuring at least one incremental motion which has been classified as a rotational motion; and

using the directional vector of the at least one measured linear motion for estimating the longitudinal axis of the vehicle;

using the rotational vector of the at least one measured rotational motion for estimating the normal to the vehicle plane so that the orientation of the camera relative to the vehicle coordinate system follows from the vehicle longitudinal axis and the normal to the vehicle plane, which have been determined in the coordinate system of the camera, respectively.

2. The method of claim 1, further comprising:

Measuring a plurality of incremental vehicle motions;

Using the plurality of estimations of the vehicle longitudinal axis and the rotational axis for determining an improved estimation for the vehicle longitudinal axis and the rotational axis.

3. The method of claim 1, further comprising:

If the rotational angle of the incremental vehicle motion is smaller than a certain threshold, classifying the motion as a linear motion;

if the rotational angle of the incremental motion is larger than a certain threshold, classifying the motion as a rotational motion.

4. The method of claim 1, further comprising:

Using the normal to the plane of the linear motion or the linear motion component as a further estimation for the normal to the vehicle longitudinal axis in addition to the rotational axis;

determining an estimation for the normal to the vehicle plane based on a combination of the normal to the determined linear motion or motion component and the rotational axis of the rotational motion.

5. The method of claim 4, wherein

the combination is determined based on the requirement that the normal to the plane of the linear motion as determined from the measurement or from the linear motion component and the rotational vector should be in parallel.

6. The method of claim 1, wherein

the estimated value for the longitudinal axis of the vehicle and the normal to the vehicle is determined iteratively by repeated measurement of the incremental vehicle motion.

7. The method of claim 6, comprising:

Determining an updated value of the normal to the vehicle by measurement i as

9. The apparatus of claim 8, further comprising:

One or more modules for performing a method according to claim 2.

10. A computer program comprising:

Computer program code which, when being executed on the computer, enables it to perform a method according to claim 1.

8. An apparatus for determining the orientation of a video camera attached to a vehicle relative to a vehicle coordinate system comprising:

A module for measuring an incremental vehicle motion in the coordinate system of the video camera based on the optical flow of the video stream of the video camera;

A module for representing the incremental vehicle motion by a motion consisting of a linear motion component and a rotational motion, wherein the linear motion component is represented by a directional vector and the rotational motion is represented by a rotation axis and a rotational angle;

A module for classifying the measured incremental vehicle motion as a linear motion or as a rotational motion based on the determined rotational angle;

A module for measuring at least one incremental motion which has been classified as a linear motion;

A module for measuring at least one incremental motion which has been classified as a rotational motion; and

A module for using the directional vector of the at least one measured linear motion for estimating the longitudinal axis of the vehicle;

A module for using the rotational vector of the at least one measured rotational motion for estimating the normal to the vehicle plane so that the orientation of the camera relative to the vehicle coordinate system follows from the vehicle longitudinal axis and the normal to the vehicle plane, which have been determined in the coordinate system of the camera, respectively.