Method and device for camera calibration

• US 20100295948A1
• Filed: 05/21/2010
• Published: 11/25/2010
• Est. Priority Date: 05/21/2009
• Status: Active Grant

• Current Assignee: Vimicro Corporation
• Original Assignee: Vimicro Corporation
• Inventors: WANG, LEI, XIE, DONGHAI
• Granted Patent: US 8,872,925 B2
• Time in Patent Office: Days
• US Class Current: 348/175
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 7/80 Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration

Abstract

Techniques pertaining to methods and devices for calibrating camera parameters are disclosed. According to one aspect of the present invention, objects on an image are detected and features on the objects, such as the top of a person, are identified. Perspective parallel lines based on the object features are constructed and vanishing points from the converged perspective parallel lines are determined. A vanishing line is defined according to the vanishing points. The intrinsic parameters, such as focal length, and the extrinsic parameters, such as tilting and pan angles, of a camera are calibrated according to the vanishing points, the vanishing line and a reference object with known dimensions, or known camera angles, The calibrated camera parameters are used for accurate photogrammetric measurements and computer vision applications.

Claims

1. A method for camera calibration, the method comprising:

detecting objects in a video sequence;

obtaining feature information from the detected objects;

extracting at least two groups of perspective horizontal parallel lines according to the obtained feature information;

estimating at least two horizontal vanishing points according to the at least two groups of perspective horizontal parallel lines;

defining a horizontal vanishing line by the at least two horizontal vanishing points; and

calculating calibration parameters of a camera according to the vanishing points and/or the vanishing line.

2. The method according to claim 1, wherein the objects comprise moving objects, the feature information comprises moving tracks of the moving objects, and the extracting at least two groups of perspective horizontal parallel lines according to the feature information comprises:

utilizing the moving tracks of the moving objects as the perspective horizontal parallel lines if the moving tracks are essentially straight.

3. The method according to claim 2, wherein the feature information comprises tops and bottoms of the moving objects, and the extracting at least two groups of perspective horizontal parallel lines according to the feature information comprises:

creating straight lines by connecting the tops and straight lines connecting the bottoms of the moving objects at different positions as the perspective horizontal parallel lines.

4. The method according to claim 1, wherein the objects have same height and are located at different positions, the feature information comprises tops and bottoms of the objects, and the extracting at least two groups of perspective horizontal parallel lines according to the feature information comprises:

extracting straight lines by connecting the tops and straight lines connecting the bottoms of the objects at different positions as the perspective horizontal parallel lines.

5. The method according to claim 1, wherein the objects comprise stationary objects, the feature information comprises straight edges or/and lines of the stationary objects, and the extracting at least two groups of perspective horizontal parallel lines according to the feature information comprises:

extracting straight lines by linear-fitting the straight edges or/and the lines of the stationary objects as the perspective parallel lines.

6. The method according to claim 1, wherein the calculating calibration parameters of a camera according to the vanishing points and/or the vanishing line comprises:

tilting the camera to at least one tilt angles within a preset angle range;

calculating calibration parameters of the camera and a visual height of a reference object according to each of the at least one tilt angle, the horizontal vanishing line, the at least two horizontal vanishing points and a known real height of the reference object; and

calculating a difference between the visual height and the known real height of the reference object for each tilt angle; and

using the calibration parameters calculated from the tilt angle, in which the difference between the visual and real heights is the smallest, as final calibrated parameters.

7. The method according to claim 6, wherein the calibration parameters comprise a focal length, the tilt angle, a pan angle, a yaw angle, a height of the camera and a known real height of a reference object, and wherein the calculating calibrations parameters of the camera and a visual height of the reference object comprise:

calculating the focal length of the camera and a vertical vanishing point according to the tilt angle and the horizontal vanishing point;

calculating the yaw angle of the camera according to the horizontal vanishing line;

calculating the pan angle of the camera according to one vanishing point of the horizontal vanishing line;

calculating a height of the camera according to the horizontal vanishing line, the vertical vanishing point and the known real height of the reference object;

determining a matrix between ground points and image points according to the focal length, the yaw angle, the tilt angle, the pan angle and the height of the camera; and

calculating the visual height of the reference object according to the matrix and the real height of the reference object.

8. The method according to claim 1, wherein the calibration parameters comprise a pan angle, and a known real height of a reference object, and wherein the calculating calibration parameters of a camera according to the vanishing points and/or the vanishing line comprises:

calculating the span angle according to the real height of the reference object, the horizontal vanishing line, and a visual height of the reference object.

9. The method according to claim 1, wherein the calibration parameters comprise a focal length, a principal point, a tilt angle, a pan angle, a yaw angle, a height of the camera and a known real height of a reference object, the at least two horizontal vanishing points comprising three orthogonal vanishing points, and wherein the calculating calibration parameters of a camera according to the vanishing points and/or the vanishing line comprises:

calculating the yaw angle according to the horizontal vanishing line;

calculating the focal length according to a vertical vanishing point;

calculating the tilt angle according to the horizontal vanishing line and the focal length;

calculating the principal point according to the three orthogonal vanishing points;

calculating the pan angle according to the principal point, the focal length and one of the at least two vanishing points of the horizontal vanishing line; and

calculating the height of the camera according to the horizontal vanishing line and the known height of the reference object.

10. A device for camera calibration, the device comprising:

an object detecting module configured for detecting objects from a video sequence captured by a camera;

a feature-obtaining module configured for obtaining feature information of the detected objects;

a parallel line extracting module configured for extracting at least two groups of perspective horizontal parallel lines according to the feature information of the detected objects;

a vanishing point and vanishing line estimating module configured for estimating at lest two vanishing points according to the groups of perspective horizontal parallel lines, and defining a vanishing line by the two vanishing points; and

a parameters calculating module configured for calculating calibration parameters of the camera according to the vanishing points and/or the vanishing line.

11. The device according to claim 10, wherein the objects comprise moving objects, the feature information comprises moving tracks of the moving objects, and wherein the parallel line extracting module extracts the moving tracks of the moving objects as the perspective horizontal parallel lines if the moving tracks are straight.

12. The device according to claim 10, wherein the objects comprise moving objects, the feature information comprises tops and bottoms of the moving objects, and wherein the parallel line extracting module extracts straight lines connecting the tops and straight lines connecting the bottoms of the moving objects at different positions as the perspective horizontal parallel lines.

13. The device according to claim 10, wherein the objects have same height and are located at different positions, the feature information comprises tops and bottoms of the objects, and wherein the parallel line extracting module extracts straight lines connecting the tops and straight lines connecting the bottoms of the objects at different positions as the perspective horizontal parallel lines.

14. The device according to claim 10, wherein the objects comprise stationary objects, the feature information comprises straight edges or/and lines of the stationary objects, and wherein the parallel line extracting module extracts straight lines by linear-fitting the straight edges or/and lines of the stationary objects as the perspective parallel lines.