METHOD OF CALIBRATING A CAMERA AND A LASER SCANNER

• US 20180315214A1
• Filed: 04/11/2018
• Published: 11/01/2018
• Est. Priority Date: 04/27/2017
• Status: Active Grant

• Current Assignee: Sick AG (Sick)
• Original Assignee: Sick AG (Sick)
• Inventors: BEINSCHOB, Patric
• Granted Patent: US 10,643,349 B2
• Time in Patent Office: Days
• CPC Class Codes: G01S 17/42 Simultaneous measurement of distance and other co-ordinates indirect measurement G01S17/46
• CPC Class Codes: G01S 17/86 Combinations of lidar systems with systems other than lidar, radar or sonar, e.g. with direction finders
• CPC Class Codes: G01S 17/89 for mapping or imaging
• CPC Class Codes: G01S 7/4808 Evaluating distance, position or velocity data
• CPC Class Codes: G01S 7/4972 Alignment of sensor
• CPC Class Codes: G06T 2207/30244 Camera pose
• CPC Class Codes: G06T 7/74 involving reference images or patches
• CPC Class Codes: G06T 7/80 Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration

Abstract

A method of calibrating a camera and a laser scanner for their sensor fusion is provided whereby the camera records a camera image of a scene and the laser scanner at least partly scans the scene with a scanning beam and records a remission image from the respective angular position of the scanning beam and from the intensity determined at the angular position of the scanning beam remitted from the scene, wherein an offset and/or a relative orientation between the camera and the laser scanner is determined from a comparison of the camera image and the remission image. Distances with respect to the remission image are here also determined by the laser scanner and distances with respect to the camera image are reconstructed; and in that a correspondence search of corresponding features in the camera image and in the remission image is carried out in three-dimensional space for the comparison.

Claims

1. A method of calibrating a camera and a laser scanner for their sensor fusion, the method comprising the steps of:

recording a camera image of a scene with the camera;

at least partly scanning the scene with a scanning beam of the laser scanner;

recording a remission image from the respective angular position of the scanning beam and from the intensity determined at the angular position of the scanning beam remitted from the scene,

determining at least one of an offset and a relative orientation between the camera and the laser scanner from a comparison of the camera image and the remission image,

determining distances with respect to the remission image by the laser scanner;

reconstructing distances with respect to the camera image; and

carrying out a correspondence search of corresponding features in the camera image and in the remission image in three-dimensional space for the comparison.

2. The method in accordance with claim 1,

wherein the distances in the camera image are reconstructed with the aid of the remission image.

3. The method in accordance with claim 1,

wherein first features are determined in the camera image and are projected onto predefined distances.

4. The method in accordance with claim 3,

wherein the first features are projected onto a unit sphere.

5. The method in accordance with claim 1,

wherein second features are determined in the remission image and are projected onto predefine distances.

6. The method in accordance with claim 5,

wherein the second features are projected onto a unit sphere.

7. The method in accordance with claim 5,

wherein first features are determined in the camera image and are projected onto predefined distances and wherein a first correspondence search is carried out in three-dimensional space between the first features and the second features.

8. The method in accordance with claim 7,

wherein the distance of the associated second feature measured by the laser scanner is assigned to the first features of a correspondence pair to acquire corrected first features.

9. The method in accordance with claim 8,

wherein a second correspondence search is carried out in three-dimensional space between the corrected first features and the second features.

10. The method in accordance with claim 1,

wherein only one-to-one correspondence pairs are permitted and/or no further features may be present in the surroundings of a correspondence pair.

11. The method in accordance with claim 10,

wherein the distance of the associated second feature measured by the laser scanner is assigned to the first features of a correspondence pair to acquire corrected first features.

12. The method in accordance with claim 11,

wherein a second correspondence search is carried out in three-dimensional space between the corrected first features and the second features.

13. The method in accordance with claim 1,

wherein the corresponding features are placed over one another as best as possible by an optimization of at least one of the offset and the orientation.

14. The method in accordance with claim 13,

wherein the corresponding features are placed over one another as best as possible by a non-linear optimization.

15. The method in accordance with claim 13,

wherein a quality of the optimization is calculated.

16. The method in accordance with claim 1,

wherein the laser scanner is a 3D laser scanner.

17. The method in accordance with claim 1,

wherein the scene for the calibration is left as it is.

18. The method in accordance with claim 17,

wherein no calibration object is arranged in the scene.

19. The method in accordance with claim 1,

wherein at least one object having a desired remission behavior is arranged in the scene for the calibration to increase the contrast.

20. A system composed of a camera and a laser scanner, and an evaluation unit that is configured to calibrate the camera and the laser scanner by using a method of calibrating the camera and the laser scanner for their sensor fusion, the method comprising the steps of:

recording a camera image of a scene with the camera;

at least partly scanning the scene with a scanning beam of the laser scanner;

recording a remission image from the respective angular position of the scanning beam and from the intensity determined at the angular position of the scanning beam remitted from the scene,

determining at least one of an offset and a relative orientation between the camera and the laser scanner from a comparison of the camera image and the remission image,

determining distances with respect to the remission image by the laser scanner;

reconstructing distances with respect to the camera image; and

carrying out a correspondence search of corresponding features in the camera image and in the remission image in three-dimensional space for the comparison.