Apparatus and method with composite sensor calibration

• US 8,639,021 B2
• Filed: 12/07/2010
• Issued: 01/28/2014
• Est. Priority Date: 01/27/2010
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd. (Samsung Group)
• Original Assignee: Samsung Electronics Co. Ltd. (Samsung Group)
• Inventors: Kim, Dong-Jo, Park, Dong-Ryeol
• Primary Examiner(s): Koziol, Stephen R
• Assistant Examiner(s): CHOI, TIMOTHY WING HO
• Application Number:
  US12/926,744
• Publication Number:
  US 20110182476A1
• Time in Patent Office: 1,148 Days
• Field of Search: None
• US Class Current: 382/154
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 7/85 Stereo camera calibration

Abstract

An apparatus and method capable of calculating a coordinate transformation parameter without having to utilize a rig are provided. The apparatus and method extract a first feature point based on a plane of first data, project the first feature point onto second data and then extract a second feature point from a part of the second data onto which the first feature point is projected. Then, calibration is performed based on the extracted feature points. Therefore, it is possible to perform calibration immediately as necessary without having to utilize a separate device such as a rig.

Claims

1. An apparatus, including one or more processing devices, the apparatus comprising:

a plane extractor to extract, using at least one of the one or more processing devices, at least one plane from first data obtained by a first sensor;

a first feature point extractor to extract, using at least one of the one or more processing devices, a first feature point from the first data based on the at least one plane;

a second feature point extractor to project, using at least one of the one or more processing devices, the first feature point onto second data obtained by a second sensor and to extract a second feature point corresponding to the first feature point from a part of the second data onto which the first feature point is projected; and

a transformation parameter calculator to calculate, using at least one of the one or more processing devices, a coordinate transformation parameter between the first sensor and the second sensor using a difference between coordinate values of the first and second feature points.

2. The apparatus of claim 1, wherein the first sensor is a 3-dimensional (3D) distance sensor and the first data is 3D range data obtained from the 3D distance sensor.

3. The apparatus of claim 1, wherein the second sensor is an image sensor and the second data is image data obtained from the image sensor.

4. The apparatus of claim 1, wherein the first feature point extractor extracts a part at which at least three planes cross each other as the first feature point.

5. The apparatus of claim 1, wherein the transformation parameter calculator registers the first feature point and the second feature point when the difference between coordinate values of the first and second feature points is determined not to meet a first threshold, and calculates, when the number of the registered first and second feature points is determined to meet a second threshold, the coordinate transformation parameter using the registered first and second feature points.

6. An apparatus, including one or more processing devices, the apparatus comprising:

a plane extractor to extract, using at least one or more processing devices, at least one plane from first data obtained by a first sensor;

a first feature point extractor to extract, using at least one of the one or more processing devices, a first feature point from the first data based on the at least one plane;

a first line extractor to extract, using at least one of the one or more processing devices, a first line from the first data based on the at least one plane;

a second line extractor to project, using at least one of the one or more processing devices, the first line onto second data obtained by a second sensor and to extract a second line corresponding to the first line from a part of the second data onto which the first line is projected;

a second feature point extractor to extract, using at least one of the one or more processing devices, a second feature point corresponding to the first feature point based on the second line; and

a transformation parameter calculator to calculate, using at least one of the one or more processing devices, a coordinate transformation parameter between the first sensor and the second sensor using a difference between coordinate values of the first and second feature points.

7. The apparatus of claim 6, wherein the first sensor is a 3-dimensional (3D) distance sensor and the first data is 3D range data obtained from the 3D distance sensor.

8. The apparatus of claim 6, wherein the second sensor is an image sensor and the second data is image data obtained from the image sensor.

9. The apparatus of claim 6, wherein the first feature point extractor extracts a part at which at least three planes cross each other as the first feature point.

10. The apparatus of claim 6, wherein the first line extractor extracts a part at which at least two planes cross each other as the first feature point.

11. The apparatus of claim 6, wherein the transformation parameter calculator registers the first feature point and the second feature point when the difference between coordinate values of the first and second feature points is determined to not meet a first threshold, and calculates, when the number of the registered first and second feature points is determined to meet a second threshold, the coordinate transformation parameter using the registered first and second feature points.

12. A method, the method comprising:

extracting at least one plane from first data obtained by a first sensor;

extracting a first feature point from the first data based on the at least one plane;

projecting the first feature point to second data obtained by a second sensor, and extracting a second feature point corresponding to the first feature point from a part of the second data to which the first feature point is projected; and

calculating a coordinate transformation parameter between the first sensor and the second sensor using a difference between coordinate values of the first and second feature points.

13. The calibration method of claim 12, wherein the extracting of the first feature point comprises extracting a part at which at least three planes cross each other as the first feature point.

19. The method of claim 12, wherein the calculating of the coordinate transformation parameter comprises:

registering, when the difference between coordinate values of the first and second feature points is determined to not meet a first threshold, the first feature point and the second feature point; and

calculating, when the number of the registered first and second feature points is determined to meet a second threshold, the coordinate transformation parameter using the registered first feature point and the second feature point.

14. A calibration method of a composite sensor, comprising:

extracting at least one plane from first data obtained by a first sensor;

extracting a first feature point from the first data based on the at least one plane;

projecting the first feature point to second data obtained by a second sensor, and extracting a second feature point corresponding to the first feature point from a part of the second data to which the first feature point is projected; and

calculating a coordinate transformation parameter between the first sensor and the second sensor using the first feature point and the second feature point,

wherein the calculating of the coordinate transformation parameter comprises;

registering, when a difference between coordinate values of the first and second feature points is equal to or less than a first threshold value, the first feature point and the second feature point; and

calculating, when the number of the registered first and second feature points is equal to or more than a second threshold value, the coordinate transformation parameter using the registered first feature point and the second feature point.

15. A method, the method comprising:

extracting at least one plane from first data obtained by a first sensor;

extracting a first feature point from the first data based on the at least one plane;

extracting a first line from the first data based on the at least one plane;

projecting the first line to second data obtained by a second sensor, and extracting a second line corresponding to the first line from a part of the second data to which the first line is projected;

extracting a second feature point corresponding to the first feature point based on the second line; and

calculating a coordinate transformation parameter between the first sensor and the second sensor using a difference between coordinate values of the first and second feature points.

16. The method of claim 15, wherein the extracting of the first feature point comprises extracting a part at which at least three planes cross each other as the first feature point.

17. The method of claim 15, wherein the extracting of the first line comprises extracting a line along which at least two planes cross each other as the first line.

18. The method of claim 15, wherein the calculating of the coordinate transformation parameter comprises:

registering the first feature point and the second feature point when the difference between coordinate values of the first feature points is equal to or less than a first threshold value; and

calculating the coordinate transformation parameter using the registered first and second feature points when the number of the registered first and second feature points is equal to or more than a second threshold value.

20. The method of claim 15, wherein the calculating of the coordinate transformation parameter comprises:

registering the first feature point and the second feature point when the difference between coordinate values of the first and second feature points is determined to not meet a first threshold; and

calculating the coordinate transformation parameter using the registered first and second feature points when the number of the registered first and second feature points is determined to meet a second threshold.