CALIBRATION DEVICE AND CALIBRATION METHOD

• US 20180058882A1
• Filed: 03/03/2017
• Published: 03/01/2018
• Est. Priority Date: 09/01/2016
• Status: Active Grant

• Current Assignee: Mitsubishi Electric Corporation
• Original Assignee: Mitsubishi Electric Corporation
• Inventors: HODOHARA, Norifumi, AOYAGI, Takahisa
• Granted Patent: US 10,641,617 B2
• Time in Patent Office: Days
• CPC Class Codes: B60R 11/04 Mounting of cameras operative during drive; Arrangement of controls thereof relative to the vehicle
• CPC Class Codes: B60R 2300/30 characterised by the type of image processing
• CPC Class Codes: G01C 21/26 specially adapted for navigation in a road network
• CPC Class Codes: G01C 25/005 initial alignment, calibration or starting-up of inertial devices
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/30208 Marker matrix
• CPC Class Codes: G06T 2207/30244 Camera pose
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06T 7/80 Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration

Abstract

A calibration device includes: a coordinate calculator that calculates coordinates of a plurality of rigid body-based points on a vehicle-based coordinate system from measurement values of respective distances between two points constituted by a combination of one of a plurality of vehicle-based points and one of the plurality of rigid body-based points, the plurality of vehicle-based points having known coordinates on the vehicle-based coordinate system and the plurality of rigid body-based points having known coordinates on a rigid body-based coordinate system; and a parameter calculator that calculates a relative attitude and a relative position of an origin as calibration parameters from a correspondence relationship between positions of image coordinates of the plurality of rigid body-based points within a captured image and the coordinates of the plurality of rigid body-based points on the vehicle-based coordinate system.

Claims

1. A calibration device for measuring relative positions and relative attitudes of a rigid body disposed on the ground and a vehicle, the calibration device comprising:

a measured distance input device that receives, based on an input operation performed by an operator, measurement values of respective distances between two points on the basis of a plurality of vehicle-based points having known coordinates on a vehicle-based coordinate system that is defined in accordance with a stopping position of the vehicle and includes an origin and two axes located on the ground, and a plurality of rigid body-based points having known coordinates on a rigid body-based coordinate system that is defined by a disposal location of a marker serving as the rigid body disposed on the ground and includes an origin and two axes located on the ground, the two points being constituted by a combination of one of the plurality of vehicle-based points and one of the plurality of rigid body-based points; and

a calibration parameter calculation device that calculates a relative attitude of the rigid body-based coordinate system and a relative position of the origin thereof on the vehicle-based coordinate system as calibration parameters on the basis of the respective measurement values and a captured image including the marker, which is captured by a camera that is mounted on the vehicle and subjected to calibration,

the calibration parameter calculation device comprising;

a coordinate calculator that calculates coordinates of the plurality of rigid body-based points on the vehicle-based coordinate system from the respective measurement values; and

a parameter calculator that calculates the relative attitude and the relative position of the origin as the calibration parameters from a correspondence relationship between positions of image coordinates of the plurality of rigid body-based points included in the captured image and the coordinates of the plurality of rigid body-based points on the vehicle-based coordinate system.

2. The calibration device according to claim 1, wherein

the plurality of vehicle-based points are constituted by a point V0, a point V1, a point V2, and a point V3 having known ground coordinates on the vehicle-based coordinate system,

the plurality of rigid body-based points are constituted by a point R0 and a point R1 having known ground coordinates on the rigid body-based coordinate system,

the measured distance input device receives four measurement values, namely a measurement value of a distance V0R0 between the point V0 and the point R0, a measurement value of a distance V1R0 between the point V1 and the point R0, a measurement value of a distance V2R1 between the point V2 and the point R1, and a measurement value of a distance V3R1 between the point V3 and the point R1, as the respective measurement values based on the input operation performed by the operator,

the coordinate calculator comprises:

a trilateration calculation unit that calculates coordinates of the point R0 on a V0-based coordinate system having the point V0 as an origin and a straight line linking the point V0 to the point V1 as one axis, and coordinates of the point R1 on a V2-based coordinate system having the point V2 as an origin and a straight line linking the point V2 to the point V3 as one axis, from the four measurement values using a trilateration method; and

a coordinate conversion unit that converts the coordinates of the point R0 on the V0-based coordinate system into coordinates on the vehicle-based coordinate system by means of rotation and parallel movement, and converts the coordinates of the point R1 on the V2-based coordinate system into coordinates on the vehicle-based coordinate system by means of rotation and parallel movement, and

the parameter calculator calculates the relative attitude and the relative position of the origin as the calibration parameters from a correspondence relationship between positions of image coordinates of the point R0 and the point R1 included in the captured image and the coordinates of the point R0 and the point R1 on the vehicle-based coordinate system.

3. The calibration device according to claim 2, wherein the coordinate conversion unit executes coordinate conversion by means of parallel movement alone when the point V2 and the point V3 are set to be identical to the point V0 and the point V1, respectively, and the point V0 and the point V1 are set such that the straight line linking the point V0 to the point V1 is parallel to one of the axes of the vehicle-based coordinate system.

4. The calibration device according to claim 2, wherein,

when the vehicle-based coordinate system and the rigid body-based coordinate system are both three-dimensional coordinate systems having a third axis in a height direction from the ground, and at least one of six points, namely the point V0, the point V1, the point V2, the point V3, the point R0, and the point R1, is a known point not located on the ground,

the trilateration calculation unit determines points obtained by dropping the point V0, the point V1, the point V2, the point V3, the point R0, and the point R1 vertically onto the ground as a point V0_G, a point V1_G, a point V2_G, a point V3_G, a point R0_G, and a point R1_G, respectively, and calculates coordinates of the point R0_G on a V0_G-based coordinate system and coordinates of the point R1_G on a V2_G-based coordinate system using a trilateration method,

the coordinate conversion unit executes coordinate conversion using the coordinates of the point R0_G on the V0_G-based coordinate system and the coordinates of the point R1_G on the V2_G-based coordinate system, calculated by the trilateration calculation unit, and

the parameter calculator calculates the calibration parameters using values obtained as a result of the coordinate conversion executed by the coordinate conversion unit.

5. The calibration device according to claim 2, wherein

the calibration parameter calculation device further comprises an input error determination unit that calculates a distance between the point R0 and the point R1 on the vehicle-based coordinate system from the respective coordinates thereof on the vehicle-based coordinate system, the coordinates having been determined by the coordinate conversion unit, as a calculated value, calculates a distance between the point R0 and the point R1 on the rigid body-based coordinate system from the respective coordinates thereof on the rigid body-based coordinate system as a design value, determines whether or not an absolute value of a difference between the calculated value and the design value is smaller than a preset threshold, and transmits a determination result to the measured distance input device, and

the measured distance input device, having determined from the received determination result that the absolute value equals or exceeds the threshold, causes a display unit to output a message prompting the operator to reset the four measurement values.

6. The calibration device according to claim 2, wherein

the measured distance input device comprises:

an input unit for setting numerical values of the four measurement values; and

a display unit for displaying the captured image captured by the camera on a screen, the captured image including a part of the vehicle and the marker.

7. The calibration device according to claim 6, wherein the measured distance input device causes the display unit to display an icon that is overlaid on the captured image so as to indicate a measurement location corresponding to an item, for which a numerical value has been set on the input unit, among the four measurement values.

8. The calibration device according to claim 7, wherein, when the icon is to be overlaid on the captured image, the measured distance input device detects the position of the marker within the captured image by implementing image processing on the captured image, and overlays the icon on the captured image after correcting a display position of the icon in accordance with a detection result.

9. The calibration device according to claim 1, wherein

the plurality of vehicle-based points are constituted by three groups of points, namely a first group constituted by (a point V0, a point V1, and a point V2), a second group constituted by (a point V3, a point V4, and a point V5), and a third group constituted by (a point V6, a point V7, and a point V8), the three groups having known three-dimensional coordinates on the vehicle-based coordinate system and the three points of each group not existing on a straight line,

the plurality of rigid body-based points are constituted by one group of points, namely (a point R0, a point R1, and a point R2), the group having known three-dimensional coordinates on the rigid body-based coordinate system and the three points of the group not existing on a straight line,

the measured distance input device receives nine measurement values, namely measurement values of respective distances between the point R0 and the three points constituting the first group, measurement values of respective distances between the point R1 and the three points constituting the second group, and measurement values of respective distances between the point R2 and the three points constituting the third group, as the respective measurement values based on the input operation performed by the operator,

the coordinate calculator comprises:

a trilateration calculation unit that calculates coordinates of the point R0 on a three-dimensional V0-based coordinate system having the point V0 as an origin, a straight line linking the point V0 to the point V1 as a first axis, and a second axis on a plane that includes the three points constituting the first group, coordinates of the point R1 on a three-dimensional V3-based coordinate system having the point V3 as an origin, a straight line linking the point V3 to the point V4 as a first axis, and a second axis on a plane that includes the three points constituting the second group, and coordinates of the point R2 on a three-dimensional V6-based coordinate system having the point V6 as an origin, a straight line linking the point V6 to the point V7 as a first axis, and a second axis on a plane that includes the three points constituting the third group, from the nine measurement values using a trilateration method; and

a coordinate conversion unit that converts the coordinates of the point R0 on the V0-based coordinate system into coordinates on the vehicle-based coordinate system by means of rotation and parallel movement, converts the coordinates of the point R1 on the V3-based coordinate system into coordinates on the vehicle-based coordinate system by means of rotation and parallel movement, and converts the coordinates of the point R2 on the V6-based coordinate system into coordinates on the vehicle-based coordinate system by means of rotation and parallel movement, and

the parameter calculator calculates the relative attitude and the relative position of the origin as the calibration parameters from a correspondence relationship between positions of image coordinates of the point R0, the point R1, and the point R2 included in the captured image and the coordinates of the point R0, the point R1, and the point R2 on the vehicle-based coordinate system.

10. A calibration method for measuring relative positions and relative attitudes of a rigid body disposed on the ground and a vehicle, the calibration method comprising:

a first step in which respective coordinates of a plurality of rigid body-based points having known coordinates on a rigid body-based coordinate system are stored in advance in a storage unit, the rigid body-based coordinate system being defined by a disposal position of a marker serving as the rigid body disposed on the ground and including an origin and two axes located on the ground;

a second step in which measurement values of respective distances between two points are received, via a measured distance input device based on an input operation performed by an operator, on the basis of the plurality of rigid body-based points and a plurality of vehicle-based points having known coordinates on a vehicle-based coordinate system that is defined in accordance with a stopping position of the vehicle and includes an origin and two axes located on the ground, the two points being constituted by a combination of one of the plurality of vehicle-based points and one of the plurality of rigid body-based points;

a third step in which a captured image including the marker, which is captured by a camera that is mounted on the vehicle and subjected to calibration, is obtained;

a fourth step in which coordinates of the plurality of rigid body-based points on the vehicle-based coordinate system are calculated from the respective measurement values; and

a fifth step in which a relative attitude of the rigid body-based coordinate system and a relative position of the origin thereof on the vehicle-based coordinate system are calculated as calibration parameters from a correspondence relationship between positions of image coordinates of the plurality of rigid body-based points included in the captured image and the coordinates of the plurality of rigid body-based points on the vehicle-based coordinate system, the coordinates having been stored in the storage unit.

11. The calibration method according to claim 10, wherein

the plurality of vehicle-based points are constituted by a point V0, a point V1, a point V2, and a point V3 having known ground coordinates on the vehicle-based coordinate system,

the plurality of rigid body-based points are constituted by a point R0 and a point R1 having known ground coordinates on the rigid body-based coordinate system,

in the second step, four measurement values, namely a measurement value of a distance V0R0 between the point V0 and the point R0, a measurement value of a distance V1R0 between the point V1 and the point R0, a measurement value of a distance V2R1 between the point V2 and the point R1, and a measurement value of a distance V3R1 between the point V3 and the point R1, are received as the respective measurement values based on the input operation performed by the operator,

the fourth step comprises:

a trilateration calculation step in which coordinates of the point R0 on a V0-based coordinate system having the point V0 as an origin and a straight line linking the point V0 to the point V1 as one axis, and coordinates of the point R1 on a V2-based coordinate system having the point V2 as an origin and a straight line linking the point V2 to the point V3 as one axis, are calculated from the four measurement values using a trilateration method; and

a coordinate conversion step in which the coordinates of the point R0 on the V0-based coordinate system are converted into coordinates on the vehicle-based coordinate system by means of rotation and parallel movement and the coordinates of the point R1 on the V2-based coordinate system are converted into coordinates on the vehicle-based coordinate system by means of rotation and parallel movement, and

in the fifth step, the relative attitude and the relative position of the origin are calculated as the calibration parameters from a correspondence relationship between positions of image coordinates of the point R0 and the point R1 included in the captured image and the coordinates of the point R0 and the point R1 on the vehicle-based coordinate system.

12. The calibration method according to claim 11, wherein, in the coordinate conversion step, coordinate conversion is executed by means of parallel movement alone when the point V2 and the point V3 are set to be identical to the point V0 and the point V1, respectively, and the point V0 and the point V1 are set such that the straight line linking the point V0 to the point V1 is parallel to one of the axes of the vehicle-based coordinate system.

13. The calibration method according to claim 11, wherein

when the vehicle-based coordinate system and the rigid body-based coordinate system are both three-dimensional coordinate systems having a third axis in a height direction from the ground, and at least one of six points, namely the point V0, the point V1, the point V2, the point V3, the point R0, and the point R1, is a known point not located on the ground,

in the trilateration calculation step, points obtained by dropping the point V0, the point V1, the point V2, the point V3, the point R0, and the point R1 vertically onto the ground are calculated as a point V0_G, a point V1_G, a point V2_G, a point V3_G, a point R0_G, and a point R1_G, respectively, whereupon coordinates of the point R0_G on a V0_G-based coordinate system and coordinates of the point R1_G on a V2_G-based coordinate system are calculated using a trilateration method,

in the coordinate conversion step, coordinate conversion is executed using the coordinates of the point R0_G on the V0_G-based coordinate system and the coordinates of the point R1_G on the V2_G-based coordinate system, the coordinates having been calculated in the trilateration calculation step, and

in the fifth step, the calibration parameters are calculated using values obtained as a result of the coordinate conversion executed in the coordinate conversion step.

14. The calibration method according to claim 11, wherein the fourth step further comprises an input error determination step in which a distance between the point R0 and the point R1 on the vehicle-based coordinate system is calculated from the respective coordinates thereof on the vehicle-based coordinate system, the coordinates having been determined in the coordinate conversion step, as a calculated value, a distance between the point R0 and the point R1 on the rigid body-based coordinate system is calculated from the respective coordinates thereof on the rigid body-based coordinate system as a design value, a determination is made as to whether or not an absolute value of a difference between the calculated value and the design value is smaller than a preset threshold, and when the absolute value is determined to equal or exceed the threshold, the measured distance input device is caused to display a message prompting the operator to reset the four measurement values.

15. The calibration method according to claim 11, wherein, in the second step, a display unit of the measured distance input device is caused to display the captured image captured by the camera, the captured image including a part of the vehicle and the marker.

16. The calibration method according to claim 15, wherein, in the second step, the display unit is caused to display an icon that is overlaid on the captured image so as to indicate a measurement location corresponding to an item, for which a numerical value has been set, among the four measurement values.

17. The calibration method according to claim 16, wherein when the icon is to be overlaid on the captured image in the second step, the position of the marker within the captured image is detected by implementing image processing on the captured image, and the icon is overlaid on the captured image after correcting a display position of the icon in accordance with a detection result.

18. The calibration method according to claim 10, wherein

the plurality of vehicle-based points are constituted by three groups of points, namely a first group constituted by (a point V0, a point V1, and a point V2), a second group constituted by (a point V3, a point V4, and a point V5), and a third group constituted by (a point V6, a point V7, and a point V8), the three groups having known three-dimensional coordinates on the vehicle-based coordinate system and the three points of each group not existing on a straight line,

the plurality of rigid body-based points are constituted by one group of points, namely (a point R0, a point R1, and a point R2), the group having known three-dimensional coordinates on the rigid body-based coordinate system and the three points of the group not existing on a straight line,

in the second step, nine measurement values, namely measurement values of respective distances between the point R0 and the three points constituting the first group, measurement values of respective distances between the point R1 and the three points constituting the second group, and measurement values of respective distances between the point R2 and the three points constituting the third group, are received as the respective measurement values based on the input operation performed by the operator,

the fourth step comprises:

a trilateration calculation step in which coordinates of the point R0 on a three-dimensional V0-based coordinate system having the point V0 as an origin, a straight line linking the point V0 to the point V1 as a first axis, and a second axis on a plane that includes the three points constituting the first group, coordinates of the point R1 on a three-dimensional V3-based coordinate system having the point V3 as an origin, a straight line linking the point V3 to the point V4 as a first axis, and a second axis on a plane that includes the three points constituting the second group, and coordinates of the point R2 on a three-dimensional V6-based coordinate system having the point V6 as an origin, a straight line linking the point V6 to the point V7 as a first axis, and a second axis on a plane that includes the three points constituting the third group, are calculated from the nine measurement values using a trilateration method; and

a coordinate conversion step in which the coordinates of the point R0 on the V0-based coordinate system are converted into coordinates on the vehicle-based coordinate system by means of rotation and parallel movement, the coordinates of the point R1 on the V3-based coordinate system are converted into coordinates on the vehicle-based coordinate system by means of rotation and parallel movement, and the coordinates of the point R2 on the V6-based coordinate system are converted into coordinates on the vehicle-based coordinate system by means of rotation and parallel movement, and

in the fifth step, the relative attitude and the relative position of the origin are calculated as the calibration parameters from a correspondence relationship between positions of image coordinates of the point R0, the point R1, and the point R2 included in the captured image and the coordinates of the point R0, the point R1, and the point R2 on the vehicle-based coordinate system.