Robot system and method for calibration

• US 9,457,470 B2
• Filed: 04/05/2013
• Issued: 10/04/2016
• Est. Priority Date: 04/05/2013
• Status: Active Grant

• Current Assignee: ABB Schweiz AG (ABB Limited)
• Original Assignee: ABB Technology Limited (ABB Limited)
• Inventors: Lundberg, Ivan
• Primary Examiner(s): LIN, ABBY YEE
• Application Number:
  US14/781,774
• Publication Number:
  US 20160039094A1
• Time in Patent Office: 1,278 Days
• US Class Current: 1/1
• CPC Class Codes: B25J 9/1607 Calculation of inertia, jacobian matrixes and inverses
• CPC Class Codes: B25J 9/1692 Calibration of manipulator
• CPC Class Codes: B25J 9/1697 Vision controlled systems
• CPC Class Codes: G05B 2219/39008 Fixed camera detects reference pattern held by end effector
• CPC Class Codes: Y10S 901/09 Closed loop, sensor feedback controls arm movement

Abstract

A method and a system for calibrating a first coordinate system R_f of a robot unit with a second coordinate system C_f of an object identification unit, wherein the robot unit includes a robot arm with an end effector and the object identification unit includes a camera unit. The calibration is performed using a calibration marker on the end effector. The method determines the intrinsic and the extrinsic parameters of the camera unit in two steps, a first step where the intrinsic parameters and a rotational part of the extrinsic parameters are determined, and a second step where a translational part of the extrinsic parameters are determined.

Claims

1. A method for calibrating a first coordinate system R_f of a robot unit with a second coordinate system C_f of an object identification unit, wherein the robot unit comprises a robot arm with an end effector and the object identification unit comprises a camera unit, the end effector further comprises a calibration marker, the method comprises

moving the end effector to a plurality of first target points wherein the first target points are chosen to include movement of the end effector in all three axes of the first coordinate system R_f of the robot unit, and wherein the end effector maintains a same orientation, while generating position measurements in the first coordinate system R_f and position measurements in an image plane coordinate system IP_f for the calibration marker with the object identification unit for each first target points;

calculating intrinsic parameters C_int of the camera unit and a rotational part of the second coordinate system C_f of the object identification unit based on the measurements in R_f and IP_f;

moving the end effector to a plurality of first orientations and for each of these orientations moving the end effector in a translation pattern to a second target point while maintaining the same first orientation of the end effector and while generating position measurements in R_f and IP_f identifying the translation pattern;

calculating a depth value Z from the camera unit to the calibration marker for each second target point based on said position measurements identifying the movement along the translation pattern;

transforming position measurements in pixels in IP_f to position values in the second coordinate system C_f based on the depth value Z and C_int;

calculating a translational part of the second coordinate system C_f of the object identification unit based on the translation and reorientation between the first orientations; and

using the rotational and the translational parts to store a relation between R_f and C_f to enable collaboration between the robot unit and the object identification unit.

2. The method according to claim 1, wherein the step of calculating the translational part of the second coordinate system C_f comprises solving a hand-eye equation A·

x=b, where A is a matrix describing rotation between different poses of the robot unit in said first orientations, and wherein b is a matrix describing a difference between the translation in C_f and the translation in R_f for the different poses.

3. The method according to claim 2, wherein the hand-eye equation is solved using a least square method.

4. The method according to claim 1, wherein the transforming step comprises transforming the position measurements in pixels in IP_f to Cartesian values X_c, Y_c, Z in the second coordinate system C_f of the object identification unit using an equation (X_c, Y_c, Z)=(C_int)⁻

1·

(u, v,

     1)·

Z.

5. The method according to claim 1, wherein Z is derived from the equation

6. The method according to claim 1, comprising teaching the robot unit a subset of said plurality of first target points before moving the end effector to the plurality of first target points.

7. The method according to claim 1, comprising moving the end effector to at least three first orientations, wherein the end effector has a different orientation in each of said first orientations.

8. The method according to claim 1, wherein the translation pattern is essentially orthogonal to an optical axis of the camera unit.

9. A robot controller having a programming unit configured to execute a method for calibrating a first coordinate system R_f of a robot unit with a second coordinate system C_f of an object identification unit, wherein the robot unit comprises a robot arm with an end effector and the object identification unit comprises a camera unit, the end effector further comprises a calibration marker, the method comprises

moving the end effector to a plurality of first target points wherein the first target points are chosen to include movement of the end effector in all three axes of the first coordinate system R_f of the robot unit, and wherein the end effector maintains a same orientation, while generating position measurements in the first coordinate system R_f and position measurements in an image plane coordinate system IP_f for the calibration marker with the object identification unit for each first target points;

calculating intrinsic parameters C_int of the camera unit and a rotational part of the second coordinate system C_f of the object identification unit based on the measurements in R_f and IP_f;

moving the end effector to a plurality of first orientations and for each of these orientations moving the end effector in a translation pattern to a second target point while maintaining the same first orientation of the end effector and while generating position measurements in R_f and IP_f identifying the translation pattern;

calculating a depth value Z from the camera unit to the calibration marker for each second target point based on said position measurements identifying the movement along the translation pattern;

transforming position measurements in pixels in IP_f to position values in the second coordinate system C_f based on the depth value Z and C_int;

calculating a translational part of the second coordinate system C_f of the object identification unit based on the translation and reorientation between the first orientations; and

using the rotational and the translational parts to store a relation between R_f and C_f to enable collaboration between the robot unit and the object identification unit.

10. A robot unit comprising an object identification unit, wherein the robot unit is configured to execute a method for calibrating a first coordinate system R_f of the robot unit with a second coordinate system C_f of the object identification unit, wherein the robot unit further comprises a robot arm with an end effector and the object identification unit comprises a camera unit, the end effector further comprises a calibration marker, the method comprises

moving the end effector to a plurality of first target points wherein the first target points are chosen to include movement of the end effector in all three axes of the first coordinate system R_f of the robot unit, and wherein the end effector maintains a same orientation, while generating position measurements in the first coordinate system R_f and position measurements in an image plane coordinate system IP_f for the calibration marker with the object identification unit for each first target points;

calculating intrinsic parameters C_int of the camera unit and a rotational part of the second coordinate system C_f of the object identification unit based on the measurements in R_f and IP_f;

moving the end effector to a plurality of first orientations and for each of these orientations moving the end effector in a translation pattern to a second target point while maintaining the same first orientation of the end effector and while generating position measurements in R_f and IP_f identifying the translation pattern;

calculating a depth value Z from the camera unit to the calibration marker for each second target point based on said position measurements identifying the movement along the translation pattern;

transforming position measurements in pixels in IP_f to position values in the second coordinate system C_f based on the depth value Z and C_int;

calculating a translational part of the second coordinate system C_f of the object identification unit based on the translation and reorientation between the first orientations; and

using the rotational and the translational parts to store a relation between R_f and C_f to enable collaboration between the robot unit and the object identification unit.

11. A computer product comprising a non-transitory computer readable medium storing computer instructions to perform, when the computer instructions are executed on a computer unit, a method for calibrating a first coordinate system R_f of a robot unit with a second coordinate system C_f of an object identification unit, wherein the robot unit comprises a robot arm with an end effector and the object identification unit comprises a camera unit, the end effector further comprises a calibration marker, the method comprises

moving the end effector to a plurality of first target points wherein the first target points are chosen to include movement of the end effector in all three axes of the first coordinate system R_f of the robot unit, and wherein the end effector maintains a same orientation, while generating position measurements in the first coordinate system R_f and position measurements in an image plane coordinate system IP_f for the calibration marker with the object identification unit for each first target points;

calculating intrinsic parameters C of the camera unit and a rotational part of the second coordinate system C_f of the object identification unit based on the measurements in R_f and IP_f;

moving the end effector to a plurality of first orientations and for each of these orientations moving the end effector in a translation pattern to a second target point while maintaining the same first orientation of the end effector and while generating position measurements in R_f and IP_f identifying the translation pattern;

calculating a depth value Z from the camera unit to the calibration marker for each second target point based on said position measurements identifying the movement along the translation pattern;

transforming position measurements in pixels in IP_f to position values in the second coordinate system C_f based on the depth value Z and C_int;

calculating a translational part of the second coordinate system C_f of the object identification unit based on the translation and reorientation between the first orientations; and

using the rotational and the translational parts to store a relation between R_f and C_f to enable collaboration between the robot unit and the object identification unit.

12. A robot system comprising

a robot unit defining a first coordinate system R_f, wherein the robot unit comprises a robot arm with an end effector;

an object identification unit defining a second coordinate system C_f, wherein the object identification unit comprises a camera unit;

a calibration marker on the end effector;

a computer unit with a programming unit and a computer readable storage medium storing computer instructions operable to cause the programming unit to perform operations comprising;

moving the end effector to a plurality of first target points wherein the first target points are chosen to include movement of the end effector in all three axes of the first coordinate system R_f of the robot unit, and wherein the end effector maintains a same orientation, while generating position measurements in the first coordinate system R_f and position measurements in an image plane coordinate system IP_f for the calibration marker with the object identification unit for each first target points;

calculating intrinsic parameters C_int of the camera unit and a rotational part of the second coordinate system C_f of the object identification unit based on the measurements in R_f and IP_f;

moving the end effector to a plurality of first orientations and for each of these first orientations moving the end effector in a translation pattern to a second target point while maintaining the same first orientation of the end effector and while generating position measurements in R_f and IP_f identifying the translation pattern;

calculating a depth value Z from the camera unit to the calibration marker for each first orientation based on said position measurements identifying the translation pattern;

transforming position measurements in pixels in IP_f to position values in the second coordinate system C_f based on the depth value Z and C_int;

calculating a translational part of the second coordinate system C_f of the object identification unit based on the translation and reorientation between the first orientations; and

using the rotational and the translational parts to store a relation between R_f and C_f to enable collaboration between the robot unit and the object identification unit.