Robot control apparatus, robot control method, program, and recording medium

• US 8,977,395 B2
• Filed: 03/19/2012
• Issued: 03/10/2015
• Est. Priority Date: 03/24/2011
• Status: Active Grant

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Negishi, Mahito, Suzuki, Hideaki, Iizuka, Shinsuke
• Primary Examiner(s): Jen, Ian
• Application Number:
  US14/006,638
• Publication Number:
  US 20140012416A1
• Time in Patent Office: 1,086 Days
• Field of Search: 700/251
• US Class Current: 700/251
• CPC Class Codes: B25J 9/1607 Calculation of inertia, jacobian matrixes and inverses
• CPC Class Codes: B25J 9/1679 characterised by the tasks executed
• CPC Class Codes: B25J 9/1697 Vision controlled systems
• CPC Class Codes: G05B 2219/39022 Transform between measuring and manipulator coordinate system
• CPC Class Codes: G05B 2219/39063 Quick calculation of coordinates by using precalculated, stored matrixes, inverses
• CPC Class Codes: G05B 2219/40082 Docking, align object on end effector with target
• CPC Class Codes: G05B 2219/40609 Camera to monitor end effector as well as object to be handled
• CPC Class Codes: H01R 43/26 for engaging or disengaging the two parts of a coupling device structural association with two-part coupling device H01R13/629

Abstract

A first coordinate system C_A of the hand unit, a second coordinate system C_B of the first workpiece, and a third coordinate system C_C of a second workpiece in a camera coordinate system are calculated (S2, S3, and S4). First and second coordinate transformation matrices ^AT_B and ^AT_C are calculated (S5 and S6). Coordinate data of a target point is set in the coordinate system of the first workpiece (S7). Coordinate data of an instruction point is set in the coordinate system of the second workpiece (S8). The coordinate data of the target point is subjected to coordinate transformation using the first coordinate transformation matrix ^AT_B (S9). The coordinate data of the instruction point is subjected to coordinate transformation using the second coordinate transformation matrix ^AT_C (S10). Operation instructions are generated using the converted coordinate data (S11).

Claims

1. A robot control apparatus controlling a robot body including an arm unit having a plurality of link members coupled with one another and a hand unit which is disposed at an end of the arm unit, which grasps a first workpiece, and which attaches the grasped first workpiece to a second workpiece, the robot control apparatus, comprising:

a camera configured to output image data obtained by capturing an image of the robot body, the first workpiece, and the second workpiece;

a first coordinate system calculation unit configured to calculate a first coordinate system representing a position and orientation of the hand unit in a camera coordinate system of the camera from the image data;

a second coordinate system calculation unit configured to calculate a second coordinate system representing a position and orientation of the first workpiece in the camera coordinate system of the camera from the image data;

a third coordinate system calculation unit configured to calculate a third coordinate system representing a position and orientation of the second workpiece in the camera coordinate system of the camera from the image data;

a first coordinate transformation matrix calculation unit configured to calculate a first coordinate transformation matrix used to transfer the second coordinate system into the first coordinate system;

a second coordinate transformation matrix calculation unit configured to calculate a second coordinate transformation matrix used to transfer the third coordinate system into the first coordinate system;

a target-point setting unit configured to set, in a coordinate system of the first workpiece, coordinate data of a target point having a constant position and orientation relative to the first workpiece;

an instruction-point setting unit configured to set, in a coordinate system of the second workpiece, coordinate data of an instruction point which allows the target point to pass in a constant position and orientation relative to the second workpiece;

a first coordinate transformation unit configured to perform coordinate transformation on the coordinate data of the target point set by the target-point setting unit using the first coordinate transformation matrix;

a second coordinate transformation unit configured to perform coordinate transformation on the coordinate data of the instruction point set by the instruction-point setting unit using the second coordinate transformation matrix;

an operation instruction generation unit configured to generate operation instructions for operating the robot body so that the robot body traces a trajectory obtained such that the coordinate data of the target point which has been subjected to the coordinate transformation by the first coordinate transformation unit passes a position of the coordinate data of the instruction point which has been subjected to the coordinate transformation by the second coordinate transformation unit; and

an operation instruction output unit configured to output the operation instructions generated by the operation instruction generation unit to the robot body.

2. A robot control method for controlling a robot body including an arm unit having a plurality of link members coupled with one another and a hand unit which is disposed at an end of the arm unit, which grasps a first workpiece, and which attaches the grasped first workpiece to a second workpiece, the robot control method, comprising:

an image capturing step of obtaining image data by capturing an image of the robot body, the first workpiece, and the second workpiece using a camera;

a first coordinate system calculation step of calculating a first coordinate system representing a position and orientation of the hand unit in a camera coordinate system of the camera from the image data;

a second coordinate system calculation step of calculating a second coordinate system representing a position and orientation of the first workpiece in the camera coordinate system of the camera from the image data;

a third coordinate system calculation step of calculating a third coordinate system representing a position and orientation of the second workpiece in the camera coordinate system of the camera from the image data;

a first coordinate transformation matrix calculation step of calculating a first coordinate transformation matrix used to transfer the second coordinate system into the first coordinate system;

a second coordinate transformation matrix calculation step of calculating a second coordinate transformation matrix used to transfer the third coordinate system into the first coordinate system;

a target-point setting step of setting, in a coordinate system of the first workpiece, coordinate data of a target point having a constant position and orientation relative to the first workpiece;

an instruction-point setting step of setting, in a coordinate system of the second workpiece, coordinate data of an instruction point which allows the target point to pass in a constant position and orientation relative to the second workpiece;

a first coordinate transformation step of performing coordinate transformation on the coordinate data of the target point set in the target-point setting step using the first coordinate transformation matrix;

a second coordinate transformation step of performing coordinate transformation on the coordinate data of the instruction point set in the instruction-point setting step using the second coordinate transformation matrix;

an operation instruction generation step of generating operation instructions for operating the robot body so that the robot body traces a trajectory obtained such that the coordinate data of the target point which has been subjected to the coordinate transformation in the first coordinate transformation step passes a position of the coordinate data of the instruction point which has been subjected to the coordinate transformation in the second coordinate transformation step; and

an operation instruction output step of outputting the operation instructions generated in the operation instruction generation step to the robot body.

8. The robot control method according to claim 2, wherein the target-point setting step includes a selection step, which is performed before the second coordinate system calculation step, of setting coordinate data of a plurality of target points in different coordinate systems prescribed by different feature points of the first workpiece and selecting a target point which has not passed the instruction point from among the plurality of target points, in the second coordinate system calculation step, the feature point which prescribes the coordinate system of the target point selected in the selection step is extracted and the second coordinate system is calculated using the extracted feature point, and in the first coordinate transformation step, the coordinate data of the target point selected in the selection step from among the plurality of target points set in the target-point setting step is subjected to coordinate transformation using the first coordinate transformation matrix.

3. A robot control method for controlling a robot body which is fixed to a base and which includes an arm unit having a plurality of link members coupled with one another and a hand unit which is disposed at an end of the arm unit, which grasps a first workpiece, and which attaches the grasped first workpiece to a second workpiece, the robot control method, comprising:

an image capturing step of obtaining image data by capturing an image of the base, the first workpiece, and the second workpiece using a camera;

a first coordinate system calculation step of calculating a base coordinate system representing a position and orientation of the base in a camera coordinate system of the camera and thereafter calculating a first coordinate system representing a position and orientation of the hand unit in the camera coordinate system of the camera from the base coordinate system in accordance with the direct kinematics;

a second coordinate system calculation step of calculating a second coordinate system representing a position and orientation of the first workpiece in the camera coordinate system of the camera from the image data;

a third coordinate system calculation step of calculating a third coordinate system representing a position and orientation of the second workpiece in the camera coordinate system of the camera from the image data;

a first coordinate transformation matrix calculation step of calculating a first coordinate transformation matrix used to transfer the second coordinate system into the first coordinate system;

a second coordinate transformation matrix calculation step of calculating a second coordinate transformation matrix used to transfer the third coordinate system into the first coordinate system;

a target-point setting step of setting, in a coordinate system of the first workpiece, coordinate data of a target point having a constant position and orientation relative to the first workpiece;

an instruction-point setting step of setting, in a coordinate system of the second workpiece, coordinate data of an instruction point which allows the target point to pass in a constant position and orientation relative to the second workpiece;

a first coordinate transformation step of performing coordinate transformation on the coordinate data of the target point set in the target-point setting step using the first coordinate transformation matrix;

a second coordinate transformation step of performing coordinate transformation on the coordinate data of the instruction point set in the instruction-point setting step using the second coordinate transformation matrix;

an operation instruction generation step of generating operation instructions for operating the robot body so that the robot body traces a trajectory obtained such that the coordinate data of the target point which has been subjected to the coordinate transformation in the first coordinate transformation step passes a position of the coordinate data of the instruction point which has been subjected to the coordinate transformation in the second coordinate transformation step; and

an operation instruction output step of outputting the operation instructions generated in the operation instruction generation step to the robot body.

4. The robot control method according to claim 3, wherein the target-point setting step includes a selection step, which is performed before the second coordinate system calculation step, of setting coordinate data of a plurality of target points in different coordinate systems prescribed by different feature points of the first workpiece and selecting a target point which has not passed the instruction point from among the plurality of target points, in the second coordinate system calculation step, the feature point which prescribes the coordinate system of the target point selected in the selection step is extracted and the second coordinate system is calculated using the extracted feature point, and in the first coordinate transformation step, the coordinate data of the target point selected in the selection step from among the plurality of target points set in the target-point setting step is subjected to coordinate transformation using the first coordinate transformation matrix.

5. A program which causes a computer which controls a robot body including an arm unit having a plurality of link members coupled with one another and a hand unit which is disposed at an end of the arm unit, which grasps a first workpiece, and which attaches the grasped first workpiece to a second workpiece to execute:

an image capturing step of obtaining image data by capturing an image of the robot body, the first workpiece, and the second workpiece using a camera;

a first coordinate system calculation step of calculating a first coordinate system representing a position and orientation of the hand unit in a camera coordinate system of the camera from the image data;

a second coordinate system calculation step of calculating a second coordinate system representing a position and orientation of the first workpiece in the camera coordinate system of the camera from the image data;

a third coordinate system calculation step of calculating a third coordinate system representing a position and orientation of the second workpiece in the camera coordinate system of the camera from the image data;

a first coordinate transformation matrix calculation step of calculating a first coordinate transformation matrix used to transfer the second coordinate system into the first coordinate system;

a second coordinate transformation matrix calculation step of calculating a second coordinate transformation matrix used to transfer the third coordinate system into the first coordinate system;

a target-point setting step of setting, in a coordinate system of the first workpiece, coordinate data of a target point having a constant position and orientation relative to the first workpiece;

an instruction-point setting step of setting, in a coordinate system of the second workpiece, coordinate data of an instruction point which allows the target point to pass in a constant position and orientation relative to the second workpiece;

a first coordinate transformation step of performing coordinate transformation on the coordinate data of the target point set in the target-point setting step using the first coordinate transformation matrix;

a second coordinate transformation step of performing coordinate transformation on the coordinate data of the instruction point set in the instruction-point setting step using the second coordinate transformation matrix;

an operation instruction generation step of generating operation instructions for operating the robot body so that the robot body traces a trajectory obtained such that the coordinate data of the target point which has been subjected to the coordinate transformation in the first coordinate transformation step passes a position of the coordinate data of the instruction point which has been subjected to the coordinate transformation in the second coordinate transformation step; and

an operation instruction output step of outputting the operation instructions generated in the operation instruction generation step to the robot body.

7. A computer-readable recording medium which records the program set forth in claim 5.

6. A program which causes a computer which controls a robot body which is fixed to a base and which includes an arm unit having a plurality of link members coupled with one another and a hand unit which is disposed at an end of the arm unit, which grasps a first workpiece, and which attaches the grasped first workpiece to a second workpiece to execute:

an image capturing step of obtaining image data by capturing an image of the base, the first workpiece, and the second workpiece using a camera;

a first coordinate system calculation step of calculating a base coordinate system representing a position and orientation of the base in a camera coordinate system of the camera and thereafter calculating a first coordinate system representing a position and orientation of the hand unit in the camera coordinate system of the camera from the base coordinate system in accordance with the direct kinematics;

a second coordinate system calculation step of calculating a second coordinate system representing a position and orientation of the first workpiece in the camera coordinate system of the camera from the image data;

a third coordinate system calculation step of calculating a third coordinate system representing a position and orientation of the second workpiece in the camera coordinate system of the camera from the image data;

a first coordinate transformation matrix calculation step of calculating a first coordinate transformation matrix used to transfer the second coordinate system into the first coordinate system;

a second coordinate transformation matrix calculation step of calculating a second coordinate transformation matrix used to transfer the third coordinate system into the first coordinate system;

a target-point setting step of setting, in a coordinate system of the first workpiece, coordinate data of a target point having a constant position and orientation relative to the first workpiece;

an instruction-point setting step of setting, in a coordinate system of the second workpiece, coordinate data of an instruction point which allows the target point to pass in a constant position and orientation relative to the second workpiece;

a first coordinate transformation step of performing coordinate transformation on the coordinate data of the target point set in the target-point setting step using the first coordinate transformation matrix;

a second coordinate transformation step of performing coordinate transformation on the coordinate data of the instruction point set in the instruction-point setting step using the second coordinate transformation matrix;

an operation instruction generation step of generating operation instructions for operating the robot body so that the robot body traces a trajectory obtained such that the coordinate data of the target point which has been subjected to the coordinate transformation in the first coordinate transformation step passes a position of the coordinate data of the instruction point which has been subjected to the coordinate transformation in the second coordinate transformation step; and

an operation instruction output step of outputting the operation instructions generated in the operation instruction generation step to the robot body.

9. A computer-readable recording medium which records the program set forth in claim 6.