Robot system

• US 7,424,341 B2
• Filed: 05/26/2004
• Issued: 09/09/2008
• Est. Priority Date: 05/29/2003
• Status: Expired due to Fees

• Current Assignee: Fanuc Ltd (Fanuc Corporation)
• Original Assignee: Fanuc Ltd (Fanuc Corporation)
• Inventors: Mizuno, Jun, Watanabe, Atsushi
• Primary Examiner(s): Tran; Khoi H.
• Assistant Examiner(s): Marc; McDieunel
• Application Number:
  US10/853,169
• Publication Number:
  US 20040243282A1
• Time in Patent Office: 1,567 Days
• Field of Search: 318/568.1, 318/568.11, 318/568.14, 318/568.16, 318/568.21, 318/568.23, 901/4, 901/43, 901/47
• US Class Current: 700/245
• CPC Class Codes: B25J 9/1697 Vision controlled systems
• CPC Class Codes: G06T 1/0014 Image feed-back for automatic industrial control, e.g. robot with camera robots B25J19/023

Abstract

A robot system efficiently performing an operation of moving a robot to close to and/or separate from a target point, such as teaching operation. A camera of a visual sensor is mounted to the robot such that a distal end of an end effector is seen within the field of view of the camera, and the end effector'"'"'s distal end and a target position on an object are specified on a monitor screen. When an approach key is depressed, a target position is detected on an image, and a difference from a position of the end effector'"'"'s distal end is calculated. Whether the difference is within an allowable range is checked. Depending on the result, an amount of robot motion is calculated, and the robot is operated. The processing is repeated until the depressed approach key is released. When a retreat key is depressed, the robot moves away from the object. The robot may be caused to stop using a distance sensor.

Claims

1. A robot system comprising:

a robot with an end effector mounted thereon;

a visual sensor having image capturing means for capturing an image including a first point set with respect to the end effector and a second point set with respect to an object;

approach commanding means for issuing an approach command for operating the robot such that the first point approaches the second point in three dimensional space in response to input of an approach instruction;

means for recognizing positions of the first and second points in the image captured by said visual sensor with the first and second points being within a field of view of the image capturing means to obtain a positional relation between the first point and the second point in the image;

means for calculating a first motion command for operating the robot such that the first point and the second point approaches each other in the captured image based on the positional relation between the first point and the second point in the image;

means for calculating a second motion command for operating the robot such that the first point approaches the second point in the three dimensional space in response to the approach command;

means for obtaining a first combined motion command by combining the first motion command and the second motion command, to operate the robot according to the first combined motion command; and

means for repeating the recognition of the positional relation between the first point and the second point, the calculations of the first motion command and the second motion command and the robot operation according to the first combined motion command while the approach instruction is inputted to said approach commanding means.

2. A robot system according to claim 1, wherein said first point is set by designating a picture element in the captured image irrespective of whether the end effector is within the field of view of said image capturing means, and the robot system comprises means for making said image capturing means approach a position of the second point the object as the first point and the second point approach each other in the captured image.

3. A robot system according to claim 1, further comprising:

means for operating the robot according to the first motion command;

means for determining whether or not the positional relation between the first point and the second point in the captured image fulfills a predetermined condition; and

means for repeating the recognition of the positional relation between the first point and the second point, the determination by said determination means, the calculation of the first motion command, and the operation of the robot according to the first motion command until said determination means determines that the predetermined condition is fulfilled, in response to the input of the approach instruction to said approach commanding means, before repeating the operation of the robot according to the first combined motion command.

4. A robot system according to claim 3, further comprising:

means for repeating the operation of the robot according to the second motion command after repeating the operation of the robot according to the first combined motion command.

5. A robot system according to claim 1, wherein said means for calculating the second motion command calculates motion information for moving the end effector in a direction along an optical axis of the image capturing means.

6. A robot system according to claim 1, wherein said means for calculating the second motion command calculates motion information for moving the end effector in a direction along a line of sight connecting the image capturing means and the second point.

7. A robot system according to claim 1, further comprising:

a distance sensor or a contact sensor mounted on the robot; and

means for stopping the robot when said distance sensor or said contact sensor detects that the first point approaches the second point within a predetermined distance while the approach instruction is inputted to said approach commanding means.

8. A robot system according to claim 7, wherein a position at which the robot stops is stored as a taught position.

9. A robot system according to claim 1, further comprising:

retreat commanding means for issuing a retreat command for operating the robot such that the first point moves remote from the second point in three dimensional space in response to input of a retreat instruction;

means for calculating a third motion command to operate the robot such that the first point moves remote from the second point in three dimensional space in response to the retreat command;

means for obtaining a second combined motion command by combining the first motion command and the third motion command, to operate the robot according to the second combined motion command; and

means for repeating the recognition of the positional relation between the first point and the second point, the calculations of the first and third motion commands, and the operation of said robot according to the second combined motion command while the retreat instruction is inputted to said retreat commanding means.

10. A robot system according to claim 9, further comprising:

means for operating the robot according to the first motion command;

determination means for determining whether or not the positional relation between the first point and the second point in the captured image fulfills a predetermined condition; and

means for repeating the recognition of the positional relation between the first point and the second point, the determination by said determination means, the calculation of the first motion command and the operation of the robot according to the first motion command until said determination means determines that the predetermined condition is fulfilled, in response to the input of the retreat instruction to said retreat commanding means before repeating the operation of the robot according to the second combined motion command.

11. A robot system according to claim 10, further comprising:

means for repeating an operation of the robot according to the third motion command after the operation of the robot according to the second combined motion command.

12. A robot system according to claim 9, wherein said means for calculating the third motion command calculates motion information for moving the end effector in a direction along an optical axis of the image capturing means.

13. A robot system according to claim 9, wherein said means for calculating the third motion command calculates motion information for moving the end effector in a direction along a line of sight connecting the image capturing means and the second point.

14. A robot system according to claim 1, further comprising display means for displaying the image captured by the image capturing means of said visual sensor,

wherein a line segment connecting the first part and the second part is displayed on the captured image by said display means.

15. A robot system according to claim 1, wherein the image capturing means is mounted on the robot.

16. A robot system comprising:

a robot with an end effector mounted thereon;

a visual sensor having image capturing means for capturing an image including a first point set with respect to the end effector and a second point set with respect to an object;

visually two-dimensional approach commanding means for issuing a visually two-dimensional approach command for operating the robot such that the first point and the second point approach each other two-dimensionally in the captured image in response to input of a visually two-dimensional approach instruction;

means for recognizing positions of the first point and the second point in the image captured by said visual sensor with the first and second points being within a field of view of said image capturing means to obtain a positional relation between the first point and the second point in the image;

means for calculating a motion command for operating the robot such that the first point and the second point approach each other in the captured image based on the positional relation between the first point and the second point;

means for operating the robot according to the calculated motion command; and

means for repeating the recognition of the positional relation between the first point and the second point, the calculation of the motion command, and the operation of the robot according to the obtained motion command while the visually two-dimensional approach instruction is inputted to said visually two-dimensional approach commanding means.

17. A robot system according to claim 16, wherein said first point is set by designating a picture element in the captured image irrespective of whether the end effector is within the field of view of said image capturing means, and the robot system comprises means for making said image capturing means approach a position of the second point the object as the first point and the second point approach each other in the captured image.

18. A robot system comprising:

a robot with an end effector mounted thereon;

a visual sensor having image capturing means for capturing an image including a first point set with respect to the end effector and a second point set with respect to an object;

visually two-dimensional approach commanding means for issuing a visually two-dimensional approach command for operating the robot such that the first point and the second point approach each other two-dimensionally in the captured image in response to input of a visually two-dimensional approach instruction;

means for specifying a direction of visually two-dimensional motion of the first point in the image captured by said visual sensor with the first point and the second point within a field of view of the image capturing means;

calculation means for calculating a motion command for operating the robot such that the first point moves in the specified direction; and

means for operating the robot according to the calculated motion command.