Robot and robot operating method

• US 20050065653A1
• Filed: 09/01/2004
• Published: 03/24/2005
• Est. Priority Date: 09/02/2003
• Status: Abandoned Application

• Current Assignee: Fanuc Ltd (Fanuc Corporation)
• Original Assignee: Fanuc Ltd (Fanuc Corporation)
• Inventors: Takizawa, Katsutoshi, Ban, Kazunori
• Application Number:
  US10/930,850
• Publication Number:
  US 20050065653A1
• Time in Patent Office: Days
• US Class Current: 700/245
• CPC Class Codes: B25J 9/1697 Vision controlled systems
• CPC Class Codes: G05B 2219/36431 Tv camera in place of tool, on display operator marks points, crosshair
• CPC Class Codes: G05B 2219/40003 Move end effector so that image center is shifted to desired position

Abstract

A robot automatically moving a distal end portion of a robot arm to an arbitrary target position, and method therefor. A camera mounted at the distal end portion of the robot arm captures an image of an object. A position R₁ corresponding to the target Q is specified in the image. Assuming that the number of pixels between the position R₁ and the center of an image screen is equal to N₁, a distance W₁ observed at a distance L₀ at the time of calibration is determined as W₁=C₀·N₁, where C₀ is a transformation coefficient. The camera is moved by the distance W₁ in an X axis direction toward the target Q. A position R₂ corresponding to the target W is specified in the image. The number, N₂, of pixels between the position R₂ and the screen center is determined. A motion vector q is determined from C₀, N₁, N₂ and L₀. The camera is moved according to the motion vector q. The robot is positioned at a position where the camera center is opposed to the target Q at the distance L₀. By specifying the target Q in the image, a motion to the specified target Q position is automatically realized.

Claims

1. A robot having a camera mounted at a distal end portion of a robot arm, comprising:

means for positioning the distal end portion of the robot arm with the camera at a first position on a plane spaced from an object by a predetermined first distance;

means for displaying an image captured by the camera at the first position on a display device;

means for allowing a manual operation to specify an arbitrary point on the object in the captured image displayed on the display device;

means for obtaining position information of the specified point in the captured image;

means for determining a direction/amount of motion of the camera to a second position where the camera confronts the specified point on the object with a predetermined second distance in between based on the obtained position information and the first predetermined distance; and

means for moving the distal end portion of the robot arm with the camera to the second position in accordance with the determined direction/amount of motion.

2. A robot having a camera mounted at a distal end portion of a robot arm, comprising:

means for displaying an image captured by the camera on a display device;

means for allowing a first manual operation to specify an arbitrary point on an object in a first image captured by the camera at a first position and displayed on the display device;

means for obtaining first position information of the specified point in the first image;

means for determining a first direction/amount of motion based on the first position information;

means for moving the distal end portion of the robot arm with the camera to a second position according to the determined first direction/amount of the first motion;

means for allowing a second manual operation to specify the same point on the object as specified by the first manual operation, in a second image captured by the camera at the second position and displayed on the display device;

means for obtaining second position information of the specified point in the second image;

means for determining a second direction/amount of motion based on the first position information and the second position information; and

means for moving the distal end portion of the robot arm with the camera to a third position according to the determined second direction/amount of motion.

3. A robot having a camera mounted at a distal end portion of a robot arm, comprising:

means for displaying an image captured by the camera on a display device;

means for allowing a first manual operation to specify an arbitrary point on an object in a first image captured by the camera at a first position and displayed on the display device;

means for obtaining first position information of the specified point in the first image;

means for determining a first direction of motion based on the first position information;

means for moving the distal end portion of the robot arm with the camera to a second position according to the determined first direction of motion and a preset amount of motion;

means for allowing a second manual operation to specify the same point on the object as specified by the first manual operation, in a second image captured by the camera at the second position and displayed on the display device;

means for obtaining second position information of the specified point on the object in the second image;

means for determining a second direction/amount of motion based on the first position information and the second position information; and

means for moving the distal end portion of the robot arm with the camera to a third position according to the determined second direction/amount of motion.

4. A robot having a camera mounted at a distal end portion of a robot arm, comprising:

means for displaying an image captured by the camera on a display device;

means for allowing a first manual operation to specify an arbitrary point on an object in a first image captured by the camera at a first position and displayed on the display device;

means for obtaining first position information of the specified point in the first image;

means for moving the distal end portion of the robot arm with the camera to a second position according to a preset first direction/amount of motion;

means for allowing a second manual operation to specify the same point on the object as specified by the first manual operation, in a second image captured by the camera at the second position and displayed on the display device;

means for obtaining second position information of the specified point on the object in the second image;

means for determining a second direction/amount of motion based on the first position information and the second position information; and

means for moving the distal end portion of the robot arm with the camera to a third position according to the determined second direction/amount of motion.

5. A robot having a camera mounted on a distal end portion of a robot arm, comprising:

means for detecting an object in a first image captured by the camera at a first position;

means for obtaining first position information of the detected object in the first image;

means for determining a first direction/amount of motion of the camera based on the first position information;

means for moving the distal end portion of the robot arm with the camera to a second position according to the determined first direction/amount of motion;

means for detecting the same object as the detected object, in a second-image captured by the camera at the second position;

means for obtaining second position information of the detected object in the second image;

means for determining a second direction/amount of motion based on the first position information and the second position information; and

means for moving the distal end portion of the robot arm with the camera to a third position based on the determined second direction/amount of motion.

6. A robot having a camera mounted at a distal end portion of a robot arm, comprising:

means for detecting an object in a first image captured by the camera at a first position;

means for obtaining first position information of the detected object in the first image;

means for determining a first direction of motion based on the first position information;

means for moving the distal end portion of the robot arm with the camera to a second position according to the determined first direction of motion and a preset amount of motion;

means for detecting the same object as the detected object, in a second image captured by the camera at the second position;

means for obtaining second position information of the detected object in the second image;

means for determining a second direction/amount of motion based on the first position information and the second position information; and

means for moving the distal end portion of the robot arm with the camera to a third position according to the determined second direction/amount of motion.

7. A robot having a camera mounted at a distal end portion of a robot arm, comprising:

means for detecting an object in a first image captured by the camera at a first position;

means for obtaining first position information of the detected object in the first image;

means for moving the distal end portion of the robot arm with the camera to a second position according to a preset first direction/amount of motion;

means for detecting the same object as the detected object, in a second image captured by the camera at the second position;

means for obtaining second position information of the detected object in the second image;

means for determining a second direction/amount of motion based on the first position information and the second position information; and

means for moving the distal end portion of the robot arm with the camera to a third position according to the determined second direction/amount of motion.

8. A robot having a camera mounted at a distal end portion of a robot arm, comprising:

means for detecting an object in a first image captured by the camera at a first position;

means for obtaining first size information of the detected object in the first image;

means for determining a first amount of motion based on the first size information;

means for moving the distal end portion of the robot arm to a second position according to a preset direction of motion and the determined first amount of motion;

means for detecting the same object as the detected object, in a second image captured by the camera at the second position;

means for obtaining second size information and position information of the detected object in the second image;

means for determining a second direction/amount of motion based on the first size information, the second size information and the position information; and

means for moving the distal end portion of the robot arm with the camera to a third position according to the determined second direction/amount of motion.

9. A robot having a camera mounted at a distal end portion of a robot arm, comprising:

means for detecting an object in a first image captured by the camera at a first position;

means for obtaining first size information of the detected object in the first image;

means for moving the distal end portion of the robot arm with the camera to a second position according to a preset first direction/amount of motion;

means for detecting the same object as the detected object, in a second image captured by the camera at the second position;

means for obtaining second size information and position information of the detected object in the second image;

means for determining a second direction/amount of motion of the camera based on the first size information, the second size information and the position information; and

means for moving the distal end portion of the robot arm with the camera to a third position according to the determined second direction/amount of motion.

10. A robot having a camera mounted at a distal end portion of a robot arm, comprising:

means for displaying an image captured by the camera on a display device;

means for allowing a manual operation to specify an arbitrary point on an object in a first image captured by the camera at a first position and displayed on the display device;

means for obtaining first position information of the specified point in the first image;

means for creating an image model based on image information in the vicinity of the specified point in the first image;

means for determining a first direction/amount of motion based on the first position information;

means for moving the distal end portion of the robot arm with the camera to a second position according to the determined first direction/amount of motion;

means for detecting the same point as the specified point, in a second image captured by the camera at the second position using the image model;

means for obtaining second position information of the detected point in the second image;

means for determining a second direction/amount of motion based on the first position information and the second position information; and

means for moving the distal end portion of the robot arm with the camera to a third position according to the determined second direction/amount of motion.

13. A robot according to any one of claims 2-12, wherein said means for determining the second direction/amount of motion determines the second direction/amount of motion for the third position where the specified point on the object is on an optical axis of the camera and spaced apart form the camera by a predetermined distance.

14. A robot according to any one of claims 2-12, wherein said means for determining the second direction/amount of motion determines the second direction/amount of motion such that an end of a tool attached to the distal end portion of the robot arm is positioned at the specified point on the object.

11. A robot having a camera mounted at a distal end portion of a robot arm, comprising:

means for displaying an image captured by the camera on a display device;

means for allowing a manual operation to specify an arbitrary point on an object in a first image captured by the camera at a first position and displayed on the display device;

means for obtaining first position information of the specified point in the first image;

means for creating an image model based on image information in the vicinity of the specified point in the first image;

means for determining a first direction of motion based on the first position information;

means for moving the distal end portion of the robot arm with the camera to a second position according to the determined first direction of motion and a preset amount of motion;

means for detecting the same point as the specified point, in a second image captured by the camera at the second position using the image model;

means for obtaining second position information of the detected point in the second image;

means for determining a second direction/amount of motion based on the first position information and the second position information; and

means for moving the distal end portion of the robot arm with the camera to a third position according to the determined second direction/amount of motion.

12. A robot having a camera mounted at a distal end portion of a robot arm, comprising:

means for displaying an image captured by the camera on a display device;

means for allowing a manual operation to specify an arbitrary point on an object in a first image captured by the camera at a first position and displayed on the display device;

means for obtaining first position information of the specified point in the first image;

means for creating an image model based on image information in the vicinity of the specified point in the first image;

means for moving the distal end portion of the robot arm with the camera to a second position according to a preset first direction/amount of motion;

means for detecting the same point as the specified point, in a second image captured by the camera at the second position using the image model;

means for obtaining second position information of the detected point in the second image;

means for determining a second direction/amount of motion based on the first position information and the second position information; and

means for moving the distal end portion of the robot arm with the camera to a third position according to the determined second direction/amount of motion.

15. A method of operating a robot having a camera mounted at a distal end portion of a robot arm, comprising the steps of:

positioning the distal end portion of the robot arm with the camera at a first position on a plane spaced from an object by a predetermined first distance;

displaying an image captured by the camera at the first position on a display device;

allowing a manual operation to specify an arbitrary point on the object in the captured image displayed on the display device;

obtaining position information of the specified point in the captured image;

determining a direction/amount of motion of the camera to a second position where the camera confronts the specified point on the object with a predetermined second distance in between based on the obtained position information and the first predetermined distance; and

moving the distal end portion of the robot arm with the camera to the second position in accordance with the determined direction/amount of motion.

16. A method of operating a robot having a camera mounted at a distal end portion of a robot arm, comprising the steps of:

displaying a first image captured by the camera at a first position on a display device;

allowing a first manual operation to specify an arbitrary point on an object in the first image displayed on the display device;

obtaining first position information of the specified point in the first image;

determining a first direction/amount of motion based on the first position information;

moving the distal end portion of the robot arm with the camera to a second position according to the determined first direction/amount of the first motion;

displaying a second image captured by the camera at the second position on the display device;

allowing a second manual operation to specify the same point on the object as specified by the first manual operation, in the second image displayed on the display device;

obtaining second position information of the specified point in the second image;

determining a second direction/amount of motion based on the first position information and the second position information; and

moving the distal end portion of the robot arm with the camera to a third position according to the determined second direction/amount of motion.

27. A method of operating a robot according to any one of claims 16-26, wherein said means for determining the second direction/amount of motion determines the second direction/amount of motion for the third position where the specified point on the object is on an optical axis of the camera and spaced apart form the camera by a predetermined distance.

28. A method of operating a robot according to any one of claims 16-26, wherein said means for determining the second direction/amount of motion determines the second direction/amount of motion such that an end of a tool attached to the distal end portion of the robot arm is positioned at the specified point on the object.

17. A method of operating a robot having a camera mounted at a distal end portion of a robot arm, comprising the steps of:

displaying a first image captured by the camera at a first position on a display device;

allowing a first manual operation to specify an arbitrary point on an object in the first image displayed on the display device;

obtaining first position information of the specified point in the first image;

determining a first direction of motion based on the first position information;

moving the distal end portion of the robot arm with the camera to a second position according to the determined first direction of motion and a preset amount of motion;

displaying a second image captured by the camera at the second position on the display device;

allowing a second manual operation to specify the same point on the object as specified by the first manual operation, in the second image displayed on the display device;

obtaining second position information of the specified point on the object in the second image;

determining a second direction/amount of motion based on the first position information and the second position information; and

moving the distal end portion of the robot arm with the camera to a third position according to the determined second direction/amount of motion.

18. A method of operating a robot having a camera mounted at a distal end portion of a robot arm, comprising the steps of:

displaying a first image captured by the camera at a first position on a display device;

allowing a first manual operation to specify an arbitrary point on an object in the first image displayed on the display device;

obtaining first position information of the specified point in the first image;

moving the distal end portion of the robot arm with the camera to a second position according to a preset first direction/amount of motion;

displaying a second image captured by the camera at the second position on the display device;

allowing a second manual operation to specify the same point on the object as specified by the first manual operation, in the second image displayed on the display device;

obtaining second position information of the specified point on the object in the second image;

determining a second direction/amount of motion based on the first position information and the second position information; and

means for moving the distal end portion of the robot arm with the camera to a third position according to the determined second direction/amount of motion.

19. A method of operating a robot having a camera mounted on a distal end portion of a robot arm, comprising the steps of:

detecting an object in a first image captured by the camera at a first position;

obtaining first position information of the detected object in the first image;

determining a first direction/amount of motion of the camera based on the first position information;

moving the distal end portion of the robot arm with the camera to a second position according to the determined first direction/amount of motion;

detecting the same object as the detected object, in a second image captured by the camera at the second position;

obtaining second position information of the detected object in the second image;

determining a second direction/amount of motion based on the first position information and the second position information; and

moving the distal end portion of the robot arm with the camera to a third position based on the determined second direction/amount of motion.

20. A method of operating a robot having a camera mounted at a distal end portion of a robot arm, comprising the steps of:

detecting an object in a first image captured by the camera at a first position;

obtaining first position information of the detected object in the first image;

determining a first direction of motion based on the first position information;

moving the distal end portion of the robot arm with the camera to a second position according to the determined first direction of motion and a preset amount of motion;

detecting the same object as the detected object, in a second image captured by the camera at the second position;

obtaining second position information of the detected object in the second image;

determining a second direction/amount of motion based on the first position information and the second position information; and

moving the distal end portion of the robot arm with the camera to a third position according to the determined second direction/amount of motion.

21. A method of operating a robot having a camera mounted at a distal end portion of a robot arm, comprising the steps of:

detecting an object in a first image captured by the camera at a first position;

obtaining first position information of the detected object in the first image;

moving the distal end portion of the robot arm with the camera to a second position according to a preset first direction/amount of motion;

detecting the same object as the detected object, in a second image captured by the camera at the second position;

obtaining second position information of the detected object in the second image;

determining a second direction/amount of motion based on the first position information and the second position information; and

moving the distal end portion of the robot arm with the camera to a third position according to the determined second direction/amount of motion.

22. A method of operating a robot having a camera mounted at a distal end portion of a robot arm, comprising the steps of:

detecting an object in a first image captured by the camera at a first position;

obtaining first size information of the detected object in the first image;

determining a first amount of motion based on the first size information;

moving the distal end portion of the robot arm to a second position according to a preset direction of motion and the determined first amount of motion;

detecting the same object as the detected object, in a second image captured by the camera at the second position;

obtaining second size information and position information of the detected object in the second image;

determining a second direction/amount of motion based on the first size information, the second size information and the position information; and

moving the distal end portion of the robot arm with the camera to a third position according to the determined second direction/amount of motion.

23. A method of operating a robot having a camera mounted at a distal end portion of a robot arm, comprising the steps of:

detecting an object in a first image captured by the camera at a first position;

obtaining first size information of the detected object in the first image;

moving the distal end portion of the robot arm with the camera to a second position according to a preset first direction/amount of motion;

detecting the same object as the detected object, in a second image captured by the camera at the second position;

obtaining second size information and position information of the detected object in the second image;

determining a second direction/amount of motion of the camera based on the first size information, the second size information and the position information; and

moving the distal end portion of the robot arm with the camera to a third position according to the determined second direction/amount of motion.

24. A method of operating a robot having a camera mounted at a distal end portion of a robot arm, comprising the steps of:

displaying a first image captured by the camera at a first position on a display device;

allowing a manual operation to specify an arbitrary point on an object in the first image displayed on the display device;

obtaining first position information of the specified point in the first image;

creating an image model based on image information in the vicinity of the specified point in the first image;

determining a first direction/amount of motion based on the first position information;

moving the distal end portion of the robot arm with the camera to a second position according to the determined first direction/amount of motion;

detecting the same point as the specified point, in a second image captured by the camera at the second position using the image model;

obtaining second position information of the detected point in the second image;

determining a second direction/amount of motion based on the first position information and the second position information; and

moving the distal end portion of the robot arm with the camera to a third position according to the determined second direction/amount of motion.

25. A method of operating a robot having a camera mounted at a distal end portion of a robot arm, comprising the steps of:

displaying a first image captured by the camera at a first position on a display device;

allowing a manual operation to specify an arbitrary point on an object in the first image displayed on the display device;

obtaining first position information of the specified point in the first image;

creating an image model based on image information in the vicinity of the specified point in the first image;

determining a first direction of motion based on the first position information;

moving the distal end portion of the robot arm with the camera to a second position according to the determined first direction of motion and a preset amount of motion;

detecting the same point as the specified point, in a second image captured by the camera at the second position using the image model;

obtaining second position information of the detected point in the second image;

determining a second direction/amount of motion based on the first position information and the second position information; and

moving the distal end portion of the robot arm with the camera to a third position according to the determined second direction/amount of motion.

26. A method of operating a robot having a camera mounted at a distal end portion of a robot arm, comprising the steps of:

displaying a first image captured by the camera at a first position on a display device;

allowing a manual operation to specify an arbitrary point on an object in the first image displayed on the display device;

obtaining first position information of the specified point in the first image;

creating an image model based on image information in the vicinity of the specified point in the first image;

moving the distal end portion of the robot arm with the camera to a second position according to a preset first direction/amount of motion;

detecting the same point as the specified point, in a second image captured by the camera at the second position using the image model;

obtaining second position information of the detected point in the second image;

determining a second direction/amount of motion based on the first position information and the second position information; and

moving the distal end portion of the robot arm with the camera to a third position according to the determined second direction/amount of motion.