Teaching model generating device

• US 7,200,260 B1
• Filed: 04/10/2000
• Issued: 04/03/2007
• Est. Priority Date: 04/08/1999
• Status: Expired due to Fees

• Current Assignee: Fanuc Ltd (Fanuc Corporation)
• Original Assignee: Fanuc Ltd (Fanuc Corporation)
• Inventors: Watanabe, Atsushi, Arimatsu, Taro
• Primary Examiner(s): Mehta; Bhavesh M.
• Assistant Examiner(s): Seth; Manav
• Application Number:
  US09/546,213
• Time in Patent Office: 2,549 Days
• Field of Search: 382/153, 382/151, 382/152, 382141-160, 382/103, 382/115, 901/47, 901/3, 901/46, 700 57- 64, 700 95- 99, 700/110, 700159-163, 700245-260
• US Class Current: 382/153
• CPC Class Codes: B25J 9/1692 Calibration of manipulator

Abstract

A workpiece W serving as an object for detection is fixed in place. A camera 20 is mounted to the end of a robot RB. The camera is turned about an axis, which passes through the center position of the workpiece W and is perpendicular to the optical axis of the camera, to take an image of the workpiece W at a plurality of positions in different directions. A teaching model is generated on the basis of each produced image data. The relative position and posture of the workpiece to the camera 20 is also stored in association with the teaching model. Thus, it is possible to easily generate the teaching model of the workpiece regardless of three-dimensional variations in posture.

Claims

1. A teaching model generating device for image processing, in which a subject object has the same or substantially similar shape as that of a reference object, the device comprising:

an image processing system with which a current three-dimensional orientation of the subject object relative to an image pickup device is recognized by carrying out pattern matching processing of an image of the subject based on a plurality of pre-determined teaching models of the reference object; and

an image-capture system, in advance of the recognizing, generating and storing the plurality of teaching models on the basis of respective image data produced by taking images of said reference object from a plurality of directions, wherein one of the reference object and said image pickup device is fixed to a movable and positionable part of a robot or is grasped with a hand of the robot, and said robot is operated for positioning to a plurality of different image pickup positions and directions, so that the image data respectively obtained at each of said different image pickup positions and direction information indicating the respective different direction, is stored as a teaching model.

3. A teaching model generating device according to claim 1, wherein said teaching model is a part of the image data of the reference object.

4. A teaching model generating device according to claim 1, wherein said teaching model comprises data obtained by performing image processing on the image data of the reference object.

2. A teaching model generating device for image processing, in which a subject object has the same or substantially similar shape as that of a reference object, the device comprising:

an image processing system with which a current three-dimensional orientation of the subject object relative to an image pickup device is recognized by carrying out pattern matching processing of an image of the subject based on a plurality of pre-determined teaching models of the reference object; and

an image-capture system, in advance of the recognizing, generating and storing the plurality of teaching models on the basis of respective image data produced by taking images of said reference object from a plurality of directions, wherein the reference object is fixed to a movable part of a first robot or is grasped with a hand of the first robot, and said image pickup device is fixed to a movable and positionable part of a second robot or is grasped with a hand of the second robot, and any one of or both of said first and second robots is operated for positioning to a plurality of different relative image pickup positions and directions, so that the image data respectively obtained at each of said different image pickup positions is stored as a teaching model.

5. A teaching model generating device according to claim 2, wherein said teaching model is generated for every direction in which said image pickup device took the image of said reference object and said teaching model is stored in association with information on the direction.

6. A teaching model generating device according to claim 2, wherein said image pickup device is a camera.

7. A teaching model generating device according to claim 2, wherein said image pickup device is a three-dimensional visual sensor that measures a distance between the image pickup device and a plurality of points on the object.

8. A method for teaching model generation and image processing, comprising:

determining a current three-dimensional orientation of a subject object relative to an image pickup device by carrying out pattern matching processing of an image of the subject based on a plurality of predetermined teaching models of a reference object; and

in advance of the determining, generating and storing the plurality of teaching models on the basis of respective image data produced by taking images of said reference object from a plurality of directions, wherein one of the reference object and said image pickup device is fixed to a movable and positionable part of a robot or is grasped with a hand of the robot, and said robot is operated for positioning to a plurality of different image pickup positions and directions, so that the image data respectively obtained at each of said image pickup positions and direction information indicating the respective different direction, is stored as a teaching model.

9. A method of automatic orientation recognition, comprising:

generating and storing a set of images of different relative orientations of a subject, the images having been captured by a plurality of robotic operations corresponding to the different relative orientations of the subject, and associating with each image information indicating its respective relative orientation of the subject;

after the generating and storing, from a known current orientation of a robot, capturing a current image of a workpiece that has an unknown orientation relative to an image pickup device on the robot before the robot has come into contact with the workpiece, where the workpiece has a shape substantially similar to the shape of the subject;

after the capturing, using pattern matching to match one of the stored images with the current image; and

after the pattern matching, and before the robot has come into contact with the workpiece, determining the orientation of the workpiece relative to the image pickup device on the robot based on the relative orientation information associated with the matched stored image, and also based on the known current orientation of the robot.

10. A method according to claim 9, further comprising automatically maneuvering the robot to the workpiece based at least on the determined orientation of the workpiece relative to the robot.

11. A method according to claim 9, wherein the generating and storing is performed for a plurality of differently shaped subjects, wherein the current image includes a plurality of differently shaped workpieces, and wherein the pattern matching further comprises identifying the workpiece from among the plurality of differently shaped workpieces using the images and orientation information of the plurality of differently shaped subjects.

12. A method of automatic orientation recognition, comprising:

generating and storing a set of images of different relative arrangements of a subject, the images having been captured by a plurality of robotic operations corresponding to the different relative arrangements, and associating with each image information indicating its respective relative arrangement of the subject;

with a known current arrangement of an image pickup device on a robot, capturing a current image of a workpiece with an unknown current arrangement relative to the robot, where the workpiece has a shape substantially similar to the shape of the subject;

using pattern matching to match one of the stored images with the current image; and

determining the current orientation of the workpiece relative to the image pickup device on the robot based on the relative arrangement information associated with the matched stored image, and also based on the known current arrangement of the image pickup device on the robot.

13. A method, comprising:

robotically taking images of a subject with different three-dimensional subject-camera arrangements that vary in three dimensions, and associating with each image or data thereof information indicating its subject-camera arrangement;

then

taking a current image of a workpiece shaped like the subject; and

then before picking up the workpiece

determining a current workpiece-camera orientation by matching one of the images or data thereof with the current image, and using predetermined subject-camera arrangement information of the matched image to determine the three-dimensional orientation of the workpiece relative to the camera.

14. At least one computer-readable medium storing instructions that when executed control a robot apparatus to perform a method comprising:

robotically taking images of a subject with a camera in different subject-camera arrangements varying in three dimensions;

associating with each image, information indicating an obtaining subject-camera arrangement used in obtaining the image;

after said associating of the images, taking a current image of a workpiece shaped like the subject;

determining a current workpiece-camera orientation by matching one of the images with the current image and using the information indicating the obtaining subject-camera arrangement associated with the one of the images; and

picking up the workpiece after said determining.