Image processing device

• US 20040190766A1
• Filed: 03/24/2004
• Published: 09/30/2004
• Est. Priority Date: 03/25/2003
• Status: Abandoned Application

• Current Assignee: Fanuc Ltd (Fanuc Corporation)
• Original Assignee: Fanuc Ltd (Fanuc Corporation)
• Inventors: Warashina, Fumikazu, Watanabe, Atsushi, Yamada, Makoto
• Application Number:
  US10/807,259
• Publication Number:
  US 20040190766A1
• Time in Patent Office: Days
• US Class Current: 382/154
• CPC Class Codes: B25J 9/1697 Vision controlled systems
• CPC Class Codes: G05B 2219/37555 Camera detects orientation, position workpiece, points of workpiece
• CPC Class Codes: G05B 2219/40053 Pick 3-D object from pile of objects
• CPC Class Codes: G06T 2207/30164 Workpiece; Machine component
• CPC Class Codes: G06T 3/00 Geometric image transformations in the plane of the image
• CPC Class Codes: G06T 7/75 involving models

Abstract

An image processing device for detecting an object whose position and orientation are unknown and for recognizing three dimensional position and/or orientation of the object. A model pattern used for a pattern matching is stored and subject to N geometrical transformations. After initial setting of an index i that specifies the i-th geometrical transformation, the i-th transformed model pattern is prepared, and, using this pattern, a pattern matching is performed. A local maximum point having a similarity equal to or higher than a preset value is searched for. The image coordinate of such a point, if any, is extracted and stored together with information on a three dimensional relative orientation used for the preparation of the transformed model pattern concerned. Based on the information on the three dimensional relative orientation corresponding to the pattern having the best similarity, the three dimensional position and/or orientation is recognized.

Claims

1. An image processing device for determining three-dimensional position and/or orientation of an object, comprising:

image data capturing means for capturing image data containing an image of the object;

model pattern creating means for creating a model pattern based on image data of a reference object with a reference orientation relatively to said image capturing means captured by said image capturing means, said reference object having a shape substantially identical to that of the object;

transformation means for performing two-dimensional and geometrical transformation of the created model pattern to generate a transformed model pattern representing an image of the object with an orientation different from the reference orientation;

pattern matching means for performing a pattern matching of the image data of the object captured by said image capturing means with the transformed model pattern;

selecting means for repeatedly performing the generation of a transformed model pattern and the pattern matching of the image data of the object with the transformed model pattern to thereby select one of the transformed model patterns in conformity with the image data of the object, and obtain information on a position of the image of the object in the image data; and

determining means for determining three-dimensional position and/or orientation of the object based on the information on the position of the image of the object in the image data and information on the orientation of the selected one of the transformed model patterns.

2. An image processing device for determining three-dimensional position and/or orientation of an object, comprising:

image data capturing means for capturing image data containing an image of the object;

model creating means for creating a model pattern based on image data of a reference object with a reference orientation relative to said image -data capturing means captured by said image data capturing means, said reference object having a shape substantially identical to that of the object;

transformation means for performing two-dimensional and geometrical transformation of the created model pattern to generate a plurality of transformed model patterns each representing an image of the object with an orientation different from the reference position;

storage means for storing the plurality of transformed model patterns and information on orientations of the respective transformed model patterns;

pattern matching means for performing pattern matching of the image data of the object captured by said image capturing means with the plurality of transformed model patterns to thereby select one of the transformed model patterns in conformity with the image data of the object, and obtain information on a position of the image of the object in the image data; and

determining means for determining three-dimensional position and/or orientation of the object based on information on the position of the image of the object in the image data and the information on an orientation of the selected one of the transformed model patterns.

3. An image processing device according to claim 1 or 2, wherein said transformation means performs the two-dimensional and geometrical transformation of an affine transformation, and said image processing device further comprises additional measuring means for obtaining a sign of inclination of the object with respect to said image capturing means.

4. An image processing device according to claim 3, wherein said additional measuring means performs dividing of a model pattern into at least two partial model patterns which are subject to the affine transformation to generate transformed partial model patterns, and pattern matching of the image data of the object with the transformed partial model patterns to determine most conformable sizes, and determines the sign of the inclination based on comparison of the sizes of the conformable partial model patterns with each other.

5. An image processing device according to claim 3, wherein said additional measuring means performs measurement of distances from a displacement sensor separately provided in the vicinity of said image capturing means to at least two points on the object using the displacement sensor, and determines the sign of the inclination based on comparison of the measured distances.

6. An image processing device according to claim 3, wherein said additional measuring means performs additional pattern matching of image data of the object captured after said image data capturing means is slightly moved or inclined and determines the sign of the inclination based on judgment whether an inclination of image of the object becomes larger or smaller than the selected one of the transformed model patterns.

7. An image processing device according to claim 1, wherein the image processing device is incorporated into a robot system comprising:

storage means storing an operating orientation of the robot relative to the object or storing an operating orientation and an operating position of the robot relative to the object; and

robot control means for determining an operating orientation of the robot or the operating orientation and an operating position of the robot based on the determined three-dimensional position and/or orientation of the object.

8. An image processing device according to claim 7, wherein said image capturing means is mounted on the robot.