Robot system for recognizing three dimensional shapes

• US 4,575,304 A
• Filed: 04/07/1983
• Issued: 03/11/1986
• Est. Priority Date: 04/07/1982
• Status: Expired due to Fees

• Current Assignee: Hitachi, Ltd.
• Original Assignee: Hitachi, Ltd.
• Inventors: Ninomiya, Takanori, Nakagawa, Yasuo
• Primary Examiner(s): Spar, Robert J.
• Assistant Examiner(s): Williamson, Dennis
• Application Number:
  US06/482,730
• Time in Patent Office: 1,069 Days
• Field of Search: 901/47, 414/730
• US Class Current: 414/730
• CPC Class Codes: B25J 19/023 including video camera means
• CPC Class Codes: B25J 9/1015 using additional, e.g. microadjustment of the end effector
• CPC Class Codes: G06T 1/0014 Image feed-back for automatic industrial control, e.g. robot with camera robots B25J19/023
• CPC Class Codes: G06V 10/10 Image acquisition document image scanning and transmission H04N1/00; control of digital cameras H04N23/60
• CPC Class Codes: G06V 10/145 Illumination specially adapted for pattern recognition, e.g. using gratings
• CPC Class Codes: G06V 10/20 Image preprocessing
• CPC Class Codes: G06V 10/24 Aligning, centring, orientation detection or correction of the image
• CPC Class Codes: G06V 10/40 Extraction of image or video features
• CPC Class Codes: G06V 10/42 Global feature extraction by analysis of the whole pattern, e.g. using frequency domain transformations or autocorrelation
• CPC Class Codes: G06V 20/64 Three-dimensional objects

Abstract

The present invention consists in a robot wherein a slit-light detector which is constructed of a light projector for projecting light so as to form a slit-light on an object to-be-handled and an image detector for detecting an image of the light-section waveform, is mounted on an operating member of the robot; means for scanning the slit-light detector is disposed; and the slit-light detector detects range data, on the basis of which a position, posture, inclination etc. of the object in a three-dimensional shape are detected, so that the robot can handle the object.

Claims

1. A robot system for operating upon an object, comprising:

a robot having an operating member movable with at least three degrees of freedom;

robot control means for moving the operating member of the robot according to a program;

a three-dimensional shape detection device mounted on the operating member of the robot, said three-dimensional shape detection device comprising a light projector which projects light so as to form a slit-light on the object, an image detector which projects in two dimensions an image of the light-segment projected by said light projector, and a scanning device which scans at least the slit-light in a predetermined direction;

circuit means for extracting a range data signal representing the image of the light-segments from the image signal detected by the image detector in accordance with scanning performed by said scanning device;

an image processor for visually recognizing the three-dimensional shape of the object including the position and the inclination, said image processor comprising differentiation means for differentiating the range data signal obtained by said circuit means to form a jump edge image signal representing a jump edge of the object, separation means for separating the range data into a region closed with the jump edge, extraction means for extracting a necessary region from the separated region and then extracting the range data of a major plane from the extracted region, and detection means for detecting a position of a center of gravity and an inclination of the major plane on the basis of the range data; and

means for modifying said program to thereby modify the movement of said operating member of the robot in response to said recognized three-dimensional shape of the object obtained by said image processor so as to be able to operate the object with said operating member of the robot.

2. A robot system according to claim 1, wherein said scanning device of said three-dimensional shape detection device is so constructed as to scan the slit-light in parallel.

3. A robot system according to claim 2, wherein said scanning device is so constructed as to integrate said light projector and said image detector therewith and to scan the slit-light image detector therewith and to scan the slit-light rectilinearly.

4. A robot system according to claim 3, wherein said robot operates with at least three degrees of freedom, and said three-dimensional shape detection device is mounted on a fore end of the member to operate with at least three degrees of freedom.

5. A robot system according to claim 4, wherein said means for moving moves said at least one of said detector means and said projector means relative to said portion of said arm on which it is mounted during the object scan.

6. A robot system according to claim 4, wherein said circuit means further provides said three-dimensional representation of the object in the coordinate system of the program employed by said control means, and said means for modifying modifies the angular position of the outer end of said manipulator arm in accordance with the determined slope of the outer edges of the object.

7. A robot system according to claim 4, wherein said circuit means further determines roof edges between the outer edges as plots of abrupt intensity changes of said plurality of image signals in the area between said outer edges.

8. A robot system according to claim 7, wherein said means for moving moves said at least one of said detector means and said projector means relative to said portion of said arm on which it is mounted during the object scan.

9. A robot system according to claim 2, wherein said robot operates with at least three degrees of freedom, and said three-dimensional shape detection device is mounted on a fore end of the member to operate with at least three degrees of freedom.

10. A robot system, according to claim 9, wherein said image signal has an intensity corresponding to the intensity of the reflected beam from the line intersection between the plane of said projected beam and the object;

said circuit means scans said line image signal along its length and produces a dimensional representation of the line on said object by determining a jump edge position on the line corresponding to the change from background reflected radiation to object reflected radiation;

said means for moving provides a scan wherein said line of intersection is moved perpendicular to itself to produce a plurality of line images;

said circuit means determines outer edges of said object from the plot of said jump edge representations, and determines the slope of said outer edges in accordance with the change in detected intensity of the jump edge representations along the outer edges.

11. A robot system according to claim 9, wherein said means for moving moves said at least one of said detector means and said projector means relative to said portion of said arm on which it is mounted during the object scan.

12. A robot system according to claim 9, wherein said circuit means provides said three-dimensional representation in the same coordinate system as said program;

and said means for modifying said program changes the angular relationship of said outer end of said manipulator arm in accordance with the posture of the object and further changes the program in accordance with the position of the object.

13. A robot system according to claim 1, wherein said robot operates with at least three degrees of freedom, and said three-dimensional shape detection device is mounted on a fore end of the operating member to operate with at least three degrees of freedom.

14. A robot system according to claim 1, wherein said robot control means is operatively responsive to information generated by said image processor, and wherein said image processor recognizes the three-dimensional shape including the position and inclination of the object, and that information of the position and inclination of the object obtained from said image processor is fed back to said robot control means and is used in said program, so that said robot can handle the object.

15. A robot system for operation upon an object, comprising:

a robot having an operating member movable with at least three degrees of freedom;

robot control means for moving the operating member of the robot according to a program;

a three-dimension shape detection device mounted on the operating member of the robot, said three-dimension shape detection device comprising a light projector which projects light so as to form a slit-light on the object, an image detector which projects in two dimensions an image of the light-segment projected by said light projector, and means for scanning the slit-light by moving the operating member of said robot in a predetermined direction whereby the slit-light is detected by said image detector as range data, so that a three-dimensional shape of the object can be visually recognized;

circuit means for extracting a range data signal representing the image of the light-segments from the image signal detected by the image detector in accordance with scanning performed by said scanning device;

an image processor for visually recognizing the three-dimensional shape of the object including the position and the inclination, said image processor comprising differentiation means for differentiating the range data signal obtained by said signal means to form a jump edge image signal representing a jump edge of the object, separation means for separating the range data into a region closed with the jump edge, extraction means for extracting a necessary region from the separated region and then extracting the range data of a major plane from the extracted region, and detection means for detecting a position of a center of gravity and an inclination of the major plane on the basis of the range data; and

means for modifying said program to thereby modify the movement of said operating member of the robot in response to said recognized three-dimensional shape of the object obtained by said image processor so as to be able to operate the object with said operating member of the robot.

16. A robot system according to claim 15, wherein said robot is constructed with at least three degrees of freedom, and that said shape detection device is mounted on the member to operate with at least three degrees of freedom.

17. A robot system according to claim 15, wherein said robot control means is operatively responsive to information generated by said image processor, and wherein said image processor visually recognizes a position and an inclination of the object, and the information of the position and inclination of the object obtained from said image processor is fed back to said robot control means and is used in said program, so that said robot can handle the object.

18. A robot system for operating upon a object comprising:

a base;

a manipulatable arm having means as its outer end for providing a desired function and to move along a desired path, means mounting the other end of said arm to said base, and means providing at least three degrees of freedom for the outer end of said arm relative to said base;

control means for moving said arm according to a program;

three-dimensional shape detector means mounted near the fore part of said arm on a portion of said arm that has at least three degrees of freedom relative to said base for movement of said detector means with said portion in accordance with the program, said shape detector means including projector means for projecting a beam of radiant energy to form a slit-light pattern on an object whose shape is to be detected, image detector means for receiving the radiant energy reflected from said object to produce in two dimensions a corresponding image signal of the light segment projected by said projector means, and means for moving at least one of said detector means and said projector means relative to the object to scan said object and produce a plurality of different image signals of the object;

circuit means for extracting a range data signal representing the image of the light-segments from the image signal detected by the image detector in accordance with scanning performed by said scanning device;

an image processor for visually recognizing the three-dimensional shape of the object, comprising differentiation means for differentiating the range data signal obtained by said circuit means to form a jump edge image signal representing a jump edge of the object, separation means for separating the range data into a region closed with the jump edge, extraction means for extracting a necessary region from the separated region and then extracting the range data of a major plane from the extracted region, and detection means for detecting a position of a center of gravity and an inclination of the major plane on the basis of the range data; and

means for modifying said program to thereby modify the movement of said robot arm in response to said detected three-dimensional shape of the object.