Gray scale vision method and system utilizing same

• US 4,707,647 A
• Filed: 05/19/1986
• Issued: 11/17/1987
• Est. Priority Date: 05/19/1986
• Status: Expired due to Term

• Current Assignee: Fanuc Robotics North America Inc (Fanuc Corporation)
• Original Assignee: GMF Robotics Corporation (Fanuc Corporation)
• Inventors: Coldren, Bruce E., Krause, Kenneth W.
• Primary Examiner(s): Shoop, Jr., William M.
• Assistant Examiner(s): Keane, Patrick C.
• Application Number:
  US06/864,416
• Time in Patent Office: 547 Days
• Field of Search: 364/513, 364/474, 364/478, 318/568, 318/640, 318/480, 358/903, 358/101, 358/139, 356/390, 382/28, 382/15, 382/14, 382/16, 382/18, 382/19, 382/22, 382/25, 382/42, 901/46, 901/47
• US Class Current: 382/151
• CPC Class Codes: B25J 9/1697 Vision controlled systems
• CPC Class Codes: G06T 1/0014 Image feed-back for automatic industrial control, e.g. robot with camera robots B25J19/023
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20061 Hough transform
• CPC Class Codes: G06T 2207/20192 Edge enhancement; Edge preservation
• CPC Class Codes: G06T 7/12 Edge-based segmentation
• CPC Class Codes: G06V 10/48 by mapping characteristic values of the pattern into a parameter space, e.g. Hough transformation
• CPC Class Codes: G06V 10/955 using specific electronic processors

Abstract

A method and system are provided for automatically locating an object at a vision station by performing an edge-detecting algorithm on at least a portion of the gray-scale digitized image of the object. Preferably, the algorithm comprises an implementation of the Hough transform which includes the iterative application of a direction-sensitive, edge-detecting convolution to the digital image. Each convolution is applied with a different convolution mask or filter, each of which is calculated to give maximum response to an edge of the object in a different direction. The method and system have the ability to extract edges from low contrast images. Also, preferably, a systolic array processor applies the convolutions. The implementation of the Hough transform also includes the steps of shifting the resulting edge-enhanced images by certain amounts in the horizontal and vertical directions, summing the shifted images together into an accumulator buffer to obtain an accumulator image and detecting the maximum response in the accumulator image which corresponds to the location of an edge. If the object to be found is permitted to rotate, at least one other feature, such as another edge, must be located in order to specify the location and orientation of the object. The location of the object when correlated with the nominal position of the object at the vision station provides the position and attitude of the object. The resultant data may be subsequently transformed into the coordinate frame of a peripheral device, such as a robot, programmable controller, numerical controlled machine, etc. for subsequent use by a controller of the peripheral device.

Claims

1. Method for automatically locating an object at a vision station, said method comprising the steps of:

generating reference data relating to at least two features of an ideal object, the data including at least two edge points, one of said points being on each of a pair of non-parallel edge segments wherein the reference data includes directional data for each of the edge points;

generating a gray-scale digital image containing the objects to be located at the vision station;

processing the reference data and the ditital image together to obtain an accumulator image, said processing step including the steps of performing an edge-detecting convolution on at least a portion of the digital image for each edge point and shifting the convoluted data by an amount and in a direction related to the directional data for each of said edge points to obtain a shifted image; and

determining the location of at least one localized bright region in the accumulator image, the location of said region corresponding to the location of one of the features within the digital image.

3. The method as claimed in claim 1 or claim 2 wherein said processing step includes the step of summing the shifted convoluted data to obtain the accumulator image.

4. The method as claimed in claim 3 including the step of scaling the gray levels of the shifted image by an amount related to the number of edge points prior to said step of summing.

5. The method as claimed in claim 1 wherein the digital image is generated by a camera and further comprising the step of correlating the location of the features within the digital image to an offset of the object from the camera.

6. The method as claimed in claim 5 further comprising the steps of generating calibration data relating the coordinate frame of the camera to the coordinate frame of a peripheral device located at the vision station and transforming the offset of the object from the camera to the device frame.

7. The method as claimed in claim 1 or claim 2 wherein said edge segments are defined by entries in an R-table having a predetermined reference point and the edge points and wherein for each edge point entry there is defined a local gradient direction and a vector to the reference point.

2. Method of automatically generating offset data for use by a programmed robot controlled to enable a robot controlled by the controller to move to an object at the vision station including a camera, the offset data relating the position of the object to the coordinate frame of the robot, the method comprising the step of:

generating calibration data relating the camera to the coordinate frame of the robot;

generating reference data relating to at least two features of an ideal object, the data including at least two edge points one of said points being on each of a pair of non-parallel edge segments wherein the reference data includes directional data for each of the edge points;

generating a gray-scale digital image containing the object to be located at the vision station;

processing the reference data and the digital image together to obtain an accumulator image, said processing step including the steps of performing an edge-detecting convolution on at least a portion of the digital image for each edge point and shifting the convoluted data by an amount and in a direction related to the directional data for each of said edge points to obtain a shifted image;

determining the location of at least one localized bright region in the accumulator image, the location of said region corresponding to the location of one of the features within the digital image;

correlating the location of the features within the digital image of an offset of the object from the camera; and

transforming the offset of the object from the camera to the robot frame.

8. A gray-scale vision system for automatically locating an object at a vision station comprising:

means for storing reference data relating to at least two features of an ideal object, the data including at least two edge points one of said edge points being on each of a pair of non-parallel edge segments wherein the reference data includes directional data for each of the edge points;

means for generating a gray-scale digital image containing the object to be located at the vision station, said means for generating including a television camera;

means for processing the reference data and the digital image together to obtain an accumulator image, said processing means including means for performing an edge-detecting convolution on at least a portion of the digital image for each edge point and shifting the convoluted data by an amount and in a direction related to the directional data for each of said edge points to obtain a shifted image; and

means for determining the location of at least one localized bright region in the accumulator image, the location of said region corresponding to the location of one of the features within the digital image.

9. The system as claimed in claim 8 further comprising means for correlating the location of the features within the digital image to an offset of the object from the camera.

10. A system for automatically generating offset data for use by a programmed robot controller to enable a robot controlled by the controller to move to an object at a vision station, the offset data relating the position of the object to the coordinate frame of the robot, the system comprising:

means for storing reference data relating to at least two features of an ideal object, the data including at least two edge points, one of said edge points being on each of a pair of non-parallel edge segments wherein the reference data includes directional data for each of the edge points;

means for generating a gray-scale digital image containing the object to be located at the vision station, said means for generating including a television camera;

means for storing calibration data relating the camera to the coordinate frame of the robot;

means for processing the reference data and the digital image together to obtain an accumulator image, said processing means including means for performing an edge detecting convolution on at least a portion of the digital image for each edge point and shifting the convoluted data by an amount and in a direction related to the directional data for each of said edge points to obtain a shifted image;

means for determining the location of at least one localized bright region in the accumulator image, the location of said region corresponding to the location of one of the features within the digital image;

means for correlating the location of the features within the digital image to an offset of the object from the camera; and

means for transforming the offset of the object from the camera to the robot frame.

11. The system as claimed in claims 8, 9 or 10 wherein said means for processing the reference data and the digital data includes a gray scale processor for performing each of the convolutions.

12. The system as claimed in claim 11 wherein at least one of said convolution is direction-sensitive and comprises a two-dimensional convolution of the digital image.

13. The system as claimed in claim 12 wherein said gray scale processor comprises a systolic array processor.