System and method of using range image data with machine vision tools

• US 6,751,338 B1
• Filed: 12/15/2000
• Issued: 06/15/2004
• Est. Priority Date: 12/15/2000
• Status: Active Grant

• Current Assignee: Cognex Corporation
• Original Assignee: Cognex Corporation
• Inventors: Wallack, Aaron S.
• Primary Examiner(s): Au, Amelia M.
• Assistant Examiner(s): KIM, CHONG R
• Application Number:
  US09/738,962
• Time in Patent Office: 1,278 Days
• Field of Search: 382/103, 382/106, 382/141, 382/153, 382/154, 382/159, 382/181, 382/190, 382/289, 382/294, 356/3
• US Class Current: 382/106
• CPC Class Codes: G06T 7/30 Determination of transform parameters for the alignment of images, i.e. image registration
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06V 20/64 Three-dimensional objects

Abstract

A system and method for using a machine vision system to locate and register patterns in an object using range data is provided. The machine vision system includes an acquisition system for acquiring a range image of an object. The system also includes a machine vision search tool coupled to the acquisition system for locating an instance of a trained pattern in the image. The tool registering the trained pattern transformed by at least two translational degrees of freedom and at least one non-translational degree of freedom with respect to an image plane. The acquisition system preferably includes a three-dimensional camera.

Claims

1. A machine vision system comprising:

an acquisition system for acquiring a first range image of an object, the first range image including a plurality of range data, wherein each datum of the range data includes a range value and an associated confidence value;

a machine vision search tool coupled to the acquisition system for locating an instance of a trained pattern in the image, the tool registering the trained pattern transformed by at least two translational degrees of freedom and at least one non-translational degree of freedom with respect to an image plane; and

processing means to construct the image, including means for producing range values for the range image by adjusting each of the range values by applying the confidence value associated with the range value to be adjusted, and the confidence values and range values of neighboring portions of the first range image.

2. The system as set forth in claim 1, wherein the acquisition system includes a three-dimensional camera.

3. The system as set forth in claim 1, wherein the processing means includes means for preventing a portion of the range image from being considered by the machine vision search tools.

4. The system as set forth in claim 3, wherein the prevention means includes means for comparing the confidence value of each range datum to a threshold value and preventing the associated range value from being considered if the confidence value does not exceed the threshold.

5. A method for locating and registering a pattern in an image comprising:

acquiring a range image of an object with a camera having a camera axis and an image plane perpendicular to the camera axis, the range image comprising a plurality of range values, each of the range values relating to a distance between an associated position on a surface of the object and the camera;

acquiring a plurality of confidence values wherein each of the confidence values is associated with a respective one of the range values;

locating and registering a trained pattern in the range image with a machine vision search tool adapted to register transformation of the pattern by at least two translational degrees of freedom within a respective image plane and at least one non-translational degree of freedom comprising a respective axis perpendicular to the image plane; and

adjusting each of the range values to derive adjusted range values, so that each of the range values is adjusted based on the confidence values each associated respectively with based on range values and range values, and each of the associated confidence values, being associated with portions of the object'"'"'s surface within a given distance of the portion of the object'"'"'s surface that is associated with the respective range value being adjusted.

6. The method as set forth in claim 5, wherein the camera is a three-dimensional camera and the step of acquiring includes interconnecting the three-dimensional camera to the machine vision search tool.

7. The method as set forth in claim 5, wherein the step of acquiring includes determining which of adjacent range values associated with neighboring portions of the object'"'"'s surface should be considered by the machine vision search tool.

8. The method as set forth in claim 7, wherein the step of determining includes comparing each of the confidence values associated with the adjacent range values to a given threshold, and not considering the corresponding adjusted range values when the associated confidence values are less than the given threshold.

9. The method as set forth in claim 5, further comprising training the search tool to recognize the pattern including exposing the search tool to a training image containing the pattern and selecting a window that contains the pattern so as to cause the search tool to store a pattern of the selected window with the pattern therein.

10. A computer-readable medium executed on a computer for locating and registering a pattern in an image, the computer-readable medium comprising program instructions for performing the steps of:

acquiring a range image of an object with a camera having a camera axis and an image plane perpendicular to the axis, the range image comprising a plurality of range values, each of the range values relating to a distance between an associated position on a surface of the object and the camera;

acquiring a plurality of confidence values wherein each of the confidence values is associated with a respective one of the range values;

locating and registering a trained pattern in the range image with a machine vision search tool adapted to register transformation of the pattern by at least two translational degrees of freedom within a respective image plane and at least one non-translational degree of freedom comprising a respective camera axis perpendicular to the image plane; and

adjusting each of the range values to derive adjusted range values so that each of the range values is adjusted based on the confidence values each associated respectively with based on range values and range values, and each of the associated confidence values being associated with portions of the object'"'"'s surface within a given distance of the portion of the object'"'"'s surface that is associated with respective of the range value being adjusted.

11. The computer-readable medium as set forth in claim 10, wherein the acquisition tool is a three-dimensional camera and the step of acquiring includes interconnecting the three-dimensional camera to the machine vision search tool.

12. The computer-readable medium as set forth in claim 10, wherein the step of acquiring includes determining which of adjacent range values associated with neighboring portions of the object'"'"'s surface should be considered by the machine vision search tool.

13. The computer-readable medium as set forth in claim 12, wherein the step of determining includes comparing each of the confidence values associated with the adjacent the range values to a given threshold, and not considering the corresponding adjusted range values when the associated confidence values are less than the given threshold.

14. The computer-readable medium as set forth in claim 10, further comprising training the search tool to recognize the pattern including exposing the search tool to a training image containing the pattern and selecting a window that contains the pattern so as to cause the search tool to store a pattern of the selected window with the pattern therein.

15. A method for locating and registering a pattern in an image comprising:

acquiring a trained pattern range image of a first object at a first position with a camera having a camera axis and an image plane perpendicular to the camera axis, including acquiring a plurality of first range values and associated confidence values, each first range value relating to a distance between an associated portion on a surface of the first object and the camera;

acquiring a run-time range image of a second object at a second position with the camera having a camera axis and an image plane perpendicular to the camera axis, including acquiring a plurality of second range values and associated second confidence values, each of the second range values relating to a distance between an associated portion on a surface of the second object and the camera;

adjusting each of the second range values based upon the second confidence values associated with the second range values and based upon neighboring second range values and second confidence values associated with the neighboring second range values;

locating and registering an instance of the trained pattern range image in the run-time range image with a machine vision search tool adapted to register transformation of the pattern by at least two translational degrees of freedom and one non-translational degree of freedom comprising a respective axis perpendicular to the image plane; and

comparing the instance with the trained pattern range image to determine relative distance of the second object at the second position from the first object at the first position.

16. The method as set forth in claim 15 wherein the step of comparing includes, respectively, averaging the first range values and the portion of the second range values corresponding to the instance to obtain first and second averages, and comparing the averaged first range values and the averaged second range values to determine the relative distance.