Fast high-accuracy multi-dimensional pattern inspection

• US 7,065,262 B1
• Filed: 11/10/2003
• Issued: 06/20/2006
• Est. Priority Date: 11/26/1997
• Status: Expired due to Term

• Current Assignee: Cognex Corporation
• Original Assignee: Cognex Corporation
• Inventors: Wallack, Aaron, Silver, William, Wagman, Adam
• Primary Examiner(s): MARIAM, DANIEL G
• Application Number:
  US10/705,563
• Time in Patent Office: 953 Days
• Field of Search: 382/103, 382/151, 382/195, 382/199, 382/203, 382/209, 382/291, 382/299, 348/169, 358/1.5
• US Class Current: 382/291
• CPC Class Codes: G06T 7/75 involving models
• CPC Class Codes: G06V 10/24 Aligning, centring, orientation detection or correction of the image
• CPC Class Codes: G06V 10/7515 Shifting the patterns to accommodate for positional errors

Abstract

A method and apparatus are provided for identifying differences between a stored pattern and a matching image subset, where variations in pattern position, orientation, and size do not give rise to false differences. The invention is also a system for analyzing an object image with respect to a model pattern so as to detect flaws in the object image. The system includes extracting pattern features from the model pattern; generating a vector-valued function using the pattern features to provide a pattern field; extracting image features from the object image; evaluating each image feature, using the pattern field and an n-dimensional transformation that associates image features with pattern features, so as to determine at least one associated feature characteristic; and using at least one feature characteristic to identify at least one flaw in the object image. The invention can find at least two distinct kinds of flaws: missing features, and extra features. The invention provides pattern inspection that is faster and more accurate than any known prior art method by using a stored pattern that represents an ideal example of the object to be found and inspected, and that can be translated, rotated, and scaled to arbitrary precision much faster than digital image re-sampling, and without pixel grid quantization errors. Furthermore, since the invention does not use digital image re-sampling, there are no pixel quantization errors to cause false differences between the pattern and image that can limit inspection performance.

Claims

1. A geometric pattern matching method for refining an estimate of a true pose of an object in a run-time image, the method comprising:

generating a low-resolution model pattern using a training image, the low-resolution model pattern including a geometric description of the expected shape of the object at a low spatial resolution, each geometric description including a list of pattern boundary points;

generating a high-resolution model pattern using the training image, the high-resolution model pattern including a geometric description of the expected shape of the object at a high spatial resolution, each geometric description including a list of pattern boundary points;

receiving a starting pose, the starting pose representing an initial estimate of the true pose of the object in the run-time image;

receiving a run-time image;

using the low-resolution model pattern, and the starting pose, analyzing the run-time image so as to provide a low-resolution pose that is a more refined estimate of the true pose than the starting pose; and

using the high-resolution model pattern, and the low-resolution pose, analyzing the run-time image so as to provide a high-resolution pose that is a more refined estimate of the true pose than the low-resolution pose.

2. The method of claim 1, wherein analyzing the run-time image so as to provide a low-resolution pose includes:

producing a low-resolution error value;

producing a low-resolution aggregate clutter value; and

producing a low-resolution aggregate coverage value.

3. The method of claim 1, wherein the low-resolution error value is a low-resolution root-mean-squares error value.

4. The method of claim 2, after analyzing the run-time image so as to provide a low-resolution pose, and before analyzing the run-time image so as to provide a high-resolution pose, further comprising:

examining the low-resolution error value, the low-resolution aggregate clutter value, and the low-resolution aggregate coverage value; and

forgoing analyzing the run-time image so as to provide a high-resolution pose if these low-resolution values do not indicate an acceptable match between the run-time image and the low-resolution model pattern.

5. The method of claim 2, after analyzing the run-time image so as to provide a low-resolution pose, and instead of analyzing the run-time image so as to provide a high-resolution pose, further comprising:

examining the low-resolution error value, the low-resolution aggregate clutter value, and the low-resolution aggregate coverage value; and

analyzing the run-time image so as to provide a high-resolution pose if these low-resolution values indicate an acceptable match between the run-time image and the low-resolution model pattern.

6. The method of claim 5, further comprising:

examining the high-resolution error value, the high-resolution aggregate clutter value, and the high-resolution aggregate coverage value; and

advising a user that the run-time image is out of focus, if these high-resolution values do not indicate an acceptable match between the run-time image and the high-resolution model pattern.

7. The method of claim 5, further comprising:

examining the high-resolution error value, the high-resolution aggregate clutter value, and the high-resolution aggregate coverage value; and

using the results of analyzing the run-time image so as to provide a low-resolution pose, if these high-resolution values do not indicate an acceptable match between the run-time image and the high-resolution model pattern.

8. The method of claim 1, wherein analyzing the run-time image so as to provide a high-resolution pose includes:

producing a high-resolution error value;

producing a high-resolution aggregate clutter value; and

producing a high-resolution aggregate coverage value.

9. The method of claim 8, wherein the high-resolution error value is a high-resolution root-mean-squares error value.

10. The method of claim 1, further comprising receiving a coordinate transformation that maps points in an orthonormal coordinate system to points in the run-time image, wherein the coordinate transformation is used in analyzing the run-time image so as to provide a low-resolution pose.

11. The method of claim 1, further comprising receiving a coordinate transformation that maps points in an orthonormal coordinate system to points in the run-time image, wherein the coordinate transformation is used in analyzing the run-time image so as to provide a high-resolution pose.

12. The method of claim 1, wherein generating low-resolution model pattern using a training image includes:

low-pass filtering and image sub-sampling so as to attenuate fine detail in the training image, thereby providing a low-resolution model pattern.

13. The method of claim 1, wherein generating high-resolution model pattern using the training image includes:

low-pass filtering and image sub-sampling so as to pass fine detail in the training image, thereby providing a high-resolution model pattern.

14. The method of claim 1, wherein analyzing the run-time image so as to provide a high-resolution pose includes:

providing an evaluated pattern boundary point list that identifies boundary points in the high-resolution model pattern that are not present in the run-time image.

15. The method of claim 1, wherein analyzing the run-time image so as to provide a high-resolution pose includes:

providing an evaluated image boundary point list that identifies boundary points in the run-time image that are not present in the high-resolution model pattern.

16. The method of claim 1,

wherein analyzing the run-time image so as to provide a low-resolution pose includes:

producing a low-resolution aggregate clutter value, and a low-resolution aggregate coverage value; and

wherein analyzing the run-time image so as to provide a high-resolution pose includes;

producing a high-resolution aggregate clutter value, and a high-resolution aggregate coverage value;

the method further including;

computing a low-resolution overall match score that is equal to the low-resolution aggregate coverage value minus half the low-resolution aggregate clutter value;

computing a high-resolution overall match score that is equal to the high-resolution aggregate coverage value minus half the high-resolution aggregate clutter value; and

using the results of analyzing the run-time image so as to provide a low-resolution pose, instead of using the results of analyzing the run-time image so as to provide high resolution pose, if the high-resolution overall match score is less than a particular fraction of the low-resolution overall match score.

17. The method of claim 16, wherein the particular fraction is equal to 0.9.

18. The method of claim 1, wherein generating a low-resolution model pattern using a training image includes:

sub-sampling by an equal amount in x and y, the amount being the largest integer that is not greater than sqrt((sqrt(wh)/8)), where w and h are width and height of the training image.

19. The method of claim 18, wherein generating a low-resolution model pattern using a training image further includes:

low-pass filtering using a filter size parameter equal to one less than the sub-sampling amount.

20. The method of claim 18, wherein generating a low-resolution model pattern using a training image further includes:

multiplying gradient magnitude values by 2.0.

21. The method of claim 1, wherein generating the high-resolution model pattern using the training image includes:

low-pass filtering and image sub-sampling so as to pass the training image unmodified.

22. The method of claim 1, wherein generating the low-resolution model pattern using the training image includes:

setting a default noise threshold for a peak detector used to create the low-resolution model pattern by using a contrast value that is the median gradient magnitude of the pixels in the training image.

23. The method of claim 1, wherein generating the high-resolution model pattern using the training image includes:

setting a default noise threshold for a peak detector used to create the high-resolution model pattern by using a contrast value that is the median gradient magnitude of the pixels in the training image.

24. The method of claim 1, wherein generating the low-resolution model pattern using the training image includes:

setting a noise threshold for a peak detector used to create the low-resolution model pattern by using a contrast value that is equal to 10 gray levels.

25. The method of claim 1, wherein generating the high-resolution model pattern using the training image includes:

setting a noise threshold for a peak detector used to create the high-resolution model pattern by using a contrast value that is one quarter of a saved train-time contrast value.

26. The method of claim 1, wherein analyzing the run-time image so as to provide a low-resolution pose includes:

using a noise threshold for a peak detector used to create the low-resolution model pattern by using a contrast value that is equal to 10 gray levels, multiplied by the ratio of run-time contrast to a saved train-time contrast.

27. The method of claim 1, wherein analyzing the run-time image so as to provide a high-resolution pose includes:

using a noise threshold for a peak detector used to create the high-resolution model pattern by using a contrast value that is one quarter of a saved train-time contrast value, multiplied by the ratio of run-time contrast to a saved train-time contrast.

28. The method of claim 1, wherein the starting pose is a coordinate transform that includes non-translational degrees of freedom.

29. The method of claim 1, wherein the a high-resolution pose is a coordinate transform that includes non-translational degrees of freedom.

30. A geometric pattern matching method for refining an estimate of a true pose of an object in a run-time image, the method comprising:

storing a plurality of model patterns, each model pattern including a geometric description of the expected shape of the object at a respective spatial resolution, each geometric description including a list of pattern boundary points;

providing a starting pose that represents an initial estimate of the true pose of the object in the run-time image;

using the starting pose and a lowest-resolution model pattern to determine an intermediate estimate of the true pose of the object in the run-time image;

using the intermediate estimate of the true pose from the previous step, and the next-higher resolution stored model pattern, to determine a further refined intermediate estimate of the true pose of the object in the run-time image; and

using a further refined intermediate estimate of the true pose as the final estimate of the true pose of the object in the run-time image.

31. The method of claim 30, wherein using the starting pose and a lowest-resolution model pattern to determine an intermediate pose includes:

producing a intermediate-resolution error value;

producing a intermediate-resolution aggregate clutter value; and

producing a intermediate-resolution aggregate coverage value.

32. The method of claim 31, wherein the intermediate-resolution error value is a intermediate-resolution root-mean-squares error value.

33. The method of claim 32, further comprising:

examining the final-resolution error value, the final-resolution aggregate clutter value, and the final-resolution aggregate coverage value; and

using the results of determining an intermediate estimate of the true pose, if these final-resolution values do not indicate an acceptable match between the run-time image and the final-resolution model pattern.

34. The method of claim 30, wherein using a further refined intermediate estimate of the true pose as the final estimate of the true pose includes:

producing a final-resolution error value;

producing a final-resolution aggregate clutter value; and

producing a final-resolution aggregate coverage value.

35. The method of claim 34, wherein the final-resolution error value is a final-resolution root-mean-squares error value.

36. The method of claim 34, further comprising:

examining the final-resolution error value, the final-resolution aggregate clutter value, and the final-resolution aggregate coverage value; and

advising a user that the run-time image is out of focus, if these final-resolution values do not indicate an acceptable match between the run-time image and the final-resolution model pattern.

37. The method of claim 30, further comprising receiving a coordinate transformation that maps points in an orthonormal coordinate system to points in the run-time image, wherein the coordinate transformation is used to determine an intermediate estimate of the true pose.

38. The method of claim 30, further comprising receiving a coordinate transformation that maps points in an orthonormal coordinate system to points in the run-time image, wherein the coordinate transformation is used to a further refined intermediate estimate of the true pose.

39. The method of claim 30, wherein storing a plurality of model patterns includes:

low-pass filtering and image sub-sampling a different amount for each model pattern so as to provide a plurality of respective spatial resolutions, and so as to attenuate detail differently in each model pattern of the plurality of model patterns.

40. The method of claim 30, wherein using a further refined intermediate estimate of the true pose as the final estimate of the true pose includes:

providing an evaluated pattern boundary point list that identifies boundary points in the final-resolution model pattern that are not present in the run-time image.

41. The method of claim 30, wherein using a further refined intermediate estimate of the true pose as the final estimate of the true pose includes:

providing an evaluated image boundary point list that identifies boundary points in the run-time image that are not present in the final-resolution model pattern.

42. The method of claim 30,

wherein using the intermediate estimate of the true pose includes:

producing an intermediate-resolution aggregate clutter value, and an intermediate-resolution aggregate coverage value; and

wherein using a further refined intermediate estimate of the true pose as the final pose includes;

producing a final-resolution aggregate clutter value, and a final-resolution aggregate coverage value;

the method further including;

computing a intermediate-resolution overall match score that is equal to the intermediate-resolution aggregate coverage value minus half the intermediate-resolution aggregate clutter value;

computing a final-resolution overall match score that is equal to the final-resolution aggregate coverage value minus half the final-resolution aggregate clutter value; and

using the results of analyzing the run-time image so as to provide a intermediate-resolution pose, instead of using the results of analyzing the run-time image so as to provide final resolution pose, if the final-resolution overall match score is less than a particular fraction of the intermediate-resolution overall match score.

43. The method of claim 42, wherein the particular fraction is equal to 0.9.

44. The method of claim 30, wherein generating an intermediate-resolution model pattern using a training image includes:

sub-sampling by an equal amount in x and y, the amount being the largest integer that is not greater than sqrt((sqrt(wh)/8)), where w and h are width and height of the training image.

45. The method of claim 44, wherein generating an intermediate-resolution model pattern using a training image further includes:

low-pass filtering using a filter size parameter equal to one less than the sub-sampling amount.

46. The method of claim 44, wherein generating an intermediate-resolution model pattern using a training image further includes:

multiplying gradient magnitude values by 2.0.

47. The method of claim 30, wherein generating the final-resolution model pattern using the training image includes:

low-pass filtering and image sub-sampling so as to pass the training image unmodified.

48. The method of claim 30, wherein generating an intermediate-resolution model pattern using the training image includes:

setting a default noise threshold for a peak detector used to create the intermediate-resolution model pattern by using a contrast value that is the median gradient magnitude of the pixels in the training image.

49. The method of claim 30, wherein generating the final-resolution model pattern using the training image includes:

setting a default noise threshold for a peak detector used to create the final-resolution model pattern by using a contrast value that is the median gradient magnitude of the pixels in the training image.

50. The method of claim 30, wherein generating the intermediate-resolution model pattern using the training image includes:

setting a noise threshold for a peak detector used to create the intermediate-resolution model pattern by using a contrast value that is equal to 10 gray levels.

51. The method of claim 30, wherein generating the final-resolution model pattern using the training image includes:

setting a noise threshold for a peak detector used to create the final-resolution model pattern by using a contrast value that is one quarter of a saved train-time contrast value.

52. The method of claim 30, wherein determining a further refined intermediate estimate of the true pose includes:

using a noise threshold for a peak detector used to create the low-resolution model pattern by using a contrast value that is equal to 10 gray levels, multiplied by the ratio of run-time contrast to a saved train-time contrast.

53. The method of claim 30, wherein determining a further refined intermediate estimate of the true pose includes:

using a noise threshold for a peak detector used to create the high-resolution model pattern by using a contrast value that is one quarter of a saved train-time contrast value, multiplied by the ratio of run-time contrast to a saved train-time contrast.

54. The method of claim 30, wherein the starting pose is a coordinate transform that includes non-translational degrees of freedom.

55. The method of claim 30, wherein the a final-resolution pose is a coordinate transform that includes non-translational degrees of freedom.