Fast high-accuracy multi-dimensional pattern inspection

• US 7,043,081 B1
• Filed: 11/10/2003
• Issued: 05/09/2006
• Est. Priority Date: 11/26/1997
• Status: Expired due to Term

• Current Assignee: Cognex Corporation
• Original Assignee: Cognex Corporation
• Inventors: Wallack, Aaron, Silver, William, Wagman, Adam
• Primary Examiner(s): MARIAM, DANIEL G
• Application Number:
  US10/705,296
• Time in Patent Office: 911 Days
• Field of Search: 382/103, 382/149, 382/151, 382/159, 382/181, 382/190, 382/199, 382/203, 382/209, 382/216, 382/218, 382/219, 382/291, 348/135
• US Class Current: 382/209
• CPC Class Codes: G06T 7/75 involving models
• CPC Class Codes: G06V 10/24 Aligning, centring, orientation detection or correction of the image
• CPC Class Codes: G06V 10/7515 Shifting the patterns to accommodate for positional errors

Abstract

A method and apparatus are provided for identifying differences between a stored pattern and a matching image subset, where variations in pattern position, orientation, and size do not give rise to false differences. The invention is also a system for analyzing an object image with respect to a model pattern so as to detect flaws in the object image. The system includes extracting pattern features from the model pattern; generating a vector-valued function using the pattern features to provide a pattern field; extracting image features from the object image; evaluating each image feature, using the pattern field and an n-dimensional transformation that associates image features with pattern features, so as to determine at least one associated feature characteristic; and using at least one feature characteristic to identify at least one flaw in the object image. The invention can find at least two distinct kinds of flaws: missing features, and extra features. The invention provides pattern inspection that is faster and more accurate than any known prior art method by using a stored pattern that represents an ideal example of the object to be found and inspected, and that can be translated, rotated, and scaled to arbitrary precision much faster than digital image re-sampling, and without pixel grid quantization errors. Furthermore, since the invention does not use digital image re-sampling, there are no pixel quantization errors to cause false differences between the pattern and image that can limit inspection performance.

Claims

1. A sequential pose refinement module in a geometric pattern matching apparatus for refining a starting pose of an object in a run-time image, the object having an expected shape and a true pose in the run-time image, the starting pose representing an initial estimate of the true pose of the object in the run-time image, the geometric pattern matching apparatus having (1) a stored model pattern, the stored model pattern including a geometric description of the expected shape of the object, the geometric description including a plurality of pattern boundary points, and a vector-valued function of position within a region that includes the pattern boundary points, and (2) a feature detector configured to detect in the run-time image a plurality of image boundary points, the sequential pose refinement module comprising:

a motion transform generator configured to use a plurality of image boundary points, and the vector-valued function of position, so as to provide a motion transform; and

a compose module configured to receive a current pose and the motion transform so as to provide a refined pose, the refined pose representing a refined estimate of the true pose of the object in the run-time image.

2. The sequential pose refinement module of claim 1, wherein the compose module provides a sequence of pose refinements that are progressively closer to the true pose in the run-time image.

3. The sequential pose refinement module of claim 1, wherein the motion transform is a six-degree-of-freedom coordinate transform that maps the current pose into a more refined current pose.

4. The sequential pose refinement module of claim 1, wherein the motion transform is a perspective transform.

5. The sequential pose refinement module of claim 1, wherein the motion transform is a transform that corrects for lens distortion.

6. The sequential pose refinement module of claim 1, wherein the motion transform generator also provides an error value.

7. The sequential pose refinement module of claim 1, further comprising:

a normal tensor module configured to receive a current pose and a client map, so as to provide a normal tensor to the motion transform generator.

8. The sequential pose refinement module of claim 1, wherein the motion transform generator includes:

a map module adapted to map the position of each image boundary point from image coordinates to coordinates of the vector-valued function.

9. The sequential pose refinement module of claim 8, further including:

a vector-valued function module configured to provide a displacement vector of a nearest pattern boundary point using the position of the nearest pattern boundary point in coordinates of the vector-valued function.

10. The sequential pose refinement module of claim 9, the motion transform generator including:

a least squares module configured to use information provided by the vector-valued function module to determine a motion transform using a least squares fitting.

11. The sequential pose refinement module of claim 9, the motion transform generator including:

a data fitting module configured to use information provided by the vector-valued function module to determine a motion transform using a data fitting method.

12. The sequential pose refinement module of claim 9, further including:

a rotate module configured to determine the position of each image boundary point in displacement vector coordinates that include a component of the position in a direction of a displacement vector of a nearest pattern boundary point.

13. The sequential pose refinement module of claim 9, further including:

an evaluate module configured to use at least a displacement vector of a nearest pattern boundary point to provide at least a weight value.

14. The sequential pose refinement module of claim 13, wherein the evaluate module also uses a prior error value.

15. The sequential pose refinement module of claim 13, wherein the evaluate module also provides a clutter value.

16. The sequential pose refinement module of claim 13, wherein the evaluate module also uses a direction of an image boundary point in coordinates of the vector-valued function.

17. The sequential pose refinement module of claim 13, wherein the evaluate module also uses a gradient magnitude of an image boundary point.

18. The sequential pose refinement module of claim 13, further including:

a sum module configured to use the displacement vector of a nearest pattern boundary point, a position of an image boundary point in coordinates of the vector-valued function, and a weight value, so as to provide a plurality of sums for a least-squares solution.

19. The sequential pose refinement module of claim 18, further including:

a solve module configured to use the plurality of sums and the normal tensor to provide the motion transform.

20. The sequential pose refinement module of claim 19, wherein the solve module also provides an error value.

21. The sequential pose refinement module of claim 20, wherein the error value is an rms error value.

22. The sequential pose refinement module of claim 1, wherein the motion transform generator includes:

a map module adapted to map the position and direction of each image boundary point from image coordinates to coordinates of the vector-valued function.

23. A geometric pattern matching method for refining a starting pose of an object in a run-time image, the object having an expected shape and a true pose in the run-time image, the starting pose representing an initial estimate of the true pose of the object in the run-time image, the method comprising:

providing a stored model pattern, the stored model pattern including a geometric description of the expected shape of the object, the geometric description including a plurality of pattern boundary points, and information about the pattern boundary points;

detecting in the run-time image a plurality of image boundary points; and

using the stored model pattern, the starting pose, and the plurality of image boundary points in a sequence of pose refinements to provide at least a refined pose representing a refined estimate of the true pose of the object in the run-time image includes;

using the plurality of image boundary points and a vector-valued function of position of the stored model pattern so as to provide a motion transform; and

using a current pose and the motion transform so as to provide a refined pose, the refined pose representing a refined estimate of the true pose of the object in the run-time image.

24. The method of claim 23, wherein a sequence of progressively refined poses are provided that are increasingly closer to the true pose in the run-time image.

25. The method of claim 23, wherein the motion transform is a six-degree-of-freedom coordinate transform that maps the current pose into a more refined current pose.

26. The method of claim 23, wherein the motion transform is a perspective transform.

27. The method of claim 23, wherein the motion transform is a transform that corrects for lens distortion.

28. The method of claim 23, wherein an error value is provided with the motion transform.

29. The method of claim 23, further comprising:

receiving a current pose and a client map, so as to provide a normal tensor to facilitate providing a motion transform.

30. The method of claim 23, wherein providing a motion transform includes:

mapping the position of each image boundary point from image coordinates to coordinates of the vector-valued function.

31. The method of claim 30, further including:

providing a displacement vector of a nearest pattern boundary point using the position of the nearest pattern boundary point in coordinates of the vector-valued function.

32. The method of claim 31, wherein providing a motion transform includes:

using information provided by the vector-valued function module to determine a motion transform using a least squares fitting.

33. The method of claim 31, wherein providing a motion transform includes:

using information provided by the vector-valued function module to determine a motion transform using a data fitting method.

34. The method of claim 31, further including:

determining the position of each image boundary point in displacement vector coordinates that include a component of the position in a direction of a displacement vector of a nearest pattern boundary point.

35. The method of claim 31, further including:

using at least a displacement vector of a nearest pattern boundary point to provide at least a weight value.

36. The method of claim 35, also using a prior error value with the displacement vector.

37. The method of claim 35, wherein a clutter value is also provided.

38. The method of claim 35, also using a direction of an image boundary point in coordinates of the vector-valued function with the displacement vector of a nearest pattern boundary point to provide at least a weight value.

39. The method of claim 35, also using a gradient magnitude of an image boundary point with the displacement vector of a nearest pattern boundary point to provide at least a weight value.

40. The method of claim 35, further including:

using the displacement vector of a nearest pattern boundary point, a position of an image boundary point in coordinates of the vector-valued function, and a weight value, so as to provide a plurality of sums for a least-squares solution.

41. The method of claim 40, further including:

using the plurality of sums and the normal tensor to provide the motion transform.

42. The method of claim 41, wherein an error value is also provided.

43. The method of claim 42, wherein the error value is an rms error value.

44. The method of claim 23, wherein providing a motion transform includes:

mapping the position and direction of each image boundary point from image coordinates to coordinates of the vector-valued function.

45. A geometric pattern matching method for refining a starting pose of an object in a run-time image, the object having an expected shape and a true pose in the run-time image, the starting pose representing an initial estimate of the true pose of the object in the run-time image, the method comprising:

providing a stored model pattern, the stored model pattern including a geometric description of the expected shape of the object, the geometric description including a plurality of pattern boundary points, and information about the pattern boundary points including a vector-valued function of position that relates displacement vectors to pattern boundary points;

detecting in the run-time image a plurality of image boundary points; and

using the stored model pattern, the starting pose, and the plurality of image boundary points in a sequence of pose refinements to provide at least a refined pose pose representing a refined estimate of the true pose of the object in the run-time image includes;

computing a normal tensor using a current pose;

mapping at least the position of an image boundary point from image coordinates to vector-valued function coordinates using the current pose to provide an image boundary point position in vector-valued function coordinates;

using the image boundary point position in vector-valued function coordinates to determine at least the displacement vector of a nearest pattern boundary point;

using the normal tensor, the displacement vector, and the position of the image boundary point in vector-valued function coordinates to obtain the position of the image boundary point in displacement vector coordinates;

using at least the displacement vector to compute a weight value;

using the displacement vector, the position of the image boundary point in vector-valued function coordinates, and weight to compute a plurality of sums;

using the plurality of sums and the normal tensor to compute a motion transform; and

using the motion transform and the current pose to provide a new pose.

46. The method of claim 45, wherein the new pose is provided as the refined pose after the last pose refinement in a sequence of pose refinements.

47. The method of claim 45, upon the last attraction event, further including:

evaluating the plurality image boundary points, and the plurality of pattern boundary points, so as to provide a plurality of evaluated pattern boundary points, an aggregate coverage value, and an aggregate clutter value.

48. The method of claim 45, wherein using at least the displacement vector to also compute a clutter value along with the weight value.

49. The method of claim 45, wherein using at least the displacement vector to also compute an evaluation value along with the weight value.

50. The method of claim 45, wherein an rms error value from a previous pose refinement is also used with the displacement vector.

51. The method of claim 45, wherein after using at least the displacement vector to compute a weight value, the evaluation and clutter values are stored in association with the image boundary point position.

52. The method of claim 45, wherein using the image boundary point position in coordinates of the vector-valued function to determine at least the displacement vector of a nearest pattern boundary point, storing at least the displacement vector in association with the image boundary point.

53. A sequential pose refinement module in a geometric pattern matching apparatus for refining a starting pose of an object in a run-time image, the object having an expected shape and a true pose in the run-time image, the starting pose representing an initial estimate of the true pose of the object in the run-time image, the geometric pattern matching apparatus having (1) a stored model pattern, the stored model pattern including a geometric description of the expected shape of the object, the geometric description including a plurality of pattern boundary points, and a vector-valued function of position within a region that includes the pattern boundary points, and (2) a feature detector configured to detect in the run-time image a plurality of image boundary points, the sequential pose refinement module comprising:

a look-up table entry generator configured to use the plurality of image boundary points, a current pose look-up table configured to map image boundary points to pattern boundary points, and the vector-valued function of position, so as to provide a look-up table entry for each image boundary point of a plurality of the image boundary points;

a look-up table for storing each entry provided by the look-up table entry generator; and

a look-up table smoother configured to apply a smoothness constraint to the look-up table so as to provide a new pose look-up table, the new pose look-up table representing a further refined estimate of the true pose of the object in the run-time image.

54. The sequential pose refinement module of claim 53, wherein the sequential pose refinement module provides a sequence of new look-up tables that are progressively more representative of the true pose in the run-time image.

55. The sequential pose refinement module of claim 53, wherein the look-up table generator includes:

a map module configured to map the position of each image boundary point from image coordinates to coordinates of the vector-valued function.

56. The sequential pose refinement module of claim 55, wherein the map module performs look-up operations and interpolation operations.

57. The sequential pose refinement module of claim 53, wherein the look-up table generator includes:

a map module configured to map the position and direction of each image boundary point from image coordinates to coordinates of the vector-valued function.

58. The sequential pose refinement module of claim 53, wherein the look-up table generator further includes:

a vector-valued function module configured to provide a displacement vector of a nearest pattern boundary point using the position of the nearest pattern boundary point in coordinates of the vector-valued function.

59. The sequential pose refinement module of claim 53, further including:

an evaluate module configured to use at least a displacement vector of a nearest pattern boundary point to provide at least a weight value.

60. The sequential pose refinement module of claim 59, wherein the evaluate module also uses a prior error value.

61. The sequential pose refinement module of claim 59, wherein the evaluate module also provides a clutter value.

62. The sequential pose refinement module of claim 59, wherein the evaluate module also uses a direction of an image boundary point in coordinates of the vector-valued function.

63. The sequential pose refinement module of claim 59, wherein the evaluate module also uses a gradient magnitude of an image boundary point.

64. The sequential pose refinement module of claim 53, wherein the lookup table generator includes:

a map module configured to map the position of each image boundary point from image coordinates to coordinates of the vector-valued function;

a vector-valued function module configured to provide a displacement vector of a nearest pattern boundary point using the position of the nearest pattern boundary point in coordinates of the vector-valued function;

an evaluate module configured to use at least a displacement vector of a nearest pattern boundary point to provide at least a weight value;

an intermediate look-up table adapted to store a weighted sum of the displacement vector of a nearest pattern boundary point and the position of each image boundary point in coordinates of the vector-valued function, the intermediate look-up table having a table entry for each image boundary point.

65. The sequential pose refinement module of claim 64, wherein the evaluate module also uses a prior error value.

66. The sequential pose refinement module of claim 64, wherein the evaluate module also provides a clutter value.

67. The sequential pose refinement module of claim 64, wherein the evaluate module also uses a direction of an image boundary point in coordinates of the vector-valued function.

68. The sequential pose refinement module of claim 64, wherein the evaluate module also uses a gradient magnitude of an image boundary point.

69. The sequential pose refinement module of claim 64, wherein the map module performs look-up operations and interpolation operations.

70. The sequential pose refinement module of claim 53, wherein the map module is configured to map the position and direction of each image boundary point from image coordinates to coordinates of the vector-valued function.