Method and apparatus for visual detection and inspection of objects

• US 20050275831A1
• Filed: 06/09/2004
• Published: 12/15/2005
• Est. Priority Date: 06/09/2004
• Status: Active Grant

• Current Assignee: Cognex Technology & Investment Corporation (Cognex Corporation)
• Original Assignee: Cognex Technology & Investment Corporation (Cognex Corporation)
• Inventors: Silver, William M.
• Granted Patent: US 9,092,841 B2
• Time in Patent Office: Days
• US Class Current: 356/237.100
• CPC Class Codes: G01N 2021/8909 Scan signal processing specially adapted for inspection of running sheets
• CPC Class Codes: G06T 2207/30164 Workpiece; Machine component
• CPC Class Codes: G06T 7/0002 Inspection of images, e.g. flaw detection
• CPC Class Codes: G06T 7/0004 Industrial image inspection
• CPC Class Codes: G06T 7/0006 using a design-rule based approach
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06V 10/443 by matching or filtering

Abstract

Disclosed are methods and apparatus for automatic optoelectronic detection and inspection of objects, based on capturing digital images of a two-dimensional field of view in which an object to be detected or inspected may be located, analyzing the images, and making and reporting decisions on the status of the object. Decisions are based on evidence obtained from a plurality of images for which the object is located in the field of view, generally corresponding to a plurality of viewing perspectives. Evidence that an object is located in the field of view is used for detection, and evidence that the object satisfies appropriate inspection criteria is used for inspection. Methods and apparatus are disclosed for capturing and analyzing images at high speed so that multiple viewing perspectives can be obtained for objects in continuous motion.

Claims

1. A method for inspection of an object, comprising:

continuously moving the object relative to a two-dimensional field of view, the object comprising a set of visible features, the set of visible features containing at least one visible feature;

capturing a plurality of frames, each frame of the plurality of frames comprising an image of the field of view, such that all visible features of the set of visible features are located in the field of view for each frame of the plurality of frames, and such that the plurality of frames comprises a plurality of locations of the set of visible features in the field of view;

determining, responsive to a first analysis of the plurality of frames, a result comprising information about the status of the object; and

reporting the result.

2. The method according to claim 1 wherein the first analysis comprises

computing a plurality of object pass scores corresponding respectively to the plurality of frames, each object pass score of the plurality of object pass scores comprising evidence that the object satisfies at least one inspection criterion.

3. The method according to claim 2 wherein

each object pass score of the plurality of object pass scores is computed responsive to a second analysis of at least one region of interest in the frame.

5. The method according to claim 2 wherein

each object pass score of the plurality of object pass scores comprises a range of at least three values indicating relative confidence that the object satisfies the at least one inspection criterion.

6. The method according to claim 3 wherein the second analysis comprises, for each object pass score of the plurality of object pass scores,

calculating a set of measurements corresponding respectively to the at least one region of interest, each measurement in the set of measurements responsive to a characteristic of one of the set of visible features of the object;

and

computing, responsive to the set of measurements, the object pass score.

7. The method according to claim 6 wherein the step of computing the object pass score comprises

converting the set of measurements to a corresponding set of logic values;

and

combining the set of logic values to produce the object pass score.

8. The method according to claim 7 wherein

each logic value of the set of logic values is a binary logic value.

9. The method according to claim 7 wherein

each logic value of the set of logic values is a fuzzy logic value.

10. The method according To claim 2 wherein

the first analysis further comprises determining, for each frame of the plurality of frames, an object detection weight comprising evidence that all visible features of the set of visible features are located in the field of view for the frame.

11. The method according to claim 2 wherein

the first analysis further comprises computing an average of the object pass scores.

12. The method according to claim 2 wherein

the first analysis further comprises calculating a percentile of the object pass scores.

13. The method according to claim 10 wherein

the first analysis further comprises computing a weighted average of the object pass scores weighted by the corresponding object detection weight.

14. The method according to claim 10 wherein

the first analysis further comprises computing a weighted average of the object pass scores weighted by the corresponding percentile of the object detection weights.

15. The method according to claim 1 wherein

the step of moving includes moving the object no more than one-quarter of the field of view between successive frames of the plurality of frames.

16. The method according to claim 1 wherein

the step of moving includes moving the object no more than five percent of the field of view between successive frames.

17. The method according to claim 1 wherein

the capturing of the plurality of frames and the first analysis of the plurality of frames is performed at a rate of not less than two hundred frames per second.

18. The method according to claim 1 wherein

the step of reporting the result comprises producing an output pulse.

19. The method according to claim 18 wherein

the step of reporting the result further comprises signaling a reject actuator with the output pulse.

20. The method according to claim 18 wherein

the step of reporting the result further comprises producing the output pulse only if the object satisfies at least one inspection criterion.

21. The method according to claim 18 wherein

the step of reporting the result further comprises producing the output pulse only if the object does not satisfy at least one inspection criterion.

22. The method according to claim 18 wherein

the step of reporting the result further comprises producing the output pulse in synchronization with the object crossing a fixed reference point.

23. The method according to claim 1, further comprising:

providing a trigger signal that indicates when the set of visible features is located in the field of view; and

starting the capturing responsive to the trigger signal.

24. The method according to claim 23 wherein

the step of reporting the result comprises producing an output pulse in synchronization with the trigger signal.

4. The method according to clam 2 wherein

each object pass score of the plurality of object pass scores comprises one of a first binary value and a second binary value.

25. A method for inspection of an object, comprising:

capturing a plurality of frames, each frame of the plurality of frames comprising an image of a two-dimensional field of view;

determining, responsive to a first analysis of the plurality of frames, a plurality of object detection weights, each object detection weight in the plurality of object detection weights comprising evidence that the object is located in the field of view for the corresponding frame, each object detection weight of the plurality of object detection weights corresponding respectively to each frame of the plurality of frames;

choosing a plurality of active frames from the plurality of frames, such that each object detection weight for each frame of the plurality of active frames indicates sufficient evidence that the object is located in the field of view;

computing, responsive to a second analysis of the plurality of active frames, a plurality of object pass scores corresponding respectively to each frame of the plurality of active frames, each object pass score in the plurality of object pass scores comprising evidence that the object satisfies at least one inspection criterion;

determining, responsive to the plurality of object pass scores, a result comprising information about whether the object satisfies the at least one inspection criterion; and

reporting the result.

26. The method according to claim 25 wherein

the step of determining the object detection weight includes determining each object detection weight in response to a third analysis of at least one region of interest in the frame.

27. The method according to claim 25 wherein

each object detection weight of the plurality of object detection weights comprises one of a first binary value and a second binary value;

and

the object detection weight indicates sufficient evidence that the object is located in the field of view when it comprises the first binary value.

28. The method according to claim 25 wherein

each object detection weight of the plurality of object detection weights comprises a range of at least three values indicating relative confidence that the object is located in the field of view for the frame;

and

the step of choosing the plurality of active frames comprises comparing the object detection weight to a threshold.

29. The method according to claim 26 wherein the third analysis comprises, for each object detection weight of the plurality of object detection weights,

computing a set of measurements corresponding respectively to the at least one region of interest, each measurement in the set of measurements responsive to a characteristic of a visible feature of the object;

and

computing, responsive to the set of measurements, the object detection weight.

30. The method according to claim 29 wherein the third analysis further comprises

converting the set of measurements to a corresponding set of logic values;

and

combining the set of logic values to produce the object detection weight.

31. The method according to claim 30 wherein

each logic value of the set of logic values is a binary logic value.

32. The method according to claim 30 wherein

each logic value of the set of logic values is a fuzzy logic value.

33. The method according to claim 25 wherein

each object pass score of the plurality of object pass scores is determined responsive to a third analysis of at least one region of interest in the frame.

34. The method according to claim 25 wherein

each object pass score of the plurality of object pass scores comprises one of a first binary value and a second binary value.

35. The method according to claim 25 wherein

each object pass score of the plurality of object pass scores comprises a range of at least three values indicating relative confidence that the object satisfies the at least one inspection criterion.

36. The method according to claim 33 wherein the third analysis comprises, for each object pass score of the plurality of object pass scores,

computing a set of measurements corresponding respectively to the at least one region of interest, each measurement in the set of measurements responsive to a characteristic of a visible feature of the object;

and

computing, responsive to the set of measurements, the object pass score.

37. The method according to claim 36 wherein the third analysis further comprises

converting the set of measurements to a corresponding set of logic values;

and

combining the set of logic values to produce the object pass score.

38. The method according to claim 37 wherein

each logic value of the set of logic values is a binary logic value.

39. The method according to claim 37 wherein

each logic value of the set of logic values is a fuzzy logic value.

40. The method according to claim 25 wherein

the second analysis further comprises computing an average of the object pass scores.

41. The method according to claim 25 wherein

the second analysis further comprises calculating a percentile of the object pass scores.

42. The method according to claim 25 further comprising

choosing a plurality of active object detection weights from the plurality of object detection weights corresponding respectively to the plurality of active frames;

and

wherein the step of determining the result is further responsive to the plurality of active object detection weights.

43. The method according to claim 42 wherein

the second analysis further comprises computing a weighted average of the object pass scores weighted by the corresponding active object detection weight.

44. The method according to claim 42 wherein

the second analysis further comprises calculating a weighted average of the object pass scores weighted by the percentile of the active object detection weights.

45. The method according to claim 25 wherein

the plurality of frames are captured and the first and second analysis are performed at a rate of not less than two hundred frames per second.

46. The method according to claim 25 wherein

the step of reporting the result comprises producing an output pulse.

47. The method according to claim 46 wherein

the step of reporting the result further comprises signaling a reject actuator with the output pulse.

48. The method according to claim 46 wherein

the step of reporting the result further comprises producing the output pulse only if the object satisfies at least one inspection criterion.

49. The method according to claim 46 wherein

the step of reporting the result further comprises producing the output pulse only if the object does not satisfy at least one inspection criterion.

50. The method according to claim 46 wherein

the step of reporting the result further comprises producing the output pulse in synchronization with the object crossing a fixed reference point.

51. A method for detection of an object, comprising:

capturing a plurality of frames, each frame of the plurality of frames comprising an image of a two-dimensional field of view;

determining, responsive to a first analysis of the plurality of frames, a plurality of object detection weights, each object detection weight in the plurality of object detection weights comprising evidence that the object is located in the field of view for the corresponding frame, each object detection weight of the plurality of object detection weights corresponding respectively to each frame of the plurality of frames;

choosing a plurality of active object detection weights from the plurality of object detection weights, such that each active object detection weight indicates sufficient evidence that the object is located in the field of view for the corresponding frame;

determining a result responsive to a second analysis of the plurality of active object detection weights, the result comprising information about whether the object has been detected; and

reporting the result.

52. The method according to claim 51 wherein

each object detection weight of the plurality of object detection weights is determined responsive to a third analysis of at least one region of interest in the frame.

53. The method according to claim 51 wherein

each object detection weight of the plurality of object detection weights comprises one of a fast binary value and a second binary value;

and

the object detection weight indicates sufficient evidence that the object is located in the field of view when it comprises the first binary value.

54. The method according to claim 51 wherein

each object detection weight of the plurality of object detection weights comprises a range of at least three values indicating relative confidence that the object is located in the field of view for the frame;

and

the step of choosing the object detection weight to be among the plurality of active object detection weights comprises comparing the object detection weight to a threshold.

55. The method according to claim 52 wherein the third analysis comprises, for each object detection weight of the plurality of object detection weights,

computing a set of measurements corresponding respectively to the at least one region of interest, each measurement in the set of measurements responsive to a characteristic of a visible feature of the object;

and

computing, responsive to the set of measurements, the object detection weight.

56. The method according to claim 55 wherein computing the object detection weight comprises

converting the set of measurements to a corresponding set of logic values;

and

combining the set of logic values to produce the object detection weight.

57. The method according to claim 56 wherein

each logic value of the set of logic values is a binary logic value.

58. The method according to claim 56 wherein

each logic value of the set of logic values is a fuzzy logic value.

59. The method according to claim 51 wherein

the second analysis is responsive to a count of the active object detection weights of the plurality of active object detection weights.

60. The method according to claim 51 wherein

the second analysis is responsive to a sum of the active object detection weights of the plurality of active object detection weights.

61. The method according to claim 51 wherein

the step of choosing the plurality of active object detection weights ends when a predetermined number of consecutive object detection weights are not chosen to be active object detection weights.

62. The method according to claim 61 wherein

the predetermined number is at least two.

63. The method according to claim 51 wherein

the plurality of frames are captured and the first and second analysis are performed at a rate of not less than two hundred frames per second.

64. The method according to claim 51 wherein

the step of reporting the result comprises producing an output pulse only if the object is detected.

65. The method according to claim 64 wherein

the step of reporting the result further comprises producing the output pulse in synchronization with the object crossing a fixed reference point.

66-72. -72. (canceled)

73. A system for inspection of an object, comprising:

a conveyer that continuously moves the object relative to a two dimensional field of view, the object comprising a set of visible features, the set of visible features containing at least one visible feature;

an image capture device that captures a plurality of frames, each frame of the plurality of frames comprising an image of the field of view, such that all visible features of the set of visible features are located in the field of view for each frame of the plurality of frames, and such that the plurality of frames comprises a plurality of locations of the set of visible features in the field of view;

an analyzer that determines, responsive to a first analysis of the plurality of frames, a result comprising information about the status of the object; and

an output signaler that reports the result.

74. The system according to claim 73, wherein the first analysis comprises computing a plurality of object pass scores corresponding respectively to the plurality of frames, each object pass score of the plurality of object pass scores comprising evidence that the object satisfies at least one inspection criterion.

75. The system according to claim 74, wherein each object pass score of the plurality of object pass scores comprises one of a first binary value and a second binary value.

76. The system according to claim 74, wherein each object pass score of the plurality of object pass scores comprises a range of at least the values indicating relative confidence that the object satisfies the at least one inspection criterion.

77. The system according to claim 74, wherein each object pass score of the plurality of object pass scores is computed responsive to a second analysis of at least one region of interest in the frame.

78. The system according to claim 77, wherein the second analysis comprises, for each object pass score of the plurality of object pass scores,

calculating a set of measurements corresponding respectively to the at least one region of interest, each measurement in the set of measurements responsive to a characteristic of one of the set of visible features of the object;

and

computing, responsive to the set of measurements, the object pass score.

79. The system according to claim 78, wherein computing the object pass score comprises

converting the set of measurements to a corresponding set of logic values;

and

combining the set of logic values to produce the object pass score.

80. The system according to claim 79, wherein each logic value of

the set of logic values is a binary logic value.

81. The system according to claim 79, wherein each logic value of

the set of logic values is a fuzzy logic value.

82. The system according to claim 74, wherein the first analysis further comprises computing an average of the object pass scores.

83. The system according to claim 74, wherein the first analysis further comprises calculating a percentile of the object pass scores.

84. The system according to claim 74, wherein the first analysis further comprises determining, for each frame of the plurality of frames, an object detection weight comprising evidence that all visible features of the set of visible features are located in the field of view for the frame.

85. The system according to claim 84, wherein the first analysis further comprises computing a weighted average of the object pass scores weighted by the corresponding object detection weight.

86. The system according to claim 84, wherein the first analysis further comprises computing a weighted average of the object pass scores weighted by the corresponding percentile of the object detection weights.

87. The system according to claim 73, wherein the conveyer moves the object no more than one-quarter of the field of view between successive frames of the plurality of frames.

88. The system according to claim 73, wherein the conveyer moves the object no more than five percent of the field of view between successive frames.

89. The system according to claim 73, wherein the image capture device and the analyzer operate at a rate of not less than two hundred frames per second.

90. The system according to claim 73, wherein the image capture device comprises an array of no more than about 40,000 pixels.

91. The system according to claim 90, wherein the image capture device comprises an LM9630.

92. The system according to claim 73, wherein the output signaler reports the result by producing an output pulse.

93. The system according to claim 92, further comprising

a reject actuator, and wherein

the output pulse signals the reject actuator.

94. The system according to claim 92, wherein the output signaler produces the output pulse only if the object satisfies at least one inspection criterion.

95. The system according to claim 92, wherein the output signaler produces the output pulse only if the object does not satisfy at least one inspection criterion.

96. The system according to claim 92, wherein the output signaler produces the output pulse in synchronization with the object crossing a fixed reference point.

97. The system according to claim 73, further comprising

a trigger signal that indicates when the set of visible features is located in the field of view;

and wherein

the image capture device captures the plurality of frames in response to the trigger signal.

98. The system according to claim 97, wherein the output signaler produces an output pulse in synchronization with the trigger signal.

99. A system for inspection of an object, comprising:

an image capture device that captures a plurality of frames, each frame of the plurality of frames comprising an image of a two-dimensional field of view;

an analyzer that

determines, responsive to a first analysis of the plurality of frames, a plurality of object detection weights, each object detection weight in the plurality of object detection weights comprising evidence that the object is located in the field of view for the corresponding frame, each object detection weight of the plurality of object detection weights corresponding respectively to each frame of the plurality of frames;

chooses a plurality of active frames from the plurality of frames, such that each object detection weight for each frame of the plurality of active frames indicates sufficient evidence that the object is located in the field of view;

computes, responsive to a second analysis of the plurality of active frames, a plurality of object pass scores corresponding respectively to each frame of the plurality of active frames, each object pass score in the plurality of object pass scores comprising evidence that the object satisfies at least one inspection criterion; and

determines, responsive to the plurality of object pass scores, a result comprising information about whether the object satisfies the at least one inspection criterion; and

an output signaler that reports the result.

100. The system according to claim 99, wherein

each object detection weight of the plurality of object detection weights comprises one of a first binary value and a second binary value;

and

the object detection weight indicates sufficient evidence that the object is located in the field of view when it comprises the first binary value.

101. The system according to claim 99, wherein

each object detection weight of the plurality of object detection weights comprises a range of at least three values indicating relative confidence that the object is located in the field of view for the frame;

and

the analyzer chooses the plurality of active frames by comparing each object detection weight to a threshold.

102. The system according to claim 99, wherein the analyzer determines each object detection weight in response to a third analysis of at least one region of interest in the frame.

103. The system according to claim 102, wherein the third analysis comprises, for each object detection weight of the plurality of object detection weights,

computing a set of measurements corresponding respectively to the at least one region of interest, each measurement in the set of measurements responsive to a characteristic of a visible feature of the object;

and

computing, responsive to the set of measurements, the object detection weight.

104. The system according to claim 103, wherein the third analysis further comprises

converting the set of measurements to a corresponding set of logic values;

and

combining the set of logic values to produce the object detection weight.

105. The system according to claim 104, wherein each logic value of the set of logic values is a binary logic value.

106. The system according to claim 104, wherein each logic value of the set of logic values is a fuzzy logic value.

107. The system according to claim 99, wherein each object pass score of the plurality of object pass scores comprises one of a fist binary value and a second binary value.

108. The system according to claim 99, wherein each object pass score of the plurality of object pass scores comprises a range of at least three values indicating relative confidence that the object satisfies the at least one inspection criterion.

109. The system according to claim 99, wherein the analyzer computes each object pass score of the plurality of object pass scores responsive to a third analysis of at least one region of interest in the frame.

110. The system according to claim 109, wherein the third analysis comprises, for each object pass score of the plurality of object pass scores,

computing a set of measurements corresponding respectively to the at least one region of interest, each measurement in the set of measurements responsive to a characteristic of a visible feature of the object;

and

computing, responsive to the set of measurements, the object pass score.

111. The system according to claim 110, wherein the third analysis further comprises

converting the set of measurements to a corresponding set of logic values;

and

combining the set of logic values to produce the object pass score.

112. The system according to claim 111, wherein each logic value of the set of logic values is a binary logic value.

113. The system according to claim 111, wherein each logic value of the set of logic values is a fuzzy logic value.

114. The system according to claim 99, wherein the second analysis further comprises computing an average of the object pass scores.

115. The system according to claim 99, wherein the second analysis further comprises calculating a percentile of the object pass scores.

116. The system according to claim 99, wherein

the analyzer further chooses a plurality of active object detection weights from the plurality of object detection weights corresponding respectively to the plurality of active frames;

and

the analyzer determines the result further responsive to the plurality of active object detection weights.

117. The system according to claim 116, wherein the second analysis further comprises computing a weighted average of the object pass scores weighted by the corresponding active object detection weight.

118. The system according to claim 116, wherein the second analysis further comprises calculating a weighted average of the object pass scores weighted by the percentile of the active object detection weights.

119. The system according to claim 99, wherein the image capture device and the analyzer operate at a rate of not less than two hundred frames per second.

120. The system according to claim 99, wherein the image capture device comprises an array of no more than about 40,000 pixels.

121. The system according to claim 120, wherein the image capture device comprises an LM9630.

122. The system according to claim 99, wherein the output signaler reports the result by producing an output pulse.

123. The system according to claim 122, further comprising

a reject actuator;

and wherein

the output pulse signals the reject actuator.

124. The system according to claim 122, wherein the output signaler produces the output pulse only if the object satisfies at least one inspection criterion.

125. The system according to claim 122, wherein the output signaler produces the output pulse only if the object does not satisfy at least one inspection criterion.

126. The system according to claim 122, wherein the output signaler produces the output pulse in synchronization with the object crossing a fixed reference point.

127. A system for detection of an object, comprising:

an image capture device that captures a plurality of frames, each frame of the plurality of frames comprising an image of a two-dimensional field of view;

an analyzer that

determines, responsive to a first analysis of the plurality of frames, a plurality of object detection weights, each object detection weight in the plurality of object detection weights comprising evidence that the object is located in the field of view for the corresponding frame, each object detection weight of the plurality of object detection weights corresponding respectively to each frame of the plurality of frames;

chooses a plurality of active object detection weights from the plurality of object detection weights, such that each active object detection weight indicates sufficient evidence that the object is located in the field of view for the corresponding frame; and

determines a result responsive to a second analysis of the plurality of active object detection weights, the result comprising information about whether the object has been detected; and

an output signaler that reports the result.

128. The system according to claim 127, wherein

each object detection weight of the plurality of object detection weights comprises one of a first binary value and a second binary value;

and

the object detection weight indicates sufficient evidence that the object is located in the field of view when it comprises the first binary value.

129. The system according to claim 127, wherein

each object detection weight of the plurality of object detection weights comprises a range of at least three values indicating relative confidence that the object is located in the field of view for the frame;

and

the analyzer chooses the object detection weight to be among the plurality of active object detection weights by comparing the object detection weight to a threshold.

130. The system according to claim 127, wherein the analyzer determines each object detection weight of the plurality of object detection weights responsive to a third analysis of at least one region of interest in the fine.

131. The system according to claim 130, wherein the third analysis comprises, for each object detection weight of the plurality of object detection weights,

computing a set of measurements corresponding respectively to the at least one region of interest, each measurement in the set of measurements responsive to a characteristic of a visible feature of the object;

and

computing, responsive to the set of measurements, the object detection weight.

132. The system according to claim 131, wherein computing the object detection weight comprises

converting the set of measurements to a corresponding set of logic values;

and

combining the set of logic values to produce the object detection weight.

133. The system according to claim 132, wherein each logic value of the set of logic values is a binary logic value.

134. The system according to claim 132, wherein each logic value of the set of logic values is a fuzzy logic value.

135. The system according to claim 127, wherein the second analysis is responsive to a count of the active object detection weights of the plurality of active object detection weights.

136. The system according to claim 127, wherein the second analysis is responsive to a sum of the active object detection weights of the plurality of active object detection weights.

137. The system according to claim 127, wherein the analyzer determines the result when a predetermined number of consecutive object detection weights are not chosen to be active object detection weights.

138. The system according to claim 137, wherein the predetermined number is at least two.

139. The system according to claim 127, wherein the image capture device and the analyzer operate at a rate of not less than two hundred frames per second.

140. The system according to claim 127, wherein the image capture device comprises an array of no more than about 40,000 pixels.

141. The system according to claim 140, wherein the image capture device comprises an LM9630.

142. The system according to claim 127, wherein the output signaler reports the result by producing an output pulse only if the object is detected.

143. The system according to claim 142, wherein the output signaler produces the output pulse in synchronization with the object crossing a fixed reference point.