Method and apparatus for three dimensional inspection of electronic components

• US 6,915,006 B2
• Filed: 04/27/2001
• Issued: 07/05/2005
• Est. Priority Date: 01/16/1998
• Status: Expired due to Fees

• Current Assignee: Scanner Technologies Corporation
• Original Assignee: Elaine Beaty, Elwin M. Beaty
• Inventors: Mork, David P., Beaty, Elwin M.
• Primary Examiner(s): Mehta, Bhavesh M.
• Assistant Examiner(s): Chawan, Sheela C
• Application Number:
  US09/844,232
• Publication Number:
  US 20020037098A1
• Time in Patent Office: 1,530 Days
• Field of Search: 382/154, 382146-151, 382/141, 382/144, 356/237.1, 356/376, 356/377, 250/359.23, 250/359.31, 250/359.34, 250/359.4, 348 86- 87, 348 94- 95, 348125-126, 720150-153, 720166-169
• US Class Current: 382/145
• CPC Class Codes: G06T 2200/04 involving 3D image data
• CPC Class Codes: G06T 2207/30148 Semiconductor; IC; Wafer
• CPC Class Codes: G06T 2207/30152 Solder
• CPC Class Codes: G06T 2207/30244 Camera pose
• CPC Class Codes: G06T 7/0002 Inspection of images, e.g. flaw detection
• CPC Class Codes: G06T 7/0004 Industrial image inspection
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G06T 7/80 Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
• CPC Class Codes: H05K 13/0818 Setup of monitoring devices prior to starting mounting operations; Teaching of monitoring devices for specific products; Compensation of drifts during operation, e.g. due to temperature shifts

Abstract

A three dimensional inspection system for inspecting ball array devices having a plurality of balls, where the ball array device is positioned in an optical system. An illuminator is located to illuminate at least one ball on the ball array device. A first optical element is positioned to transmit light to the sensor. A second optical element is positioned to direct light from the at least one ball to the sensor, where the sensor, the first optical element and the second optical element cooperate to obtain at least two differing views of the at least one ball, the sensor providing an output representing the at least two differing views. A processor is coupled to receive the output, where the processor processes the output by using a triangulation method to calculate a three dimensional position of the at least one ball with reference to a pre-calculated calibration plane.

Claims

1. A three dimensional inspection system for inspecting ball grid array devices having a plurality of balls, wherein the ball grid array device is positioned in an optical system, the inspection system comprising:

a) a fixed illuminator located to illuminate at least one ball on the ball grid array device;

b) a single sensor;

c) a first optical element positioned to transmit light to the sensor;

d) a second optical element positioned to direct light from the at least one ball to the sensor, where the sensor, the first optical element and the second optical element cooperate to obtain at least two differing views of the at least one ball, the sensor providing an output representing the at least two differing views; and

e) a processor, coupled to receive the output, where the processor processes the output by using a triangulation method to calculate a three dimensional position of the at least one ball with reference to a pre-calculated calibration plane.

2. The three dimensional inspection apparatus of claim 1 wherein the calibration plane comprises a coordinate system having X, Y and Z axes and wherein an X measurement value is proportional to a Z measurement value.

3. The three dimensional inspection apparatus of claim 1 wherein the calibration plane comprises a coordinate system having X, Y and Z axes and wherein an XY measurement value is proportional to a Z measurement value.

4. The three dimensional inspection apparatus of claim 1 wherein the calibration plane comprises a coordinate system having X, Y and Z axes and wherein a Y measurement value is proportional to a Z measurement value.

5. The three dimensional inspection apparatus of claim 1 wherein the triangulation calculations are based on determining a center of the ball in a first view and determining a ball top location in a second view.

6. The three dimensional inspection apparatus of claim 1 wherein the pre-calculated calibration plane is defined by measuring a calibration pattern.

7. The three dimensional inspection apparatus of claim 1 wherein the second optical element comprises a mirror.

8. The three dimensional inspection apparatus of claim 1 wherein the second optical element comprises a prism.

9. The three dimensional inspection apparatus of claim 1 wherein one of the at least two differing views is obtained at a low angle of view.

10. The three dimensional inspection apparatus of claim 1 wherein the sensor and the second optical element are positioned to receive light from different angles relative to the calibration plane.

11. The three dimensional inspection apparatus of claim 1 wherein the sensor comprises a charged coupled device array.

12. The three dimensional inspection apparatus of claim 1 wherein the sensor comprises a complementary metal oxide semiconductor device array.

13. The three dimensional inspection apparatus of claim 1 wherein the triangulation method comprises measurements derived from the at least two differing views include grayscale edge detection to locate ball positions.

14. The three dimensional inspection apparatus of claim 1 wherein the measurements include threshold analysis.

15. The three dimensional inspection apparatus of claim 1 wherein the first optical element comprises a lens.

16. The three dimensional inspection apparatus of claim 1 wherein the first optical element comprises a pin-hole lens.

17. The three dimensional inspection apparatus of claim 1 wherein the first optical element comprises a plurality of lens elements.

18. The three dimensional inspection apparatus of claim 1 wherein the first optical element comprises a telecentric lens.

19. The three dimensional inspection apparatus of claim 1 wherein the ball grid array devices comprise bump on wafer devices.

20. The three dimensional inspection apparatus of claim 1 wherein the processor comprises a personal computer.

21. The three dimensional inspection apparatus of claim 1 wherein the sensor includes a solid state sensor array.

22. The three dimensional inspection apparatus of claim 1 wherein one of the views comprises a segment having a crescent shape.

23. The three dimensional inspection system of claim 1 further comprising a diffuser disposed to provide illumination for imaging of a perimeter of the ball grid array device.

24. The three dimensional inspection system of claim 1, wherein the at least one ball on the ball grid array device being inspected comprises a contact.

25. The three dimensional inspection system of claim 1, wherein the at least one ball on the ball grid array device being inspected comprises a pin.

26. The three dimensional inspection system of claim 1, wherein the at least one ball on the ball grid array device being inspected is selected from the group consisting of:

bump contact, ball contact, pad, and pedestal.

27. The three dimensional inspection system of claim 1, wherein the two different views of the at least one ball are obtained in a single image.

28. The three dimensional inspection system of claim 1, wherein the two different views of the at least one ball are each obtained in a separate image.

33. The three dimensional inspection apparatus of claim 27 wherein the pre-calculated calibration plane is defined by measuring a calibration pattern.

29. A three dimensional inspection apparatus for ball grid array devices having a plurality of balls, the apparatus comprising:

a) a fixed illuminator positioned to produce reflections from the ball gird array device;

b) a single sensor disposed to receive light at a first angle relative to the ball array device;

c) a first optical element positioned to transmit light to the sensor, where the sensor obtains a first view of the ball grid array device;

d) a second optical element disposed to receive light at a second angle different from the first angle and to transmit a second view of the ball array device to the sensor;

e) a frame grabber coupled to the sensor to transmit image information from the sensor; and

f) a processor, coupled to receive the image information, where the processor applies triangulation calculations to measurements of the image information so as to calculate a three dimensional position of at least one ball with reference to a pre-calculated calibration plane.

30. The three dimensional inspection apparatus of claim 29 wherein the calibration plane comprises a coordinate system having X, Y and Z axes and wherein an X measurement value is proportional to a Z measurement value.

31. The three dimensional inspection apparatus of claim 29 wherein the calibration plane comprises a coordinate system having X, Y and Z axes and wherein an XY measurement value is proportional to a Z measurement value.

32. The three dimensional inspection apparatus of claim 29 wherein the calibration plane comprises a coordinate system having X, Y and Z axes and wherein a Y measurement value is proportional to a Z measurement value.

34. The three dimensional inspection apparatus of claim 29 wherein the second optical element comprises a mirror.

35. The three dimensional inspection apparatus of claim 29 wherein the second optical element comprises a prism.

36. The three dimensional inspection apparatus of claim 29 wherein the illuminator comprises a ring light.

37. The three dimensional inspection apparatus of claim 29 wherein the illuminator comprises a plurality of light emitting diodes.

38. The three dimensional inspection apparatus of claim 37, wherein at least two of the plurality of light emitting diodes are spectrally diverse from one another.

39. The three dimensional inspection apparatus of claim 29 wherein the illuminator comprises reflected light.

40. The three dimensional inspection apparatus of claim 29 wherein the sensor comprises a charged coupled device array.

41. The three dimensional inspection apparatus of claim 29 wherein the sensor comprises a complementary metal oxide semiconductor device array.

42. The three dimensional inspection apparatus of claim 29 wherein the ball grid array devices comprise bump on wafer devices.

43. The three dimensional inspection apparatus of claim 29 wherein the measurements from the first image and the second image include grayscale edge detection to locate ball positions.

44. The three dimensional inspection apparatus of claim 29 wherein the measurements include threshold analysis.

45. The three dimensional inspection apparatus of claim 29 wherein the first optical element comprises a lens.

46. The three dimensional inspection apparatus of claim 29 wherein the first optical element comprises a pin-hole lens.

47. The three dimensional inspection apparatus of claim 29 wherein the first optical element comprises a plurality of lens elements.

48. The three dimensional inspection apparatus of claim 29 wherein the first optical element comprises a telecentric lens.

49. The three dimensional inspection apparatus of claim 29 wherein the sensor includes a solid state sensor array.

50. The three dimensional inspection apparatus of claim 29 wherein the processor comprises a personal computer.

51. The three dimensional inspection apparatus of claim 29 wherein the second optical element reflects a view to the sensor where at least one ball of the ball grid array device exhibits a crescent shape.

52. The three dimensional inspection apparatus of claim 29 further comprising a diffuser disposed to provide illumination for imaging of a perimeter of the ball grid array device.

53. The three dimensional inspection apparatus of claim 29, wherein the at least one ball on the ball grid array device being inspected comprises a contact.

54. The three dimensional inspection apparatus of claim 29, wherein the at least one ball on the ball grid array device being inspected comprises a pin.

55. The three dimensional inspection apparatus of claim 29, wherein the at least one ball on the ball grid array device being inspected is selected from the group consisting of:

bump contact, ball contact, pad, and pedestal.

56. The three dimensional inspection apparatus of claim 29, wherein the first view of the ball grid array device and the second view of the ball grid array device are obtained in a single image.

57. The three dimensional inspection apparatus of claim 29, wherein the first view of the ball grid array device and the second view of the ball grid array device are each obtained in a separate image.

58. A three dimensional inspection apparatus for ball grid array devices having a plurality of balls, the apparatus comprising:

a) a fixed illuminator disposed to illuminate a ball grid array device;

b) a single sensor disposed to receive light at a first angle relative to the ball grid array device, and wherein the sensor includes a solid state sensor array;

c) a first optical element positioned to transmit light to the sensor, where the sensor obtains a first view of the ball grid array device;

d) a second optical element disposed to receive light at a second angle different from the first angle and to transfer a second view of the ball array device to the sensor;

e) an image acquisition apparatus coupled to the sensor to transmit image information representing the first view and the second view; and

f) a processor, coupled to receive the image information, where the processor applies triangulation calculations to measurements of the image information so as to calculate a three dimensional position of at least one ball with reference to a pre-calculated calibration plane, wherein the calibration plane comprises a coordinate system having X, Y and Z axes, and wherein an X measurement value is proportional to a Z measurement value.

59. The three dimensional inspection apparatus of claim 58 wherein an XY measurement value is proportional to a Z measurement value.

60. The three dimensional inspection apparatus of claim 58 wherein a Y measurement value is proportional to a Z measurement value.

61. The three dimensional inspection apparatus of claim 58 wherein the pre-calculated calibration plane is defined by measuring a calibration pattern.

62. The three dimensional inspection apparatus of claim 58 wherein the measurements include grayscale edge detection to locate ball positions.

63. The three dimensional inspection apparatus of claim 58 wherein the measurements include threshold analysis.

64. The three dimensional inspection apparatus of claim 58 wherein the illuminator comprises a plurality of light emitting diodes.

65. The three dimensional inspection apparatus of claim 64, wherein at least two of the plurality of light emitting diodes are spectrally diverse from one another.

66. The three dimensional inspection apparatus of claim 58 wherein the illuminator comprises reflected light.

67. The three dimensional inspection apparatus of claim 58 wherein the ball grid array devices comprise bump on wafer devices.

68. The three dimensional inspection apparatus of claim 58 wherein the solid state sensor array includes a charged coupled device array.

69. The three dimensional inspection apparatus of claim 58 wherein the solid state sensor array includes a complementary metal oxide semiconductor array.

70. The three dimensional inspection apparatus of claim 58 wherein the second optical element comprises a mirror.

71. The three dimensional inspection apparatus of claim 58 wherein the second optical element comprises a prism.

72. The three dimensional inspection apparatus of claim 58 wherein the second view comprises a segment having a crescent shape.

73. The three dimensional inspection apparatus of claim 58 wherein the image acquisition apparatus comprises a frame grabber.

74. The three dimensional inspection apparatus of claim 58 wherein the processor comprises a personal computer.

75. The three dimensional inspection apparatus of claim 58 wherein the first optical element comprises a lens.

76. The three dimensional inspection apparatus of claim 58 wherein the first optical element comprises a pin-hole lens.

77. The three dimensional inspection apparatus of claim 58 wherein the first optical element comprises a plurality of lens elements.

78. The three dimensional inspection apparatus of claim 58 wherein the first optical element comprises a telecentric lens.

79. The three dimensional inspection apparatus of claim 58 further comprising a diffuser disposed to provide illumination for imaging of a perimeter of the ball grid array device.

80. The three dimensional inspection apparatus of claim 58, wherein the at least one ball on the ball grid array device being inspected comprises a contact.

81. The three dimensional inspection apparatus of claim 58, wherein the at least one ball on the ball grid array device being inspected comprises a pin.

82. The three dimensional inspection apparatus of claim 58, wherein the at least one ball on the ball grid array device being inspected is selected from the group consisting of:

bump contact, ball contact, pad, and pedestal.

83. The three dimensional inspection apparatus of claim 58, wherein the first view of the ball grid array device and the second view of the ball grid array device are obtained in a single image.

84. The three dimensional inspection apparatus of claim 58, wherein the first view of the ball grid array device and the second view of the ball grid array device are each obtained in a separate image.

85. A three dimensional inspection apparatus for ball grid array devices having a plurality of balls, wherein the ball grid array device is positioned in a fixed optical system, the apparatus comprising:

a) an illumination apparatus positioned for illuminating the ball grid array device;

b) a camera disposed in a fixed focus position relative to the ball grid array device for taking a first image of the ball grid array device to obtain a circular doughnut shape image from at least one ball;

c) an optical element disposed in a fixed focus position relative to the ball array device for transmitting a second image of the ball array device to the camera to obtain a side view image of the at least one ball; and

d) a processor, coupled to receive the first image and the second image, that applies triangulation calculations on related measurements of the first image and the second image to calculate a three dimensional position of the at least one ball with reference to a pre-calculated calibration plane.

86. The three dimensional inspection apparatus of claim 85 wherein the second image comprises a segment having a crescent shape.

87. The three dimensional inspection apparatus of claim 85 wherein the calibration plane comprises a coordinate system having X, Y and Z axes and wherein an X measurement value is proportional to a Z measurement value.

88. The three dimensional inspection apparatus of claim 85 wherein the triangulation calculations are based on determining a center of the ball in the first image and determining a ball top location in the second image.

89. The three dimensional inspection apparatus of claim 85 wherein the pre-calculated calibration plane is defined by measuring a calibration pattern.

90. The three dimensional inspection apparatus of claim 85 wherein the optical element comprises a mirror that reflects light between the ball grid array device and the camera.

91. The three dimensional inspection apparatus of claim 85 wherein the second image is obtained at a low angle of view.

92. The three dimensional inspection apparatus of claim 85 wherein the camera and the optical element are fixed at different angles relative to the calibration plane.

93. The three dimensional inspection apparatus of claim 85 wherein the camera comprises a charged coupled device array.

94. The three dimensional inspection apparatus of claim 85 wherein the measurements from the first image and the second image include grayscale edge detection to locate ball positions.

95. The three dimensional inspection apparatus of claim 85 wherein the illumination apparatus further comprises a diffuser.

96. The three dimensional inspection apparatus of claim 85 wherein the ball grid array devices comprise bump on wafer devices.

97. The three dimensional inspection apparatus of claim 85 wherein the camera comprises a complementary metal oxide semiconductor device array.

98. The three dimensional inspection apparatus of claim 85 wherein the triangulation calculations include threshold analysis.

99. The three dimensional inspection apparatus of claim 85, is wherein the at least one ball on the ball grid array device being inspected comprises a contact.

100. The three dimensional inspection apparatus of claim 85, wherein the at least one ball on the ball grid array device being inspected comprises a pin.

101. The three dimensional inspection apparatus of claim 85, wherein the at least one ball on the ball grid array device being inspected is selected from the group consisting of:

bump contact, ball contact, pad, and pedestal.

102. The three dimensional inspection apparatus of claim 85, wherein the circular doughnut shape image and the side view image of the at least one ball are obtained in a single image.

103. The three dimensional inspection apparatus of claim 85, wherein the circular doughnut shape image and the side view image of the at least one ball are each obtained in a separate image.

104. The three dimensional inspection apparatus of claim 85, wherein the illumination apparatus comprises a plurality of light emitting diodes.

105. The three dimensional inspection apparatus of claim 104, wherein at least two of the plurality of light emitting diodes are spectrally diverse from one another.

109. The apparatus of claim 89, wherein at least two of the more than one light source are spectrally diverse from one another.

126. The apparatus of claim 87, wherein the lead is selected from the group consisting of:

bump contact, ball contact, pad, and pedestal.

142. The apparatus of claim 104, wherein the one or more light sources provide a diffuse light for the bottom view of the lead.

106. An apparatus for three dimensional inspection of a lead on a ball grid array device, the apparatus comprising:

one or more light sources providing illumination to the lead;

fixed optical elements disposed so as to obtain both a bottom view of the lead and a side perspective view of the lead;

a single camera disposed so as to receive from the fixed optical elements at least the bottom view and the side perspective view of the lead;

a memory connected to receive from the camera as pixel values the bottom view and the side perspective view of the lead;

a processor connected to the memory, the processor implementing software instructions adapted to cause the processor to execute the following actions;

determining a first lead reference pixel position in the bottom view;

determining a second lead reference pixel position in the side view;

converting the first and second lead reference pixel positions into a world value by using pixel values and parameters determined during a calibration.

107. The apparatus of claim 106, wherein a single light source illuminates the lead.

108. The apparatus of claim 106, wherein more than one light source illuminates the lead.

110. The apparatus of claim 106, wherein the bottom view of the lead and a side perspective view of the lead are obtained in a single image.

111. The apparatus of claim 106, wherein the bottom view of the lead and a side perspective view of the lead are obtained in more than one image.

112. The apparatus of claim 106, wherein the parameters determined during the calibration are selected from the group consisting of:

pixel scale factors, an angle at a particular point in a view, and correspondence of one or more pixel values to world values.

113. The apparatus of claim 106, wherein the calibration includes resolving missing state values of an inspection system by imaging a precision pattern of known dimensions and spacing.

114. The apparatus of claim 106, wherein the calibration includes determining and storing pixel values of features of a precision pattern of known dimensions and spacing.

115. The apparatus of claim 106, wherein the calibration includes determining and storing deviations from ideal world locations of features of a precision pattern of known dimensions and spacing.

116. The apparatus of claim 106, wherein a z value is calculated by combining a deviation of the first lead reference pixel position from its ideal position with a deviation of the second lead reference pixel position from its ideal position.

117. The apparatus of claim 106, wherein the software instructions are further adapted to cause the processor to execute the further action of:

converting world values to Z deviations by calculating deviation values that represent the deviation of the lead from its ideal position.

118. The apparatus of claim 106, wherein the software instructions are further adapted to cause the processor to execute the further action of:

converting world values to coplanarity values by calculating deviation values that represent the deviation of the lead from a reference plane.

119. The apparatus of claim 106, wherein the software instructions are further adapted to cause the processor to execute the further action of:

converting world values to coplanarity values by calculating deviation values that represent the deviation of the lead from a seating plane.

120. The apparatus of claim 106, wherein the one or more light sources comprise a diffuse light.

121. The apparatus of claim 106, wherein the one or more light sources provide a diffuse light for the bottom view of the lead.

122. The apparatus of claim 106, wherein the one or more light sources provide a diffuse light for the side perspective view of the lead.

123. The apparatus of claim 106, wherein the one or more light sources comprise an overhead reflective diffuser to enhance an image of the outline of the ball grid array device.

124. The apparatus of claim 106, wherein the lead comprises a contact.

125. The apparatus of claim 106, wherein the lead comprises a pin.

127. An apparatus for three dimensional inspection of a lead on a ball grid array device, the method comprising:

one or more light sources providing illumination to the lead;

fixed optical elements disposed so as to obtain both a bottom view of the lead and a side perspective view of the lead;

a single camera disposed so as to receive from the fixed optical elements at least the bottom view and the side perspective view of the lead;

a memory connected to receive from the camera as pixel values the bottom view and the side perspective view of the lead;

a processor connected to the memory, the processor implementing software instructions adapted to cause the processor to execute the following actions;

determining a first lead reference pixel position in the bottom view;

determining a second lead reference pixel position in the side view;

converting the first lead reference pixel position into a first world value and the second lead reference pixel position into a second world value by using pixel values and parameters determined during a calibration.

128. The apparatus of claim 127, wherein a single light source illuminates the lead.

129. The apparatus of claim 127, wherein more than one light source illuminates the lead.

130. The apparatus of claim 129, wherein at least two of the more than one light source are spectrally diverse from one another.

131. The apparatus of claim 127, wherein the bottom view of the lead and a side perspective view of the lead are obtained in a single image.

132. The apparatus of claim 127, wherein the bottom view of the lead and a side perspective view of the lead are obtained in more than one image.

133. The apparatus of claim 127, wherein the parameters determined during the calibration are selected from the group consisting of:

pixel scale factors, an angle at a particular point in a view, and correspondence of one or more pixel values to world values.

134. The apparatus of claim 127, wherein the calibration includes resolving missing state values of an inspection system by imaging a precision pattern of known dimensions and spacing.

135. The apparatus of claim 127, wherein the calibration includes determining and storing pixel values of features of a precision pattern of known dimensions and spacing.

136. The apparatus of claim 127, wherein the calibration includes determining and storing deviations from ideal world locations of features of a precision pattern of known dimensions and spacing.

137. The apparatus of claim 127, wherein a Z value is calculated by combining a deviation of the first world value from its ideal position with a deviation of the second world value from its ideal position.

138. The apparatus of claim 127, wherein the software instructions are further adapted to cause the processor to execute the further action of:

converting world values to Z deviations by calculating deviation values that represent the deviation of the lead from its ideal position.

139. The apparatus of claim 127, wherein the software instructions are further adapted to cause the processor to execute the further action of:

converting world values to coplanarity values by calculating deviation values that represent the deviation of the lead from a reference plane.

140. The apparatus of claim 127, wherein the software instructions are further adapted to cause the processor to execute the further action of:

converting world values to coplanarity values by calculating deviation values that represent the deviation of the lead from a seating plane.

141. The apparatus of claim 127, wherein the one or more light sources comprise a diffuse light.

143. The apparatus of claim 127, wherein the one or more light sources provide a diffuse light for the side perspective view of the lead.

144. The apparatus of claim 127, wherein the one or more light sources comprise an overhead reflective diffuser to enhance an image of the outline of the ball grid array device.

145. The apparatus of claim 127, wherein the lead comprises a contact.

146. The apparatus of claim 127, wherein the lead comprises a pin.

147. The apparatus of claim 127, wherein the lead is selected from the group consisting of:

bump contact, ball contact, pad, and pedestal.