METHOD AND APPARATUS FOR 3-DIMENSIONAL VISION AND INSPECTION OF BALL AND LIKE PROTRUSIONS OF ELECTRONIC COMPONENTS

• US 20100328435A1
• Filed: 03/29/2007
• Published: 12/30/2010
• Est. Priority Date: 06/21/2006
• Status: Active Grant

• Current Assignee: Mit Semiconductor Company Limited
• Original Assignee: Generic Power Pte., Ltd. (Manufacturing Integration Technology Ltd. /2 Subsidiaries/)
• Inventors: Teo, Soon Poo, Puah, Yong Joo, Tang, Hak Wee
• Granted Patent: US 8,885,040 B2
• Time in Patent Office: Days
• US Class Current: 348/47
• CPC Class Codes: G01B 11/0608 Height gauges
• CPC Class Codes: G01N 21/8806 Specially adapted optical and illumination features
• CPC Class Codes: G01N 21/951 Balls
• CPC Class Codes: G01N 21/95684 Patterns showing highly reflecting parts, e.g. metallic elements
• CPC Class Codes: G06T 2200/04 involving 3D image data
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 2207/10152 Varying illumination
• CPC Class Codes: G06T 2207/30148 Semiconductor; IC; Wafer
• CPC Class Codes: G06T 2207/30152 Solder
• CPC Class Codes: G06T 2207/30244 Camera pose
• CPC Class Codes: G06T 7/0004 Industrial image inspection
• CPC Class Codes: G06T 7/0006 using a design-rule based approach
• CPC Class Codes: G06T 7/593 from stereo images
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G06T 7/80 Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 13/243 using three or more 2D image sensors
• CPC Class Codes: H04N 13/246 Calibration of cameras
• CPC Class Codes: H04N 13/254 in combination with electromagnetic radiation sources for illuminating objects
• CPC Class Codes: H05K 13/0818 Setup of monitoring devices prior to starting mounting operations; Teaching of monitoring devices for specific products; Compensation of drifts during operation, e.g. due to temperature shifts

Abstract

A method for 3-dimensional vision inspection of objects, such as microelectronic components, which have protrusions as features to be inspected, such as input/output contact balls, is disclosed. The method comprising the steps of determining interior and exterior parameters of a pair of cameras, forming a rectified coordinate system from the parameters of both cameras determined above with a relative pose of second image capturing means with respect to the first image capturing means. A pair of images of the object having a plurality of co-planar protrusions on one surface are captured by the cameras system wherein image coordinates of the images are transformed into coordinates of the rectified coordinate system. Conjugate points of each protrusion are then determined for the measurement of its co-planarity and height. Various configurations of the apparatus for capturing the images and processing the image data are also disclosed.

Claims

1. A method for 3-dimensional vision inspection of objects, including microelectronic components, having protrusions as features to be inspected, wherein said protrusions having surfaces that are non-planar and/or curvatures, said method comprising the steps of:

(a) providing at least a pair of image capturing means with similar interior parameters arranged in a stereovision configuration;

(b) calibrating each of said image capturing means to determine the interior and exterior parameters of said image capturing means;

(c) forming a rectified coordinate system from the parameters of both image capturing means determined above with a relative pose of second image capturing means with respect to the first image capturing means;

(d) capturing an image of an object to be inspected using each of said image-capturing means, wherein said object having a plurality of co-planar protrusions and said image captured is of at least one surface curvature of each of said protrusion;

(e) rectifying coordinates of each image to produce a pair of rectified images in the said rectified coordinate system;

(f) determining a conjugate point for the centre of each protrusion on each of the pair of images captured; and

(g) measuring the distance of each protrusion from at least one of the image capturing means within the rectified coordinate system and the co-planarity of the protrusions.

2. A method for 3-dimensional vision according to claim 1 wherein step (g) measuring the distance of each protrusion from at least one of the image capturing means and calculation of co-planarity of the protrusions comprising:

determining from the rectified coordinates using established mathematical functions the distance of each of the protrusions from at least one of the image capturing means.

3. A method for 3-dimensional vision according to claim 1 further including the step of:

(h) calculating the co-planarity of the protrusions based on the distance of each protrusion to form a seating plane using standard best-fit plane algorithm.

4. A method for 3-dimensional vision according to claim 3 further including the step of:

(i) calculating the height of the protrusions wherein;

the substrate plane is determined with reference to the rectified coordinates; and

perpendicular distance of protrusion to the substrate plane is taken as the protrusion height.

5. A method for 3-dimensional vision according to claim 4

wherein the protrusion height is calculated as follows:

(i) approximating with a second plane, in addition to the first seating plane, that is constructed with protrusions having good co-planarity values;

(ii) shifting said second plane by the amount of nominal protrusion height to form the substrate plane; and

(iii) taking the height of the protrusion as the perpendicular distance of said protrusion to the substrate plane.

6. A method for 3-dimensional vision according to claim 1 wherein the calibration in step (b) is determined in relation to the measurement plane.

7. A method according to claim 6 wherein the measurement plane is provided in the form of a calibration grid.

8. A method for 3-dimensional vision according to claim 1 wherein the conjugate points of each protrusion being determined in step (f) includes

(i) locating in each rectified image, the edges of the object to draw up the boundaries of the object;

and

(ii) performing standard blob analysis to locate the gross position of each protrusion.

9. A method for 3-dimensional vision according to claim 1 wherein step (f) further includes

(iii) projecting two edge-detecting windows onto each protrusion to locate its top and bottom edges up to sub-pixel accuracy;

(iv) projecting a third edge-detecting window from a central vertical position of the protrusion to detect the left edge of the protrusion in right image and the right edge of the protrusion in the left image up to sub-pixel accuracy; and

(v) assigning conjugate points to be right and left edges of the protrusion.

10. A method for 3-dimensional vision according to claim 9 wherein step (g) further includes

(vi) calculating disparity of the protrusion to be the difference between the positions of the said right and left edges;

and

(vii) determining the distance of the protrusion from the stereoscopic camera configuration utilizing the disparity and the relative pose computed during calibration.

11. A method for 3-dimensional vision according to any one of the preceding claims wherein the protrusion to be inspected has at least one of a round edge or ball tip, including pins of a pin grid array (PGA) package, balls of ball grid array (BGA), and like microelectronic packages.

23. An object which has been inspected according any one of the methods of claims 1-11 or with an apparatus according to any one of claims 12-22.

24. An object according to claim 23 which is any one of a microelectronic pin ball array (PGA) and ball grid array (BGA) packages.

12. An apparatus for 3-dimensional vision inspection of non-planar objects, including microelectronic components, wherein, for calibration and/or measurement purposes, said apparatus comprising:

(a) at least a pair of image capturing devices with similar interior parameters arranged in a stereovision configuration;

(b) a pair of reflective surfaces for reflecting an image of an object to be inspected to be captured by each of said image-capturing means; and

(c) means for mounting said object to be inspected to enable said object'"'"'s image to be so reflected and so captured;

wherein said object has a plurality of co-planar protrusions on at least one surface or substrate.

13. An apparatus for 3-dimensional vision inspection according to claim 12 wherein the object to be inspected is placed within a ring of light source and is illuminated thereby.

14. An apparatus for 3-dimensional vision inspection according to claim 13 wherein the light source comprises a plurality of light-emitting diodes (LED) with their light beam positioned at about an inclination of 25°

towards the object to be inspected.

15. An apparatus for 3-dimensional vision inspection according to claim 14 wherein a background light source is further provided.

16. An apparatus for 3-dimensional vision inspection according to claim 12 wherein the image capturing device comprises a camera and a frame grabber.

17. An apparatus for 3-dimensional vision inspection according to claim 16 wherein the frame grabbers are connected to a data processing means via a high-speed data transfer bus including an IEEE-1394 bus (also known as “

Firewire”

).

18. An apparatus for 3-dimensional vision inspection according to claim 17 wherein the data processing means is a computer capable of running at least a program for calibrating, rectifying and processing captured images.

19. An apparatus for 3-dimensional vision inspection according to any one of claims 12 to 18 further including a third image capturing device for another imaging process.

20. An apparatus for 3-dimensional vision inspection according to claim 19 wherein the third image capturing device is placed in between the pair of first two image capturing devices and is configured for inspecting another type of feature of the object, or another type of object.

21. An apparatus for 3-dimensional vision inspection according to claim 12 wherein a calibration reticle for having its image captured for calibration process is provided removably mounted.

22. An apparatus for 3-dimensional vision inspection according to claim 12 wherein the reflective surfaces are positioned at an inclination of about 22.5°

from parallel axis towards the object to be inspected so that images of said object are reflected at parallel into the image capturing devices.

25. A facility wherein an apparatus according to any one of claims 12 to 22 is installed.