Pick and place machine with component placement inspection

• US 20030110610A1
• Filed: 11/08/2002
• Published: 06/19/2003
• Est. Priority Date: 11/13/2001
• Status: Active Grant

• Current Assignee: CyberOptics Corporation (Nordson Corporation)
• Original Assignee: CyberOptics Corporation (Nordson Corporation)
• Inventors: Liberty, Todd D., Haugen, Paul R., Duquette, David W., Bushman, Thomas W., Fishbaine, David, Madsen, David D., Gaida, John D., Roth, Scott D., Dale, Theodore Paul, Buchika, Brant O.
• Granted Patent: US 7,239,399 B2
• Time in Patent Office: Days
• US Class Current: 29/407.09
• CPC Class Codes: H04N 13/194 Transmission of image signals
• CPC Class Codes: H04N 13/207 using a single 2D image sensor
• CPC Class Codes: H04N 13/218 using spatial multiplexing
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 13/254 in combination with electromagnetic radiation sources for illuminating objects
• CPC Class Codes: H04N 2013/0081 Depth or disparity estimation from stereoscopic image signals
• CPC Class Codes: H05K 13/0812 the monitoring devices being integrated in the mounting machine, e.g. for monitoring components, leads, component placement
• CPC Class Codes: H05K 13/0818 Setup of monitoring devices prior to starting mounting operations; Teaching of monitoring devices for specific products; Compensation of drifts during operation, e.g. due to temperature shifts
• CPC Class Codes: Y10T 29/49778 with aligning, guiding, or instruction
• CPC Class Codes: Y10T 29/53087 with signal, scale, illuminator, or optical viewer
• CPC Class Codes: Y10T 29/53191 Means to apply vacuum directly to position or hold work part

Abstract

Improved component placement inspection and verification is performed by a pick and place machine. Improvements include stereovision imaging of the intended placement location; enhanced illumination to facilitate the provision of relatively high-power illumination in the restricted space near the placement nozzle(s); optics to allow image acquisition device to view the placement location from an angle relative to a plane of the placement location, thereby reducing the possibility of such images being obstructed by the component; techniques for rapidly acquiring images with commercially available CCD arrays such that acquisition of before and after images does not substantially impact system throughput; and image processing techniques to provide component inspection and verification information.

Claims

1. A pick and place machine for placing a component upon a workpiece, the machine comprising:

a placement head having at least one nozzle for releasably holding the component;

a robotic system for generating relative movement between the placement head and the workpiece;

a first image acquisition device disposed to obtain a first view before-image of an intended placement location before placement of the component, and a first view after-image of the intended placement location after placement of the component;

a second image acquisition device disposed to obtain a second view before-image of an intended placement location before placement of the component, and a second view after-image of the intended placement location after placement of the component; and

wherein the first and second views are spaced apart to provide stereovision inspection of the intended placement location.

2. The machine of claim 1, wherein:

the first view is disposed at an angle relative to a plane of the intended placement location; and

a detector array within the first image acquisition device is disposed at an angle relative to an optical axis of the first image acquisition system.

3. The machine of claim 1, and further comprising:

an illuminator including a light source mounted remote from the nozzle; and

a light pipe optically coupled to the light source and having an output end directed to illuminate the intended placement location.

4. The machine of claim 3, wherein the illuminator provides light having a band of wavelengths, and wherein the at least the first image acquisition system includes a filter to pass light within the band and obstruct light outside the band.

5. The machine of claim 1, wherein the first image acquisition system employs non-telecentric optics.

6. The machine of claim 1, and further comprising a structured illuminator adapted to direct structured illumination upon the intended placement location, and wherein at least one of the first and second image acquisition devices obtains a structured illumination image of the intended placement location after component placement.

7. The machine of claim 1, wherein the first image acquisition device includes a CCD array having a frame rate, and wherein the first image acquisition device causes the CCD array to obtain the first view before image and the first view after image at a rate that exceeds the frame rate.

8. The machine of claim 7, wherein the CCD array is an interline transfer CCD array.

9. The machine of claim 7, wherein the CCD array is a frame transfer CCD array.

10. The machine of claim 7, wherein the first image acquisition device further includes an electronically controlled optical shutter device to obstruct excess light after the first view after image integration time has elapsed.

11. The machine of claim 1, and further comprising a structured illuminator adapted to direct structured illumination upon the intended placement location, and wherein at least one of the first and second image acquisition devices obtains a structured illumination image of the intended placement location before component placement.

12. The device of claim 1, wherein inspection of the intended placement location includes pre-placement inspection of the intended placement location.

13. An illuminator for a pick and place machine, the illuminator comprising:

a light source; and

a light pipe optically coupled to the light source, the light pipe having an output end adapted to be mounted to direct illumination upon an intended placement location within the pick and place machine.

14. The device of claim 13, wherein the light source is a light emitting diode.

15. The device of claim 13, wherein the light pipe is flexible.

16. The device of claim 13, wherein the light pipe is a fiberoptic light pipe.

17. The device of claim 13, and further comprising:

at least one additional light source; and

at least one additional light pipe coupled to the at least on additional light source, and having an output end.

18. The device of claim 17, wherein the light source and the at least one additional light source are embodied in an LED array.

19. The device of claim 17, wherein the output end of the light pipe and the at least one additional light pipe are mounted to direct illumination upon the intended placement location at a desired angle.

20. The device of claim 17, wherein the output end of the light pipe and the at least one additional light pipe are mounted to direct illumination upon the intended placement location with a desired shape.

21. The device of claim 17, wherein the light source and the at least one additional light source provide light of substantially the same wavelengths.

22. The device of claim 17, wherein the light source and the at least one additional light source provide light in substantially different wavelengths.

23. The device of claim 13, wherein the light source is adapted to mount on a placement head of the pick and place machine.

24. An image acquisition device for use in a pick and place machine, the device comprising:

a detector array;

optics having an optical axis to view an intended placement location at an angle with respect to a plane of the intended placement location and adapted to focus an image of the intended placement location on the detector array; and

wherein the detector array is disposed at an angle relative to the optical axis.

25. The device of claim 24 and further comprising a mirror optically interposed between the detector array and the intended placement location.

26. The device of claim 24, wherein the angle is selected to satisfy the Scheimpflug condition.

27. The device of claim 24, wherein the optics are non-telecentric.

28. The device of claim 27, and further comprising an optical stop interposed between the optics and the placement location.

29. The device of claim 28, wherein the stop is integral with the image acquisition device.

30. The device of claim 24, and further comprising:

an illuminator including;

a light source; and

a light pipe optically coupled to the light source, the light pipe having an output end adapted to be mounted to direct illumination upon an intended placement location within the pick and place machine.

31. The device of claim 30, wherein the illuminator is packaged together with the device.

32. A method of acquiring a plurality of images in a pick and place machine, the method comprising:

resetting photosites of a photosensitive array;

acquiring a first image on the photosites;

transferring the first image to a non-photosensitive area of the array;

resetting the photosites of the photosensitive array to clear the first image; and

generating a controlled exposure to acquire a second image before the first image is read completely from the non-photosensitive area of the array.

33. The method of claim 32, wherein transferring the first image to the non-photosensitive area includes transferring the first image to vertical registers of an interline transfer CCD array.

34. The method of claim 32, wherein transferring the first image to the non-photosensitive area includes transferring the first image to a storage area of a frame transfer CCD array.

35. The method of claim 32, wherein the first and second images are smaller than the size of an imaging area of the array.

36. The method of claim 35, and further comprising positioning the photosensitive array at an angle relative to an optical axis providing the first and second images.

37. The method of claim 32, wherein generating a controlled exposure of the second image includes obstructing all light from falling upon the photosensitive array after an integration period of the second image has elapsed.

38. The method of claim 37, wherein obstructing all light includes causing an electronically controlled optical shutter to enter a closed state thereby preventing light from passing therethrough.

39. The method of claim 32, and further comprising:

hindering illumination not within a selected wavelength band from falling upon the photosensitive array; and

wherein generating the controlled exposure includes providing illumination within the selected wavelength band upon a placement location for a selected exposure time.

40. A method of acquiring a plurality of images of a placement location in a pick and place machine, the method comprising:

resetting photosites of a first photosensitive array;

acquiring a first image on the photosites of the first photosensitive array;

resetting photosites of a second photosensitive array; and

acquiring a second image on the photosites of the second photosensitive array;

wherein the first and second arrays acquire images sequentially through common optics.

41. The method of claim 40, wherein the first and second photosensitive arrays acquire images based on substantially monochromatic illumination.

42. A method of processing a plurality of images of an intended placement location in a pick and place machine to verify component placement, the method comprising:

acquiring before-placement and after-placement images of an intended placement location;

selecting a region of interest in the before-placement and after-placement images;

generating a difference image based upon differences between the before-image and the after-image;

thresholding the difference image to produce a binary difference image;

performing connectivity analysis on the binary difference image to obtain an image metric; and

comparing the image metric with expected object metric to provide component placement verification.

43. The method of claim 42, and further comprising masking at least one area known to change in the before and after images.

44. The method of claim 43, wherein the area includes a nozzle of the pick and place machine.

45. The method of claim 42, and further comprising masking at least one area not within an expected envelope of the component.

46. The method of claim 42, and further comprising:

aligning the before-image with the after-image before obtaining the difference image.

47. The method of claim 46, wherein the step of aligning includes applying a correlation.

48. The method of claim 42, and further comprising:

performing a morphological erosion operation on the binary difference image.

49. The method of claim 48, wherein the morphological erosion operation is repeated.

50. The method of claim 42, and further comprising performing a morphological dilation operation on the binary difference image.

51. The method of claim 50, wherein the morphological dilation operation is repeated.

52. The method of claim 42, wherein the metric includes object centroid.

53. The method of claim 42, wherein the metric includes object length.

54. The method of claim 42, wherein the metric includes object width.

55. The method of claim 42, wherein the metric includes object area.

56. The method of claim 42, wherein the metric includes object rotation.

57. The method of claim 42, wherein the method is performed within an image acquisition device.

58. A method of processing a plurality of images of an intended placement location in a pick and place machine to verify component placement, the method comprising:

acquiring before-placement and after-placement images of an intended placement location;

selecting a region of interest in the before-placement and after-placement images;

generating a difference image based upon differences between the before-image and the after-image;

thresholding the difference image to produce a binary difference image;

calculate pixel change count of the intended placement location;

calculate total pixel count; and

comparing the pixel change count of the intended placement location with the total pixel count to provide component placement verification.

59. A method of processing a plurality of images of an intended placement location in a pick and place machine to verify component placement, the method comprising:

acquiring before-placement and after-placement images of an intended placement location;

selecting a region of interest in the before-placement and after-placement images;

generating a difference image based upon differences between the before-image and the after-image;

calculate difference intensity sum of the intended placement location;

calculate total pixel difference intensity sum; and

comparing the pixel difference intensity sum of the intended placement location with the total pixel difference intensity sum to provide component placement verification.

60. A pick and place machine for placing a component upon a workpiece, the machine comprising:

an image acquisition system having a CCD array;

optics including multiple image paths to simultaneously focus a plurality of images of an intended placement location upon the CCD array;

wherein the image acquisition system acquires a plurality of before placement images of the intended location before the component is placed, and acquires a plurality of after placement images of the intended placement location after the component is placed; and

a host controller coupled to the image acquisition system compares the before placement images with the after placement images to verify component placement.

61. A method of calibrating an off-axis image system in a pick and place machine, the method comprising:

placing a known target in an intended placement location;

acquiring an off-axis image of the known target;

determining initial positions of known artifacts on the known target from the acquired image; and

calculating a transformation matrix.

62. The method of claim 61, and further comprising:

after the step of acquiring an off-axis image, presenting the known target at a different position along a z-axis;

acquiring at least one additional image of the known target at the different position; and

determining adjusted positions of known artifacts on the known target from the at least one additional image; and

comparing the initial positions to the adjusted positions to refine the transformation matrix.

63. A method of calibrating a plurality of image acquisition devices in a pick and place machine, the method comprising:

placing a known target in an intended placement location;

acquiring an image of the known target with each image acquisition device;

determining positions of known artifacts on the known target from the acquired images; and

comparing the determined positions from each acquired image to calibrate for depth.