Method and apparatus using image subtraction and dynamic thresholding

• US 6,061,476 A
• Filed: 11/24/1997
• Issued: 05/09/2000
• Est. Priority Date: 11/24/1997
• Status: Expired due to Term

• Current Assignee: Cognex Corporation
• Original Assignee: Cognex Corporation
• Inventors: Nichani, Sanjay
• Primary Examiner(s): LEE, TOMMY D
• Application Number:
  US08/976,931
• Time in Patent Office: 897 Days
• Field of Search: 382/270, 382/141, 382/145, 382/149, 358/465, 358/466
• US Class Current: 382/270
• CPC Class Codes: G06T 2207/30141 Printed circuit board [PCB]
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/136 involving thresholding
• CPC Class Codes: G06T 7/174 involving the use of two or more images
• CPC Class Codes: G06V 10/28 Quantising the image, e.g. histogram thresholding for discrimination between background and foreground patterns

Abstract

A method and apparatus implementing dynamic thresholding techniques in processes that use image subtraction. The subtracted image is separated into a positive difference image using a positive image map and a negative difference image using a negative image map. A positive threshold map and a negative threshold map process a Gaussian filtered first image to generate respective positive and negative dynamic threshold images. The positive and negative threshold map mask out irrelevant areas and implement adaptive gain in pixel thresholds proportional to a difference or contrast between foreground or feature of interest and local background. Dynamic thresholding is a subtraction providing a subtracted positive image and a subtracted negative image. The subtracted positive and subtracted negative images are combined to provide a merged image which is binarized. Edge or Interface detection involves edge detection and true peak detection to find the edge peaks in a pre-image and post image. A binarization map converts edge peak pre and post images and the post image is then dilated to compensate any mis-registration. The dilated post image is subtracted from the binary edge pre-image to form an edge subtraction image. Pixels absent from the post image are assumed to be occluded by a feature of interest. The two binary images are merged, filtered, and connectivity is effected to obtain position and size of contiguous regions of interest.

Claims

1. A method for providing dynamic thresholding in an automated inspection system:

said method comprising the steps of;

capturing with an image acquisition device a first image comprising a plurality of first image pixels and a second image comprising a plurality of second image pixels;

subtracting gray level values of said second image pixels from gray level values of said first image pixels to provide a subtracted image having a plurality of subtracted image pixel values;

processing said subtracted image pixel values with a positive image pixel map to generate a positive difference image consisting of positive difference values from said subtracted image pixels values;

processing said subtracted image pixel values with a negative image pixel map to generate a negative difference image consisting of negative difference values from said subtracted image pixels values;

generating a positive dynamic threshold image as a function of said gray level values of said first image pixels and said gray level values of said second image pixels;

generating a negative dynamic threshold image as a function of said gray level values of said first image pixels and said gray level values of said second image pixels;

subtracting said positive difference image from said positive dynamic threshold image, to implement thresholding and provide a subtracted positive image;

subtracting said negative difference image from said negative dynamic threshold image, to implement thresholding and provide a subtracted negative image;

converting said subtracted positive image and said subtracted negative image to provide, respectively, a positive results image and a negative results image wherein pixels that exceed respective threshold values are set to a first number and pixels that do not exceed respective threshold values are set to a second number;

merging said positive results image and said negative results image to provide a merged image; and

binarizing said merged image to provide a binarized foreground/background detection output wherein all positive values representing features of interest are set to a first binarization number and all zero values are set to a second binarization number.

2. The method of claim 1 wherein prior to said step of subtracting, said first and said second image are filtered with a median filter.

3. The method of claim 1 wherein said first image is subjected to a Gaussian filter prior to said steps of generating said positive threshold map and generating said negative threshold map.

4. The method of claim 1 wherein said negative difference values are absolute values of negative difference values from said subtracted image pixels values.

5. The method of claim 1 wherein said step of converting said subtracted positive image and said subtracted negative image is effected by passing each of said subtracted positive image and said subtracted negative image through a common conversion map to eliminate negative values and wherein pixels that exceed respective threshold values are set to 255 and pixels that do not exceed respective threshold values are set to zero.

6. The method of claim 1 further including interface detection processing, comprising the steps of:

processing said plurality of first image pixels and said plurality of second image pixels through an edge detector to determine, respectively, a first pixel interface map including a pixel value and orientation of at least one first image interface pixel and a second pixel interface map including a pixel value and orientation of any second image interface pixel;

eliminating spurious and superfluous interface pixels from said first pixel interface map and said second pixel interface map to provide, respectively, an interface peak first image and an interface peak second image;

binarizing said interface peak first image and said interface peak second image to provide a binarized interface first image wherein a first interface binarization value indicates interface peak pixels and a second interface binarization value indicates non-interface pixels;

dilating said interface peak second image to compensate for sub-pixel mis-registration and provide a dilated second image; and

subtracting said dilated second image from said binarized interface first image to form an interface subtraction image, wherein any interface peak pixel present in said interface peak first image and absent from said interface peak second image yields an interface value indicating that it is occluded in said interface peak second image.

7. The method of claim 6 further including the steps of,

processing said interface subtraction image to get rid of any negative values and provide an interface difference image having all positive and zero values;

dilating said interface difference image to provide a dilated interface difference image; and

binarizing said dilated interface difference image to provide a binarized interface detection output wherein all positive values representing features of interest are set to said first binarization number and all zero values are set to said second binarization number.

8. The method of claim 7 further including the steps of:

merging said binarized foreground/background detection output and said binarized interface detection to form a merged binary image; and

processing said merged binary image in a connectivity operation in accordance with a specified minimum feature area, to obtain an indication of position and size of contiguous features of interest.

9. The method of claim 8 wherein, prior to processing said merged binary image in a connectivity operation said merged binary image is subjected to morphological filtering to effect cleanup of noise.

10. The method of claim 7 wherein said binarized foreground/background detection output and said binarized interface detection output are merged using a binary-OR operation, to form said merged binary image.

11. The method of claim 6 wherein a step of filtering said first image and said second image with a median filter to reduce noise is performed prior to said step of processing said first and second images through said edge detector.

12. The method of claim 6 wherein in said binarizing step a binarization map is used to convert said interface peak first image and said interface peak second image to respective 8-bit binary images wherein each of said interface peak pixels is denoted by a value of 255 constituting said first interface pixel binarization value and non-interface pixels are denoted by a value of 0 constituting said second interface pixel binarization value.

13. The method of claim 6 wherein said method is implemented in a pre-post solder paste inspection system and said first image is a pre-image of a substrate before solder is deposited thereon and said second image is a post image of said substrate after solder is deposited thereon.

14. The method of claim 1 wherein said steps of generating said positive dynamic threshold image and said negative dynamic threshold image as a function of said gray level values of said first image pixels and said gray level values of said second image pixels involves using a positive threshold map and a negative threshold map which are pixel maps that:

mask out areas that are not relevant, and implement adaptive gain in a form of pixel thresholds that are proportional to a difference in gray level value between a gray level value of features of interest in a foreground and gray level value of a local background.

15. A method for inspecting solder paste deposited on a circuit board having solder pads disposed thereon for receiving solder paste, comprising:

capturing a pre-image of said circuit board before solder paste is deposited thereon, said pre-image comprising a plurality of pre-image pixels, said image an image acquisition device capturing a post-image of said circuit board after solder paste is deposited thereon, said post-image comprising a plurality of post-image pixels;

filtering said pre-image and said post-image to provide, respectively filtered pre-image pixels and filtered post-image pixels;

subtracting gray level values of said filtered post-image pixels from gray level values of said filtered pre-image pixels to provide a subtracted image having a plurality of subtracted image pixel values;

processing said subtracted image pixel values with a positive image pixel map to generate a positive difference image consisting of positive difference values from said subtracted image pixels values;

processing said subtracted image pixel values with a negative image pixel map to generate a negative difference image consisting of negative difference values from said subtracted image pixels values;

filtering said pre-image with an averaging filter to generate average filtered pre-image pixels;

generating a positive dynamic threshold image as a function gray level values of said average filtered pre-image pixels and said gray level values of said post-image pixels;

generating a negative dynamic threshold image as a function of gray level values of said average filtered pre-image pixels and said gray level values of said post-image pixels;

subtracting said positive difference image from said positive dynamic threshold image, to implement thresholding and provide a subtracted positive image;

subtracting said negative difference image from said negative dynamic threshold image, to implement thresholding and provide a subtracted negative image;

converting said subtracted positive image and said subtracted negative image by passing each of said subtracted positive image and said subtracted negative image through a common conversion map to eliminate negative values to provide, respectively, a positive results image and a negative results image wherein pixels that exceed respective threshold values are set to a first number and pixels that do not exceed respective threshold values are set to a second number;

merging said positive results image and said negative results image to provide a merged image; and

binarizing said merged image to provide a binarized foreground/background detection output wherein all positive values represent one of solder paste disposed on solder pads and solder paste disposed on said circuit board, said positive values being set to a first binarization number and all zero values being set to a second binarization number.

16. The method of claim 15 wherein said negative difference values are absolute values of negative difference values from said subtracted image pixels values.

17. The method of claim 15 wherein pixels that exceed respective threshold values are set to 255 and pixels that do not exceed respective threshold values are set to zero.

18. The method of claim 15 further including interface detection processing to detect interfaces where solder pads meet said circuit board, comprising the steps of:

processing said plurality of pre-image pixels and said plurality of post-image pixels through an edge detector to determine, respectively, a first pixel interface map including a pixel value and orientation of at least one first image interface pixel and a second pixel interface map including a pixel value and orientation of any second image interface pixel;

eliminating spurious and superfluous interface pixels from said first pixel interface map and said second pixel interface map to provide, respectively, an interface peak pre-image and an interface peak post-image;

binarizing said interface peak pre-image and said interface peak post-image to provide a binarized interface image wherein a first interface binarization value indicates interface peak pixels and a second interface binarization value indicates non-interface pixels;

dilating said interface peak post-image to compensate for sub-pixel mis-registration and provide a dilated post-image; and

subtracting said dilated post-image from said binarized interface image to form an interface subtraction image, wherein any interface peak pixel present in said interface peak image and absent from said interface peak post-image yields an interface value indicating that it is occluded by solder paste in said interface peak post-image.

19. The method of claim 18 further including the steps of,

processing said interface subtraction image to get rid of any negative values and provide an interface difference image having all positive and zero values;

dilating said interface difference image to provide a dilated interface difference image; and

binarizing said dilated interface difference image to provide a binarized interface detection output wherein all positive values representing features of interest are set to said first binarization number and all zero values are set to said second binarization number.

20. The method of claim 19 further including the steps of:

merging said binarized foreground/background detection output and said binarized interface detection to form a merged binary image; and

processing said merged binary image in a connectivity operation in accordance with a specified minimum area, to obtain an indication of position and size of contiguous solder paste regions.

21. The method of claim 20 wherein, prior to processing said merged binary image in a connectivity operation said merged binary image is subjected to morphological filtering to effect cleanup of noise.

22. The method of claim 15 wherein said steps of generating said positive dynamic threshold image and said negative dynamic threshold image as a function of said gray level values of said first image pixels and said gray level values of said second image pixels involves using a positive threshold map and a negative threshold map which are pixel maps that:

mask out areas that are not relevant, and implement adaptive gain in a form of pixel thresholds that are proportional to a difference in gray level value between solder paste in a foreground and a local background.