System and method for dynamic image recognition

• US 6,771,819 B2
• Filed: 04/22/2002
• Issued: 08/03/2004
• Est. Priority Date: 07/28/1999
• Status: Expired due to Fees

• Current Assignee: Intelligent Reasoning Systems, Inc. (KLA Corporation)
• Original Assignee: Intelligent Reasoning Systems, Inc. (KLA Corporation)
• Inventors: Eskridge, Thomas C., Newberry, Jeff E., DeYong, Mark R., Grace, John W.
• Primary Examiner(s): Mehta, Bhavesh M.
• Assistant Examiner(s): CHAWAN, SHEELA C
• Application Number:
  US10/127,518
• Publication Number:
  US 20020159643A1
• Time in Patent Office: 834 Days
• Field of Search: 382/141, 382/148, 382/159, 382/281, 382/228, 382/282, 382/149, 382/173, 382/190, 382/294, 382/151, 382/225, 382/305, 348/92, 348/126, 348/125, 348/128, 348/87, 702/35, 356/239.3, 706/900, 706/908
• US Class Current: 382/228
• CPC Class Codes: G06F 18/00 Pattern recognition
• CPC Class Codes: G06T 2207/30141 Printed circuit board [PCB]
• CPC Class Codes: G06T 7/0008 checking presence/absence
• CPC Class Codes: G06V 2201/06 Recognition of objects for industrial automation
• CPC Class Codes: H05K 13/0815 Controlling of component placement on the substrate during or after manufacturing
• CPC Class Codes: Y10S 706/908 Electronic or computer, internal or network, circuit

Abstract

A system and method for object inspection that includes an image recognition program stored on a tangible medium for classifying and subclassifying regions of interest on an image. The image recognition program can be used in an image inspection system to determine defects on objects such as printed wiring assemblies. The image recognition program is executable to collect raw image data, segment out rectangular regions of interest that can be component sites defined by CAD data, preprocess each region of interest by scaling, gain and offset correction, and gamma correction, generating a set of image spaces for each region of interest using a set of spatial image transforms, generating features on the image spaces, scoring the features, comparing the feature scores to a knowledge base of feature scores to make a class determination for the features, generating a presence/absence decision confidence for the features, calculating a class determination and decision confidence for each region of interest. The class determinations can include present or absent, and right or wrong polarity. Another aspect of the image recognition software includes the ability to subclassify the defects. Another aspect of the present invention includes the ability to incrementally train the existing knowledge base as more images are processed.

Claims

1. A method for dynamic image recognition, comprising:

collecting raw image data from at least one imaged object;

segmenting out a region of interest;

performing at least one spatial image transform on the raw image data to generate a set of derived spaces for the region of interest; and

clustering the region of interest to generate a set of features for each derived space based on a set of clustering rules to generate arbitrary features on the region of interest for each derived space.

2. The method of claim 1, wherein the region of interest is a component site defined by CAD data.

3. The method of claim 1, further comprising preprocessing the region of interest, comprising;

performing a region of interest scaling;

performing a gain and offset correction; and

performing a gamma correction.

4. The method of claim 1, wherein each step is performed for a plurality of regions of interest.

5. The method of claim 1, wherein said clustering the region of interest to generate a set of features occurs prior to defect detection.

6. The method of claim 1, further comprising:

scoring each feature for each region of interest on each of the derived spaces;

classifying each feature for presence/absence;

calculating a decision confidence for each feature;

calculating a relative decision confidence based on past decision confidences; and

classifying a feature as a defect based on the decision confidence and relative decision confidence for that feature.

7. A method for dynamic image recognition, comprising:

collecting raw image data from at least one imaged object;

segmenting out a region of interest;

performing at least one spatial image transform to generate a set of derived spaces for the region of interest; and

clustering the region of interest to generate a set of features for each derived space based on a set of clustering rules to generate arbitrary features on the region of interest for each derived space; and

wherein performing a spatial image transform to generate a set of derived spaces for the region of interest, comprises;

deriving a reduced zero order magnitude space;

deriving a reduced first order magnitude space;

deriving a first order direction space;

deriving a reduced second order magnitude space; and

deriving a second order direction space.

8. The method of claim 7, wherein the deriving the set of derived spaces further comprises:

deriving the reduced zero order magnitude space by reducing each raw image pixel gray level value to a zero order reduced gray level value according to a zero order reduction transform;

deriving the reduced first order magnitude space by creating a first order magnitude space by determining the maximum difference between the raw pixel gray level value for each pixel and its eight immediately adjacent pixels and then reducing the first order magnitude space according to a first order reduction transform;

deriving the first order direction space from the raw image by determining a first order direction code for each pixel based on the direction of the lowest of the eight adjacent pixels from each pixel according to a first order direction map;

deriving the reduced second order magnitude space by creating a second order magnitude space from the first order magnitude space by determining the maximum difference between the first order pixel gray level value for each pixel and its eight immediately adjacent pixels and then reducing the second order magnitude space according to a second order reduction transform;

deriving the second order direction space from the first order magnitude space by determining a second order direction code for each pixel based on the direction of the lowest of the eight adjacent pixels from each pixel according to a second order direction map.

9. A method for dynamic image recognition, comprising:

collecting raw image data from at least one imaged object;

segmenting out a region of interest;

performing at least one spatial image transform to generate a set of derived spaces for the region of interest;

clustering the region of interest to generate a set of features for each derived space based on a set of clustering rules to generate arbitrary features on the region of interest for each derived space;

creating a presence/absence knowledge base by;

generating each presence/absence feature for each derived space from at least one blank object;

scoring each presence/absence feature for each derived space from a first blank object; and

scoring each presence/absence feature for each derived space from every other blank object and each assembled object imaged;

creating a polarity knowledge base by;

generating each polarity feature for each derived space from a first assembled object;

rotating the feature images to generate wrong polarity features; and

scoring each polarity feature for each derived space for every assembled object imaged.

10. The method of claim 9, wherein the number of presence/absence features and the number of polarity features generated in each derived space is not predetermined prior to generation.

11. The method of claim 9, wherein executing the image recognition software to score the features further comprises:

scoring features in each direction space by giving each feature a score equal to the number of direction codes on the image that match the direction codes of the feature in the knowledge base according to the formula;

12. The method of claim 11, further comprising executing the image recognition software to:

determine a mean and standard deviation for a predetermined sample of imaged objects for both presence/absence feature scores and polarity feature scores;

calculate a pruning constant for both presence/absence and polarity;

selecting the presence/absence features and the polarity features to be used for scoring based upon predetermined pruning rules; and

pruning the features in the knowledge base.

13. A system for dynamic image recognition, comprising:

a processor; and

an image recognition computer software program stored in computer-readable form on a storage medium and executable to;

collect raw image data from at least one imaged object;

segment out a region of interest;

perform at least one spatial image transform on the raw image data to generate a set of derived spaces for the region of interest; and

cluster the region of interest to generate a set of features for each derived space based on a set of clustering rules to generate arbitrary features on the region of interest for each derived space.

14. The system of claim 13, wherein the image recognition software performs each step on a plurality of regions of interest.

15. The system of claim 13, wherein said image recognition computer software program is executable to cluster the region of interest to generate a set of features prior to defect detection.

16. The system of claim 13, wherein said image recognition computer software program is executable to:

score each feature for each region of interest on each of the derived spaces;

classify each feature for presence/absence;

calculate a decision confidence for each feature;

calculate a relative decision confidence based on past decision confidences; and

classify a feature as a defect based on the decision confidence and relative decision confidence for that feature.

17. A system for dynamic image recognition, comprising:

a processor; and

an image recognition computer software program stored in computer-readable form on a storage medium and executable to;

collect raw image data from at least one imaged object;

segment out a region of interest;

perform at least one spatial image transform to generate a set of derived spaces for the region of interest, wherein said spatial image transform to generate a set of derived spaces for the region of interest, comprises;

deriving a reduced zero order magnitude space;

deriving a reduced first order magnitude space;

deriving a first order direction space;

deriving a reduced second order magnitude space; and

deriving a second order direction space; and

cluster the region of interest to generate a set of features for each derived space based on a set of clustering rules to generate arbitrary features on the region of interest for each derived space.

18. The system of claim 17, wherein the image recognition program is further executable to:

derive the reduced zero order magnitude space by reducing each raw image pixel gray level value to a zero order reduced gray level value according to a zero order reduction transform;

derive the reduced first order magnitude space by creating a first order magnitude space by determining the maximum difference between the raw pixel gray level value for each pixel and its eight immediately adjacent pixels and then reducing the first order magnitude space according to a first order reduction transform;

derive the first order direction space from the raw image by determining a first order direction code for each pixel according to a first order direction map;

derive the reduced second order magnitude space by creating a second order magnitude space from the first order magnitude space by determining the maximum difference between the first order pixel gray level value for each pixel and its eight immediately adjacent pixels and then reducing the second order magnitude space according to a second order reduction transform; and

derive the second order direction space from the first order magnitude space by determining a second order direction code for each pixel based on the direction of the lowest of the eight adjacent pixels from each pixel according to a second order direction map.

19. A system for dynamic image recognition, comprising:

a processor; and

an image recognition computer software program stored in computer-readable form on a storage medium and executable to;

collect raw image data from at least one imaged object;

segment out a region of interest;

perform at least one spatial image transform to generate a set of derived spaces for the region of interest; and

cluster the region of interest to generate a set of features for each derived space based on a set of clustering rules to generate arbitrary features on the region of interest for each derived space;

create a presence/absence knowledge base by;

generating each presence/absence feature for each derived space from at least one blank object;

scoring each presence/absence feature for each derived space from a first blank object; and

scoring each presence/absence feature for each derived space from every other blank object and each assembled object imaged; and

create a polarity knowledge base by;

generating each polarity feature for each derived space from a first assembled object;

rotating the feature images to generate wrong polarity features; and

scoring each polarity feature for each derived space for every assembled object imaged.

20. The system of claim 19, wherein the image recognition software is further executable to:

score features in each direction space by giving each feature a score equal to the number of direction codes on the image that match the direction codes of the feature in the knowledge base according to the formula;

21. The system of method of claim 20, wherein the presence/absence defect subclassifications include skewed, tombstoned, billboarded, damaged, new, and wrong components and foreign material.