Integration of automatic and manual defect classification

• US 10,043,264 B2
• Filed: 04/19/2012
• Issued: 08/07/2018
• Est. Priority Date: 04/19/2012
• Status: Active Grant

• Current Assignee: Applied Materials Israel Limited (Applied Materials Incorporated)
• Original Assignee: Applied Materials Israel Limited (Applied Materials Incorporated)
• Inventors: Greenberg, Gadi, Kaizerman, Idan, Rozenman, Efrat
• Primary Examiner(s): Wang, Carol
• Application Number:
  US13/451,496
• Publication Number:
  US 20130279794A1
• Time in Patent Office: 2,301 Days
• CPC Class Codes: G06F 18/2411 based on the proximity to a decision surface, e.g. support vector machines
• CPC Class Codes: G06F 18/2433 Single-class perspective, e.g. one-against-all classification; Novelty detection; Outlier detection
• CPC Class Codes: G06T 2207/10061 from scanning electron microscope
• CPC Class Codes: G06T 2207/30148 Semiconductor; IC; Wafer
• CPC Class Codes: G06T 7/001 using an image reference approach
• CPC Class Codes: G06V 10/764 using classification, e.g. of video objects
• CPC Class Codes: G06V 10/987 with the intervention of an operator

Abstract

A method for defect classification includes storing definitions of defect classes in terms of a classification rules in a multi-dimensional feature space. Inspection data associated with defects detected in one or more samples under inspection is received. A plurality of first classification results is generated by applying an automatic classifier to the inspection data based on the definitions, the plurality of first classification results comprising a class label and a corresponding confidence level for a defect. Upon determining that a confidence level for a defect is below a predetermined confidence threshold, a plurality of second classification results are generated by applying at least one inspection modality to the defect. A report is generated comprising a distribution of the defects among the defect classes by combining the plurality of first classification results and the plurality of second classification results.

Claims

1. A method for defect classification comprising:

storing, by a processor, a plurality of definitions of a plurality of defect classes in terms of a plurality of classification rules in a multi-dimensional feature space, wherein the plurality of classification rules, for each given defect class, defines in a feature space a boundary of a region associated with the given class and provides a confidence measure associated with classification of a defect to the given defect class, the confidence measure being indicative of a level of confidence as a function of the location of the defect in the feature space with respect to the respective boundaries;

receiving, by the processor, inspection data associated with a plurality of defects detected in one or more samples under inspection;

receiving, by the processor and from an operator, a classification performance measure selected from a plurality of performance measures, wherein the plurality of performance measures comprises at least one of a maximum rejection rate or a target purity level;

determining at least one confidence threshold corresponding to the classification performance measure;

applying, by the processor, an automatic classifier to the inspection data, the automatic classifier based on the plurality of definitions, and identifying a plurality of defects each classified with a low level of confidence based on the at least one confidence threshold and indicative of the defect being located in an overlap region between the respective boundaries of at least two of the defect classes;

generating, by the processor, a plurality of classification results by applying, to the identified plurality of defects classified with the low level of confidence, at least one inspection modality that is different than the automatic classifier to assign each of the identified plurality of defects to one of the at least two of the defect classes associated with the overlap region;

refining, by the processor, the automatic classifier to adjust boundaries of one or more defect classes of the plurality of classes when a threshold amount of the identified plurality of defects located in the overlap region have been classified by the at least one inspection modality that is different than the automatic classifier, wherein the refining is provided by training the automatic classifier using each classification result of the plurality of classification results of the identified plurality of defects classified with the low level of confidence.

2. The method of claim 1, further comprising:

determining to each given defect class a confidence threshold of the given defect class, wherein a confidence threshold of the given defect class is based on the loci of the respective boundary in the feature space.

3. The method of claim 2, wherein an extent of a boundary in the feature space is controlled by a confidence level of a respective defect class.

4. The method of claim 2, wherein the plurality of definitions of the plurality of defect classes comprises a kernel function having a parameter, and wherein applying the automatic classifier to the inspection data comprises selecting a value of the parameter for the plurality of defect classes based on the at least one confidence threshold.

5. The method of claim 2, wherein at least two defect classes have differently determined confidence thresholds.

6. The method of claim 1, wherein the at least one inspection modality that is different than the automatic classifier corresponds to a visual inspection, and wherein applying the automatic classifier to the inspection data comprises applying a multi-class classifier to the inspection data to assign defects of the plurality of defects to a defect class of the plurality of defect classes, wherein the multi-class classifier is configured to identify the defects in the overlap region.

7. An apparatus comprising:

a memory to store a plurality of definitions of a plurality of defect classes in terms of a plurality of classification rules in a multi-dimensional feature space, wherein the plurality of classification rules, for each given defect class, defines in a feature space a boundary of a region associated with the given class and provides a confidence measure associated with classification of a defect to the given defect class, the confidence measure being indicative of a level of confidence as a function of the location of the defect in the feature space with respect to the respective boundaries; and

a processor, operatively coupled with the memory, to;

receive inspection data associated with a plurality of defects detected in one or more samples under inspection;

receive, from an operator, a classification performance measure selected from a plurality of performance measures, wherein the plurality of performance measures comprises at least one of a maximum rejection rate or a target purity level;

determine at least one confidence threshold corresponding to the classification performance measure;

apply an automatic classifier to the inspection data, the automatic classifier based on the plurality of definitions, and identifying a plurality of defects each classified with a low level of confidence based on the at least one confidence threshold and indicative of the defect being located in an overlap region between the respective boundaries of at least two of the defect classes;

generate a plurality of classification results by applying, to the identified plurality of defects classified with the low level of confidence, at least one inspection modality that is different than the automatic classifier to assign each of the identified plurality of defects to one of the at least two of the defect classes associated with the overlap region; and

refine the automatic classifier to adjust boundaries of one or more defect classes of the plurality of classes when a threshold amount of the identified plurality of defects located in the overlap region have been classified by the at least one inspection modality that is different than the automatic classifier, wherein the refining is provided by training the automatic classifier using each classification result of the plurality of classification results of the identified plurality of defects classified with the low level of confidence.

8. The apparatus of claim 7, wherein the processor is further to:

determine to each given defect class a confidence threshold of the given defect class, wherein a confidence threshold of the given defect class is based on the loci of the respective boundary in the feature space.

9. The apparatus of claim 8, wherein an extent of a boundary in the feature space is controlled by a confidence level of a respective defect class.

10. The apparatus of claim 8, wherein the plurality of definitions of the plurality of defect classes comprises a kernel function having a parameter, and wherein applying the automatic classifier to the inspection data comprises selecting a value of the parameter for the plurality of defect classes based on the at least one confidence threshold.

11. The apparatus of claim 8, wherein at least two defect classes have differently determined confidence thresholds.

12. The apparatus of claim 7, wherein the at least one inspection modality that is different than the automatic classifier corresponds to a visual inspection, and wherein applying the automatic classifier to the inspection data comprises applying a multi-class classifier to the inspection data to assign defects of the plurality of defects to a defect class of the plurality of defect classes, wherein the multi-class classifier is configured to identify the defects in the overlap region.

13. A non-transitory computer readable storage medium having instructions that, when executed by a processor, cause the processor to perform operations comprising:

storing, by the processor, a plurality of definitions of a plurality of defect classes in terms of a plurality of classification rules in a multi-dimensional feature space, wherein the plurality of classification rules, for each given defect class, defines in a feature space a boundary of a region associated with the given defect class and provides a confidence measure associated with classification of a defect to the given class, the confidence measure being indicative of a level of confidence as a function of the location of the defect in the feature space with respect to the respective boundaries;

receiving, by the processor, inspection data associated with a plurality of defects detected in one or more samples under inspection;

receiving, by the processor and from an operator, a classification performance measure selected from a plurality of performance measures, wherein the plurality of performance measures comprises at least one of a maximum rejection rate or a target purity level;

determining at least one confidence threshold corresponding to the classification performance measure;

applying, by the processor, an automatic classifier to the inspection data, the automatic classifier based on the plurality of definitions, and identifying a plurality of defects each classified with a low level of confidence based on the at least one confidence threshold and indicative of the defect being located in an overlap region between the respective boundaries of at least two of the defect classes;

generating, by the processor, a plurality of classification results by applying, to the identified plurality of defects classified with the low level of confidence, at least one inspection modality that is different than the automatic classifier to assign each of the identified plurality of defects to one of the at least two of the defect classes associated with the overlap region; and

refining, by the processor, the automatic classifier to adjust boundaries of one or more defect classes of the plurality of classes when a threshold amount of the identified plurality of defects located in the overlap region have been classified by the at least one inspection modality that is different than the automatic classifier, wherein the refining is provided by training the automatic classifier using each classification result of the plurality of classification results of the identified plurality of defects classified with the low level of confidence.

14. The non-transitory computer readable storage medium of claim 13, wherein the processing device further performs operations comprising:

determining to each given defect class a confidence threshold of the given defect class, wherein a confidence threshold of a given defect class is based on the loci of the respective boundary in the feature space.

15. The non-transitory computer readable storage medium of claim 14, wherein an extent of a boundary in feature space is controlled by a confidence level of a respective defect class.

16. The non-transitory computer readable storage medium of claim 14, wherein the plurality of definitions of the plurality of defect classes comprises a kernel function having a parameter, and wherein applying the automatic classifier to the inspection data comprises selecting a value of the parameter for the plurality of defect classes based on the at least one confidence threshold.

17. The non-transitory computer readable storage medium of claim 14, wherein at least two defect classes have differently determined confidence thresholds.

18. The non-transitory computer readable storage medium of claim 13, wherein applying the automatic classifier to the inspection data comprises applying a multi-class classifier to the inspection data to assign defects of the plurality of defects to a defect class of the plurality of defect classes, wherein the multi-class classifier is configured to identify the defects in the overlap region.