Unified neural network for defect detection and classification

• US 10,607,119 B2
• Filed: 09/06/2017
• Issued: 03/31/2020
• Est. Priority Date: 09/06/2017
• Status: Active Grant

• Current Assignee: KLA-Tencor Corporation
• Original Assignee: KLA-Tencor Corporation
• Inventors: He, Li, Mahadevan, Mohan, Venkataraman, Sankar, Ying, Huajun, Yang, Hedong
• Primary Examiner(s): Alam, Fayyaz
• Application Number:
  US15/697,426
• Publication Number:
  US 20190073568A1
• Time in Patent Office: 937 Days
• Field of Search: None
• CPC Class Codes: G06F 18/214 Generating training patterns; Bootstrap methods, e.g. bagging or boosting
• CPC Class Codes: G06F 18/24 Classification techniques
• CPC Class Codes: G06F 18/28 Determining representative reference patterns, e.g. by averaging or distorting; Generating dictionaries
• CPC Class Codes: G06F 18/40 Software arrangements specially adapted for pattern recognition, e.g. user interfaces or toolboxes therefor
• CPC Class Codes: G06T 2207/10061 from scanning electron microscope
• CPC Class Codes: G06T 2207/20081 Training; Learning
• CPC Class Codes: G06T 2207/20084 Artificial neural networks [ANN]
• CPC Class Codes: G06T 2207/30148 Semiconductor; IC; Wafer
• CPC Class Codes: G06T 7/0006 using a design-rule based approach
• CPC Class Codes: G06T 7/001 using an image reference approach

Abstract

Methods and systems for detecting and classifying defects on a specimen are provided. One system includes one or more components executed by one or more computer subsystems. The one or more components include a neural network configured for detecting defects on a specimen and classifying the defects detected on the specimen. The neural network includes a first portion configured for determining features of images of the specimen generated by an imaging subsystem. The neural network also includes a second portion configured for detecting defects on the specimen based on the determined features of the images and classifying the defects detected on the specimen based on the determined features of the images.

Claims

1. A system configured to detect and classify defects on a specimen, comprising:

one or more computer subsystems; and

one or more components executed by the one or more computer subsystems, wherein the one or more components comprise;

a neural network configured for detecting defects on a specimen and classifying the defects detected on the specimen, wherein the neural network comprises;

a first portion configured for determining features of images of the specimen generated by an imaging subsystem; and

a second portion configured for detecting defects on the specimen based on the determined features of the images and classifying the defects detected on the specimen based on the determined features of the images, wherein the second portion comprises a proposal network configured for detecting defects on the specimen based on the features determined for the images, and wherein the proposal network detects the defects without classifying the defects; and

wherein the one or more computer subsystems are configured for generating results of the detecting and classifying.

2. The system of claim 1, wherein the neural network is further configured as a deep learning network.

3. The system of claim 1, wherein the neural network is further configured as an AlexNet.

4. The system of claim 1, wherein the neural network is further configured as a GoogleNet.

5. The system of claim 1, wherein the neural network is further configured as a VGG network.

6. The system of claim 1, wherein the neural network is further configured as a deep residual network.

7. The system of claim 1, wherein the first portion comprises one or more convolutional layers.

8. The system of claim 1, wherein the one or more computer subsystems are further configured to train the neural network by inputting class labels assigned by a user to training defect images and the training defect images to the neural network.

9. The system of claim 8, wherein the one or more computer subsystems are further configured to train the neural network by inputting the class labels and the training defect images to the neural network with one or more training reference images, a design for a specimen for which the training defect images were generated, or the one or more training reference images and the design.

10. The system of claim 1, wherein the one or more computer subsystems are further configured for fine tuning the neural network using pre-trained weights.

11. The system of claim 10, wherein the pre-trained weights are obtained by training an image classification network.

12. The system of claim 10, wherein the pre-trained weights are obtained by training the second portion configured for classifying the defects detected on the specimen.

13. The system of claim 1, wherein the one or more computer subsystems are further configured for training the neural network from scratch.

14. The system of claim 1, wherein the second portion further comprises one or more fully connected layers configured for selecting one or more of the determined features and classifying the defects based on the one or more selected features.

15. The system of claim 1, wherein the one or more computer subsystems are further configured to train the neural network by inputting class labels assigned by a user to bounding boxes in training defect images and the training defect images to the neural network.

16. The system of claim 15, wherein the one or more computer subsystems are further configured to train the neural network by inputting the class labels and the training defect images to the neural network with one or more training reference images, a design for a specimen for which the training defect images were generated, or the one or more training reference images and the design.

17. The system of claim 1, wherein the proposal network is further configured for generating hounding boxes for each of the detected defects.

18. The system of claim 17, wherein the second portion further comprises one or more region of interest pooling layers followed by one or more fully connected layers, wherein the one or more region of interest pooling layers are configured for generating fixed length representations of the generated bounding boxes, wherein the fixed length representations are input to the one or more fully connected layers, and wherein the one or more fully connected layers are configured for selecting one or more of the determined features and classifying the detected defects based on the one or more selected features.

19. The system of claim 18, wherein the one or more computer subsystems are further configured for fine tuning the one or more region of interest pooling layers and the one or more fully connected layers using the proposal network.

20. The system of claim 1, wherein the imaging subsystem is configured as an electron beam based imaging subsystem.

21. The system of claim 1, wherein e imaging subsystem is configured as an optical based imaging subsystem.

22. The system of claim 1, wherein the specimen is a wafer.

23. The system of claim 1, wherein the specimen is a reticle.

24. A system configured to detect and classify defects on a specimen, comprising:

an imaging subsystem configured for generating images of a specimen;

one or more computer subsystems configured for acquiring the images; and

one or more components executed by the one or more computer subsystems, wherein the one or more components comprise;

a neural network configured for detecting defects on the specimen and classifying the defects detected on the specimen, wherein the neural network comprises;

a first portion configured for determining features of the images of the specimen generated by the imaging subsystem; and

a second portion configured for detecting defects on the specimen based on the determined features of the images and classifying the defects detected on the specimen based on the determined features of the images, wherein the second portion comprises a proposal network configured for detecting defects on the specimen based on the features determined for the images, and wherein the proposal network detects the defects without classifying the defects; and

wherein the one or more computer subsystems are configured for generating results of the detecting and classifying.

25. A non-transitory computer-readable medium, storing program instructions executable on one or more computer systems for performing a computer-implemented method for detecting and classifying defects on a specimen, wherein the computer-implemented method comprises:

acquiring images of a specimen generated by an imaging subsystem;

determining features of the images of the specimen by inputting the images into a first portion of a neural network configured for detecting defects on the specimen and classifying the defects detected on the specimen, wherein one or more components are executed by the one or more computer systems, and wherein the one or more components comprise the neural network;

detecting defects on the specimen based on the determined features of the images and classifying the defects detected on the specimen based on the determined features of the images by inputting the determined features of the images into a second portion of the neural network, wherein the second portion comprises a proposal network configured for detecting defects on the specimen based on the features determined for the images, and wherein the proposal network detects the defects without classifying the defects; and

generating results of the detecting and classifying.

26. A computer-implemented method for detecting and classifying defects on a specimen, comprising:

acquiring images of a specimen generated by an imaging subsystem;

determining features of the images of the specimen by inputting the images into a first portion of a neural network configured for detecting defects on the specimen and classifying the defects detected on the specimen, wherein one or more components are executed by one or more computer systems, and wherein the one or more components comprise the neural network;

detecting defects on the specimen based on the determined features of the images and classifying the defects detected on the specimen based on the determined features of the images by inputting the determined features of the images into a second portion of the neural network, wherein the second portion comprises a proposal network configured for detecting defects on the specimen based on the features determined for the images, and wherein the proposal network detects the defects without classifying the defects; and

generating results of the detecting and classifying.