Training a neural network for defect detection in low resolution images

• US 10,599,951 B2
• Filed: 03/25/2019
• Issued: 03/24/2020
• Est. Priority Date: 03/28/2018
• Status: Active Grant

• Current Assignee: KLA-Tencor Corporation (KLA Corporation)
• Original Assignee: KLA-Tencor Corporation (KLA Corporation)
• Inventors: Bhaskar, Kris, Karsenti, Laurent, Ries, Brad, Nicolaides, Lena, Yeoh, Richard (Seng Wee), Hiebert, Stephen
• Primary Examiner(s): Seth, Manav
• Application Number:
  US16/364,140
• Publication Number:
  US 20190303717A1
• Time in Patent Office: 365 Days
• CPC Class Codes: G06F 18/214 Generating training patterns; Bootstrap methods, e.g. bagging or boosting
• CPC Class Codes: G06F 18/217 Validation; Performance evaluation; Active pattern learning techniques
• CPC Class Codes: G06T 2207/20081 Training; Learning
• CPC Class Codes: G06T 2207/20084 Artificial neural networks [ANN]
• CPC Class Codes: G06T 2207/30148 Semiconductor; IC; Wafer
• CPC Class Codes: G06T 7/0004 Industrial image inspection
• CPC Class Codes: G06V 10/82 using neural networks
• CPC Class Codes: G06V 2201/06 Recognition of objects for industrial automation
• CPC Class Codes: G06V 30/1916 Validation; Performance evaluation
• CPC Class Codes: G06V 30/19173 Classification techniques
• CPC Class Codes: G06V 30/2504 Coarse or fine approaches, e.g. resolution of ambiguities or multiscale approaches

Abstract

Methods and systems for training a neural network for defect detection in low resolution images are provided. One system includes an inspection tool that includes high and low resolution imaging subsystems and one or more components that include a high resolution neural network and a low resolution neural network. Computer subsystem(s) of the system are configured for generating a training set of defect images. At least one of the defect images is generated synthetically by the high resolution neural network using an image generated by the high resolution imaging subsystem. The computer subsystem(s) are also configured for training the low resolution neural network using the training set of defect images as input. In addition, the computer subsystem(s) are configured for detecting defects on another specimen by inputting the images generated for the other specimen by the low resolution imaging subsystem into the trained low resolution neural network.

Claims

1. A system configured to train a neural network for defect detection in low resolution images, comprising:

an inspection tool comprising a high resolution imaging subsystem and a low resolution imaging subsystem, wherein the high and low resolution imaging subsystems comprise at least an energy source and a detector, wherein the energy source is configured to generate energy that is directed to a specimen, and wherein the detector is configured to detect energy from the specimen and to generate images responsive to the detected energy;

one or more computer subsystems configured for acquiring the images of the specimen generated by the high and low resolution imaging subsystems; and

one or more components executed by the one or more computer subsystems, wherein the one or more components comprise a high resolution neural network and a low resolution neural network; and

wherein the one or more computer subsystems are further configured for;

generating a training set of defect images, wherein at least one of the defect images is generated synthetically by the high resolution neural network using at least one of the images generated by the high resolution imaging subsystem;

training the low resolution neural network using the training set of defect images as input; and

detecting defects on another specimen by inputting the images generated for the other specimen by the low resolution imaging subsystem into the trained low resolution neural network.

2. The system of claim 1, wherein the training set of defect images comprises images of the specimen generated by more than one mode of the low resolution imaging subsystem.

3. The system of claim 2, wherein the more than one mode of the low resolution imaging subsystem comprises all of the modes of the low resolution imaging subsystem.

4. The system of claim 2, wherein the one or more computer subsystems are further configured for selecting one or more of the more than one mode of the low resolution imaging subsystem used for detecting the defects on the other specimen based on results of training the low resolution neural network with the images generated by the more than one mode of the low resolution imaging subsystem.

5. The system of claim 1, wherein the inspection t configured as a macro inspection tool.

6. The system of claim 1, wherein the defects detected on the other specimen are defects of a back end layer of the other specimen.

7. The system of claim 1, wherein the defects detected on the other specimen are defects of a redistribution layer of the other specimen.

8. The system of claim 1, wherein the defects detected on the other specimen are defects of a high noise layer of the other specimen.

9. The system of claim 1, wherein the defects detected on the other specimen are defects of a layer comprising metal lines of the other specimen.

10. The system of claim 1, wherein the other specimen on which the defects are detected is a post-dice specimen.

11. The system of claim 1, wherein the high and low resolution neural networks are configured for single image defect detection.

12. The system of claim 1, wherein the training set of defect images comprises one or more images of one or more programmed defects on the specimen, wherein the one or more computer subsystems are further configured for generating the one or more programmed defects by altering a design for the specimen to create the one or more programmed defects in the design, and wherein the altered design is printed on the specimen to create the one or more programmed defects on the specimen.

13. The system of claim 1, wherein the training set of defects comprises one or more images of one or more synthetic defects, and wherein the one or more computer subsystems are further configured for generating the one or more synthetic defects by altering a design for the specimen to create the one or more synthetic defects in the design, generating simulated high resolution images for the one or more synthetic defects based on the one or more synthetic defects in the design, and adding the simulated high resolution images to the training set.

14. The system of claim 13, wherein the one or more computer subsystems are further configured for generating the simulated high resolution images using the high resolution neural network, and wherein the high resolution neural network is configured as a deep generative model.

15. The system of claim 1, wherein the training set of defects comprises one or more images of one or more synthetic defects, wherein the one or more computer subsystems are further configured for generating the one or more images of the one or more synthetic defects by altering a design for the specimen to create the one or more synthetic defects in the design, and wherein the one or more computer subsystems are further configured for generating simulated low resolution images for the one or more synthetic defects based on the one or more synthetic defects in the design.

16. The system of claim 15, wherein the one or more computer subsystems are further configured for generating the simulated low resolution images using a deep generative model.

17. The system of claim 15, wherein generating the simulated low resolution images is performed with a generative adversarial network or a variational Bayesian method.

18. The system of claim 1, wherein the training set of defects comprises one or more synthetic defects, and wherein the one or more computer subsystems are further configured for generating the one or more synthetic defects by altering one or more of the images generated by the high resolution imaging subsystem and one or more of the images generated by the low resolution imaging subsystem to create a segmentation image, altering the one or more of the images generated by the high resolution imaging subsystem based on the segmentation image, and generating simulated low resolution images for the one or more synthetic defects based on the altered one or more images.

19. The system of claim 18, wherein generating the simulated low resolution images is performed with a generative adversarial network or a variational Bayesian method.

20. The system of claim 1, wherein the one or more computer subsystems are further configured for generating the at least one of the defect images synthetically by altering the at least one of the images generated by the high resolution imaging subsystem for the specimen to create high resolution images for known defects of interest.

21. The system of claim 1, wherein the training set of defect images comprises one or more images of one or more artificial defects on the specimen generated by performing a process on the specimen known to cause the one or more artificial defects on the specimen.

22. The system of claim 1, wherein the training set of defect images comprises one or more defects detected on the specimen in one or more of the images generated by the high resolution imaging subsystem.

23. The system of claim 22, wherein the one or more computer subsystems are further configured for detecting the defects on the specimen in the images generated by the high resolution imaging subsystem by single image detection.

24. The system of claim 22, wherein the one or more computer subsystems are further configured for detecting the defects on the specimen in the images generated by the high resolution imaging subsystem by die-to-database detection.

25. The system of claim 1, wherein the inspection tool is configured for scanning swaths on the specimen while detecting energy from the specimen, and wherein the one or more computer subsystems are further configured for acquiring and storing at least three of the swaths of the images generated by the high resolution imaging subsystem such that the at least three of the swaths are available for use in generating the training set of defect images.

26. The system of claim 1, wherein the one or more computer subsystems are further configured for training the high resolution neural network, and wherein training the high resolution neural network and training the low resolution neural network are performed using a generative adversarial network or a variational Bayesian method.

27. The system of claim 1, wherein the high resolution neural network is configured as a semi-supervised deep learning framework.

28. The system of claim 1, wherein the low resolution neural network is configured as a semi-supervised deep learning framework.

29. The system of claim 1, wherein the images generated by the low resolution imaging subsystem and acquired by the one or more computer subsystems comprise images taken through focus, wherein the one or more computer subsystems are further configured for mapping the images taken through focus to the images generated by the high resolution imaging subsystem, and wherein training the low resolution neural network is further performed based on the results of training the high resolution neural network and results of the mapping.

30. A non-transitory computer-readable medium, storing program instructions executable on one or more computer systems for performing a computer-implemented method for training a neural network for defect detection in low resolution images, wherein the computer-implemented method comprises:

generating images for a specimen with high and low resolution imaging subsystems of an inspection tool, wherein the high and low resolution imaging subsystems comprise at least an energy source and a detector, wherein the energy source is configured to generate energy that is directed to the specimen, and wherein the detector is configured to detect energy from the specimen and to generate images responsive to the detected energy;

wherein one or more components are executed by the one or more computer systems, and wherein the one or more components comprise a high resolution neural network and a low resolution neural network;

generating a training set of defect images, wherein at least one of the defect images is generated synthetically by the high resolution neural network using at least one of the images generated by the high resolution imaging subsystem;

training the low resolution neural network using the training set of defect images as input; and

detecting defects on another specimen by inputting the images generated for the other specimen by the low resolution imaging subsystem into the trained low resolution neural network, wherein generating the training set, training the low resolution neural network, and detecting the defects are performed by the one or more computer systems.

31. A computer-implemented method for training a neural network for defect detection in low resolution images, comprising:

generating images for a specimen with high and low resolution imaging subsystems of an inspection tool, wherein the high and low resolution imaging subsystems comprise at least an energy source and a detector, wherein the energy source is configured to generate energy that is directed to the specimen, and wherein the detector is configured to detect energy from the specimen and to generate images responsive to the detected energy;

wherein one or more components are executed by one or more computer systems, and wherein the one or more components comprise a high resolution neural network and a low resolution neural network;

generating a training set of defect images, wherein at least one of the defect images is generated synthetically by the high resolution neural network using at least one of the images generated by the high resolution imaging subsystem;

training the low resolution neural network using the training set of defect images as input; and

detecting defects on another specimen by inputting the images generated for the other specimen by the low resolution imaging subsystem into the trained low resolution neural network, wherein generating the training set, training the low resolution neural network, and detecting the defects are performed by the one or more computer systems.