Generating simulated images from input images for semiconductor applications

• US 10,395,356 B2
• Filed: 05/23/2017
• Issued: 08/27/2019
• Est. Priority Date: 05/25/2016
• Status: Active Grant

• Current Assignee: KLA-Tencor Corporation (KLA Corporation)
• Original Assignee: KLA-Tencor Corporation (KLA Corporation)
• Inventors: Zhang, Jing, Bhaskar, Kris
• Primary Examiner(s): Youssef, Menatoallah
• Application Number:
  US15/603,249
• Publication Number:
  US 20170345140A1
• Time in Patent Office: 826 Days
• CPC Class Codes: G01N 2021/8883 involving the calculation of gauges, generating models
• CPC Class Codes: G01N 21/956 Inspecting patterns on the surface of objects contactless testing of electronic circuits G01R31/308; testing currency G07D; manufacturing processes per se of semiconductor devices implementing a measuring step H01L22/10
• CPC Class Codes: G06T 2207/20081 Training; Learning
• CPC Class Codes: G06T 2207/20084 Artificial neural networks [ANN]
• CPC Class Codes: G06T 2207/30148 Semiconductor; IC; Wafer
• CPC Class Codes: G06T 7/0004 Industrial image inspection
• CPC Class Codes: G06V 10/454 Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]
• CPC Class Codes: G06V 10/764 using classification, e.g. of video objects
• CPC Class Codes: G06V 10/82 using neural networks

Abstract

Methods and systems for generating a simulated image from an input image are provided. One system includes one or more computer subsystems and one or more components executed by the one or more computer subsystems. The one or more components include a neural network that includes two or more encoder layers configured for determining features of an image for a specimen. The neural network also includes two or more decoder layers configured for generating one or more simulated images from the determined features. The neural network does not include a fully connected layer thereby eliminating constraints on size of the image input to the two or more encoder layers.

Claims

1. A system configured to generate a simulated image from an input image, comprising:

one or more computer subsystems configured to acquire an image for a specimen by directing energy to the specimen and detecting energy from the specimen using the specimen itself and imaging hardware; and

one or more components executed by the one or more computer subsystems, wherein the one or more components comprise;

a neural network, wherein the neural network comprises;

two or more encoder layers configured for determining features of the image for the specimen, wherein the image is a low resolution image of the specimen; and

two or more decoder layers configured for generating one or more simulated images from the determined features, wherein the one or more simulated images are one or more high resolution images of the specimen, wherein the neural network is configured as a deep generative model, and wherein the neural network does not comprise a fully connected layer thereby eliminating constraints on size of the image input to the two or more encoders layers.

2. The system of claim 1, wherein the neural network is further configured as a fully convolutional model.

3. The system of claim 1, wherein the neural network is further configured as a generative adversarial net.

4. The system of claim 1, wherein the neural network is further configured as a conditional generative adversarial net.

5. The system of claim 1, wherein the neural network is further configured as a generative adversarial network and variational autoencoder.

6. The system of claim 1, wherein a part of the neural network is further configured as a convolutional neural network.

7. The system of claim 1, wherein the one or more computer subsystems, the one or more components, and the neural network do not crop the image input to the two or more encoder layers.

8. The system of claim 1, wherein the one or more computer subsystems, the one or more components, and the neural network do not reconstruct the one or more simulated images from two or more cropped images.

9. The system of claim 1, wherein the one or more computer subsystems are further configured for setting up the neural network by replacing a fully connected layer in a preexisting neural network with a group of convolutional layers thereby creating the neural network.

10. The system of claim 1, wherein the one or more computer subsystems are configured for training the neural network using a fovy-decay weighted loss function to alter boundary effects.

11. The system of claim 1, wherein the one or more computer subsystems are configured for training the neural network using a hatch of training images, each having the same arbitrary size.

12. The system of claim 1, wherein the one or more computer subsystems are configured for training the neural network using a batch of training images, and wherein two or more of the training images in the batch have different arbitrary sizes.

13. The system of claim 1, wherein the image input to the two or more encoder layers is an entire frame image generated for the specimen.

14. The system of claim 1, wherein the image input to the two or more encoder layers is an entire die image for the specimen.

15. The system of claim 1, wherein the image input to the two or more encoder layers is generated by an electron beam based imaging system.

16. The system of claim 1, wherein the image input to the two or more encoder layers is generated by an optical based imaging system.

17. The system of claim 1, wherein the image input to the two or more encoder layers is generated by an inspection system.

18. The system of claim 1, wherein the image input to the two or more encoder layers is generated by a metrology system.

19. The system of claim 1, wherein the specimen is a wafer.

20. The system of claim 1, wherein the specimen is a reticle.

21. The system of claim 1, wherein the one or more computer subsystems are configured for detecting a defect on the specimen based on the one or more simulated images.

22. The system of claim 1, wherein the one or more computer subsystems are configured for classifying a defect detected in the image input to the two or more encoder layers or the one or more simulated images, and wherein said classifying is performed based on the one or more simulated images.

23. The system of claim 1, wherein the one or more computer subsystems are configured for measuring one or more features of the specimen or a defect detected on the specimen based on the one or more simulated images.

24. The system of claim 1, wherein the one or more computer subsystems are configured for learning a representation of one or more structures on the specimen by determining values for the features that render the one or more simulated images substantially the same as the image input to the two or more encoder layers.

25. A system configured to generate a simulated image from an input image, comprising:

an imaging subsystem configured for generating an image of a specimen by directing energy to the specimen and detecting energy from the specimen using the specimen itself and imaging hardware, wherein the image is a low resolution image of the specimen;

one or more computer subsystems configured for acquiring the image; and

one or more components executed by the one or more computer subsystems, wherein the one or more components comprise;

a neural network, wherein the neural network comprises;

two or more encoder layers configured for determining features of the image; and

two or more decoder layers configured for generating one or more simulated images from the determined features, wherein the one or more simulated images are one or more high resolution images of the specimen, wherein the neural network is configured as a deep generative model, and wherein the neural network does not comprise a fully connected layer thereby eliminating constraints on size of the image input to the two or more encoder layers.

26. A non-transitory computer-readable medium, storing program instructions executable on one or more computer systems for performing a computer-implemented method for generating a simulated image from an input image, wherein the computer-implemented method comprises:

acquiring an image for a specimen by directing energy to the specimen and detecting energy from the specimen using the specimen itself and imaging hardware;

determining features of the image for the specimen by inputting the image into two or more encoder layers of a neural network, wherein the image is a low resolution image of the specimen, wherein the neural network is configured as a deep generative model, and wherein the neural network does not comprise a fully connected layer thereby eliminating constraints on size of the image input to the two or more encoder layers; and

generating one or more simulated images from the determined features, wherein the one or more simulated images are one or more high resolution images of the specimen, wherein generating the one or more simulated images is performed by two or more decoder layers of the neural network, wherein said acquiring, said determining, and said generating are performed by the one or more computer systems, wherein one or more components are executed by the one or more computer systems, and wherein the one or more components comprise the neural network.

27. A computer-implemented method for generating a simulated image from an input image, comprising:

acquiring an image for a specimen by directing energy to the specimen and detecting energy from the specimen using the specimen itself and imaging hardware;

determining features of the image for the specimen by inputting the image into two or more encoder layers of a neural network, wherein the image is a low resolution image of the specimen, wherein the neural network is configured as a deep generative model, and wherein the neural network does not comprise a fully connected layer thereby eliminating constraints on size of the image input to the two or more encoder layers; and

generating one or more simulated images from the determined features, wherein generating the one or more simulated images is performed by two or more decoder layers of the neural network, wherein the one or more simulated images are one or more high resolution images of the specimen, wherein said acquiring, said determining, and said generating are performed by one or more computer systems, wherein one or more components are executed by the one or more computer systems, and wherein the one or more components comprise the neural network.