Generating simulated images from design information

• US 9,965,901 B2
• Filed: 06/07/2016
• Issued: 05/08/2018
• Est. Priority Date: 11/19/2015
• Status: Active Grant

• Current Assignee: KLA-Tencor Corporation (KLA Corporation)
• Original Assignee: KLA-Tencor Corporation (KLA Corporation)
• Inventors: Zhang, Jing, Bhaskar, Kris
• Primary Examiner(s): Pham, Nam
• Application Number:
  US15/176,139
• Publication Number:
  US 20170148226A1
• Time in Patent Office: 700 Days
• CPC Class Codes: G06F 18/2148 characterised by the process organisation or structure, e.g. boosting cascade
• CPC Class Codes: G06F 30/398 Design verification or optimisation, e.g. using design rule check [DRC], layout versus schematics [LVS] or finite element methods [FEM] optical proximity correction [OPC] design processes G03F1/36
• CPC Class Codes: G06N 3/044 Recurrent networks, e.g. Hopfield networks
• CPC Class Codes: G06N 3/045 Combinations of networks
• CPC Class Codes: G06N 3/047 Probabilistic or stochastic networks
• CPC Class Codes: G06N 3/088 Non-supervised learning, e.g. competitive learning
• CPC Class Codes: G06T 19/20 Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
• CPC Class Codes: G06T 2200/08 involving all processing steps from image acquisition to 3D model generation
• CPC Class Codes: G06T 2207/10061 from scanning electron microscope
• CPC Class Codes: G06T 2207/20084 Artificial neural networks [ANN]
• CPC Class Codes: G06T 2207/30148 Semiconductor; IC; Wafer
• CPC Class Codes: G06T 7/00 Image analysis
• CPC Class Codes: G06T 7/001 using an image reference approach
• CPC Class Codes: G06T 9/00 Image coding bandwidth or redundancy reduction for static pictures H04N1/41; coding or decoding of static colour picture signals H04N1/64; methods or arrangements for coding, decoding, compressing or decompressing digital video signals H04N19/00
• CPC Class Codes: G06V 10/454 Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]

Abstract

Methods and systems for generating simulated images from design information are provided. One system includes one or more computer subsystems and one or more components executed by the computer subsystem(s), which include a generative model. The generative model includes two or more encoder layers configured for determining features of design information for a specimen. The generative model also includes two or more decoder layers configured for generating one or more simulated images from the determined features. The simulated image(s) illustrate how the design information formed on the specimen appears in one or more actual images of the specimen generated by an imaging system.

Claims

1. A system configured to generate simulated images from design information, comprising:

one or more computer subsystems; and

one or more components executed by the one or more computer subsystems, wherein the one or more components comprise;

a generative model, wherein the generative model comprises;

two or more encoder layers configured for determining features of design information for a specimen; and

two or more decoder layers configured for generating one or more simulated images from the determined features, wherein the one or more simulated images illustrate how the design information formed on the specimen appears in one or more actual images of the specimen generated by an imaging system.

2. The system of claim 1, wherein the generative model is a deep generative model.

3. The system of claim 1, wherein the generative model is a convolution and deconvolution neural network.

4. The system of claim 1, wherein the two or more encoder layers comprise two or more convolutional and pooling layers, and wherein the two or more decoder layers comprise two or more deconvolutional and up-pooling layers.

5. The system of claim 1, wherein the two or more encoder layers determine the features by dimension reduction of the design information.

6. The system of claim 1, wherein the design information is computer aided design data.

7. The system of claim 1, wherein the one or more computer subsystems are configured for generating a training dataset used for training the generative model, and wherein the training dataset comprises a set of pairs of a portion of other design information and an actual image generated for the portion of the other design information.

8. The system of claim 1, wherein the one or more computer subsystems are configured for modifying a portion of other design information to create a predetermined defect in the portion of the other design information and generating a training dataset used for training the generative model, and wherein the training dataset comprises:

a pair of the modified portion of the other design information and an actual image generated for the modified portion of the other design information; and

other pairs of a portion of the other design information and an actual image generated for the portion of the other design information.

9. The system of claim 1, wherein the one or more computer subsystems are configured for pre-training the two or more encoder layers and a portion of the two or more decoder layers.

10. The system of claim 1, wherein the one or more computer subsystems are configured for training the two or more encoder layers and a portion of the two or more decoder layers by learning the features for design information in a training dataset, and wherein the learning comprises inferring the features for the design information in the training dataset using the two or more encoder layers and regenerating the design information in the training dataset from the inferred features using the portion of the two or more decoder layers.

11. The system of claim 10, wherein the training is unsupervised.

12. The system of claim 10, wherein the one or more components further comprise an additional component, wherein the training is performed using the additional component, and wherein the additional component comprises an auto-encoder, a stacked auto-encoder, an auto-encoder with regularization, a denoising auto-encoder, stacked denoising auto-encoders, a sparse auto-encoder, a variational auto-encoder, an auto-encoder configured to learn a distribution of features, a Restricted Boltzmann Machine, a Restricted Boltzmann Machine configured to learn a distribution of features, a gated Markov Random Field, a Deep Boltzmann Machine, a Deep Belief Network, a convolutional Restricted Boltzmann Machine, a convolution auto-encoder, a convolutional neural network, a recurrent neural network, a long short-term memory, generative adversarial nets, or some combination thereof.

13. The system of claim 10, wherein the learned features comprise vectors of scalar values, vectors of independent distributions, or joint distributions.

14. The system of claim 10, wherein the learned features comprise vectors of independent distributions, and wherein the vectors of independent distributions comprise Bernoulli, binomial, multinomial, Poisson binomial, beta-binomial, multinomial, Boltzmann, Poisson, Conway-Maxwell-Poisson, Beta, Gaussian/Normal, Skew normal, Rayleigh, Laplace, gamma, pareto, or student-t distributions.

15. The system of claim 10, wherein the learned features comprise joint distributions, and wherein the joint distributions comprise Multivariate Gaussian/Normal, multivariate student-t, Dirichlet, matrix Gaussian/normal, or matrix t-distributions.

16. The system of claim 1, wherein the one or more computer subsystems are configured for pre-training the two or more encoder layers and a portion of the two or more decoder layers and, subsequent to the pre-training, training the two or more encoder layers, the portion of the two or more decoder layers, and another portion of the two or more decoder layers.

17. The system of claim 1, wherein the one or more computer subsystems are configured for training the two or more encoder layers and the two or more decoder layers by learning the features for design information in a training dataset, and wherein the learning comprises inferring the features for the design information in the training dataset using the two or more encoder layers, regenerating the design information in the training dataset from the inferred features using the two or more decoder layers, generating simulated images for the design information in the training dataset from the inferred features using the two or more decoder layers, and modifying one or more parameters of the two or more decoder layers until the simulated images for the design information in the training dataset match actual images for the design information in the training dataset.

18. The system of claim 17, wherein the training is supervised.

19. The system of claim 17, wherein the one or more parameters comprise weights.

20. The system of claim 1, wherein the two or more encoder layers are fixed.

21. The system of claim 1, wherein the two or more encoder layers are trainable.

22. The system of claim 1, wherein the two or more decoder layers comprise first and second portions of the two or more decoder layers, and wherein the first and second portions are jointly constructed.

23. The system of claim 1, wherein the two or more decoder layers comprise first and second portions of the two or more decoder layers, and wherein the one or more computer subsystems are configured to jointly train the first and second portions.

24. The system of claim 1, wherein the two or more decoder layers comprise a first portion of the two or more decoder layers configured in a first pass and a second portion of the two or more decoder layers configured in a second pass.

25. The system of claim 1, wherein the two or more decoder layers comprise first and second portions of the two or more decoder layers configured as a single pass via channel configuration.

26. The system of claim 1, wherein the two or more decoder layers are further configured for regenerating the design information from the determined features.

27. The system of claim 1, wherein the one or more actual images are generated for the specimen by the imaging system after a process is performed on the specimen, and wherein the process comprises a final fabrication process step that is not a lithography process.

28. The system of claim 1, wherein at least one of the one or more simulated images illustrates one or more statistics for the one or more simulated images.

29. The system of claim 28, wherein the one or more statistics comprise one or more distributions selected from a group consisting of Bernoulli, binomial, multinomial, Poisson binomial, beta-binomial, beta-multinomial, Boltzmann, Poisson, Conway-Maxwell-Poisson, Beta, Gaussian/Normal, Skew normal, Rayleigh, Laplace, gamma, pareto, student-t, Multivariate Gaussian/Normal, multivariate student-t, Dirichlet, matrix Gaussian/Normal, and matrix t-distribution.

30. The system of claim 1, wherein generating the one or more simulated images comprises directly deriving the one or more simulated images from the determined features.

31. The system of claim 1, wherein the imaging system is an electron beam based imaging system.

32. The system of claim 1, wherein the imaging system is an optical based imaging system.

33. The system of claim 1, wherein the imaging system is an inspection system.

34. The system of claim 1, wherein the imaging system is a defect review system.

35. The system of claim 1, wherein the imaging system is a metrology system.

36. The system of claim 1, wherein the specimen is a wafer.

37. The system of claim 1, wherein the specimen is a reticle.

38. The system of claim 1, wherein the imaging system is configured to acquire the one or more actual images and the one or more simulated images and to detect defects on the specimen by comparing the one or more actual images to the one or more simulated images.

39. The system of claim 1, wherein the imaging system is configured to perform calibration of the imaging system using the one or more simulated images.

40. The system of claim 1, wherein the imaging system is configured to perform alignment of the one or more actual images to the one or more simulated images.

41. A system configured to generate simulated images from design information, comprising:

one or more computer subsystems;

one or more components executed by the one or more computer subsystems, wherein the one or more components comprise;

a generative model, wherein the generative model comprises;

two or more encoder layers configured for determining features of design information for a specimen; and

two or more decoder layers configured for generating one or more simulated images from the determined features, wherein the one or more simulated images illustrate how the design information formed on the specimen appears in one or more actual images of the specimen; and

an imaging subsystem configured to generate the one or more actual images of the specimen, wherein the one or more computer subsystems are configured to acquire the one or more actual images and the one or more simulated images and to perform one or more functions for the specimen based on the one or more actual images and the one or more simulated images.

42. A non-transitory computer-readable medium, storing program instructions executable on a computer system for performing a computer-implemented method for generating simulated images from design information, wherein the computer-implemented method comprises:

determining features of design information for a specimen by inputting the design information to two or more encoder layers of a generative model; and

generating one or more simulated images by inputting the determined features to two or more decoder layers of the generative model, wherein the one or more simulated images illustrate how the design information formed on the specimen appears in one or more actual images of the specimen generated by an imaging system.

43. A computer-implemented method for generating simulated images from design information, comprising:

determining features of design information for a specimen by inputting the design information to two or more encoder layers of a generative model; and

generating one or more simulated images by inputting the determined features to two or more decoder layers of the generative model, wherein the one or more simulated images illustrate how the design information formed on the specimen appears in one or more actual images of the specimen generated by an imaging system, and wherein said determining and said generating are performed with one or more computer systems.