Defect classification apparatus and defect classification method

• US 10,203,851 B2
• Filed: 11/26/2012
• Issued: 02/12/2019
• Est. Priority Date: 12/28/2011
• Status: Active Grant

• Current Assignee: Hitachi High-Tech Company Limited (Hitachi, Ltd.)
• Original Assignee: Hitachi High-Technologies Corporation (Hitachi, Ltd.)
• Inventors: Minekawa, Yohei, Takagi, Yuji, Harada, Minoru, Hirai, Takehiro, Nakagaki, Ryo
• Primary Examiner(s): Bashore, William L
• Assistant Examiner(s): Parcher, Daniel W
• Application Number:
  US14/360,636
• Publication Number:
  US 20140331173A1
• Time in Patent Office: 2,269 Days
• CPC Class Codes: G02B 21/365 Control or image processing arrangements for digital or video microscopes G02B21/361, G02B21/362 take precedence
• CPC Class Codes: G06F 16/353 into predefined classes
• CPC Class Codes: G06F 16/355 Class or cluster creation or modification
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 9/451 Execution arrangements for user interfaces
• CPC Class Codes: G06T 7/0004 Industrial image inspection
• CPC Class Codes: H01J 2237/2817 Pattern inspection
• CPC Class Codes: H01J 37/265 Controlling the tube; circuit arrangements adapted to a particular application not otherwise provided, e.g. bright-field-dark-field illumination

Abstract

Provided is a GUI including: an unadded pane region that hierarchically displays folders which are sets of images having no class information added thereto; an image pane region that displays the images displayed in the unadded pane region, the displayed images having no classification added thereto; and a class pane region that displays images having classification added thereto, wherein by externally inputting class information for one image having the class information added thereto, the input class information is displayed.

Claims

1. A classification apparatus including a Graphical User Interface (GUI), comprising:

an image capturing apparatus that emits an electron beam to a surface of an object to be checked, detects electrons generated from the surface of the object, and converts the detected electrons into an image; and

a processor including an image feature calculator that calculates a feature amount of the image by processing the image in the image capturing apparatus, a hierarchical structure generator that generates a hierarchical structure of folders including the image using the feature amount of the image calculated by the image feature calculator, and a class information adder that, in a case in which defect class information is externally input for the folders having the hierarchical structure generated by the hierarchical structure generator, adds the defect class information to images in the folders,

wherein the processor is configured to output the GUI comprising

an unadded pane region that displays the hierarchical structure of folders, said folders being associated with sets of images having no defect class information added thereto;

an image pane region that displays images from the sets of images associated with said folders displayed in the unadded pane region, the images displayed in the image pane having no defect class information added thereto; and

a class pane region that displays the images having defect class information added thereto,

wherein by externally inputting defect class information for one image having no defect class information added thereto, the input defect class information is displayed,

wherein said image feature calculator is further configured to calculate a distance between folders based on said calculated feature amounts of a plurality of combinations of two uppermost folders in the hierarchical structure of folders, for all combinations of any two of said uppermost folders,

wherein said hierarchical structure generator is further configured to create a new folder in a new uppermost layer associated with folders in which said calculated distance is shorter than a threshold value,

wherein said image feature calculator is further configured to calculate a first evaluation value based on a distribution of said calculated feature amounts before said new folder in the new uppermost layer is created, and a second evaluation value based on a distribution of said calculated feature amounts after said new folder in the new uppermost layer is created, and

wherein said hierarchical structure generator is further configured to create a further new folder in a further new uppermost layer when said second evaluation value is greater than said first evaluation value by a predetermined threshold amount, and to not create a further new folder in a further new uppermost layer when said second evaluation value does not exceed said first evaluation value by the predetermined threshold amount.

2. The classification apparatus according to claim 1,

wherein the processor is further configured to output, via the GUI, an added pane region that hierarchically displays the folders which are sets of images having defect class information added thereto.

3. The classification apparatus according to claim 1,

wherein the processor is further configured to receive, via the GUI, a selection of a folder corresponding to the images displayed in the class pane region, and is further configured to output, via the GUI, images included in the selected folder for display in the image pane region.

4. The classification apparatus according to claim 1,

wherein when a folder displayed in the unadded pane region is to be moved to the class pane region, the processor is further configured to output, via the GUI, an input screen configured which accepts externally inputted defect class information associated with the folder to be moved,

wherein in a case in which the defect class information on a folder to be moved is not displayed in the class pane region, the processor is further configured to output, via the GUI, the defect class information on the folder to be moved,

wherein in a case in which a folder in the unadded pane region is to be moved to the class pane region, the processor is further configured to receive, via the GUI, defect class information corresponding to the folder to be moved, and

wherein for images corresponding to the folders displayed in the unadded pane region, the processor is further configured to output, via the GUI, folders having a hierarchical structure and sorted into folders having a hierarchical structure on the basis of types of feature amounts externally input.

5. The classification apparatus according to claim 1,

wherein the defect class information includes at least one of a class ID and an assigned input means, and

wherein the processor is further configured to output, via the GUI, the defect class information including at least one of the class ID and the assigned input means.

6. The classification apparatus according to claim 1,

wherein the processor is further configured to provide a sorting function of rearranging the images displayed in the image pane region on the basis of feature amounts of the images.

7. The classification apparatus according to claim 1,

wherein the processor is further configured to receive, via the GUI, a selection of a predetermined layer for display in the unadded pane region, and is further configured to output, via the GUI, a representative image of folders existing in the selected folder.

8. A classification method comprising:

an image capturing step comprising emitting an electron beam to a surface of an object to be checked, detecting electrons generated from the surface of the object, and converting the detected electrons into an image; and

a processing step including an image feature calculating step of calculating a feature amount of the image by processing the image obtained in the conversion in the image capturing step, a hierarchical structure generating step of generating a hierarchical structure of folders including the image by using a feature amount of the image calculated in the image feature calculating step; and

a class information adding step of, in a case in which defect class information is externally input for the folders having the hierarchical structure, adding the defect class information to images in the folders; and

a display step of displaying via a Graphical User Interface (GUI)

an unadded pane region that displays the hierarchical structure of folders, said folders being associated with sets of images having no defect class information added thereto;

an image pane region that displays images from the sets of images associated with said folders displayed in the unadded pane region, the images displayed in the image pane having no defect class information added thereto; and

a class pane region that displays the images having defect class information added thereto,

wherein by externally inputting defect class information for one image having no defect class information added thereto, the input defect class information is displayed,

wherein said image feature calculating step further comprises calculating a distance between folders based on said calculated feature amounts of a plurality of combinations of two uppermost folders in the hierarchical structure of folders, for all combinations of any two of said uppermost folders,

wherein said hierarchical structure generating step further comprises creating a new folder in a new uppermost layer associated with folders in which said calculated distance is shorter than a threshold value,

wherein said image feature calculator is further configured to calculate a first evaluation value based on a distribution of said calculated feature amounts before said new folder in the new uppermost layer is created, and a second evaluation value based on a distribution of said calculated feature amounts after said new folder in the new uppermost layer is created, and

wherein said hierarchical structure generator is further configured to create a further new folder in a further new uppermost layer when said second evaluation value is greater than said first evaluation value by a predetermined threshold amount, and to not create a further new folder in a further new uppermost layer when said second evaluation value does not exceed said first evaluation value by the predetermined threshold amount.

9. The method according to claim 8,

further comprising outputting, via the GUI, an added pane region that hierarchically displays the folders which are sets of images having defect class information added thereto.

10. The method according to claim 8,

further comprising receiving, via the GUI, a selection of a folder corresponding to the images displayed in the class pane region, and outputting, via the GUI, images included in the selected folder for display in the image pane region.

11. The method according to claim 8,

wherein when a folder displayed in the unadded pane region is to be moved to the class pane region, further comprising outputting, via the GUI, an input screen configured which accepts externally inputted defect class information associated with the folder to be moved,

wherein in a case in which the defect class information on a folder to be moved is not displayed in the class pane region, further comprising outputting, via the GUI, the defect class information on the folder to be moved,

wherein in a case in which a folder in the unadded pane region is to be moved to the class pane region, further comprising receiving, via the GUI, defect class information corresponding to the folder to be moved, and

wherein for images corresponding to the folders displayed in the unadded pane region, further comprising outputting, via the GUI, folders having a hierarchical structure and sorted into folders having a hierarchical structure on the basis of types of feature amounts externally input.

12. The method according to claim 8,

wherein the defect class information includes at least one of a class ID and an assigned input means, and

further comprising outputting, via the GUI, the defect class information including at least one of the class ID and the assigned input means.

13. The method according to claim 8,

further comprising a sorting function of rearranging the images displayed in the image pane region on the basis of feature amounts of the images.

14. The method according to claim 8,

further comprising receiving, via the GUI, a selection of a predetermined layer for display in the unadded pane region, and outputting, via the GUI, a representative image of folders existing in the selected folder.

15. A non-transitory computer-readable storage medium upon which is embodied a sequence of programmed instructions which, when executed by a processor, cause the processor to perform steps comprising:

emitting an electron beam to a surface of an object to be checked;

detecting electrons generated from the surface of the object;

converting the detected electrons into an image;

calculating a feature amount of the image by processing the image obtained in the step of converting;

generating a hierarchical structure of folders including the image by using a feature amount of the image calculated in the image feature calculating step;

receiving defect class information externally input for the folders having the hierarchical structure;

adding the defect class information to images in the folders; and

a display step of displaying in a Graphical User Interface (GUI) an image of the GUI, comprising

an unadded pane region that displays the hierarchical structure of folders, said folders being associated with sets of images having no defect class information added thereto;

an image pane region that displays images from the sets of images associated with said folders displayed in the unadded pane region, the images displayed in the image pane having no defect class information added thereto; and

a class pane region that displays the images having defect class information added thereto,

wherein by externally inputting defect class information for one image having no defect class information added thereto, the input defect class information is displayed,

wherein said calculating the image feature amount further comprises calculating a distance between folders based on said calculated feature amounts of a plurality of combinations of two uppermost folders in the hierarchical structure of folders, for all combinations of any two of said uppermost folders,

wherein said generating the hierarchical structure generating step further comprises creating a new folder in a new uppermost layer associated with folders in which said calculated distance is shorter than a threshold value

wherein said image feature calculator is further configured to calculate a first evaluation value based on a distribution of said calculated feature amounts before said new folder in the new uppermost layer is created, and a second evaluation value based on a distribution of said calculated feature amounts after said new folder in the new uppermost layer is created, and

wherein said hierarchical structure generator is further configured to create a further new folder in a further new uppermost layer when said second evaluation value is greater than said first evaluation value by a predetermined threshold amount, and to not create a further new folder in a further new uppermost layer when said second evaluation value does not exceed said first evaluation value by the predetermined threshold amount.

16. The non-transitory computer-readable storage medium according to claim 15,

further comprising outputting, via the GUI, an added pane region that hierarchically displays the folders which are sets of images having defect class information added thereto.

17. The non-transitory computer-readable storage medium according to claim 15,

further comprising receiving, via the GUI, a selection of a folder corresponding to the images displayed in the class pane region, and outputting, via the GUI, images included in the selected folder for display in the image pane region.

18. The non-transitory computer-readable storage medium according to claim 15,

wherein when a folder displayed in the unadded pane region is to be moved to the class pane region, further comprising outputting, via the GUI, an input screen configured which accepts externally inputted defect class information associated with the folder to be moved,

wherein in a case in which the defect class information on a folder to be moved is not displayed in the class pane region, further comprising outputting, via the GUI, the defect class information on the folder to be moved,

wherein in a case in which a folder in the unadded pane region is to be moved to the class pane region, further comprising receiving, via the GUI, defect class information corresponding to the folder to be moved, and

wherein for images corresponding to the folders displayed in the unadded pane region, further comprising outputting, via the GUI, folders having the hierarchical structure and sorted according to the hierarchical structure on the basis of types of feature amounts externally input.

19. The non-transitory computer-readable storage medium according to claim 15,

wherein the defect class information includes at least one of a class ID and an assigned input means, and

further comprising outputting, via the GUI, the defect class information including at least one of the class ID and the assigned input means.

20. The non-transitory computer-readable storage medium according to claim 15,

further comprising a sorting function of rearranging the images displayed in the image pane region on the basis of feature amounts of the images.

21. The non-transitory computer-readable storage medium according to claim 15,

further comprising receiving, via the GUI, a selection of a predetermined layer for display in the unadded pane region, and outputting, via the GUI, a representative image of folders existing in the selected folder.