Computerized scheme for distinction between benign and malignant nodules in thoracic low-dose CT

• US 20060018524A1
• Filed: 07/15/2005
• Published: 01/26/2006
• Est. Priority Date: 07/15/2004
• Status: Abandoned Application

• Current Assignee: University of Chicago
• Original Assignee: UC Tech
• Inventors: Suzuki, Kenji, Doi, Kunio
• Application Number:
  US11/181,884
• Publication Number:
  US 20060018524A1
• Time in Patent Office: Days
• US Class Current: 382/128
• CPC Class Codes: G06F 18/254 of classification results, e.g. of results related to same input data
• CPC Class Codes: G06T 2207/30061 Lung
• CPC Class Codes: G06T 7/0012 Biomedical image inspection
• CPC Class Codes: G06V 10/25 Determination of region of interest [ROI] or a volume of interest [VOI]
• CPC Class Codes: G06V 2201/03 Recognition of patterns in medical or anatomical images

Abstract

A system, method, and computer program product for classifying a target structure in an image into abnormality types. The system has a scanning mechanism that scans a local window across sub-regions of the target structure by moving the local window across the image to obtain sub-region pixel sets. A mechanism inputs the sub-region pixel sets into a classifier to provide output pixel values based on the sub-region pixel sets, each output pixel value representing a likelihood that respective image pixels have a predetermined abnormality, the output pixel values collectively determining a likelihood distribution output image map. A mechanism scores the likelihood distribution map to classify the target structure into abnormality types. The classifier can be, e.g., a single-output or multiple-output massive training artificial neural network (MTANN).

Claims

1. A method of classifying a target structure in an image into predetermined abnormality types, comprising:

scanning a local window across sub-regions of the image to obtain respective sub-region pixel sets;

inputting the sub-region pixel sets into a classifier, wherein the classifier provides, corresponding to the sub-regions, respective output pixel values that each represent a likelihood that respective image pixels have a predetermined abnormality, the output pixel values collectively determining a likelihood distribution map; and

scoring the likelihood distribution map to classify the target structure into the predetermined abnormality types.

2. The method of claim 1, wherein the classifier includes plural output units so that the classifier provides, corresponding to the sub-regions, respective output pixel values for each of the plural output units that each represent a likelihood that respective image pixels have one of the predetermined abnormality types, the output pixel values for each output unit collectively determining a likelihood distribution map, so that plural likelihood distribution maps are determined, and

wherein the scoring step comprises scoring each likelihood distribution map to classify the target structure.

3. The method of claim 2, further comprising:

comparing the scores from the plural output units of the classifier to classify the target structure into one of the predetermined abnormality types.

4. The method of claim 3, wherein the comparing step comprises:

calculating a maximum score among the scores determined in the scoring step.

5. A system for classifying a target structure in an image into predetermined abnormality types, comprising:

a scanning mechanism configured to scan a local window across sub-regions of the image to obtain respective sub-region pixel sets;

a mechanism configured to input the sub-region pixel sets into a classifier configured to provide output pixel values based on the sub-region pixel sets, each output pixel value representing a likelihood that respective image pixels have a predetermined abnormality, the output pixel values collectively determining a likelihood distribution map; and

a mechanism configured to score the likelihood distribution map to classify the target structure into the predetermined abnormality types.

6. A computer program product storing instructions which when executed by a computer programmed with the stored instructions causes the computer to execute a process for classifying a target structure in an image into predetermined abnormality types by performing the steps comprising:

scanning a local window across sub-regions of the image to obtain respective sub-region pixel sets;

inputting the sub-region pixel sets into a classifier, wherein the classifier provides, corresponding to the sub-regions, respective output pixel values that each represent a likelihood that respective image pixels have a predetermined abnormality, the output pixel values collectively determining a likelihood distribution map; and

scoring the likelihood distribution map to classify the target structure into the predetermined abnormality types.

7. A method for determining a likelihood of a predetermined abnormality for a target structure in an image, comprising:

scanning a local window across sub-regions of the image to obtain respective sub-region pixel sets;

inputting the sub-region pixel sets to N classifiers, N being an integer greater than 1, the N classifiers being configured to output N respective outputs, wherein each of the N classifiers provides, corresponding to the sub-regions, respective output pixel values that each represent a likelihood that respective image pixels have the predetermined abnormality, the output pixel values collectively determining a likelihood distribution map;

scoring the N likelihood distribution maps determined by the N classifiers in the inputting step to generate N respective scores indicating whether the target structure is the predetermined abnormality; and

combining the N scores determined in the scoring step to determine an output value indicating a likelihood that the target structure is the predetermined abnormality.

8. The method of claim 7, wherein the combining step comprises:

combining the N scores to determine the output value, wherein the output value is a continuous, non-binary value indicating a likelihood that a nodule structure in the image is malignant.

9. A system for determining a likelihood of a predetermined abnormality for a target structure in an image, comprising:

a scanning mechanism configured to scan a local window across sub-regions of the image to obtain respective sub-region pixel sets;

N classifiers configured to receive the sub-region pixel sets obtained by the scanning mechanism, N being an integer greater than 1, and to output N respective outputs, wherein each of the N classifiers provides, corresponding to the sub-regions, respective output pixel values that each represent a likelihood that respective image pixels have the predetermined abnormality, the output pixel values collectively determining a likelihood distribution map;

a mechanism configured to score the N likelihood distribution maps determined by the N classifiers to generate N respective scores indicating whether the target structure is the predetermined abnormality; and

a combining classifier configured to combine the N scores determined by the mechanism configured to score to determine an output value indicating a likelihood that the target structure is the predetermined abnormality.

10. The system of claim 9, further comprising:

a mechanism configured to identify structures in the image.

11. The system of claim 9, wherein at least one of the classifiers is a massive training artificial neural network (MTANN).

12. The system of claim 9, further comprising:

a graphical user interface configured to display the output value indicating the likelihood that the target structure is the predetermined abnormality.

13. A computer program product storing instructions which when executed by a computer programmed with the stored instructions causes the computer to execute a process for determining a likelihood of a predetermined abnormality for a target structure in an image by performing steps comprising:

scanning a local window across sub-regions of the image to obtain respective sub-region pixel sets;

inputting the sub-region pixel sets to N classifiers, N being an integer greater than 1, the N classifiers being configured to output N respective outputs, wherein each of the N classifiers provides, corresponding to the sub-regions, respective output pixel values that each represent a likelihood that respective image pixels have the predetermined abnormality, the output pixel values collectively determining a likelihood distribution map;

scoring the N likelihood distribution maps determined by the N classifiers in the inputting step to generate N respective scores indicating whether the target structure is the predetermined abnormality; and

combining the N scores determined in the scoring step to determine an output value indicating a likelihood that the target structure is the predetermined abnormality.

14. A method for determining likelihoods of predetermined abnormality types for a target structure in an image, comprising:

scanning a local window across sub-regions of the image to obtain respective sub-region pixel sets;

inputting the sub-region pixel sets to N classifiers, N being an integer greater than 1, each of the N classifiers being configured to output N outputs, wherein each output of each of the N classifiers provides, corresponding to the sub-regions, respective output pixel values that each represent a likelihood that respective image pixels have one of the predetermined abnormality types, the output pixel values for each output of each of the N classifiers collectively determining a likelihood distribution map, so that N² likelihood distribution maps are determined for the image;

scoring, for each of the N classifiers, the N likelihood distribution maps determined by each classifier in the inputting step to generate N respective scores for each classifier indicating, for each classifier, whether the target structure is one of the predetermined abnormality types, so that N² scores are determined for the image; and

combining, for each abnormality type of the predetermined abnormality types, N scores, one score associated with each of the N classifiers and indicating whether the target structure is of the abnormality type, to obtain an output value indicating a likelihood that the target structure is of the abnormality type, so that N output values are determined, one for each abnormality type of the predetermined abnormality types.

15. A system for determining likelihoods of predetermined abnormality types for a target structure in an image, comprising:

a scanning mechanism configured to scan a local window across sub-regions of the image to obtain respective sub-region pixel sets;

N classifiers, each of the N classifiers configured to receive the sub-region pixel sets, N being an integer greater than 1, and to output N outputs, wherein each output of each of the N classifiers provides, corresponding to the sub-regions, respective output pixel values that each represent a likelihood that respective image pixels have one of the predetermined abnormality types, the output pixel values for each output of each of the N classifiers collectively determining a likelihood distribution map so that N² likelihood distribution maps are determined for the image;

N scoring mechanisms, each scoring mechanism configured to score, for a corresponding classifier, the N likelihood distribution maps determined by each classifier to generate N respective scores for each classifier indicating, for each classifier, whether the target structure is one of the predetermined abnormality types, so that N² scores are determined for the image; and

N combining classifiers, each combining classifier configured to combine, for each abnormality type of the predetermined abnormality types, N scores, one score associated with each of the N classifiers and indicating whether the target structure is of the abnormality type, to obtain an output value indicating a likelihood that the target structure is of the abnormality type, so that N output values are determined, one for each abnormality type of the predetermined abnormality types.

16. The system of claim 15, further comprising:

means for displaying the N output values.

17. The system of claim 15, further comprising:

a graphical user interface configured to display the N output values indicating the likelihood that the target structure is of the predetermined abnormality types.

18. The system of claim 17, further comprising:

means for displaying the N output values in the image adjacent to the target structure.

19. The system of claim 15, wherein N is greater than two.

20. A computer program product storing instructions which when executed by a computer programmed with the stored instructions causes the computer to execute a process for determining likelihoods of predetermined abnormality types for a target structure in an image by performing steps comprising:

scanning a local window across sub-regions of the image to obtain respective sub-region pixel sets;

inputting the sub-region pixel sets to N classifiers, N being an integer greater than 1, each of the N classifiers being configured to output N outputs, wherein each output of each of the N classifiers provides, corresponding to the sub-regions, respective output pixel values that each represent a likelihood that respective image pixels have one of the predetermined abnormality types, the output pixel values for each output of each of the N classifiers collectively determining a likelihood distribution map so that N² likelihood distribution maps are determined for the image;

scoring, for each of the N classifiers, the N likelihood distribution maps determined by each classifier in the inputting step to generate N respective scores for each classifier indicating, for each classifier, whether the target structure is one of the predetermined abnormality types so that N² scores are determined for the image; and

combining, for each abnormality type of the predetermined abnormality types, N scores, one score associated with each of the N classifiers and indicating whether the target structure is of the abnormality type, to obtain an output value indicating a likelihood that the target structure is of the abnormality type, so that N output values are determined, one for each abnormality type of the predetermined abnormality types.

21. A system for indicating the likelihood that a lesion in a medical image is one of a first or second type of abnormality, comprising:

a first classifier, configured to analyze a subset of the image, the first classifier being optimized to recognize the first type of abnormality, and configured to output a first score indicative of the likelihood that the lesion is of the first or second type of abnormality;

a second classifier, configured to analyze a subset of the image, the second classifier being optimized to recognize the second type of abnormality, and configured to output a second score indicative of the likelihood that the lesion is of the first or second type; and

a third classifier, configured to combine the first and second scores and to output a third score indicative of the likelihood that the lesion is of the first or second type.

22. The system of claim 21, wherein the first type of abnormality is a benign lesion, and the second type of abnormality is a malignant lesion.

23. A system for indicating at least one score indicative of the likelihood that a target lesion in a medical image is one of a first, second, or third type of abnormality, comprising:

a first classifier, configured to analyze a subset of the image, the first classifier being optimized to recognize the first type of abnormality, and configured to output a first set of three scores, which indicate, respectively, the likelihood that the target lesion is of the first, second, or third type of abnormality;

a second classifier, configured to analyze a subset of the image, the second classifier being optimized to recognize the second type of abnormality, and configured to output a second set of three scores, which indicate, respectively, the likelihood that the target lesion is of the first, second, or third type of abnormality;

a third classifier, configured to analyze a subset of the image, the third classifier being optimized to recognize the third type of abnormality, and configured to output a third set of three scores, which indicate, respectively, the likelihood that the target lesion is of the first, second, or third type of abnormality;

a fourth classifier, configured to combine the three scores from the first, second, and third classifiers that indicate that the target lesion is of the first type of abnormality, and to output a tenth score indicative of the likelihood that the target lesion is of the first type of abnormality;

a fifth classifier, configured to combine the three scores from the first, second, and third classifiers that indicate that the target lesion is of the second type of abnormality and to output a eleventh score indicative of the likelihood that the target lesion is of the second type of abnormality;

a sixth classifier, configured to combine the three scores from the first, second, and third classifiers that indicate that the target lesion is of the third type of abnormality and to output a twelfth score indicative of the likelihood that the target lesion is of the third type of abnormality; and

a graphical user interface configured to display a representation of at least one of the tenth, eleventh, and twelfth scores.

24. The system of claim 23, wherein the displayed representation is at least one numerical value.

25. The system of claim 23, wherein the displayed representation is a graphical representation indicating which of the first, second, and third types of abnormality have the highest likelihood.

26. The system of claim 25, wherein the displayed representation is a color;

and

the system further comprises a means to indicate to a user the correspondence between the color and the type of abnormality having the highest likelihood.

27. The system of claim 23, wherein the displayed representation is displayed adjacent to the image of the target lesion.

28. The system of claim 23, wherein the displayed representation is superimposed on the image of the target lesion.

29. The system of claim 23, wherein the displayed representation is at least two numerical values.

30. A system for indicating at least one score indicative of the likelihood that a target lesion in a medical image is one of N types of abnormality, comprising:

a first set of N classifiers, wherein each classifier in the first set is configured to analyze a subset of the image, and each classifier is optimized to recognize a different one of the N types of abnormalities, and each classifier in the first set is configured to output a first set of N scores, wherein each of the N scores outputted by each classifier indicates the likelihood that the target lesion is one of a different one of the N types of abnormalities;

a second set of N classifiers, wherein each classifier in the second set is configured to combine the one score outputted by each of the first set of N classifiers that indicates that the target lesion is of a single type of abnormality, and wherein each classifier in the second set is configured to combine a different set of N scores; and

wherein each of the second set of N classifiers is configured to output one element of a set of N combined scores each indicating the likelihood that the target lesion is of the said single type of abnormality; and

a graphical user interface configured to display a representation of at least one of the set of N combined scores.

31. A system for indicating the likelihood that an identified region in a medical image is a malignant lesion, or one of a plurality of benign types of abnormalities, comprising:

a first classifier configured to analyze a subset of the image, the first classifier optimized to output a first score indicating whether the identified region is a malignant lesion;

a plurality of additional classifiers each configured to analyze a subset of the image and each optimized to output additional scores indicating whether the suspicious region is one of the different benign types of abnormalities;

a combining classifier configured to combine the first score and the additional scores and to output a set of final scores indicating the likelihoods that the identified region contains a malignant lesion, or one of the plurality of benign types of abnormalities.

32. A system for indicating the likelihood that an identified region in a medical image is one of a plurality of types of abnormalities, comprising:

a plurality of classifiers each configured to analyze a subset of the image and each optimized to output a first score indicating whether the identified region is one of the different types of abnormalities;

a combining classifier configured to combine the set of first scores and to output a set of final scores indicating the likelihoods that the identified region contains one of the plurality of types of abnormalities; and

a graphical user interface configured to display at least one indicator representative of at least one final score of the set of final scores.

33. The system of claim 32, wherein the plurality of abnormalities are indicative of diseases selected from a group comprising fibrosis, scleroderma, polymyositis, rheumatoid arthritis, dermatopolymyositis, aspiration pneumonia, pleural effusion, pulmonary fibrosis, pulmonary hypertension, scleroderma pulmonary, autoimmune interstitial pneumonia, pulmonary veno-occlusive disease, shrinking lung syndrome, lung cancer, and pulmonary embolism.

34. A system for indicating the likelihood that an identified region in an image of a lung is one of N types of abnormalities, comprising:

N classifiers each configured to analyze a subset of the image and each optimized to output one of a first set of N scores indicating whether the identified region is one of the different types of abnormalities;

an additional combining classifier, configured to combine the first set of scores and to output at least one final score indicating at least one likelihood that the identified region is one of the plurality of types of abnormalities; and

a graphical user interface configured to display at least one indicator representative of the at least one final score.