Method and apparatus for automated detection of target structures from medical images using a 3d morphological matching algorithm

• US 20070140541A1
• Filed: 12/03/2003
• Published: 06/21/2007
• Est. Priority Date: 12/04/2002
• Status: Active Grant

• Current Assignee: Jinsung Kim, Kyongtae Bae
• Original Assignee: Jinsung Kim, Kyongtae Bae
• Inventors: Kim, Jinsung, Bae, Kyongtae
• Granted Patent: US 7,949,169 B2
• Time in Patent Office: Days
• US Class Current: 382/131
• CPC Class Codes: G06T 2207/30061 Lung
• CPC Class Codes: G06T 2207/30101 Blood vessel; Artery; Vein; Vascular
• CPC Class Codes: G06T 7/0012 Biomedical image inspection

Abstract

A method for the detection of target structures shown in digital medical images, the method of comprising: (1) generating a three dimensional (3D) volumetric data set of a patient region within which the target structure resides from a plurality of segmented medical image slices; (2) grouping contiguous structures that are depicted in the 3D volumetric data set to create corresponding grouped structure data sets; (3) assigning each grouped structure data set to one of a plurality of detection algorithms, each detection algorithm being configured to detect a different type of target structure; and (4) processing each grouped structure data set according to its assigned detection algorithm to thereby detect whether any target structures are present in the medical images. Preferably, the target structures are pulmonary nodules, and a specialized detection algorithm is applied to image data classified as a candidate for depicting perivascular nodules. To segment perivascular nodule candidates from surrounding vessels, the image data is preferably correlated with a plurality of 3D morphological filters.

Claims

1. A method for automated detection of target structures shown in digital medical images, the method of comprising:

generating a three dimensional (3D) volumetric data set of a patient region within which the target structure resides from a plurality of segmented medical image slices;

grouping contiguous structures that are depicted in the 3D volumetric data set to create corresponding grouped structure data sets;

assigning each grouped structure data set to one of a plurality of detection algorithms, each detection algorithm being configured to detect a different type of target structure; and

processing each grouped structure data set according to its assigned detection algorithm to thereby detect whether any target structures are present in the medical images.

2. The method of claim 1 wherein the patient region being imaged is a region that includes a plurality of vessels, further comprising:

classifying each grouped structure data set as either a vessel group data set or a non-vessel group data set; and

wherein the assigning step comprises assigning each vessel group data set to a first detection algorithm and assigning each non-vessel group data set to a second detection algorithm.

3. The method of claim 2 wherein the processing step comprises:

for each non-vessel group data set, determining a target structure status for the structure depicted therein based on geometric criteria; and

for each vessel group data set, (i) convolving the vessel group data set with at least one predetermined 3D morphological filter to thereby compute a correlation value between the vessel group data set and the 3D morphological filter, (ii) selecting a vessel group data set or subset thereof if its correlation value is within a predetermined range, and (iii) determining a target structure status for the structure depicted in the selected data set based on geometric criteria.

4. The method of claim 2 wherein the target structure is a pulmonary nodule, wherein the medical images are computed tomography (CT) images, and wherein the patient region depicted in the CT images is the patient'"'"'s lung region.

5. The method of claim 4 wherein the geometric criteria includes at least one of the group consisting of structure size, structure compactness, and structure elongation.

6. The method of claim 4 wherein the convolving step comprises convolving the vessel group data set with a plurality of predetermined 3D morphological filters to thereby compute correlation values between the vessel group data set and the 3D morphological filters.

7. The method of claim 6 wherein the predetermined morphological filters comprise a plurality of spherical filters, each spherical filter being tuned to a different predetermined diameter.

8. The method of claim 4 further comprising segmenting the regions depicted the CT images to identify the patient'"'"'s thorax and lung region, including any isolated nodules, perivascular nodules, and pleural nodules that may be present therein.

9. The method of claim 4 wherein the target structure status determining steps comprise eliminating structures that are not pulmonary nodules and identifying structures not eliminated by the eliminating step as nodules.

10. The method of claim 1 wherein the target structures are colon polyps.

11. A device for detecting whether pulmonary nodules are present in a patient'"'"'s lung region from a three-dimensional (3D) data set representative of a volumetric image of the patient'"'"'s lung region, the device comprising:

a processor configured to (1) identify contiguous structures in the 3D data set, (2) classify the identified contiguous structures according to a plurality of classifications, the classifications comprising a vessel contiguous structure classification and a non-vessel contiguous structure classification, (3) apply a first nodule detection operation to each vessel contiguous structure to determine a nodule status therefor, and (4) apply a second nodule detection operation to each non-vessel contiguous structure to determine a nodule status therefor, wherein the first nodule detection operation is different than the second nodule detection algorithm.

12. The device of claim 11 wherein the processor is further configured to apply the first nodule detection operation by:

segmenting nodule candidate structures from surrounding vessel structures through a correlation of each vessel contiguous structure with a 3D morphological filter; and

determining a nodule status for each segmented nodule candidate.

13. The device of claim 12 wherein the processor is further configured to perform the correlation by convolving each vessel contiguous structure with a 3D morphological filter.

14. The device of claim 13 wherein the processor is further configured to perform the correlation by convolving each vessel contiguous structure with a plurality of 3D morphological filters.

15. The device of claim 14 wherein the filters comprise a plurality of spherical filters, each filter being tuned with a different diameter.

16. The device of claim 14 wherein the processor is further configured to determine nodule status for the segmented nodule candidates by determining a nodule status for each nodule candidate at least partially on the basis of geometric criteria.

17. The device of claim 16 wherein the geometric criteria includes size.

18. The device of claim 16 wherein the geometric criteria includes compactness.

19. The device of claim 16 wherein the geometric criteria includes elongation.

20. The device of claim 16 wherein the geometric criteria includes size, compactness, and elongation.

21. The device of claim 14 wherein the processor is further configured to perform the segmentation by determining that a nodule candidate exists if the correlation results in a correlation within a predetermined range of correlation values.

22. The device of claim 12 wherein each non-vessel contiguous structure comprises a nodule candidate, and wherein the processor is further configured to apply the second nodule detection operation by determining a nodule status for each non-vessel nodule candidate at least partially on the basis of geometric criteria.

23. The device of claim 22 wherein the processor is further configured to determine the nodule status for each non-vessel nodule candidate by comparing each nodule candidate with a size criteria.

24. The device of claim 23 wherein the size criteria is diameter.

25. The device of claim 22 wherein the processor is further configured to determine the nodule status for each non-vessel nodule candidate by comparing each nodule candidate with a compactness criteria.

26. The device of claim 25 wherein the processor is further configured to compare each non-vessel nodule candidate with a compactness criteria by:

for each non-vessel nodule candidate, (1) determining a volume of that non-vessel nodule candidate, (2) determining a volume of the smallest 3D box that encloses that non-vessel nodule candidate, and (3) computing a ratio of the determined non-vessel nodule candidate volume to the determined box volume; and

for each non-vessel nodule candidate having a determined volume ratio less than approximately 0.5 or greater than approximately 1.5, determining that that non-vessel nodule candidate is not a pulmonary nodule.

27. The device of claim 22 wherein the processor is further configured to determine the nodule status for each non-vessel nodule candidate by comparing each nodule candidate with an elongation criteria.

28. The device of claim 27 wherein the processor is further configured to compare each non-vessel nodule candidate with an elongation criteria by comparing each non-vessel nodule candidate with a two-dimensional (2D) elongation criteria and a 3D elongation criteria.

29. The device of claim 28 wherein the processor is further configured to compare each non-vessel nodule candidate with an elongation criteria by comparing a non-vessel nodule candidate with the 3D elongation criteria only if that non-vessel nodule candidate satisfies the 2D elongation criteria.

30. The device of claim 27 wherein the processor is further configured to compare each non-vessel nodule candidate with an elongation criteria by:

for each non-vessel nodule candidate, (1) determining a length of a major axis of the smallest rectangle or ellipse that encloses that non-vessel nodule candidate, (2) determining a length of a minor axis of the smallest rectangle or ellipse that encloses that non-vessel nodule candidate, (3) computing a ratio of the major axis length to the minor axis length; and

for each non-vessel nodule candidate having a determined elongation axis ratio greater than approximately 3.0, determining that that non-vessel nodule candidate is not a pulmonary nodule.

31. The device of claim 27 wherein the processor is further configured to compare each non-vessel nodule candidate with an elongation criteria by:

for each non-vessel nodule candidate, (1) determining a maximum eigenvalue from coordinates of the voxels of that non-vessel nodule candidate, (2) determining a minimum eigenvalue from coordinates of the voxels of that non-vessel nodule candidate, (3) computing a ratio of the maximum eigenvalue to the minimum eigenvalue; and

for each non-vessel nodule candidate having a determined elongation eigenvalue ratio greater than approximately 3.0, determining that that non-vessel nodule candidate is not a pulmonary nodule.

32. The device of claim 22 wherein the processor is further configured to determine a nodule status for each non-vessel nodule candidate by comparing each nodule candidate with a size criteria, a compactness criteria, and an elongation criteria.

33. The device of claim 11 wherein the processor is further configured to generate the 3D data set from a plurality of 2D image slices of the patient'"'"'s lung region.

34. The device of claim 33 wherein the 2D image slices comprise a plurality of computed tomography slices.

35. The device of claim 33 wherein the 2D image slices comprise a plurality of magnetic resonance slices.

36. The device of claim 33 wherein the 2D image slices comprise a plurality of ultrasound slices.

67. The device of claim 34 wherein the processor is further configured to accommodate processing CT slices of different slice thicknesses.

68. The device of claim 34 wherein the processor is further configured to accommodate processing CT slices of different reconstruction intervals.

37. A device for analyzing a 3D data set representative of a patient'"'"'s lung region, the device comprising:

a processor configured to (1) group the data set into data subsets, each subset being representative of a contiguous structure, (2) identify each data subset that corresponds to a vessel, and (3) segment any perivascular nodule candidates from each identified subset by correlating that identified subset with at least one 3D morphological filter that is tuned to an expected shape of a perivascular nodule.

38. The device of claim 37 wherein the processor is configured to perform the correlation by correlating each identified subset with a plurality of 3D morphological filters, each filter being tuned to a different expected shape of a perivascular nodule.

39. The device of claim 38 wherein at least one filter is a spherical filter tuned with a predetermined diameter.

40. The device of claim 39 wherein the predetermined diameter is approximately 3 mm.

41. The device of claim 38 wherein a plurality of the filters are spherical filters, each spherical filter being tuned with a different predetermined diameter.

42. The device of claim 38 wherein the processor is further configured to perform the correlation by performing the correlations in parallel, each parallel correlation being configured to correlate an identified subset with one of the filters.

43. The device of claim 37 wherein the processor is further configured to perform the correlation by convolving each identified subset with the at least one filter to thereby compute a correlation value.

44. The device of claim 43 wherein the processor is further configured to perform the segmentation by determining whether a perivascular nodule candidate exists at least partially according to the computed correlation value.

45. The device of claim 44 wherein the processor is further configured to determine whether a perivascular nodule candidate exists by determining that a perivascular nodule candidate does exist if the computed correlation value lies within a predetermined range of correlation values.

46. The device of claim 43 wherein the processor is further configured to perform the convolution by computing the correlation value between the identified subset (I) and the filter (F) by a Fast Fourier Transform (FFT) according to the formula:

$\left(I*F\right)<br><br>\left(t\right)=\sum <br><br>I<br><br>\left(t\right)·<br><br>F<br><br>\left(x-t\right)=\frac{1}{n_i<br><br>n_f}<br><br>\underset{x=0}{\overset{n-1}{\sum <br><br>}}<br><br>I<br><br>\left(u_i\right)·<br><br>\exp <br><br>\left(2<br><br>\mathrm{\pi <br><br>}<br><br>\text{\hspace{1em}<br><br>}<br><br>i<br><br>\text{\hspace{1em}<br><br>}<br><br>u_i<br><br>x/n_i\right)·<br><br>F<br><br>\left(u_f\right)·<br><br>\exp <br><br>\left(2<br><br>\mathrm{\pi <br><br>}<br><br>\text{\hspace{1em}<br><br>}<br><br>i<br><br>\text{\hspace{1em}<br><br>}<br><br>u_f<br><br>\left(t-x\right)/n_f\right)$

wherein u_i is a value of an i^th voxel in I, wherein u_f is a value of an f^th voxel in F, wherein n_i is a value for a total number of voxels in I, and wherein n_f is a value for a total number of voxels in F.

47. The device of claim 38 wherein the processor is further configured to, for each segmented perivascular nodule candidate, determine a nodule status therefor at least partially on the basis of geometric criteria.

48. The device of claim 47 wherein the geometric criteria comprises at least one selected from the group consisting of candidate size, candidate compactness, and candidate elongation.

49. The device of claim 37 wherein the processor is further configured to generate the 3D data set from one selected from the group consisting of a plurality of computed tomography (CT) slices, a plurality of magnetic resonance (MR) slices, and a plurality of ultrasound slices.

50. The device of claim 37 wherein the processor is further configured to (1) identify each data subset that corresponds to a non-vessel, and (2) for each subset identified as corresponding to a non-vessel, determine a nodule status therefor at least partially on the basis of geometric criteria.

51. A computer-readable medium for detecting whether pulmonary nodules are present in a patient'"'"'s lung region from a 3D data set representative of a volumetric image of the patient'"'"'s lung region, the computer-readable medium comprising:

a plurality of instructions executable by a processor for;

(1) identifying contiguous structures in the data set;

(2) classifying the identified contiguous structures according to a plurality of classifications, the classifications comprising a vessel contiguous structure classification and a non-vessel contiguous structure classification;

(3) applying a first nodule detection operation to each vessel contiguous structure to determine a nodule status therefor; and

(4) applying a second nodule detection operation to each non-vessel contiguous structure to determine a nodule status therefor, wherein the first nodule detection operation is different than the second nodule detection operation.

52. The computer-readable medium of claim 51 wherein the instructions for applying the first nodule detection operation comprise a plurality of instructions executable by a processor for (1) segmenting nodule candidate structures from surrounding vessel structures by correlating each vessel contiguous structure with a 3D morphological filter;

and (2) determining a nodule status for each segmented nodule candidate.

53. The computer-readable medium of claim 52 wherein the correlating instructions comprise a plurality of instructions executable by a processor for convolving each vessel contiguous structure with a plurality of 3D morphological filters.

54. The computer-readable medium of claim 53 wherein the nodule status determining instructions for the segmented nodule candidates comprise a plurality of instructions executable by a processor for determining a nodule status for each nodule candidate at least partially on the basis of geometric criteria.

55. The computer-readable medium of claim 54 wherein each non-vessel contiguous structure comprises a nodule candidate, and wherein the instructions for applying the second nodule detection operation comprise a plurality of instructions executable by a processor for determining a nodule status for each non-vessel nodule candidate at least partially on the basis of geometric criteria.

56. A device configured to automatically detect the presence of pulmonary nodules depicted within a three-dimensional data set representative of a patient'"'"'s lung region, the data set comprising a plurality of contiguous structures associated with a first classification and a plurality of contiguous structures associated with a second classification, the device comprising:

a processor configured to (1) apply a first nodule detection algorithm to contiguous structures associated with the first classification, and (2) apply a second nodule detection algorithm to contiguous structures associated with the second classification.

57. The device of claim 56 wherein the first classification comprises a vessel classification, and wherein the second classification comprises a non-vessel classification.

58. The device of claim 57 wherein the processor is further configured to apply the first nodule detection algorithm by segmenting perivascular nodule candidates from the vessel contiguous structures through a correlation of each vessel contiguous structure with a plurality of 3D morphological filters, each filter being tuned to an expected shape of a perivascular nodule.

59. A system for automatic detection of pulmonary nodules shown in CT images, the system comprising:

a CT scanner for scanning a patient'"'"'s lung region to generate a CT image data set;

a computer configured to (1) segment CT image data corresponding to the patient'"'"'s lung region from the CT image data set, (2) generate a three dimensional volumetric data set of a patient'"'"'s lung region from the segmented CT image data, (3) group contiguous structures that are depicted in the volumetric data set to create corresponding grouped structure data sets, (4) classify each grouped structure data set as either a vessel group data set or a non-vessel group data set, (5) for each non-vessel group data set, determine a nodule status for the structure depicted therein based on geometric criteria, and (6) for each vessel group data set, (a) convolve the vessel group data set with a predetermined morphological filter to thereby compute a correlation value between the vessel group data set and the morphological filter, (b) select a vessel group data set if its correlation value is within a predetermined range, and (c) determine a nodule status for the structure depicted in the selected vessel group data set based on geometric criteria.

60. A computer readable medium for use with computer-aided diagnosis of pulmonary nodules present in computed tomography (CT) images, the computer readable medium comprising:

a code segment configured to generate a three dimensional (3D) volumetric data set of a patient'"'"'s lung region from a plurality of segmented CT images;

a code segment configured to classify each grouped structure data set as either a vessel group data set or a non-vessel group data set;

a code segment configured to, for each non-vessel group data set, determine a nodule status for the structure depicted therein based on geometric criteria; and

a code segment configured to, for each vessel group data set, (1) convolve the vessel group data set with a predetermined morphological filter to thereby compute a correlation value between the vessel group data set and the morphological filter, (2) select a vessel group data set if its correlation value is within a predetermined range, and (3) determine a nodule status for the structure depicted in the selected vessel group data set based on geometric criteria.

61. A method of analyzing a data set representative of a region of the patient'"'"'s body, the method comprising:

grouping contiguous structures depicted in the data set;

identifying contiguous structures that correspond to a predefined classification; and

segmenting target structure candidates from the identified structures by correlating each identified structure with at least one filter that is tuned to an expected shape of a target structure.

62. The method of claim 61 wherein the body region is the patient'"'"'s lung region, wherein the predefined classification is a vessel classification, wherein the target structure candidates are perivascular nodule candidates, and wherein the at least one filter is a 3D morphological filter.

63. The method of claim 62 wherein the lung region data set is a 3D data set.

64. The method of claim 63 wherein the correlating step comprises correlating each identified vessel structure with a plurality of 3D morphological filters, each filter being tuned to a different expected shape of a perivascular nodule.

65. The method of claim 61 wherein the data set comprises a plurality of 2D image slices, and wherein at least one of the group consisting of the grouping step, the identifying step, and the segmenting step is performed on a 2D slice-by-slice basis.

66. The method of claim 61 wherein the body region is the patient'"'"'s colonic region, wherein the target structure candidates are colon polyp candidates, and wherein the at least one filter is a 3D morphological filter.