Method, computer program product, and system for the automated analysis of lesions in magnetic resonance, mammogram and ultrasound images

• US 6,317,617 B1
• Filed: 07/25/1997
• Issued: 11/13/2001
• Est. Priority Date: 07/25/1997
• Status: Expired due to Term

• Current Assignee: ARCH Development Corporation (University of Chicago)
• Original Assignee: ARCH Development Corporation (University of Chicago)
• Inventors: Bick, Ulrich, Gilhuijs, Kenneth, Giger, Maryellen L.
• Primary Examiner(s): Smith, Ruth S.
• Application Number:
  US08/900,188
• Time in Patent Office: 1,572 Days
• Field of Search: 128/920, 128/922, 600/407, 600/408, 600/410, 600/411, 600/420, 600/425, 600/427, 600/431, 600/437, 600/443, 600/458, 706/15, 706/20, 706/21, 382/128, 382/131, 382/132
• US Class Current: 600/408
• CPC Class Codes: A61B 5/055 involving electronic [EMR] or nuclear [NMR] magnetic resonance, e.g. magnetic resonance imaging
• CPC Class Codes: A61B 5/7264 Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems neural networks per se G06N3/00; expert systems per se G06N5/00
• CPC Class Codes: G06T 2207/10096 Dynamic contrast-enhanced magnetic resonance imaging [DCE-MRI]
• CPC Class Codes: G06T 2207/30068 Mammography; Breast
• CPC Class Codes: G06T 7/0012 Biomedical image inspection
• CPC Class Codes: Y10S 128/922 including image analysis

Abstract

A method and system for the computerized automatic analysis of lesions in magnetic resonance (MR) images, a computer programmed to implement the method, and a data structure for storing required parameters is described. Specifically the system includes the computerized analysis of lesions in the breast using spatial, temporal and/or hybrid measures. Techniques include novel developments and implementations of two-dimensional and three-dimensional features to assess the characteristics of the lesions and in some cases give an estimate of the likelihood of malignancy or of prognosis. The system can also allow for the enhanced visualization of the breast and its pathological states. The system also includes an option to merge the extracted features with those from x-ray and/or ultrasound images in order to further characterize the lesion and/or make a diagnosis and/or a prognosis.

Claims

1. A method for the analysis of a lesion in an anatomy, comprising:

obtaining plural image data, representative of plural images of a same portion of the anatomy, derived from at least two imaging modalities selected from the group consisting of magnetic resonance imaging, x-ray imaging, and ultrasound imaging;

identifying from the plural image data a possible lesion in said plural images;

extracting, for each of said plural images derived from said at least two imaging modalities, at least one feature related to characterization of a lesion from image data corresponding to the identified possible lesion; and

merging in a common image classifier a plurality of extracted features, including at least one feature related to characterization of a lesion from each of said plural images derived from said at least two imaging modalities to characterize said possible lesion based on the merged plurality of extracted features and yield a corresponding classification.

2. The method according to claim 1, further comprising:

extracting features from at least one of the interior or the surface of the identified possible lesion.

3. The method according to claim 1, further comprising:

extracting geometric based features of the identified possible lesion.

4. The method according to claim 3, further comprising:

extracting at least one of circularity and irregularity geometric based features of the identified possible lesion.

5. The method according to claim 4, further comprising:

extracting the circularity geometric based features of the identified possible lesion based on the following equation;

$C=\frac{E<br><br>\text{\hspace{1em}<br><br>}<br><br>F<br><br>\text{\hspace{1em}<br><br>}<br><br>V}{V}$

 

where C is a measure of circularity of the identified possible lesion, EFV is an effective volume of the identified possible lesion, and V is a sphere having a same volume as the effective volume of the identified possible lesion.

6. The method according to claim 4, further comprising:

extracting the irregularity geometric based features of the identified possible lesion based on the following equation;

$I=1-\mathrm{\pi <br><br>}·<br><br>\frac{{\left(E<br><br>\text{\hspace{1em}<br><br>}<br><br>D\right)}^2}{S}$

 

where I is a measure of irregularity of the lesion, ED is an effective diameter of the identified possible lesion, and S is a surface area of the identified possible lesion.

7. The method according to claim 1, further comprising:

extracting gray level based features of the identified possible lesion.

8. The method according to claim 1, further comprising:

extracting from each of said images derived from said at least two modalities gradient based features of the identified possible lesion.

9. The method according to claim 1, further comprising:

extracting from each of said images derived from said at least two modalities gradient based features of the identified possible lesion based on at least one of the following equations;

$G<br><br>\text{\hspace{1em}<br><br>}<br><br>\mathrm{F1}=\underset{i\in <br><br>\mathrm{lesion}}{\sum <br><br>}<br><br>\frac{R_i·<br><br>G_i}{G_i}$$G<br><br>\text{\hspace{1em}<br><br>}<br><br>\mathrm{F2}=\underset{i\in <br><br>\mathrm{shell}}{\sum <br><br>}<br><br>\frac{R_i·<br><br>G_i}{G_i}$

 

where GF1 and GF2 are a gradient based features of the identified possible lesion, and a 3-voxel thick shell around the identified possible lesion, respectively, R_i is an effective diameter of the lesion, and G_i is a surface area of the identified possible lesion.

10. The method according to claim 1, further comprising:

extracting from each of said images derived from said at least two modalities a texture based feature of the identified possible lesion.

11. The method according to claim 10, further comprising:

extracting from each of said images derived from said at least two modalities the texture based feature based on the following equation;

12. The method according to anyone of claim 1, further comprising:

extracting and comparing features of the identified possible lesion with features of a surrounding area of the identified possible lesion for each of said images derived from said at least two different modalities.

13. The method according to anyone of claim 1, wherein said extracting step comprises:

extracting from each of said images derived from said at least two different modalities temporal based features including at least one of Gd-DTPA uptake, speed of Gd-DTPA uptake, and inhomogeneity of Gd-DTPA uptake in the lesion calculated in terms of volume, uptake in the center of lesion, uptake in the margin of the lesion, a flow of contrast agent into the lesion, and a flow of contrast agent out of the lesion.

14. The method according to anyone of claim 13, wherein said characterizing step comprises:

providing at least one of a probability of malignancy, diagnosis, and prognosis of the identified possible lesion based on the extracted features.

15. The method according to anyone of claim 13, wherein said characterizing step comprises:

using an artificial neural network to provide at least one of a probability of malignancy, diagnosis, and prognosis of the identified possible lesion based on the at least one feature extracted from the plural images derived from said at least two imaging modalities.

16. The method according to claim 1, wherein said characterizing step comprises:

providing at least one of a probability of malignancy, diagnosis, and prognosis of the identified possible lesion based on the extracted features.

17. The method according to claim 1, wherein said characterizing step comprises:

using an artificial neural network to provide at least one of a probability of malignancy, diagnosis, and prognosis of the identified possible lesion based on the at least one feature extracted from the plural images derived from said at least two imaging modalities.

18. The method according to anyone of claim 1, further comprising:

extracting features that characterize a lesion within the image data in at least one of two dimensions and three dimensions.

32. A storage medium storing a program for performing the steps recited in one of claims 1-31.

19. A method for the analysis of a lesion in an anatomy, comprising:

obtaining plural image data, representative of plural images of a same portion of the anatomy, derived from at least one imaging modality selected from the group consisting of magnetic resonance imaging, x-ray imaging, and ultrasound imaging, said plural images derived at different times during introduction of a contrast agent into said anatomy;

identifying from the image data a possible lesion in said plural images;

extracting at least one feature related to inhomogeneity of uptake of a lesion from said plural image data corresponding to the identified possible lesion ; and

characterizing said possible lesion based at least in part on the at least one feature extracted from the plural images derived from said at least one imaging modality.

20. The method according to claim 19, further comprising:

extracting geometric based features of the identified possible lesion; and

characterizing said possible lesion based at least in part on the extracted geometric based features.

21. The method according to claim 20, further comprising:

extracting at least one of circularity and irregularity geometric based features from the identified possible lesion.

22. The method according to claim 19, further comprising:

extracting gray level based features from the identified possible lesion; and

characterizing said possible lesion based at least in part on the extracted gray level based features.

23. The method according to claim 19, further comprising:

extracting from each of said images gradient based features of the identified possible lesion; and

characterizing said possible lesion based at least in part on the extracted gradient based features.

24. The method according to claim 19, further comprising:

extracting from each of said images a texture based feature of the identified possible lesion; and

characterizing said possible lesion based at least in part on the extracted texture based features.

25. The method according to claim 19, further comprising:

extracting and comparing features of the identified possible lesion with features of a surrounding area of the identified possible lesion for each of said images; and

characterizing said possible lesion based at least in part on the extracted and compared features.

26. The method according to claim 19, wherein said determining step comprises:

determining from each of said images a variation over time of at least one of Gd-DTPA uptake, speed of Gd-DTPA uptake, and inhomogeneity of Gd-DTPA uptake in the lesion calculated in terms of volume, uptake in the center of lesion, uptake in the margin of the lesion, a flow of contrast agent into the lesion, and a flow of contrast agent out of the lesion.

27. The method according to claim 26, wherein said characterizing step comprises:

providing at least one of a probability of malignancy, diagnosis, and prognosis of the identified possible lesion based on the extracted features.

28. The method according to claim 26, wherein said characterizing step comprises:

using an artificial neural network to provide at least one of a probability of malignancy, diagnosis, and prognosis of the identified possible lesion based on the at least one feature extracted from the plural images.

29. The method according to claim 19, wherein said characterizing step comprises:

providing at least one of a probability of malignancy, diagnosis, and prognosis of the identified possible lesion based on the extracted features.

30. The method according to claim 19, wherein said characterizing step comprises:

using an artificial neural network to provide at least one of a probability of malignancy, diagnosis, and prognosis of the identified possible lesion based on the at least one feature extracted from the plural images.

31. The method according to claim 19, further comprising:

extracting features that characterize a lesion within the image data in at least one of two dimensions and three dimensions.

33. The method according to claim 19, wherein said extracting step comprises extracting a feature related to a time variation in a voxel value of the lesion.

34. The method according to claim 19, wherein said extracting step comprises extracting a feature related to a time variation in a standard deviation of a voxel value in the lesion.

35. The method according to claim 19, wherein said extracting step comprises extracting a feature related to a radial gradient frequency variation in the lesion.

36. The method according to claim 19, wherein:

said extracting step comprises extracting from said plural image data corresponding to the identified possible lesion at least one further feature related to an uptake of a lesion;

and said characterizing step comprises characterizing said possible lesion based at least in part on the one further feature related to the uptake of the lesion.

37. The method according to claim 36, wherein said extracting step comprises extracting a feature related to uptake in a margin of the lesion.

38. The method according to claim 36, wherein said extracting step comprises extracting a feature related to uptake in a center of the lesion.

39. The method according to claim 36, wherein said extracting step comprises extracting a feature related to a speed of uptake in the lesion.

40. The method according to claim 19, wherein:

said extracting step comprises extracting at least one further feature related to a spatial characteristic of a lesion from said plural image data corresponding to the identified possible lesion;

said characterizing step comprises characterizing said possible lesion based at least in part on the one further feature related to the spatial characteristic of the lesion.

41. The method according to claim 40, wherein said extracting step comprises extracting a feature related to a textural characteristic of the lesion.

42. The method according to claim 40, wherein said extracting step comprises extracting a feature related to a power spectrum of voxel values in the lesion.

43. The method according to claim 40, wherein said extracting step comprises extracting a feature related to a variation in a margin characteristic of the lesion.

44. The method according to claim 40, wherein said extracting step comprises extracting a feature related to a radial gradient index of the lesion.

45. The method according to claim 40, wherein said extracting step comprises extracting a feature related to a margin gradient of the lesion.

46. The method according to claim 40, wherein said extracting step comprises extracting a feature related to a shape feature of the lesion.

47. The method according to claim 46, wherein said extracting step comprises extracting a feature related to an irregularity of the lesion.

48. The method according to claim 46, wherein said extracting step comprises extracting a feature related to a circularity of the lesion.

49. A system for the analysis of a lesion in an anatomy, comprising:

means for obtaining plural image data, representative of plural images of a same portion of the anatomy, derived from at least two imaging modalities selected from the group consisting of magnetic resonance imaging, x-ray imaging, and ultrasound imaging;

means for identifying from the image data a possible lesion in said plural images;

means for extracting, for each of said plural images derived from said at least two imaging modalities, at least one feature related to characterization of a lesion from said plural image data corresponding to the identified possible lesion; and

means for merging in a common image classifier a plurality of extracted features, including at least one feature extracted from each of the plural images derived from said at least two imaging modalities to characterize said possible lesion based on the merged extracted features and yield a corresponding classification.

50. A system for the analysis of a lesion in an anatomy, comprising:

means for obtaining plural image data, representative of plural images of a same portion of the anatomy, derived from at least one imaging modality selected from the group consisting of magnetic resonance imaging, x-ray imaging and ultrasound imaging, said plural images derived at different times during introduction of a contrast agent into said anatomy;

means for identifying from the image data a possible lesion in said plural images;

means for extracting at least one feature related to inhomogeneity of uptake of a lesion from said image data corresponding to the identified possible lesion;

and means for characterizing said possible lesion based at least in part on the at least one feature extracted from the plural images derived from said at least one imaging modality.

51. A method for the analysis of a lesion in an anatomy, comprising:

obtaining plural image data, representative of plural images of a same portion of the anatomy, derived from at least one imaging modality selected from the group consisting of magnetic resonance imaging, x-ray imaging, and ultrasound imaging;

identifying from the image data a possible lesion in said plural images;

extracting a radial gradient peak distinction and a variance of a gradient along a margin of said possible lesion; and

characterizing said possible lesion based at least in part on said radial gradient peak distinction and said variance of a gradient along a margin extracted from the plural images derived from said at least one imaging modality.