Systems and methods for automated diagnosis and grading of tissue images

• US 20060064248A1
• Filed: 08/09/2005
• Published: 03/23/2006
• Est. Priority Date: 08/11/2004
• Status: Active Grant

• Current Assignee: David Sans
• Original Assignee: Aureon Biosciences, Inc.
• Inventors: Saidi, Olivier, Teverovskiy, Mikhail, Tabesh, Ali
• Granted Patent: US 7,761,240 B2
• Time in Patent Office: Days
• US Class Current: 702/19
• CPC Class Codes: G06T 2207/30024 Cell structures in vitro; Tissue sections in vitro
• CPC Class Codes: G06T 7/0012 Biomedical image inspection
• CPC Class Codes: G06V 20/695 Preprocessing, e.g. image segmentation

Abstract

Systems and methods are provided for automated diagnosis and grading of tissue images based on morphometric data extracted from the images by a computer. The morphometric data may include image-level morphometric data such as fractal dimension data, fractal code data, wavelet data, and/or color channel histogram data. The morphometric data may also include object-level morphometric data such as color, structural, and/or textural properties of segmented image objects (e.g., stroma, nuclei, red blood cells, etc.).

Claims

1. A method for evaluating a tissue image for a medical condition, the method comprising:

receiving morphometric data for the tissue image; and

evaluating the morphometric data with a model predictive of the medical condition, wherein the model is based on one or more morphometric features from the group of morphometric features consisting of one or more fractal dimension features, one or more fractal code features, one or more wavelet features, and one or more color channel histogram features, thereby evaluating the tissue image for the medical condition.

2. The method of claim 1, wherein the model is based on one or more fractal dimension features from one or more binary images, each binary image corresponding to a particular color channel of the image.

3. The method of claim 2, further comprising extracting the one or more fractal dimension features from the tissue image, wherein said extracting comprises:

thresholding the tissue image at one or more N_b fixed, equally-spaced thresholds in one or more of its red, green, and blue (RGB) color channels to obtain one or more binary images; and

computing a fractal dimension value γ

_kⁱ for each of the one or more binary images.

4. The method of claim 1, wherein the model is based on one or more fractal code features from the group of fractal code features consisting of a mean square error (MSE) between a domain block and a range block, a shift parameter of an affine transform, a scaling parameter of an affine transform, a shuffling transform, and a Euclidean distance between a domain block and a range block in an image plane.

5. The method of claim 4, further comprising extracting the one or more fractal code features from the tissue image, wherein said extracting comprises:

partitioning the tissue image into non-overlapping blocks of two different sizes;

range blocks B_j and domain blocks {overscore (A)}_i; and

searching for a domain block {overscore (A)}ⁱ and a transformation T_i such that the transformed domain block T_i{overscore (A)}_i is an approximation of B_j,

wherein one or more characteristics of the transformation comprise the one or more fractal code features.

6. The method of claim 5, wherein said searching comprises searching for a domain block {overscore (A)}_i and a transformation T_i such that a mean square error (MSE) between the domain block and range block is minimized.

7. The method of claim 1, wherein the model is based on one or more wavelet features comprising one or more measurements of variance of wavelet coefficients for one or more wavelet representation subbands.

8. The method of claim 7, further comprising extracting the one or more wavelet features from the tissue image, wherein said extracting comprises:

generating a wavelet representation of the tissue image; and

computing the variance of the wavelet subband coefficients for at least one of the wavelet representation subbands.

9. The method of claim 1, wherein the model is based on one or more color channel histogram features comprising one or more pixel counts at one or more intensities of one or more color channels.

10. The method of claim 9, further comprising extracting the one or more channel histogram features from the tissue image, wherein said extracting comprises:

separating the tissue image into one or more of its red, green, and blue (RGB) color channels to produce one or more color channel images;

separating one or more of the color channel images into bins representing levels of color intensity; and

computing a pixel count for one or more of the bins.

11. The method of claim 1, wherein the model predicts whether the tissue in the tissue image is cancerous or non-cancerous.

12. The method of claim 1, wherein the model predicts a cancer grade for the tissue in the tissue image.

13. The method of claim 12, wherein the cancer grade comprises a Gleason grade.

14. A method for evaluating a tissue image, the method comprising:

evaluating a first set of morpometric data from the tissue image with a model that predicts whether the tissue in the tissue image is cancerous or non-cancerous; and

when the model classifies the tissue as cancerous tissue, evaluating a second set of morphometric data from the tissue image with a model that predicts a cancer grade for the tissue.

15. The method of claim 14, wherein said first set of morphometric data comprises image-level morphometric data from the group of image-level morphometric data consisting of fractal dimension data, fractal code data, wavelet data, and color channel histogram data.

16. The method of claim 15, wherein said second set of morphometric data comprises object-level data comprising one or more color, structural, and/or textural properties of one or more segmented image objects.

17. The method of claim 14, wherein the tissue image comprises a tissue image of a portion of a tissue section, and wherein said method comprises:

performing, for each portion of said tissue section, said evaluating of said first set of morpometric data from a tissue image for that portion with said model that predicts whether tissue in the tissue image is cancerous or non-cancerous; and

performing, only for portion(s) of the tissue section classified as cancerous, said evaluating a second set of morphometric data from the tissue image(s) with a model that predicts a cancer grade for the tissue.

18. A method for generating a model predictive of a medical condition, the method comprising:

receiving morphometric data from two or more tissue images with known classification with respect to the medical condition, wherein the morphometric data comprises morphometric data from the group of morphometric data consisting of fractal dimension data, fractal code data, wavelet data, and color channel histogram data; and

performing feature selection and/or classification on the morphometric data, thereby generating the model predictive of the medical condition.

19. The method of claim 18, wherein the model predicts whether the tissue in the tissue image is cancerous or non-cancerous.

20. The method of claim 18, wherein the model predicts a cancer grade for the tissue in the tissue image.

21. The method of claim 20, wherein the cancer grade is a Gleason grade.

42. The apparatus of claim 20, wherein the cancer grade is a Gleason grade.

22. An apparatus for evaluating a tissue image for a medical condition, the apparatus comprising:

a model predictive of the medical condition, wherein the model is based on one or more morphometric features from the group of morphometric features consisting of one or more fractal dimension features, one or more fractal code features, one or more wavelet features, and one or more color channel histogram features, wherein the model is configured to;

receive morphometric data for the tissue image; and

evaluate the morphometric data for the tissue image according to the model, thereby evaluating the tissue image for the medical condition.

23. The apparatus of claim 22, wherein the model is based on one or more fractal dimension features from one or more binary images, each binary image corresponding to a particular color channel of the image.

24. The apparatus of claim 23, further comprising an image processing tool configured to extract the one or more fractal dimension features from tissue image, wherein said image processing tool is configured to:

threshold the tissue image at one or more N_b fixed, equally-spaced thresholds in one or more of its red, green, and blue (RGB) color channels to obtain one or more binary images; and

compute a fractal dimension value γ

_kⁱ for each of the one or more binary images.

25. The apparatus of claim 22, wherein the model is based on one or more fractal code features from the group of fractal code features consisting of a mean square error (MSE) between a domain block and a range block, a shift parameter of an affine transform, a scaling parameter of an affine transform, a shuffling transform, and a Euclidean distance between a domain block and a range block in an image plane.

26. The apparatus of claim 25, further comprising an image processing tool configured to extract the one or more fractal code features from the tissue image, wherein the image processing tool is configured to:

partition the tissue image into non-overlapping blocks of two different sizes;

range blocks B_j and domain blocks {overscore (A)}_i; and

search for a domain block {overscore (A)}_i and a transformation T_i such that the transformed domain block T_i{overscore (A)}_i is an approximation of B_j,

wherein one or more characteristics of the transformation comprise the one or more fractal code features.

27. The apparatus of claim 26, wherein said image processing tool is configured to search for a domain block {overscore (A)}_i and a transformation T_i such that a mean square error (MSE) between the domain block and range block is minimized.

28. The apparatus of claim 22, wherein the model is based on one or more wavelet features comprising one or more measurements of variance of wavelet coefficients for one or more wavelet representation subbands.

29. The apparatus of claim 28, further comprising an image processing tool configured to extract the one or more wavelet features from the tissue image, wherein the image processing tool is configured to:

generate a wavelet representation of the tissue image; and

compute the variance of the wavelet subband coefficients for at least one of the wavelet representation subbands.

30. The apparatus of claim 22, wherein the model is based on one or more color channel histogram features comprising one or more pixel counts at one or more intensities of one or more color channels.

31. The apparatus of claim 30, further comprising an image processing tool configured to extract the one or more channel histogram features from the tissue image, wherein said image processing tool is configured to:

separate the tissue image into one or more of its red, green, and blue (RGB) color channels to produce one or more color channel images;

separate one or more of the color channel images into bins representing levels of color intensity; and

compute a pixel count for one or more of the bins.

32. The apparatus of claim 22, wherein the model predicts whether the tissue in the tissue image is cancerous or non-cancerous.

33. The apparatus of claim 22, wherein the model predicts a cancer grade for the tissue in the tissue image.

34. The apparatus of claim 33, wherein the cancer grade comprises a Gleason grade.

35. An apparatus for evaluating a tissue image, the apparatus comprising:

a first model configured to evaluate a first set of morpometric data from the tissue image in order to predict whether the tissue in the tissue image is cancerous or non-cancerous; and

a second model configured to evaluate, subsequent to the first model classifying the tissue as cancerous tissue, a second set of morphometric data from the tissue image in order to predict a cancer grade for the tissue.

36. The apparatus of claim 35, wherein said first set of morphometric data comprises image-level morphometric data from the group of image-level morphometric data consisting of fractal dimension data, fractal code data, wavelet data, and color channel histogram data.

37. The apparatus of claim 36, wherein said second set of morphometric data comprises object-level data comprising one or more color, structural, and/or textural properties of one or more segmented image objects.

38. The apparatus of claim 35, wherein the tissue image comprises a tissue image of a portion of a tissue section, and wherein:

said first model is configured to perform, for each portion of said tissue section, said evaluating of said first set of morpometric data from a tissue image for that portion; and

said second model is configured to perform, only for portion(s) of the tissue section classified as cancerous by said first model, said evaluating of a second set of morphometric data from the tissue image(s).

39. An apparatus for generating a model predictive of a medical condition, the apparatus comprising:

an analytical tool configured to;

receive morphometric data from two or more tissue images with known classification with respect to the medical condition, wherein the morphometric data comprises morphometric data from the group of morphometric data consisting of fractal dimension data, fractal code data, wavelet data, and color channel histogram data; and

perform feature selection and/or classification on the morphometric data, thereby generating the model predictive of the medical condition.

40. The apparatus of claim 39, wherein the model predicts whether the tissue in the tissue image is cancerous or non-cancerous.

41. The apparatus of claim 39, wherein the model predicts a cancer grade for the tissue in the tissue image.

43. A computer readable medium comprising computer executable instructions recorded thereon for performing the method comprising:

receiving morphometric data for a tissue image; and

evaluating the morphometric data with a model predictive of a medical condition, wherein the model is based on one or more morphometric features from the group of morphometric features consisting of one or more fractal dimension features, one or more fractal code features, one or more wavelet features, and one or more color channel histogram features, thereby evaluating the tissue image for the medical condition.