Malignancy diagnosis using content-based image retreival of tissue histopathology

• US 8,280,132 B2
• Filed: 08/01/2007
• Issued: 10/02/2012
• Est. Priority Date: 08/01/2006
• Status: Active Grant

• Current Assignee: Rutgers University, Trustees Of The University Of Pennsylvania (University of Pennsylvania)
• Original Assignee: Rutgers University, Trustees Of The University Of Pennsylvania (University of Pennsylvania)
• Inventors: Madabhushi, Anant, Doyle, Scott, Feldman, Michael D., Tomaszewski, John E.
• Primary Examiner(s): Bella, Matthew
• Assistant Examiner(s): Li, Ruiping
• Application Number:
  US12/375,981
• Publication Number:
  US 20100098306A1
• Time in Patent Office: 1,889 Days
• Field of Search: 382/128, 382/190, 382/224, 600/410
• US Class Current: 382/128
• CPC Class Codes: G06T 2207/30096 Tumor; Lesion
• CPC Class Codes: G06T 7/0012 Biomedical image inspection
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06V 20/695 Preprocessing, e.g. image segmentation

Abstract

This invention relates to computer-aided diagnostics using content-based retrieval of histopathological image features. Specifically, the invention relates to the extraction of image features from a histopathological image based on predetermined criteria and their analysis for malignancy determination.

Claims

1. A computer-aided diagnostic method to predict a probability that a histological image contains a malignant region comprising:

a. obtaining said histological image, wherein the histological image is a first of a series of further histological images ordered in increasing magnification;

b. identifying a region or regions of said histological image classified as suspect;

c. extracting one or more image features from at least one of said identified region or regions;

d. reducing a dimensionality of said extracted feature image features; and

e. classifying said identified region or regions as either benign, malignant, or suspect based on at least one of said extracted image features, wherein if said identified region or regions are classified as malignant, the histological image has a malignant region, otherwise;

i. if the identified region or regions are classified as benign, then the histological image does not have a malignant region;

or

ii. if the identified region or regions are classified as suspect, a next histological image in the series is obtained and the steps of identifying, extracting, reducing and classifying are repeated.

2. The method of claim 1, wherein said series of images are generated by a pyramidal decomposition.

3. The method of claim 1, wherein said image feature is a statistical feature, a Haralick co-occurrence feature, a Haar wavelet feature, or a Gabor feature, wherein said image feature is calculated independently from each channel of the respective image using a sliding window of 3×

3, 5×

5, or 7×

7 pixels.

4. The method of claim 3, wherein said statistical feature is an average, a median, a standard deviation, a difference, a Sobel filter, a Kirsch filter, a horizontal derivative, a vertical derivative, or a diagonal derivative of a pre-selected determinant.

5. The method of claim 3, wherein said Haralick co-occurrence feature is an angular second moment, a contrast, a correlation, a variance, an entropy, an inverse difference moment, a sum average, a sum variance, a sum entropy, a difference variance, or a difference entropy.

6. The method of claim 3, wherein said Haar wavelet feature is a result of a convolution of said respective image with a Haar wavelet kernel.

7. The method of claim 3, wherein said Gabor feature is a result of convolving said respective image with a bank of Gabor filters, generated by convolving a Gaussian function with a sinusoid at a range of scale and orientation parameter values.

8. The method of claim 1, wherein the step of reducing the dimensionality is done by using a principal component analysis, a linear dimensional analysis, a multidimensional scaling, a graph embedding, an ISOMAP, a local linear embedding, a kernel-based principal component analysis, a semidefinite embedding, Laplacian eigenmaps, or a combination thereof.

9. The method of claim 1, wherein said step of classifying further comprises the steps of:

a. creating a likelihood scene for a selected image of the series of images;

b. creating a mask identifying regions of said selected image as benign or suspect by thresholding said likelihood scene at a value determined through system training; and

c. classifying said identified region or regions as either benign, malignant, or suspect based on said mask, wherein if the identified region or regions are classified as suspect, said mask is resized by interpolation to a size of a subsequent image in the series to identify suspect regions on said subsequent image.

10. The method of claim 9 wherein said likelihood scene is determined by:

a Decision Tree algorithm, a Support Vector Machine algorithm, an Adaboost algorithm using Bayes Decision Theorem, or at least one image feature extracted from one or more training images.

11. A method for histopathological analysis of a gland tissue sample, comprising:

a. selecting an image of the gland tissue sample, wherein said image comprises regions classified by the methods of claim 9 as malignant, benign or suspect;

b. identifying regions of said selected image classified as suspect;

c. extracting at least one image feature from at least one of said identified regions;

d. reducing the dimensionality of said at least one extracted image feature; and

e. classifying said identified regions as one of the group consisting of a cancerous tissue of a specific clinical stage of malignancy, a benign epithelium, and a benign stroma based on said at least one extracted image feature.

12. The method of claim 11, wherein said at least one image feature is at least one of the group consisting of an architectural feature, a nuclear density feature, a gland morphology feature, and a global texture feature.

13. The method of claim 12, wherein said architectural feature is selected from the group of features calculated from a Voronoi diagram, a Delaunay graph, and a minimum spanning tree constructed from nuclei centers in the image, and a co-adjacency matrix constructed from the gland centers in the image.

14. The method of claim 13, wherein said architectural feature computed from the Voronoi diagram is a standard deviation, an average, a ratio of minimum to maximum, or a disorder of areas, perimeters, or chords of polygons in the Voronoi diagram of said image.

15. The method of claim 13, wherein said architectural feature computed from the Delaunay graph is a standard deviation, an average, a ratio of minimum to maximum, or a disorder of side lengths or areas of triangles in the Delaunay graph of said image.

16. The method of claim 13, wherein said architectural feature computed from the minimum spanning tree is a standard deviation, an average, a ratio of minimum to maximum, or a disorder of edge lengths in the minimum spanning tree of said image.

17. The method of claim 13, wherein said co-adjacency matrix constructed from the gland centers in said image is used to calculate an angular second moment, a contrast, a correlation, a variance, an entropy, an inverse difference moment, a sum average, a sum variance, a sum entropy, a difference variance, or a difference entropy.

18. The method of claim 12, wherein said nuclear density feature is a number or density of nuclei in said image;

an average, a standard deviation, or a disorder of distances from each of the nuclei in said image to 3, 5, or 7 nearest neighboring nuclei;

or the average, the standard deviation, or the disorder of the number of neighboring nuclei within a circle with a radius of 10, 20, 30, 40, or 50 pixels centered on each nucleus within said image.

19. The method of claim 12, wherein said gland morphology feature is calculated from a boundary of either a gland lumen or an interior nuclei as a ratio of a gland area to an area of a smallest circle that circumscribes a gland of the image;

a standard deviation, a variance, or a ratio of a maximum to an average distance between a gland center and a boundary;

a ratio of an estimated boundary length (the boundary calculated using a fraction of the boundary pixels) to an actual boundary length;

a ratio of a boundary length to an area enclosed by the boundary; and

a sum of a difference between a distance from the gland center to the boundary and an average of the distances from the gland center to two adjacent points.

20. The method of claim 12, wherein said global texture feature is an average, a median, a standard deviation, a range, an angular second moment, a contrast, a correlation, a variance, an entropy, a sum average, a sum variance, a sum entropy, a difference variance, a difference entropy, a difference moment, or a Gabor filter.

21. The method of claim 11, wherein the dimensionality of the at least one extracted image feature is reduced through use of principal component analysis, linear dimensional analysis, multidimensional scaling, graph embedding, ISOMAP, local linear embedding, kernel-based principal component analysis, semidefinite embedding, or Laplacian eigenmaps.

22. The method of claim 11, wherein said step of classification is performed by:

a Support Vector Machine algorithm, a Decision Tree algorithm, an Adaboost algorithm using Bayes Decision Theorem or combination thereof.

24. The method of claim 1, wherein the histological image is of a prostate.

25. The method of claim 11, wherein the gland tissue sample is of a prostate.

23. A system for predicting a probability that a histological image contains a malignant region, the system comprising:

a database for storing a series of further histological images ordered in increasing magnification, the histological image being a first image in the series of further histological images;

means for identifying a region or regions of said histological image classified as suspect;

a feature extraction module for extracting one or more image features from at least one of said identified region or regions;

a manifold learning module for reducing a dimensionality of said extracted image features; and

means for classifying said identified region or regions as either benign, malignant, or suspect based on at least one of said extracted image features,

wherein if said identified region or regions are classified as malignant, the histological image has a malignant region, otherwise,

i. if the identified region or regions are classified as benign, then the histological image does not have a malignant region;

or

ii. if the identified region or regions are classified as suspect, a next histological image in the series is obtained, the next histological image being processed via the means for identifying, the feature extraction module, the manifold learning module and the means for classifying to classify at least one region of the next histological image as benign, malignant or suspect.