MALIGNANCY DIAGNOSIS USING CONTENT - BASED IMAGE RETREIVAL OF TISSUE HISTOPATHOLOGY

• US 20100098306A1
• Filed: 08/01/2007
• Published: 04/22/2010
• Est. Priority Date: 08/01/2006
• Status: Active Grant

• Current Assignee: Rutgers University, Trustees Of The University Of Pennsylvania (University of Pennsylvania)
• Original Assignee: Trustees Of The University Of Pennsylvania (University of Pennsylvania)
• Inventors: Tomaszewski, John E., Doyle, Scott, Feldman, Michael D., Madabhushi, Anant
• Granted Patent: US 8,280,132 B2
• Time in Patent Office: Days
• US Class Current: 382/128
• CPC Class Codes: G06T 2207/30096 Tumor; Lesion
• CPC Class Codes: G06T 7/0012 Biomedical image inspection
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06V 20/695 Preprocessing, e.g. image segmentation

Abstract

This invention relates to computer-aided diagnostics using content-based retrieval of histopathological image features. Specifically, the invention relates to the extraction of image features from a histopathological image based on predetermined criteria and their analysis for malignancy determination.

Claims

1. A computer-aided diagnostic method to predict the probability that a histological image contains a malignant region comprising:

a. obtaining a histological image, wherein the histological image is a first of a series of images ordered in increasing magnification;

b. identifying a region or regions of said histological image classified as suspect;

c. extracting one or more image features from at least one of said identified regions;

d. reducing a dimensionality of said extracted feature data; and

e. classifying said extracted region or regions as either benign, malignant, or suspect based on at least one said extracted image feature, whereby if said extracted region or regions are classified as malignant, the histological image has a malignant region, otherwise;

i. if the extracted region or regions are classified as benign, then the histological image does not have a malignant region;

or

ii. if the extracted region or regions are classified as suspect, the next histological image in the series is obtained and the steps of identifying, extracting, reducing and classifying are repeated.

2. The method of claim 1, whereby said ordered series of images are generated by a pyramidal decomposition.

3. The method of claim 1, whereby said image feature is a statistical feature, a Haralick co-occurrence feature, a Haar wavelet feature, or a Gabor feature, whereby said image feature is calculated independently from each channel of the image using a sliding window of 3×

3, 5×

5, or 7×

7 pixels.

4. The method of claim 3, whereby said statistical feature is an average, a median, a standard deviation, a difference, a Sobel filter, a Kirsch filter, a horizontal derivative, a vertical derivative, or a diagonal derivative of a pre-selected determinant.

5. The method of claim 3, whereby said Haralick co-occurrence feature is an angular second moment, a contrast, a correlation, a variance, an entropy, an inverse difference moment, a sum average, a sum variance, a sum entropy, a difference variance, or a difference entropy.

6. The method of claim 3, whereby said Haar wavelet feature is the result of a convolution of said image with a Haar wavelet kernel.

7. The method of claim 3, whereby said Gabor feature is the result of convolving said image with a bank of Gabor filters, generated by convolving a Gaussian function with a sinusoid at a range of scale and orientation parameter values.

8. The method of claim 1, whereby the step of reducing the dimensionality is done by using a principal component analysis, a linear dimensional analysis, a multidimensional scaling, a graph embedding, an ISOMAP, a local linear embedding, a kernel-based principal component analysis, a semidefinite embedding, Laplacian eigenmaps, or a combination thereof.

9. The method of claim 1, whereby said step of classifying further comprises the steps of:

a. creating a likelihood scene for said selected image;

b. creating a mask identifying regions of said selected image as benign or suspect by thresholding said likelihood scene at a value determined through system training; and

c. classifying said extracted region or regions as either benign, malignant, or suspect based on said mask, whereby if the region or regions are classified as suspect, said mask is resized by interpolation to the size of the subsequent image in the ordered series to identify suspect regions on said subsequent image

10. The method of claim 9 whereby said likelihood scene is determined by:

a Decision Tree algorithm, a Support Vector Machine algorithm, an Adaboost algorithm using Bayes Decision Theorem, or at least one image feature extracted from one or more training images.

14. A method for histopathological analysis of a gland tissue sample, comprising:

a. selecting an image of the tissue sample, wherein said image comprises regions classified by the methods of claim 9 as malignant, benign or suspect;

b. identifying regions of said selected image classified as suspect;

c. extracting at least one image feature from at least one of said identified regions;

d. reducing the dimensionality of said extracted feature data; and

e. classifying said selected image region as one of the group consisting of a cancerous tissue of a specific clinical stage of malignancy, a benign epithelium, and a benign stroma based on said at least one extracted image feature.

15. The method of claim 14, wherein said at least one image feature is at least one of the group consisting of an architectural feature, a nuclear density, a gland morphology, and a global texture feature.

16. The method of claim 15, wherein said architectural feature is selected from the group of features calculated from a Voronoi diagram, a Delaunay graph, and a minimum spanning tree constructed from the nuclei centers in the image, and a co-adjacency matrix constructed from the gland centers in the image.

17. The method of claim 16, wherein said architectural feature computed from a Voronoi diagram is the standard deviation, average, ratio of minimum to maximum, or disorder of the areas, perimeters, or chords of the polygons in the Voronoi diagram of said image.

18. The method of claim 16, wherein said architectural feature computed from a Delaunay graph is the standard deviation, average, ratio of minimum to maximum, or disorder of the side lengths or areas of the triangles in the Delaunay graph of said image.

19. The method of claim 16, wherein said architectural feature computed from a minimum spanning tree is the standard deviation, average, ratio of minimum to maximum, or disorder of the edge lengths in the minimum spanning tree of said image.

20. The method of claim 16, wherein said co-adjacency matrix constructed from the gland centers in said image is used to calculate an angular second moment, a contrast, a correlation, a variance, an entropy, an inverse difference moment, a sum average, a sum variance, a sum entropy, a difference variance, or a difference entropy.

21. The method of claim 15, wherein said nuclear density feature is the number or density of nuclei in said image;

the average, standard deviation, or disorder of the distances from each of the nuclei in said image to the 3, 5, or 7 nearest neighboring nuclei;

or the average, standard deviation, or disorder of the number of neighboring nuclei within a circle with a radius of 10, 20, 30, 40, or 50 pixels centered on each nucleus within said image.

22. The method of claim 15, wherein said gland morphology feature is calculated from the boundary of either the gland lumen or interior nuclei as the ratio of the gland area to the area of the smallest circle that circumscribes the gland;

the standard deviation, variance, or ratio of the maximum to the average distance between the gland center and the boundary;

the ratio of the estimated boundary length (the boundary calculated using a fraction of the boundary pixels) to the actual boundary length;

ratio of the boundary length to the area enclosed by the boundary; and

the sum of the difference between the distance from the center of the gland to the boundary and the average of the distances from the center to the two adjacent points.

23. The method of claim 15, wherein said global texture feature is an average, a median, a standard deviation, a range, an angular second moment, a contrast, a correlation, a variance, an entropy, a sum average, a sum variance, a sum entropy, a difference variance, a difference entropy, a difference moment, or a Gabor filter.

24. The method of claim 14, whereby the dimensionality of the extracted features is reduced through the use of principal component analysis, linear dimensional analysis, multidimensional scaling, graph embedding, ISOMAP, local linear embedding, kernel-based principal component analysis, semidefinite embedding, or Laplacian eigenmaps.

25. The method of claim 14, whereby said step of classification is performed by:

a Support Vector Machine algorithm, a Decision Tree algorithm, an Adaboost algorithm using Bayes Decision Theorem or their combination.

98. A system for performing the method of claim 1, the system comprising:

a database of histopathological images, an image query module, a feature extraction module, a manifold learning module and an integration module.

99. The method of claim 1, whereby the histological image is of a prostate.

100. The method of claim 15, whereby the gland tissue sample is of a prostate.

11-13. -13. (canceled)

26-97. -97. (canceled)