SYSTEMS AND METHODS FOR AUTOMATED SCREENING AND PROGNOSIS OF CANCER FROM WHOLE-SLIDE BIOPSY IMAGES

• US 20140233826A1
• Filed: 09/26/2012
• Published: 08/21/2014
• Est. Priority Date: 09/27/2011
• Status: Abandoned Application

• Current Assignee: Board of Regents of the University of Texas System (University of Texas System)
• Original Assignee: Board of Regents of the University of Texas System (University of Texas System)
• Inventors: Agaian, Sos S., Lopez, Clara M. Mosquera, Almuntashri, Ali, Metzler, Richard
• Application Number:
  US14/347,912
• Publication Number:
  US 20140233826A1
• Time in Patent Office: Days
• US Class Current: 382/133
• CPC Class Codes: A61B 10/0241 for prostate
• CPC Class Codes: A61B 5/4381 Prostate evaluation or disorder diagnosis
• CPC Class Codes: A61B 5/7203 for noise prevention, reduction or removal
• CPC Class Codes: A61B 5/725 using specific filters therefor, e.g. Kalman or adaptive filters A61B5/316 take precedence
• CPC Class Codes: A61B 5/7257 using Fourier transforms
• CPC Class Codes: A61B 5/726 using Wavelet transforms
• CPC Class Codes: A61B 5/7267 involving training the classification device
• CPC Class Codes: G01N 15/1433 using image recognition
• CPC Class Codes: G06F 18/211 Selection of the most significant subset of features
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/10056 Microscopic image
• CPC Class Codes: G06T 2207/20036 Morphological image processing
• CPC Class Codes: G06T 2207/20081 Training; Learning
• CPC Class Codes: G06T 2207/30024 Cell structures in vitro; Tissue sections in vitro
• CPC Class Codes: G06T 2207/30081 Prostate
• CPC Class Codes: G06T 2207/30096 Tumor; Lesion
• CPC Class Codes: G06T 5/20 using local operators
• CPC Class Codes: G06T 5/70 Denoising; Smoothing
• CPC Class Codes: G06T 7/0012 Biomedical image inspection
• CPC Class Codes: G06T 7/13 Edge detection
• CPC Class Codes: G06T 7/181 : involving edge growing; involving edge linking
• CPC Class Codes: G06T 7/90 : Determination of colour characteristics
• CPC Class Codes: G06V 20/69 : Microscopic objects, e.g. biological cells or cellular parts
• CPC Class Codes: G06V 20/695 : Preprocessing, e.g. image segmentation
• CPC Class Codes: G06V 20/698 : Matching; Classification
• CPC Class Codes: G16H 30/20 : for handling medical images, e.g. DICOM, HL7 or PACS
• CPC Class Codes: G16H 50/20 : for computer-aided diagnosis, e.g. based on medical expert systems
• CPC Class Codes: G16H 50/30 : for calculating health indices; for individual health risk assessment
• CPC Class Codes: G16H 50/70 : for mining of medical data, e.g. analysing previous cases of other patients

Abstract

The invention provides systems and methods for detection, grading, scoring and tele-screening of cancerous lesions. A complete scheme for automated quantitative analysis and assessment of human and animal tissue images of several types of cancers is presented. Various aspects of the invention are directed to the detection, grading, prediction and staging of prostate cancer on serial sections/slides of prostate core images, or biopsy images. Accordingly, the invention includes a variety of sub-systems, which could be used separately or in conjunction to automatically grade cancerous regions. Each system utilizes a different approach with a different feature set. For instance, in the quantitative analysis, textural-based and morphology-based features may be extracted at image- and (or) object-levels from regions of interest. Additionally, the invention provides sub-systems and methods for accurate detection and mapping of disease in whole slide digitized images by extracting new features through integration of one or more of the above-mentioned classification systems. The invention also addresses the modeling, qualitative analysis and assessment of 3-D histopathology images which assist pathologists in visualization, evaluation and diagnosis of diseased tissue. Moreover, the invention includes systems and methods for the development of a tele-screening system in which the proposed computer-aided diagnosis (CAD) systems. In some embodiments, novel methods for image analysis (including edge detection, color mapping characterization and others) are provided for use prior to feature extraction in the proposed CAD systems.

Claims

1. A method, performed by a processing unit, for computing pixels along object edges and producing a deinterlaced image from an interlaced source, the method comprising:

performing image filtering on a collected image depending on the nature of noise in the collected image;

smoothing the filtered image using shape-dependent filters;

calculating gradient vectors in the image using different kernels;

selecting an edge angle;

determine threshold values within a local dynamic range,

generating several edge maps, and

fusing the generated edge maps together.

2. The method of claim 1, further comprising applying an algorithm for image edge-preserving contrast enhancement which is based on HVS, Parameterized Logarithm Image Processing operations, and is integrated with morphological log-ratio approach in order to come up with an effective edge detection operator that is sensitive to edges of dark areas in the image.

3. Methods and systems for computation of fractal-like dimension for modifying input data/image into data describing the image, the method comprising:

performing image filtering depending on the nature of noise in the image;

binarizing the image by using the method described in claim 1 or 2, or by using an image bit-plane decomposition method;

calculating the fractal dimension of binary images by using different grid sizes based on a Differential Box Counting (DBC) algorithm; and

fusing resulting fractal dimensions.

8. The method of claim 7, wherein forming texture-based feature vectors comprises extraction of a set of features from Fourier and wavelet transforms such us wavelet energy and entropy, phase information of coefficients, and spatial domain algorithms such as statistical spatial filtering, wavelet-based fractal dimension according to the method of claim 3 or others, and gray level histogram.

4. A method for computing the fractal dimension of color images comprising:

performing a color model transformation by using arbitrary functions operating on the original RGB components of the image;

splitting the image in smaller windows and computing for each block the probability of having m points/pixels in a hypercube of the size of the window;

estimating the color fractal dimension by using a weighting function and fitting the curve of logarithm of the window sizes against the logarithm of the total number of boxes of each window size needed to cover the image.

6. A method for image color quantization comprising:

performing an image color standardization procedure;

calculating the distance between each pixel and the colors considered in the color map of claim 4 by using a distance metric;

minimizing the calculated distance; and

assigning the nearest reference color or class to the each pixel.

9. A textural-based system and methods for automatically detect, classify, and grade cancerous regions of a histology image (particularly a prostate biopsy images) comprising:

performing an image color standardization procedure;

describing the image data and forming texture-based feature vectors by using spatial and transforms domain image information along with color fractal analysis as claimed in claim 4;

selecting the group of features that best describe the images;

training a classifier by using the generated feature vectors;

classifying histology images according to the Gleason grading system;

using the result of classification to determine the Gleason score of the image;

assessing the accuracy of the Gleason grading/scoring system by using cross-validation methods.

10. The description step of the system of claim 9, further comprising the extraction of a set of features from real, logarithmic, or complex wavelets and multi-wavelets and a new color fractal dimension algorithm developed in this invention. Particularly, transform features include joint probability of detail coefficients, phase information of coefficients, energy distribution of detail coefficients, and energy of color channels of the biopsy image. A new color fractal dimension is also extracted to complete the feature space.

5. A method for carcinomas color region mapping comprising:

constructing 2-D projections of the RGB color model of a large variety of biopsy images including normal and cancerous tissue;

computing 2-D histograms of biopsy images;

selecting the most prominent colors by using an algorithm for local maxima location in 3-D surfaces;

constructing a color region mapping, in which each color corresponds to one class or tissue structure.

11. A morphology- and architectural-based system and methods for automatically detect, classify, and grade cancerous regions of a histology image (particularly a prostate biopsy images) comprising:

performing an image color standardization procedure;

segmenting the image by applying any proper algorithm including the method proposed in claims 5 and 6;

describing the image data and forming morphology and architectural-based feature vectors by extracting the shape, size and arrangement of tissue structures;

selecting the group of features that best describe the images;

training a classifier by using the generated feature vectors;

classifying cancerous images according to grading systems, for example Gleason grading system for prostate carcinomas;

using the result of classification to determine the Gleason score of the image;

assessing the accuracy of the Gleason grading/scoring system by using cross-validation methods.

12. The description step of the system of claim 11, further comprising the extraction of features:

color information such as color distribution entropy and 2-D color histograms along with tissue morphology and architectural features from segmented tissue structures.

13. Systems and methods for automatically detecting, classifying and grading cancerous regions of 3-D histological images (particularly a prostate image) comprising:

performing an image color standardization procedure,

segmenting image by applying any proper algorithm including the method proposed in claims 5 and 6;

mapping of 2-D images or an image into 3-D space by using pre-developed mapping algorithms,

describing the image data and forming color-, texture-, morphology-, and architectural-based feature vectors;

training a classifier within an framework by using generated feature vectors;

accessing the cancer grade or scale, for example, according to the Gleason grading scale;

accessing the most prominent, the second most prominent pattern and the third prominent pattern in said image data and computing a Gleason 3-D score as a sum of Gleason 3-D grades of said patterns;

accessing the accuracy of Gleason 3-D grading/scoring system;

estimating volume of a tumor region.

15. The systems of claims 7, 9, 11 and 13 wherein automatically classifying the biopsy images is based on machine learning algorithms:

linear discriminants, Gaussian models, Multivariate Adaptive Regression Splines (MARS), Classification an Regression trees (CART), decision trees, k-nearest neighbor, Bayesian, neural networks, and/or SVM. Boosting algorithms such as AdaBoost, SVM Boost, etc. can be used to improve the classification performance.

17. A system and method for automatically detecting, classifying and grading cancerous regions from digitized Whole-Slide (particularly a prostate image) comprising:

performing an image color standardization procedure;

splitting image into smaller regions;

determining and grading a location of a tumor in the prostate using one or more of the systems of claim 7 or 9 or 11 or 13;

accessing a cancer stage;

accessing the most prominent, the second most prominent pattern and third prominent pattern in said image data and computing a Gleason score as a sum of Gleason grades of said patterns;

generating a cancer map from whole digitized slides according to Gleason grading/scoring levels associated with slides processing block-elements;

accessing the accuracy of Gleason grading/scoring system;

accessing the estimating size of a tumor region.

18. The system according to claim 17, wherein said grade estimating module is configured for estimating a size of a tumor in said at least one tissue region or whole digitized slides.

19. The system of claim 17, wherein a complete diagnosis is performed for providing useful information for pathologists about the disease severity and the localization, size and volume of cancerous tissue.

21. The system of claim 17 further comprising methods for surgery quality control, which allow a surgeon to be immediately aware of whether the surrounding tissue of a specimen is or not positive for cancer, prior the patient is closed up.

7. A textural-based system and methods for automatically detecting, classifying, and grading cancerous regions of a histology image comprising:

performing an image color standardization procedure;

forming texture-based feature vectors by using spatial and transforms domain image information along with fractal analysis;

selecting the group of features that best describe the images;

training a classifier by using the generated feature vectors;

classifying histology images according to the Gleason grading system;

using the result of classification to determine the Gleason score of the image;

assessing the accuracy of the Gleason grading/scoring system by using cross-validation methods.

14. The systems as claimed in 13, wherein the mapping from 2-D image to 3-D images includes algorithms for 3-D reconstruction from a single 2-D image or from several 2-D slides. The processed images nay be gray scaled or colored images.

16. A method to obtain a multidimensional Gleason grading wherein a revised Gleason value is generated as follows:

Revised Gleason Value=2D Gleason Grade+3rd Dimensional Core Grade

20. The system of claim 17 wherein complete diagnosing is performed, the outputs of the systems include:

tumor extent in the needle biopsy, number and identification of positive needle cores, percentage of positive cores for cancer, length of the tumor in positive cores and length of the core, percentage of needle core tissue affected by carcinoma and area of tissue positive for cancer, Gleason grade of each selected cancerous region of interest, percentage of each Gleason pattern in biopsy specimens, Gleason score of each positive core, including tertiary pattern information, the most frequent Gleason score in slides, numbers of identical Gleason scores in the positive cores, revised Gleason value (claim 16), and localized cancer map including the grade of each cancerous patch in all needle biopsy slides.

40. The system of claim 37 wherein complete diagnosing is performed, the outputs of the systems include:

tumor extent in the needle biopsy, number and identification of positive needle cores, percentage of positive cores for cancer, length of the tumor in positive cores and length of the core, percentage of needle core tissue affected by carcinoma and area of tissue positive for cancer, Gleason grade of each selected cancerous region of interest, percentage of each Gleason pattern in biopsy specimens, Gleason score of each positive core, including tertiary pattern information, the most frequent Gleason score in slides, numbers of identical Gleason scores in the positive cores, revised Gleason value (claim 16), and localized cancer map including the grade of each cancerous patch in all needle biopsy slides.

22. A system for cancer prognosis, wherein the pathology report including all the outputs of systems claimed in 7, 9, 11 and 13 is integrated with patient information and PSA analysis results to construct feature vectors. The resulting feature vectors are used to aid pathologists in cancer prognosis, risk assessment and treatment planning.

23. The system of claim 22, further comprising the use of feature vectors for prostate cancer risk assessment by implementing risk calculators according to the patient'"'"'s available information.

24. The system of claim 22 further comprising a method for cancer prediction based on 2-D and 3-D histopathology images using cancer map or Gleason graded block and pixel features to predict the state of the tissue in missing blocks within the scope of the image or to produce extrapolated values to know the cancer grade in the neighborhoods outside the core extracted from the prostate gland during a needle biopsy procedure.

25. The method of claim 24 wherein Kalman filtering or similar techniques are used for estimation and prediction of cancer grade (Gleason grade) outside of the scope of observed histopathology biopsy images.

26. A system and methods for cancer tele-screening and diagnosis based on biopsy image and an adjudication scheme, wherein expert pathologists and CAD systems are integrated.

27. The adjudication methods for prostate cancer diagnoses used in the system of claim 26 consist of replacing one or more pathologists by CAD systems and evaluate the agreement among them to produce a faster and less expensive prostate cancer diagnosis.

28. The systems of claim 26 wherein said adjudication methods is configure for using morphology- or textural-based 2-D classification systems for cancer detection and grading, such that the CADs integrated into the systems perform independent tissue assessment.

29. The tele-screening system of claim 26 uses communication networks such as internet for image and patient data acquisition, pathologists'"'"' report acquisition. The final assessment results are stored in a centralized database available for pathologist consultation.

30. A method and system constructing an optimal signal detector using a learning algorithm in tandem with a prediction algorithm (which may be one in the same) and data compression algorithm.

31. A method and system for detecting carcinomas in a sample image based on compression rates obtained using a detector trained on previously classified carcinoma images as specified in claim 30.

32. A method and system for using detection results to compute a hard, integer Gleason score of a query image through selection of a best compressing pattern compressors (PCs) using one of the systems of claims 30 and 31.

33. A method and system for using detection results to compute a hard, integer Gleason scores of query image pixels and regions through selection of a best compressing pattern compressors (PCs) using one of the systems of claims 30 and 31.

34. A method and system for using detection results to compute a soft, continuous-scale Gleason score of an image (query), comprising:

compressing query image data using specific pattern PCs and optionally a self PC according to systems in claims 30 and 31;

computing pattern weights determined by;

PC compression rates,

PC compression rate statistics,

and/or functions of each pattern PC compression rates or statistics and

query image self PC compression rates or statistics;

combining pattern values and weights into a single Gleason score.

35. A method and system for using detection results to compute a soft, continuous-scale Gleason scores of query image pixels and regions, comprising:

compressing query image pixels using specific pattern PCs and optionally a self PC according to systems in claims 30 and 31;

computing pattern weights determined by either by pattern PC compression rate or by comparison of each pattern PC prediction to a query image'"'"'s self PC prediction on a pixel by pixel basis;

combining pattern values and weights into a single, soft Gleason score on a pixel by pixel basis.

37. A system and method for automatically detecting, classifying and grading cancerous regions from digitized, Whole-Slides (particularly a prostate image) comprising:

segmenting the image into prominent features;

optionally downsizing the image;

converting the image into a 1 dimensional signal;

compressing the signal using the systems of claims 30 and 31;

assigning Gleason scores to each pixel using the system of claims 33 or 35, and 36;

determining, grading, and scoring a location of a tumor in the prostate using one or more of the systems of claim 33, 35 or 36;

assessing the and grading a location of a tumor in the prostate using one or more of the systems of claim 33, 35 or 36;

estimating the current carcinoma pattern distribution of a selected region using the systems of claim 33, 35 or 36;

estimating the future carcinoma pattern distribution of a selected region using the systems of claim 33, 35 or 36;

generating a cancer map from whole, digitized slides according to the systems of claim 33, 35 or 36;

assessing the accuracy of Gleason grading/scoring system;

38. The system according to claim 37, wherein said grade estimating module is configured for estimating a size of a tumor in said at least one tissue region or whole digitized slides.

39. The system of claim 37, wherein a complete diagnosis is performed for providing useful information for pathologists about the disease severity and the localization, size and volume of cancerous tissue.

41. The system of claim 37 further comprising methods for surgery quality control, which allow a surgeon to be immediately aware of whether the surrounding tissue of a specimen is or not positive for cancer, prior the patient is closed up.

36. A system and method for determining the most likely distribution of Gleason patterns within a query image, comprising:

the systems of claim in 30, 31 and 33 or 35;

determining pairwise uncertainty statistics between pixel scores or grades;

forming the uncertainty statistics into a matrix;

computing the eigenvalues of said matrix;

computing the eigenvectors of said matrix;

using all or some of the above features to estimate the most likely current distribution of Gleason patterns within the sample;

using all or some of the above features to estimate the most likely future distribution of Gleason patterns within the sample;

42. A system for cancer prognosis, wherein the pathology report including all the outputs of systems claimed in 30 through 41 is integrated with patient information and PSA analysis results to construct feature vectors. The resulting feature vectors are used to aid pathologists in cancer prognosis, risk assessment and treatment planning.