Systems and methods for segmentation and processing of tissue images and feature extraction from same for treating, diagnosing, or predicting medical conditions

• US 9,754,152 B2
• Filed: 12/28/2015
• Issued: 09/05/2017
• Est. Priority Date: 07/30/2010
• Status: Active Grant

• Current Assignee: Fundação D. Anna De Sommer Champalimaud E Dr. Carlos Montez Champalimaud
• Original Assignee: Fundação D. Anna De Sommer Champalimaud E Dr. Carlos Montez Champalimaud
• Inventors: Ajemba, Peter, Scott, Richard, Ramachandran, Janakiramanan, Zeineh, Jack, Donovan, Michael, Fernandez, Gerardo, Liu, Qiuhua, Khan, Faisal
• Primary Examiner(s): Rahmjoo, Manuchehr
• Application Number:
  US14/981,352
• Publication Number:
  US 20160188954A1
• Time in Patent Office: 617 Days
• Field of Search: None
• CPC Class Codes: G06T 2207/10056 Microscopic image
• CPC Class Codes: G06T 2207/20041 Distance transform
• CPC Class Codes: G06T 2207/20212 Image combination
• CPC Class Codes: G06T 2207/30024 Cell structures in vitro; Tissue sections in vitro
• CPC Class Codes: G06T 5/94 based on local image properties, e.g. for local contrast enhancement
• CPC Class Codes: G06T 7/0012 Biomedical image inspection
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/155 involving morphological operators
• CPC Class Codes: G06T 7/45 using co-occurrence matrix computation
• CPC Class Codes: G06V 20/698 Matching; Classification

Abstract

Apparatus, methods, and computer-readable media are provided for segmentation, processing (e.g., preprocessing and/or postprocessing), and/or feature extraction from tissue images such as, for example, images of nuclei and/or cytoplasm. Tissue images processed by various embodiments described herein may be generated by Hematoxylin and Eosin (H&E) staining, immunofluorescence (IF) detection, immunohistochemistry (IHC), similar and/or related staining processes, and/or other processes. Predictive features described herein may be provided for use in, for example, one or more predictive models for treating, diagnosing, and/or predicting the occurrence (e.g., recurrence) of one or more medical conditions such as, for example, cancer or other types of disease.

Claims

1. A computer implemented method comprising:

receiving an image of at least one tissue within the memory of a computer;

performing, using code executing in at least one or more processors of the computer, triangulation on the tissue image with epithelial nuclei centers as vertices, thereby identifying one or more triangles having one or more regions;

identifying one or more polygonal areas within the tissue by merging one or more regions of the one or more triangles;

classifying, with the at least one or more processors, at least one of the one or more polygonal areas as at least one of;

(a) gland rings,

(b) glandular non-rings,

(c) stroma regions,

(d) under-segmented regions, and

(e) incomplete regions;

storing the classified image to at least one data storage device as a patient dataset;

assigning, with at least one or more processors, a respective depth to each of the one or more triangles; and

sorting the one or more triangles based on the respective depths;

wherein merging one or more regions of the one or more triangles comprises merging one or more regions of the one or more triangles starting with the triangle having the greatest respective depth.

2. The method of claim 1, wherein the step of performing triangulation on the image comprises:

segmenting nuclei of a cellular region within the image into individually contiguous regions;

extracting the one or more epithelial nuclei centers; and

partitioning the image into polygonal regions with the one or more epithelial nuclei centers at the vertices.

3. The method of claim 1, wherein the step of classifying at least one of the one or more polygonal areas comprises classifying at least one of the one or more polygonal areas based on one or more of a size, stromal area, lumen area, ring density, and cytoplasm connectivity around a ring.

4. The method of claim 1, wherein assigning a respective depth to each of the one or more triangles comprises assigning a respective depth to each of the one or more triangles that is equal or substantially equal to a length of a longest side of the triangle.

5. The method of claim 1, wherein merging one or more regions of the one or more triangles comprises merging two of the one or more regions of the one or more triangles based on (a) a determination that a length of a common side between the two triangles is at least 90% of a depth of a neighbor, and (b) based on a determination that both of the two regions touch the same epithelial units.

6. The method of claim 1, further comprising:

applying, using code executing in at least one or more processors of the computer a perturbation to the image to generate one or more variant images;

performing segmentation on the tissue image and the one or more variant images to produce segmented versions of the tissue image and the one or more variant images; and

computing one or more metrics of similarity for the segmented versions of the tissue image and the one or more variant images in order to assess the stability of the segmentation.

7. A system comprising:

one or more processors configured to interact with a computer-readable medium in order to perform operations comprising;

receiving within a memory of the processor, an image of at least one tissue;

segmenting nuclei of a cellular region within the image into individually contiguous regions;

extracting one or more epithelial nuclei centers;

partitioning the image into polygonal regions with the one or more epithelial nuclei centers at the vertices, thereby identifying one or more triangles;

merging one or more regions of the one or more triangles, thereby identifying one or more polygonal areas within the tissue;

classifying at least one of the one or more polygonal areas as at least one of;

(a) gland rings,

(b) glandular non-rings,

(c) stroma regions

(d) under-segmented regions, and

(e) incomplete regions;

evaluating a dataset with a predictive model stored within a memory of the processor, the model being based on one or more ring features selected from the group of ring features consisting of;

one or more ring metrics, feature(s) derived from one or more ring metrics, and feature(s) representing an adjacency relationship between rings; and

generating an evaluation of a medical condition by outputting a value indicative of the medical condition in the patient.

8. A computer implemented method comprising:

receiving an image of at least one tissue within the memory of a computer;

performing, using code executing in at least one or more processors of the computer, triangulation on the tissue image with epithelial nuclei centers as vertices, thereby identifying one or more triangles having one or more regions;

identifying one or more polygonal areas within the tissue by merging one or more regions of the one or more triangles;

classifying, with the at least one or more processors, at least one of the one or more polygonal areas as at least one of;

(a) gland rings,

(b) glandular non-rings,

(c) stroma regions,

(d) under-segmented regions, and

(e) incomplete regions;

storing the classified image to at least one data storage device as a patient dataset;

evaluating the patient dataset with a predictive model using code executing in at least one or more processors of the computer, the model being based on one or more ring features selected from the group of ring features consisting of;

one or more ring metrics,

feature(s) derived from one or more ring metrics, and

feature(s) representing an adjacency relationship between rings; and

providing, based on the evaluating, output data indicative of the medical condition of the patient.

9. The method of claim 8, wherein the predictive model is further based on at least one additional feature selected from the group of features consisting of:

one or more clinical features,

one or more molecular features, and

one or more computer generated morphometric feature(s).

10. The method of claim 8, wherein the one or more ring features comprises one or more ring features selected from the group of ring features consisting of:

outer diameter of ring, inner diameter of ring, border gap, lumen or clearing diameter, border density, lumen ratio, proportion of border touching inner clearing, proportion of border touching stroma, ratio of border less than a predefined number of pixels from stroma, mean distance of border pixels from stroma, width of epithelial padding between ring and stroma, feature(s) derived from any of the foregoing features, and feature(s) representing an adjacency relationship between rings.

11. The method of claim 8, wherein providing output data indicative of a medical condition comprises generating a value indicative of the medical condition in the patient.