OBJECT AND SPATIAL LEVEL QUANTITATIVE IMAGE ANALYSIS

• US 20100111396A1
• Filed: 11/06/2008
• Published: 05/06/2010
• Est. Priority Date: 11/06/2008
• Status: Active Grant

• Current Assignee: TRIAD National Security, LLC
• Original Assignee: Los Alamos National Security LLC (Government of the United States of America)
• Inventors: Boucheron, Laura E.
• Granted Patent: US 8,488,863 B2
• Time in Patent Office: Days
• US Class Current: 382/133
• CPC Class Codes: G06F 18/2115 by evaluating different subsets according to an optimisation criterion, e.g. class separability, forward selection or backward elimination
• CPC Class Codes: G06F 18/29 Graphical models, e.g. Bayesian networks
• CPC Class Codes: G06T 2207/10056 Microscopic image
• CPC Class Codes: G06T 2207/30024 Cell structures in vitro; Tissue sections in vitro
• CPC Class Codes: G06T 2207/30068 Mammography; Breast
• CPC Class Codes: G06T 7/0012 Biomedical image inspection
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/187 involving region growing; involving region merging; involving connected component labelling
• CPC Class Codes: G06V 10/426 Graphical representations
• CPC Class Codes: G06V 10/771 Feature selection, e.g. selecting representative features from a multi-dimensional feature space
• CPC Class Codes: G06V 10/84 using probabilistic graphical models from image or video features, e.g. Markov models or Bayesian networks
• CPC Class Codes: G06V 20/695 Preprocessing, e.g. image segmentation

Abstract

Quantitative object and spatial arrangement-level analysis of tissue are detailed using expert (pathologist) input to guide the classification process. A two-step method is disclosed for imaging tissue, by classifying one or more biological materials, e.g. nuclei, cytoplasm, and stroma, in the tissue into one or more identified classes on a pixel-by-pixel basis, and segmenting the identified classes to agglomerate one or more sets of identified pixels into segmented regions. Typically, the one or more biological materials comprises nuclear material, cytoplasm material, and stromal material. The method further allows a user to markup the image subsequent to the classification to re-classify said materials. The markup is performed via a graphic user interface to edit designated regions in the image.

Claims

1. A method for analyzing an image of tissue, comprising:

classifying one or more biological materials in said tissue into one or more identified classes;

wherein materials are classified on a pixel-by-pixel basis; and

segmenting the identified classes to agglomerate one or more sets of identified pixels into segmented regions.

2. A method as recited in claim 1, wherein one or more biological materials comprises nuclear material, cytoplasm material, and stromal material.

3. A method as recited in claim 1, further comprising:

allowing a user to markup said image subsequent to said classification to re-classify said materials.

4. A method as recited in claim 3, wherein said markup is performed via a graphic user interface to edit designated regions in the image.

5. A method as recited in claim 3, wherein said markup is used to develop one or more classifiers for specific classes of materials;

wherein said one or more classifiers are used to classify materials in subsequent imaging.

6. A method as recited in claim 1, further comprising:

displaying said segmented regions; and

allowing the user to markup said image to re-segment the identified classes.

7. A method as recited in claim 6, wherein:

one or more algorithms are used to classify and segment said biological materials; and

wherein the image markup is used as input for said algorithms.

8. A method as recited in claim 1, further comprising;

performing object-level classification of said segmented regions.

9. A method as recited in claim 8, wherein said object-level classification distinguishes segmented nuclei as cancerous or non-cancerous based on one or more predetermined object-level classifiers.

10. A method as recited in claim 8, further comprising:

using said object-level classification as input for cellular-level segmentation.

11. A method as recited in claim 10, wherein said cellular-level segmentation comprises grouping nuclear and cytoplasmic material into identified cells.

12. A method as recited in claim 1, wherein segmenting the identified classes comprises applying a plurality of segmentation algorithms, each segmentation algorithm providing a segmentation output, the method further comprising:

applying a segmentation metric to quantitatively compare the outputs of the individual segmentation algorithms; and

selecting an output based on said segmentation metric comparison.

13. A method as recited in claim 12, wherein the segmentation metric is a function of:

number of segmented regions;

size and shape of the region of pixels missed;

size and shape of excess pixels;

fraction of nuclei detected; and

number of extra segmented regions.

14. A method as recited in claim 13, wherein the segmentation is a function of the equation:

15. A method as recited in claim 1, wherein classifying one or more biological materials comprises classifying cytoplasm materials and stroma materials by subtracting out pixels related to background and nuclei.

16. A machine readable medium on which are provided program instructions for image analysis, the program instructions comprising:

program code for receiving a cell image showing the biological components of one or more cells;

program code for classifying on a pixel-by-pixel basis one or more biological components in said tissue into one or more identified classes;

program code for segmenting the identified classes to agglomerate one or more sets of identified pixels into segmented regions of the cell image to obtain a segmented digital image.

17. A machine readable medium as recited in claim 16, further comprising:

program code for extracting one or more parameters from the segmented digital image to characterize the nucleus, cytoplasm or stroma of at least one of the cells of the cell image.

18. A machine readable medium as recited in claim 16, further comprising:

a graphical user interface configured to allow user markup of said image subsequent to said classification to re-classify said components.

19. A machine readable medium as recited in claim 18, further comprising:

program code for inputting said user markup as training data; and

program code incorporating machine learning using said training data to develop one or more classifiers for specific classes of components;

wherein said one or more classifiers are used to classify components in subsequent imaging.

20. A machine readable medium as recited in claim 16, further comprising:

program code for user markup of said image to re-segment the identified classes.

21. A machine readable medium as recited in claim 16, further comprising;

program code for allowing additional higher-level segmentation and classification iterations;

22. A machine readable medium as recited in claim 21, wherein higher-level classification comprises object-level classification of said segmented regions.

23. A machine readable medium as recited in claim 22, further comprising:

program code using said object-level classification as input for cellular-level segmentation.

24. A machine readable medium as recited in claim 23, wherein said cellular-level segmentation comprises grouping nuclear and cytoplasmic material into identified cells.

25. A machine readable medium as recited in claim 16, wherein program code for segmenting is configured to apply a plurality of segmentation algorithms, each segmentation algorithm providing a segmentation output, further comprising:

program code having a segmentation metric to quantitatively compare the outputs of the individual segmentation algorithms, and for selecting an output based on said segmentation metric comparison.

26. A machine readable medium as recited in claim 25, wherein the segmentation metric is a function of:

number of segmented regions;

size and shape of the region of pixels missed;

size and shape of excess pixels;

fraction of nuclei detected; and

number of extra segmented regions.

27. A method for analyzing an image of tissue, comprising:

providing a digital cell image showing the biological components of one or more cells;

providing a graphical user interface allowing a user to markup the image for classification and segmentation;

inputting, as training data, the image markup to build one or more classifiers;

based on said one or more classifiers, classifying one or more biological materials in said tissue into one or more identified classes on a pixel-by-pixel basis; and

segmenting the identified classes to agglomerate one or more sets of identified pixels into segmented regions.

28. A method as recited in claim 27, wherein classifying one or more biological components comprises classifying cytoplasm materials and stroma materials by subtracting out pixels related to background and nuclei.

29. A method as recited in claim 27, further comprising;

performing object-level classification of said segmented regions.

30. A method as recited in claim 29, further comprising:

using said object-level classification as input for cellular-level segmentation.

31. A method as recited in claim 27, wherein segmenting the identified classes comprises applying a plurality of segmentation algorithms, each segmentation algorithm providing a segmentation output, the method further comprising:

applying a segmentation metric to quantitatively compare the outputs of the individual segmentation algorithms; and

selecting an output based on said segmentation metric comparison.

32. A method as recited in claim 31, wherein the segmentation metric is a function of:

number of segmented regions;

size and shape of the region of pixels missed;

size and shape of excess pixels;

fraction of nuclei detected; and

number of extra segmented regions.