Interactive and automated tissue image analysis with global training database and variable-abstraction processing in cytological specimen classification and laser capture microdissection applications

• US 8,346,483 B2
• Filed: 09/15/2003
• Issued: 01/01/2013
• Est. Priority Date: 09/13/2002
• Status: Active Grant

• Current Assignee: Life Technologies Corporation (Thermo Fisher Scientific Incorporated)
• Original Assignee: Life Technologies Corporation (Thermo Fisher Scientific Incorporated)
• Inventors: Kil, David H.
• Primary Examiner(s): SIMS, JASON M
• Application Number:
  US10/662,765
• Publication Number:
  US 20040093166A1
• Time in Patent Office: 3,396 Days
• Field of Search: None
• US Class Current: 702/19
• CPC Class Codes: G01N 1/06 providing a thin slice, e.g. microtome
• CPC Class Codes: G01N 1/30 Staining; Impregnating ; Fixation; Dehydration; Multistep processes for preparing samples of tissue, cell or nucleic acid material and the like for analysis
• CPC Class Codes: G01N 15/1433 using image recognition
• CPC Class Codes: G01N 2001/284 using local activation of adhesive, i.e. Laser Capture Microdissection
• CPC Class Codes: G02B 21/365 Control or image processing arrangements for digital or video microscopes G02B21/361, G02B21/362 take precedence
• CPC Class Codes: G06F 18/211 Selection of the most significant subset of features
• CPC Class Codes: G06F 18/40 Software arrangements specially adapted for pattern recognition, e.g. user interfaces or toolboxes therefor
• CPC Class Codes: G06T 2207/10056 Microscopic image
• CPC Class Codes: G06T 2207/20132 Image cropping
• CPC Class Codes: G06T 2207/30024 Cell structures in vitro; Tissue sections in vitro
• CPC Class Codes: G06T 7/00 Image analysis
• CPC Class Codes: G06T 7/0012 Biomedical image inspection
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/143 involving probabilistic approaches, e.g. Markov random field [MRF] modelling
• CPC Class Codes: G06V 10/771 Feature selection, e.g. selecting representative features from a multi-dimensional feature space
• CPC Class Codes: G06V 10/945 User interactive design; Environments; Toolboxes
• CPC Class Codes: G06V 20/69 Microscopic objects, e.g. biological cells or cellular parts
• CPC Class Codes: G06V 20/695 Preprocessing, e.g. image segmentation
• CPC Class Codes: G06V 20/698 Matching; Classification

Abstract

A system and method for performing tissue image analysis and region of interest identification for further processing applications such as laser capture microdissection is provided. The invention provides three-stage processing with flexible state transition that allows image recognition to be performed at an appropriate level of abstraction. The three stages include processing at one or more than one of the pixel, subimage and object levels of processing. Also, the invention provides both an interactive mode and a high-throughput batch mode which employs training files generated automatically.

Claims

1. A method for image analysis, the method comprising:

receiving a first image at a processor;

transforming the first image into a feature space;

selecting a region of interest (ROI) at a pixel level of processing from the first image, wherein the ROI is a portion of the first image;

extracting two or more features from the ROI at a pixel level of processing;

selecting a non-ROI at a pixel level of processing from the first image, wherein the non-ROI is a portion of the first image and independent of the selected ROI;

extracting two or more features from the non-ROI at a pixel level of processing;

ranking, in a combinatorial manner, the extracted features from the ROI and the non-ROI based on feature performance for successful detection of a selected ROI at a pixel level of processing;

recording the ranked extracted features;

selecting a classification algorithm;

running the classification algorithm to classify the first image or a second image into one or more ROIs at a pixel level of processing based in part on comparing the selected ROI and non-ROI, wherein the first or second image selected for classification is a classified image;

determining a size of one or more of the ROIs based on pixel level processing; and

outputting analysis results to a computing device.

2. The method of claim 1, wherein selecting at least one ROI comprises selecting one or more pixels from the image;

and wherein the step of selecting at least one non-ROI comprises selecting one or more pixels from the image.

3. The method of claim 1, further comprising transmitting the recorded ROIs at a pixel level of processing for laser capture microdissection.

4. The method of claim 1, further comprising selecting a second level of processing.

5. The method of claim 4, wherein the second level of processing is subimage level processing.

6. The method of claim 5, further comprising the steps of:

selecting at least one polygonal ROI from the classified image at a subimage level of processing;

extracting one or more features from the polygonal ROI at a subimage level of processing;

selecting at least one polygonal non-ROI at a subimage level of processing;

extracting one or more features from the non-ROI at a subimage level of processing;

ranking the extracted features based on feature performance for successful detection of a selected ROI;

recording the ranked features based on subimage processing;

selecting a classification algorithm;

running the classification algorithm to classify the image into ROIs based on subimage level processing; and

recording the ROIs based on subimage level processing.

7. The method of claim 6, further comprising transmitting the recorded regions of interest based on subimage level processing for laser capture microdissection.

8. The method of claim 4, wherein the second level of processing is object processing.

9. The method of claim 8, further comprising:

selecting at least one polygonal ROI from the classified image at an object level of processing.

10. The method of claim 9, further comprising:

recording the at least one polygonal ROI at an object level of processing; and

transmitting the at least one polygonal region of interest based on object level processing for laser capture microdissection.

11. The method of claim 9, further comprising:

extracting one or more features from the ROI at an object level of processing;

selecting at least one polygonal non-ROI at an object level of processing;

extracting one or more features from the non-ROI at an object level of processing;

ranking the extracted features based on feature performance for successful detection of a selected ROI;

recording the ranked features based on object level processing;

selecting a classification algorithm;

running the classification algorithm to classify the image into ROIs based on object level processing; and

recording the ROIs based on object level processing.

12. The method of claim 11, further comprising transmitting the ROIs based on object level processing for laser capture microdissection.

13. The method of claim 4, further comprising selecting a third level of processing.

14. The method of claim 13, wherein the third level of processing is object level processing.

15. The method of claim 14, further comprising:

selecting at least one polygonal ROI from the classified image at an object level of processing.

16. The method of claim 15, further comprising:

recording the at least one polygonal ROI at an object level of processing;

transmitting the at least one polygonal ROIs based on object level processing for laser capture microdissection.

17. The method of claim 15, further comprising:

extracting one or more features from the ROI at an object level of processing;

selecting at least one polygonal non-ROI at an object level of processing;

extracting one or more features from the non-ROI at the object level of processing;

ranking the extracted features based on feature performance for successful detection of a selected ROI;

recording the ranked extracted features based on object level processing;

selecting a classification algorithm;

running the classification algorithm to classify the image into ROIs based on object level processing; and

recording the ROIs based on object level processing.

18. The method of claim 17, further comprising transmitting the regions of interest based on object level processing for laser capture microdissection.

19. The method of claim 6, wherein the further steps are performed prior to outputting the analysis results.

20. The method of claim 6, wherein the further steps are performed after outputting the analysis results, and wherein the method further comprises outputting the analysis results after performing the further steps.

21. The method of claim 11, wherein the further steps are performed prior to outputting the analysis results.

22. The method of claim 11, wherein the further steps are performed after outputting the analysis results, and wherein the method further comprises outputting the analysis results after performing the further steps.

23. The method of claim 17, wherein the further steps are performed prior to outputting the analysis results.

24. The method of claim 17, wherein the further steps are performed after outputting the analysis results, and wherein the method further comprises outputting the analysis results after performing the further steps.

25. The method of claim 1, wherein ranking comprises first iteratively ordering each individual feature for a given ROI or non-ROI according to ability of the single feature to detect an ROI and then iteratively ordering pairs of features for a given ROI or non-ROI according to ability of the pair of features to detect an ROI.