Anomaly Detection for Medical Samples under Multiple Settings

• US 20180253589A1
• Filed: 03/02/2017
• Published: 09/06/2018
• Est. Priority Date: 03/02/2017
• Status: Active Grant

• Current Assignee: Hong Kong Applied Science And Technology Research Institute Co., Ltd.
• Original Assignee: Hong Kong Applied Science And Technology Research Institute Co., Ltd.
• Inventors: WU, Xiaohua, LAW, Yan Nei
• Granted Patent: US 10,108,845 B2
• Time in Patent Office: Days
• CPC Class Codes: G01N 21/64 Fluorescence; Phosphorescence
• CPC Class Codes: G01N 33/483 Physical analysis of biological material
• CPC Class Codes: G06F 18/2113 by ranking or filtering the set of features, e.g. using a measure of variance or of feature cross-correlation
• CPC Class Codes: G06F 18/2411 based on the proximity to a decision surface, e.g. support vector machines
• CPC Class Codes: G06F 18/24143 Distances to neighbourhood prototypes, e.g. restricted Coulomb energy networks [RCEN]
• CPC Class Codes: G06F 18/2415 based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate
• CPC Class Codes: G06F 18/24323 Tree-organised classifiers
• CPC Class Codes: G06F 18/254 of classification results, e.g. of results related to same input data
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/10056 Microscopic image
• CPC Class Codes: G06T 2207/10064 Fluorescence image
• CPC Class Codes: G06T 2207/20084 Artificial neural networks [ANN]
• CPC Class Codes: G06T 2207/30024 Cell structures in vitro; Tissue sections in vitro
• CPC Class Codes: G06T 7/0012 Biomedical image inspection
• CPC Class Codes: G06T 7/0014 using an image reference approach
• CPC Class Codes: G06V 10/507 Summing image-intensity values; Histogram projection analysis
• CPC Class Codes: G06V 10/764 using classification, e.g. of video objects
• CPC Class Codes: G06V 10/809 of classification results, e.g. where the classifiers operate on the same input data
• CPC Class Codes: G06V 10/82 using neural networks
• CPC Class Codes: G06V 20/69 Microscopic objects, e.g. biological cells or cellular parts
• CPC Class Codes: G06V 20/693 : Acquisition
• CPC Class Codes: G06V 20/695 : Preprocessing, e.g. image segmentation
• CPC Class Codes: G06V 20/698 : Matching; Classification
• CPC Class Codes: G06V 2201/03 : Recognition of patterns in medical or anatomical images

Abstract

Using multiple imaging modes in whole slide image screening is potentially useful to reduce false positives. To use multiple imaging modes, a method for locating anomalies on a medical sample from an image thereof uses an anomaly-detection process that comprises using plural base classifiers individually to classify an object-of-interest suspected to be an anomaly. Each base classifier respectively extracts features of the object-of-interest and generates, according to the extracted features, a score indicating a likelihood of the object-of-interest being anomalous. The anomaly-detection process further comprises using an aggregate classifier to combine the scores generated by the base classifiers to determine whether the object-of-interest is the anomaly. The aggregate classifier determines a dependability measure for each base classifier according to setting-based variables of a setting under which the sample and the image are obtained, and then selectively combines the scores of the base classifiers according to the dependability measures.

Claims

1. A computer-implemented method for locating one or more anomalies on a medical sample from an image thereof, the medical sample and the image being prepared under a setting comprising one or more setting-based variables, the method comprising:

locating, from the image, any object-of-interest suspected to be anomalous on the medical sample; and

when one or more objects-of-interest are located, determining whether an individual object-of-interest is an individual anomaly by an anomaly-detection process and repeating the anomaly-detection process for each of the one or more objects-of-interest;

wherein the anomaly-detection process comprises;

using plural base classifiers individually to classify the individual object-of-interest, wherein each base classifier respectively extracts features of the individual object-of-interest and generates, according to the extracted features, a score indicating a likelihood of the individual object-of-interest being anomalous; and

using an aggregate classifier to combine the scores generated by the base classifiers to determine whether the individual object-of-interest is the individual anomaly, wherein the aggregate classifier determines a dependability measure of an individual score for each base classifier according to the one or more setting-based variables, and selectively combines the scores of all the base classifiers according to the dependability measures of the scores.

2. The method of claim 1, wherein the image is obtained by imaging an entire slide on which the medical sample resides.

3. The method of claim 1, wherein one of the base classifiers is arranged to extract morphological features of the individual object-of-interest, and to use one or more decision trees to classify the morphological features for generating the score.

4. The method of claim 1, wherein one of the base classifiers is arranged to extract local histogram features of the individual object-of-interest, and to use a support vector machine (SVM) to classify the local histogram features for generating the score.

5. The method of claim 4, wherein when the image is a color image, one or more color histograms are used for extracting the local histogram features.

6. The method of claim 4, wherein when the image is a grayscale image, one or more intensity histograms are used for extracting the local histogram features.

7. The method of claim 1, wherein one of the base classifiers is a convolutional neural network (CNN).

8. The method of claim 1, wherein:

the base classifiers include a first base classifier and a second base classifier;

the first classifier is arranged to extract morphological features of the individual object-of-interest, and to use one or more decision trees to classify the morphological features for generating the score; and

the second classifier is arranged to extract local histogram features of the individual object-of-interest, and to use a support vector machine (SVM) to classify the local histogram features for generating the score.

9. The method of claim 1, wherein:

the base classifiers include a first base classifier and a second base classifier;

the first classifier is arranged to extract morphological features of the individual object-of-interest, and to use one or more decision trees to classify the morphological features for generating the score; and

the second classifier is a convolutional neural network (CNN).

10. The method of claim 1, wherein the image is a brightfield image.

11. The method of claim 1, wherein the image is a fluorescent image.

12. The method of claim 1, wherein the one or more setting-based variables include a type of the image, the type of the image being selected front a brightfield image and a fluorescent image.

13. The method of claim 12, wherein the one or more setting-based variables further include art exposure time in imaging the medical sample.

14. The method of claim 1, wherein the one or more setting-based variables include a staining agent applied to the medical sample for staining the one or more anomalies before the medical sample is imaged.

15. The method of claim 1, further comprising:

before the anomaly-detection process is performed, training the base classifiers and the aggregate classifier with images of plural training medical samples each having pre--identified one or more anomalies, wherein an individual training medical sample and the image thereof are obtained under a pre-selected setting.

16. The method of claim 1, wherein the method is adapted for the medical sample suspected to contain one or more types of acid-fast bacilli (AFB).

17. A method for detecting one or more targeted anomalies having a pre-selected characteristic on a medical sample, comprising:

applying a first staining agent to the medical sample for staining anomalies including the one or more targeted anomalies;

after the anomalies are stained, imaging the medical sample to give a first image;

locating, by one or more computers, the anomalies from the first image by a locating process according to one or more first setting-based variables of a first setting under which the medical sample and the first image are obtained, wherein the locating process is arranged according to the method of claim 1;

applying a second staining agent to the medical sample for re-staining the one or more targeted anomalies, wherein the second staining agent is formulated to preferentially stain a subset of the anomalies where the anomalies in the subset have the pre-selected characteristic;

after the one or more targeted anomalies are re-stained, imaging the medical sample to give a second image; and

locating, by the one or more computers, the one or more targeted anomalies from the second image by the locating process according to one or more second setting-based variables of a second setting under which the medical sample and the second image are obtained, whereby the one or more targeted anomalies are detected.

18. The method of claim 17, wherein after the one or more targeted anomalies are re-stained, the medical sample is imaged only on areas surrounding the anomalies that are located from the first image.

19. A method for detecting one or more targeted anomalies having a pre-selected characteristic on a medical sample, comprising:

applying a first staining agent to the medical sample for staining anomalies including the one or more targeted anomalies;

after the anomalies are stained, imaging the medical sample to give a first image;

locating, by one or more computers, the anomalies from the first image by a locating process according to one or more first setting-based variables of a first setting under which the medical sample and the first image are obtained, wherein the locating process is arranged according to the method of claim 8;

applying a second staining agent to the medical sample for re-staining the one or more targeted anomalies, wherein the second staining agent is formulated to preferentially stain a subset of the anomalies where the anomalies in the subset have the pre-selected characteristic;

after the one or more targeted anomalies are re-stained, imaging the medical sample to give a second image; and

locating, by the one or more computers, the one or more targeted anomalies from the second image by the locating process according to one or more second setting-based variables of a second setting under which the medical sample and the second image are obtained, whereby the one or more targeted anomalies are detected.

20. A method for detecting one or more targeted anomalies having a pre-selected characteristic on a medical sample, comprising:

applying a first staining agent to the medical sample for staining anomalies including the one or more targeted anomalies;

after the anomalies are stained, imaging the medical sample to give a first image;

locating, by one or more computers, the anomalies from the first image by a locating process according to one or more first setting-based variables of a first setting under which the medical sample and the first image are obtained, wherein the locating process is arranged according to the method of claim 9;

applying a second staining agent to the medical sample for re-staining the one or more targeted anomalies, wherein the second staining agent is formulated to preferentially stain a subset of the anomalies where the anomalies in the subset have the pre-selected characteristic;

after the one or more targeted anomalies are re-stained, imaging the medical sample to give a second image; and

locating, by the one or more computers, the one or more targeted anomalies from the second image by the locating process according to one or more second setting-based variables of a second setting under which the medical sample and the second image are obtained, whereby the one or more targeted anomalies are detected.