Object classification in digital images

• US 10,614,571 B2
• Filed: 12/09/2016
• Issued: 04/07/2020
• Est. Priority Date: 12/10/2015
• Status: Active Grant

• Current Assignee: Qiagen GmbH (Qiagen NV)
• Original Assignee: Qiagen GmbH (Qiagen NV)
• Inventors: Unterer, Thomas, Schuster, Guido
• Primary Examiner(s): Akinyemi, Ajibola A
• Application Number:
  US16/060,879
• Publication Number:
  US 20180374212A1
• Time in Patent Office: 1,215 Days
• Field of Search: 382128
• CPC Class Codes: G06T 2207/10064 Fluorescence image
• CPC Class Codes: G06T 2207/30072 Microarray; Biochip, DNA array; Well plate
• CPC Class Codes: G06T 7/0012 Biomedical image inspection
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/136 involving thresholding
• CPC Class Codes: G16Z 99/00 Subject matter not provided for in other main groups of this subclass

Abstract

System and method for distinguishing at least one first object from at least one second object in a plurality of digital images is provided. The at least one first object having received at least one molecule comprising genetic information, the at least one second object not having received a molecule comprising genetic information. The at least one molecule is configured to receive one of a plurality of fluorescent compounds in each of a plurality of cycles. The digital images being determined by an optical imaging system during emission of electromagnetic radiation by the fluorescent compounds, wherein the plurality of digital images comprises a plurality of series of images, each image of a series referring to the emission spectrum of a respective fluorescent compound and wherein the series of images is repeatedly taken for each of the plurality of cycles.

Claims

1. A computer implemented method for distinguishing at least one first object from at least one second object in a plurality of digital images wherein the method comprises:

determining intensity values from the plurality of digital images for each object, wherein the at least one first object having received at least one molecule comprising genetic information, the at least one second object having not received a molecule comprising genetic information, wherein the at least one molecule is configured to receive one of a plurality of fluorescent compounds in each of a plurality of cycles, the plurality of digital images being determined by an optical imaging system during emission of electromagnetic radiation by the fluorescent compounds, wherein the plurality of digital images comprise a plurality of series of images, each image of a series of images referring to an emission spectrum of a respective fluorescent compound, and wherein the series of images is repeatedly taken for each of the plurality of cycles;

correcting autofluorescence effects of the determined intensity values by subtracting a predetermined intensity value from the determined intensity values;

correcting crosstalk effects of the determined intensity values, caused by overlapping fluorescent spectra of the fluorescent compounds, by applying a predetermined crosstalk matrix to the determined intensity values;

computing standard deviation values of the corrected intensity values for each object and each of the plurality of cycles;

computing a mean value of the standard deviation values for each object;

normalizing the mean values for each object using a linear transformation; and

determining classification thresholds from the normalized mean values using a threshold selection method to obtain a binary digital image, wherein the at least one first object is represented by a first intensity value above the corresponding classification threshold and the at least one second object is represented by a second intensity value below the corresponding classification threshold.

2. The method according to claim 1, further comprising:

determining minimum intensity values within a plurality of predetermined areas of each of the digital images; and

correcting illumination caused by the optical imaging system for each object in each of the digital images by dividing the determined intensity values by the corresponding minimum intensity values.

3. The method according to claim 2, further comprising:

applying a median filter to the determined minimum intensity values of the predetermined areas; and

applying a Gaussian filter to the determined minimum intensity values of the predetermined areas after applying the median filter.

4. The method according to claim 2, further comprising:

interpolating the determined minimum intensity values of the predetermined areas to an original digital image size using a bicubic interpolation.

5. The method according to claim 1, wherein the predetermined intensity value for correcting the autofluorescence effects is determined by taking a predetermined smallest intensity value from the determined intensity values, by taking the 10% smallest intensity value from the determined intensity values.

6. The method according to claim 1, wherein the crosstalk matrix accounts for an overlap of a green fluorescent spectrum with a yellow fluorescent spectrum of the fluorescent compounds by applying a factor of 0.4.

7. The method according to claim 1, wherein normalizing the mean values comprises:

subdividing the mean values into a plurality of groups; and

determining minimum mean values and maximum mean values for each group.

8. The method according to claim 7, further comprising:

applying a region growing approach to the minimum mean values and the maximum mean values,

or a median filter to the minimum mean values and the maximum mean values,

or a Gaussian filter to the minimum mean values and the maximum mean values.

9. The method according to claim 7, further comprising:

interpolating the minimum mean values and the maximum mean values to an original digital image size using a bicubic interpolation.

10. The method according to claim 7, wherein the linear transformation is performed for each object by dividing differences between the mean values and the corresponding minimum mean values by the corresponding differences between the maximum mean values and the corresponding mean minimum values.

11. The method according to claim 1, further comprising:

subdividing the normalized mean values in a predetermined number of subgroups;

determining corresponding mean values for each subgroup before determining the classification thresholds; and

applying a region growing approach, a median filter, or a Gaussian filter to the determined mean values of the subgroups.

12. The method according to claim 11, further comprising:

interpolating the thresholds determined by the threshold selection method to an original digital image size using a bicubic interpolation.

13. The method according to claim 1, wherein the threshold selection method comprises an Otsu segmentation method applied to the determined mean values of the subgroups.

14. A system for distinguishing at least one first object from at least one second object in a plurality of digital images, wherein the system comprises:

an intensity determining unit configured to determine intensity values from the plurality of digital images for each object, wherein the at least one first object having received at least one molecule comprising genetic information, the at least one second object having not received a molecule comprising genetic information, wherein the at least one molecule is configured to receive one of a plurality of fluorescent compounds in each of a plurality of cycles, the plurality of digital images being determined by an optical imaging system during emission of electromagnetic radiation by the fluorescent compounds, wherein the plurality of digital images comprises a plurality of series of images, each image of a series of images referring to an emission spectrum of a respective fluorescent compound, and wherein the series of images is repeatedly taken for each of the plurality of cycles;

a correction unit configured to;

correct autofluorescence effects of the determined intensity values by subtracting a predetermined intensity value from the determined intensity values; and

correct crosstalk effects of the determined intensity values, caused by overlapping fluorescent spectra, by applying a predetermined crosstalk matrix to the determined intensity values; and

a computation unit configured to;

compute standard deviation values of the corrected intensity values for each object and each of the plurality of cycles,

compute mean values of the standard deviations for each object;

compute normalized mean values for each object using a linear transformation; and

compute classification thresholds using a threshold selection method to obtain a binary digital image, wherein the at least one first object is represented by a first intensity value above the corresponding classification threshold and the at least one second object is represented by a second intensity value below the corresponding classification threshold.

15. The system according to claim 14, wherein the computation unit is further configured to:

determine minimum intensity values within a plurality of predetermined areas of each of the digital images; and

correct illumination caused by the optical imaging system for each object in each of the digital images by dividing the determined intensity values by the corresponding minimum intensity values.

16. The system according to claim 15, wherein the computation unit is further configured to:

apply a median filter to the determined minimum intensity values of the predetermined areas; and

applying a Gaussian filter to the determined minimum intensity values of the predetermined areas after applying the median filter.

17. The system according to claim 15, wherein the computation unit is further configured to:

interpolate the determined minimum intensity values of the predetermined areas to an original digital image size using a bicubic interpolation.

18. A non-transitory computer readable media having instructions stored thereon that, when executed by at least one computing device, cause the at least one computing device to perform operations comprising:

determining intensity values from a plurality of digital images for each object, wherein at least one first object having received at least one molecule comprising genetic information, at least one second object having not received a molecule comprising genetic information, wherein the at least one molecule is configured to receive one of a plurality of fluorescent compounds in each of a plurality of cycles, the plurality of digital images being determined by an optical imaging system during emission of electromagnetic radiation by the fluorescent compounds, wherein the plurality of digital images comprise a plurality of series of images, each image of a series of images referring to an emission spectrum of a respective fluorescent compound, and wherein the series of images is repeatedly taken for each of the plurality of cycles;

correcting autofluorescence effects of the determined intensity values by subtracting a predetermined intensity value from the determined intensity values;

correcting crosstalk effects of the determined intensity values, caused by overlapping fluorescent spectra of the fluorescent compounds, by applying a predetermined crosstalk matrix to the determined intensity values;

computing standard deviation values of the corrected intensity values for each object and each of the plurality of cycles;

computing a mean value of the standard deviations for each object;

normalizing the mean values for each object using a linear transformation; and

determining classification thresholds from the normalized mean values using a threshold selection method to obtain a binary digital image, wherein the at least one first object is represented by a first intensity value above the corresponding classification threshold and the at least one second object is represented by a second intensity value below the corresponding classification threshold.

19. The non-transitory computer readable media according to claim 18, wherein the operations further comprise:

determining minimum intensity values within a plurality of predetermined areas of each of the digital images; and

correcting illumination caused by the optical imaging system for each object in each of the digital images by dividing the determined intensity values by the corresponding minimum intensity values.

20. The non-transitory computer readable media according to claim 19, wherein the operations further comprise:

applying a median filter to the determined minimum intensity values of the predetermined areas; and

applying a Gaussian filter to the determined minimum intensity values of the predetermined areas-after applying the median filter.