Method and apparatus for automated image analysis of biological specimens

• US 6,215,892 B1
• Filed: 11/27/1996
• Issued: 04/10/2001
• Est. Priority Date: 11/30/1995
• Status: Expired due to Term

• Current Assignee: Carl Zeiss Microscopy GmbH (Carl-Zeiss-Stiftung)
• Original Assignee: Chromavision Medical Systems Inc. (NeoGenomics, Inc.)
• Inventors: Ring, James E., Riding, Thomas J., Douglass, James W.
• Primary Examiner(s): Bella, Matthew
• Assistant Examiner(s): PATEL, KANJIBHAI B
• Application Number:
  US08/758,436
• Time in Patent Office: 1,595 Days
• Field of Search: 382/308, 382/128, 382/129, 382/130, 382/133, 382/134, 382/199, 382/203, 382/274, 382/144, 382/291, 345/150, 348/645, 348/650, 348/577, 348/150, 364/550
• US Class Current: 382/128
• CPC Class Codes: G01N 1/312 for samples mounted on planar substrates
• CPC Class Codes: G01N 15/1433 using image recognition
• CPC Class Codes: G01N 15/1468 with spatial resolution of the texture or inner structure of the particle
• CPC Class Codes: G01N 2015/1472 with colour
• CPC Class Codes: G01N 2015/1488 Methods for deciding
• CPC Class Codes: G01N 2015/1497 Particle shape
• CPC Class Codes: G02B 21/244 using image analysis techniques
• CPC Class Codes: G02B 21/367 providing an output produced by processing a plurality of individual source images, e.g. image tiling, montage, composite images, depth sectioning, image comparison
• CPC Class Codes: G06T 7/60 Analysis of geometric attributes
• CPC Class Codes: G06V 20/69 Microscopic objects, e.g. biological cells or cellular parts
• CPC Class Codes: G06V 20/693 Acquisition

Abstract

A method and apparatus for automated cell analysis of biological specimens automatically scans at a low magnification to acquire images which are analyzed to determine candidate cell objects of interest. The low magnification images are converted from a first color space to a second color space. The color space converted image is then low pass filtered and compared to a threshold to remove artifacts and background objects from the candidate object of interest pixels of the color converted image. The candidate object of interest pixels are morphologically processed to group candidate object of interest pixels together into groups which are compared to blob parameters to identify candidate objects of interest which correspond to cells or other structures relevant to medical diagnosis of the biological specimen. The location coordinates of the objects of interest are stored and additional images of the candidate cell objects are acquired at high magnification. The high magnification images are analyzed in the same manner as the low magnification images to confirm the candidate objects of interest which are objects of interest. A high magnification image of each confirmed object of interest is stored for later review and evaluation by a pathologist.

Claims

1. A method of automatic analysis of an image of a magnified area of a slide having a biological specimen prepared with a reagent comprising the steps of:

(a) transforming substantially all pixels of an image of a biological specimen on a slide, said pixels being transformed from a first color space to a second color space; and

(b) identifying rare event candidate objects of interest from said transformed pixels in said second color space with reference to a predetermined threshold,

wherein the first color space includes red, green, and blue components for each pixel and the transforming step includes forming a ratio between two components of the red, blue and green components for each pixel in the first color space to transform the pixels to the second color space.

2. The method of claim 1 further comprising the step of:

selecting a grayscale value for each pixel from a look-up table which corresponds to the ratio of two components in the first color space.

3. A method of automatic analysis of an image of a magnified area of a slide having a biological specimen prepared with a reagent comprising the steps of:

(a) transforming substantially all pixels of an image of a biological specimen on a slide, said pixels being transformed from a first color space to a second color space;

(b) identifying rare event candidate objects of interest from said transformed pixels in said second color space with reference to a predetermined threshold;

(c) positioning the optical system at an initial Z stage position;

(d) acquiring at low magnification an image of a slide having a stained biological specimen thereon and calculating a pixel variance about a pixel mean for the acquired image;

(e) incrementing the position of the Z stage;

(f) repeating steps (d) and (e) for a fixed number of coarse iterations to form a first set of variance data;

(g) performing a least squares fit of the first set of variance data to a first function;

(h) positioning the Z stage at a position near the peak of the first function;

(i) repeating steps (d) and (e) for a fixed number of fine iterations to form a second set of variance data;

(j) performing a least squares fit of the second set of variance data to a second function;

(k) selecting the peak value of the least squares fit curve as an estimate of the best focal position; and

(l) performing the above steps for an array of X-Y stage positions to form an array of focal positions and performing a least squares fit of the array of focal positions to yield a least squares fit focal plane prior to said transforming step.

4. A method of automatic analysis of an image of a magnified area of a slide having a biological specimen prepared with a reagent comprising the steps of:

(a) transforming substantially all pixels of an image of a biological specimen on a slide, said pixels being transformed from a first color space to a second color space;

(b) identifying rare event candidate objects of interest from said transformed pixels in said second color space with reference to a predetermined threshold;

(c) positioning the optical system at an initial Z stage position;

(d) acquiring an image and calculating a pixel variance about a pixel mean for the acquired image;

(e) incrementing the position of Z stage;

(f) repeating steps (d) and (e) for a fixed number of iterations;

(g) performing a least squares fit of the variance data to a known function; and

(h) selecting the peak value of the least squares fit curve as an estimate of the best focal position prior to said transforming step.

5. A method of automatic analysis of an image of a magnified area of a slide having a biological specimen prepared with a reagent comprising the steps of:

(a) transforming substantially all pixels of an image of a biological specimen on a slide, said pixels being transformed from a first color space to a second color space;

(b) identifying rare event candidate objects of interest from said transformed pixels in said second color space with reference to a predetermined threshold;

(c) morphologically processing the rare event candidate object of interest pixels to identify artifact pixels;

(d) identifying the rare event candidate objects of interest from the remaining candidate object of interest pixels not identified as artifact pixels, comprising the steps of;

(d1) grouping said morphologically processed rare event candidate object of interest pixels into regions of connected rare event candidate object of interest pixels to identify rare event objects of interest;

(d2) comparing said rare event objects of interest to blob analysis parameters; and

(d3) storing location coordinates of the rare event candidate objects of interest having an area corresponding to the blob analysis parameters;

(e) filtering said rare event candidate object of interest pixels with a low pass filter prior to morphologically processing the low pass filtered rare event candidate object of interest pixels,

wherein said previously performed method steps (d1), (d2), and (d3) are performed on images acquired at low magnification and the method further comprises the steps of;

(e1) adjusting an optical system viewing the slide from which the rare event objects of interest were identified to high magnification, comprising the steps of;

(aa) positioning the optical system at an initial Z stage position;

(bb) acquiring an image and selecting a center pixel of a candidate object of interest;

(cc) defining a region of interest centered about the selected center pixel;

(dd) performing a fast fourier transform of said region of interest to identify frequency components for the region of interest and complex magnitudes for the frequency components;

(ee) computing a power value by summing the square of the complex magnitudes for the frequency components that are within the range of frequencies of 25% to 75% of a maximum frequency component for the fast fourier transform of the region of interest;

(ff) incrementing the position of the Z stage;

(gg) repeating steps (bb)-(ee) for a fixed number of iterations; and

(hh) selecting the Z stage position corresponding to the largest power value as the best focal position;

(e2) acquiring a high magnification image of the slide at the corresponding location coordinates for each rare event candidate object of interest;

(e3) transforming pixels of the high magnification image in the first color space to a second color space; and

(e4) identifying high magnification rare event objects of interest from the rare event candidate object of interest pixels in the second color space.

6. A method of automatic analysis of an image of a magnified area of a slide having a biological specimen prepared with a reagent comprising the steps of:

(a) transforming substantially all pixels of an image of a biological specimen on a slide, said pixels being transformed from a first color space to a second color space;

(b) identifying rare event candidate objects of interest from said transformed pixels in said second color space with reference to a predetermined threshold;

(c) morphologically processing the rare event candidate object of interest pixels to identify artifact pixels;

(d) identifying the rare event candidate objects of interest from the remaining candidate object of interest pixels not identified as artifact pixels, comprising the steps of;

(d1) grouping said morphologically processed rare event candidate object of interest pixels into regions of connected rare event candidate object of interest pixels to identify rare event objects of interest;

(d2) comparing said rare event objects of interest to blob analysis parameters; and

(d3) storing location coordinates of the rare event candidate objects of interest having an area corresponding to the blob analysis parameters;

(e) filtering said rare event candidate object of interest pixels with a low pass filter prior to morphologically processing the low pass filtered rare event candidate object of interest pixels,

wherein said previously performed method steps (d1), (d2), and (d3) are performed on images acquired at low magnification and the method further comprises the steps of;

(e1) adjusting an optical system viewing the slide from which the rare event objects of interest were identified to high magnification, comprising the steps of;

(aa) positioning the optical system at an initial Z stage position;

(bb) acquiring an image and selecting a center pixel of a candidate object of interest;

(cc) defining a region of interest centered about the selected center pixel;

(dd) applying a Hanning window function to the region of interest;

(ee) performing a fast fourier transform of said region of interest following the application of the Hanning window function to identify frequency components for the region of interest and complex magnitudes for the frequency components;

(ff) computing a power value by summing the square of the complex magnitudes for the frequency components for the fast fourier transform of the region of interest;

(gg) incrementing the position of the Z stage;

(hh) repeating steps (bb)-(ff) for a fixed number of iterations; and

(ii) selecting the Z stage position corresponding to the largest power value as the best focal position;

(e2) acquiring a high magnification image of the slide at the corresponding location coordinates for each rare event candidate object of interest;

(e3) transforming pixels of the high magnification image in the first color space to a second color space; and

(e4) identifying high magnification rare event objects of interest from the rare event candidate object of interest pixels in the second color space.

7. A method for initially focusing an optical system of an automated microscopy system comprising the steps of:

(a) positioning an optical system at an initial Z stage position;

(b) acquiring an image and calculating a pixel variance about a pixel mean for the acquired image;

(c) incrementing the position of the Z stage;

(d) repeating steps (b) and (c) for a fixed number of iterations;

(e) performing a least squares fit of the variance data to a known function; and

(f) selecting the peak value of the least squares fit curve as an estimate of the best focal position.

8. A method for initially focusing an optical system of an automated microscopy system comprising the steps of:

(a) positioning an optical system at an initial Z stage position;

(b) acquiring at low magnification an image of a slide having a stained biological specimen thereon and calculating a pixel variance about a pixel mean for the acquired image;

(c) incrementing the position of the Z stage;

(d) repeating steps (b) and (c) for a fixed number of coarse iterations to form a first set of variance data;

(e) performing a least squares fit of the first set of variance data to a first function;

(f) positioning the Z stage at a position near the peak of the first function;

(g) repeating steps (b) and (c) for a fixed number of fine iterations to form a second set of variance data;

(h) performing a least squares fit of the second set of variance data to a second function;

(i) selecting the peak value of the least squares fit curve as an estimate of the best focal position; and

(j) performing the above steps for an array of X-Y stage positions to form an array of focal positions and performing a least squares fit of the array of focal positions to yield a least squares fit focal plane.

9. A method for focusing an optical system of an automated microscopy system at high magnification comprising the steps of:

(a) positioning an optical system at an initial Z stage position;

(b) acquiring an image and selecting a center pixel of a candidate object of interest;

(c) defining a region of interest centered about the selected center pixel;

(d) performing a fast fourier transform of said region of interest to identify frequency components for the region of interest and complex magnitudes for the frequency components;

(e) computing a power value by summing the square of the complex magnitudes for the frequency components that are within the range of frequencies of 25% to 75% of a maximum frequency component for the fast fourier transform of the region of interest;

(f) incrementing the position of the Z stage;

(g) repeating steps (b)-(e) for a fixed number of iterations; and

(h) selecting the Z stage position corresponding to the largest power value as the best focal position.

10. A method for focusing an optical system of an automated microscopy system at high magnification comprising the steps of:

(a) positioning an optical system at an initial Z stage position;

(b) acquiring an image and selecting a center pixel of a candidate object of interest;

(c) defining a region of interest centered about the selected center pixel;

(d) applying a Hanning window function to the region of interest;

(e) performing a fast fourier transform of said region of interest following the application of the Hanning window function to identify frequency components for the region of interest and complex magnitudes for the frequency components;

(f) computing a power value by summing the square of the complex magnitudes for the frequency components for the fast fourier transform of the region of interest;

(g) incrementing the position of the Z stage;

(h) repeating steps (b)-(f) for a fixed number of iterations; and

(i) selecting the Z stage position corresponding to the largest power value as the best focal position.

11. Apparatus for automatic image analysis of a slide having a biological specimen, comprising:

an optical system having an X-Y stage;

means for scanning over a scan area of the slide at a plurality of locations at low magnification of the optical system;

means for acquiring a low magnification image at each location in the scan area;

a processor for processing each low magnification image to detect candidate objects of interest;

means for storing X-Y coordinates of each location for each candidate object of interest;

means for adjusting the optical system to a high magnification;

means for repositioning the X-Y stage to the location for each candidate object of interest;

means for acquiring a high magnification image of each candidate object of interest; and

a storage device for storing each high magnification image.