Method and apparatus for automated image analysis of biological specimens

• US 6,718,053 B1
• Filed: 02/01/2000
• Issued: 04/06/2004
• Est. Priority Date: 11/27/1996
• Status: Expired due to Term

• Current Assignee: Carl Zeiss Microscopy GmbH (Carl-Zeiss-Stiftung)
• Original Assignee: Chromavision Medical Systems Inc. (NeoGenomics, Inc.)
• Inventors: Decker, William J., Ellis, Bob, McLaren, Gina
• Primary Examiner(s): Mehta, Bhavesh M.
• Assistant Examiner(s): PATEL, KANJIBHAI B
• Application Number:
  US09/495,461
• Time in Patent Office: 1,526 Days
• Field of Search: 382/128, 382/129, 382/130, 382/131, 382/132, 382/133, 382/134, 382/255, 382/264, 382/275, 382/280, 382/308, 382/254, 382/162, 382/299, 422/63, 435/41, 435/7.2, 345/604, 345/611
• US Class Current: 382/128
• CPC Class Codes: G01N 15/1433 using image recognition
• CPC Class Codes: G02B 21/367 providing an output produced by processing a plurality of individual source images, e.g. image tiling, montage, composite images, depth sectioning, image comparison
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/10056 Microscopic image
• CPC Class Codes: G06T 2207/20012 Locally adaptive
• CPC Class Codes: G06T 2207/20016 Hierarchical, coarse-to-fine, multiscale or multiresolution image processing; Pyramid transform
• CPC Class Codes: G06T 2207/30024 Cell structures in vitro; Tissue sections in vitro
• CPC Class Codes: G06T 5/30 Erosion or dilatation, e.g. thinning
• CPC Class Codes: G06T 5/70 Denoising; Smoothing
• CPC Class Codes: G06T 5/77 Retouching; Inpainting; Scratch removal
• CPC Class Codes: G06T 7/136 involving thresholding
• CPC Class Codes: G06V 10/10 Image acquisition document image scanning and transmission H04N1/00; control of digital cameras H04N23/60
• CPC Class Codes: G06V 10/25 Determination of region of interest [ROI] or a volume of interest [VOI]
• CPC Class Codes: G06V 10/267 by performing operations on regions, e.g. growing, shrinking or watersheds
• CPC Class Codes: G06V 10/443 by matching or filtering
• CPC Class Codes: G06V 10/56 relating to colour
• CPC Class Codes: G06V 20/69 Microscopic objects, e.g. biological cells or cellular parts
• CPC Class Codes: G06V 20/693 Acquisition

Abstract

A method and apparatus for automated cell analysis of biological specimens automatically scans at a low magnification to acquire images (288) which are analyzed to determine candidate cell objects of interest. The low magnification images are converted from a first color space to a second color space (290). The color space converted image is then low pass filtered (292) and compared to a threshold (294) to remove artifacts and background objects from the candidate object of interest pixels of the color converted image. The candidate object of interest pixels are morphologically processed (296) to group candidate object of interest pixels together into groups which are compared to blob parameters (298) to identify candidate objects of interest which correspond to cells or other structures relevant to medical diagnosis of the biological specimen. The location coordinates of the objects of interest are stored and additional images of the candidate cell objects are acquired at high magnification. The high magnification images are analyzed in the same manner as the low magnification images to confirm the candidate objects of interest which are objects of interest. A high magnification image of each confirmed object of interest is stored for later review and evaluation by a pathologist.

Claims

1. An apparatus for automatic image analysis of a slide having a biological specimen, comprising:

a computer comprising;

at least one system processor;

a monitor in operable communication with the computer; and

an input device in communication with the computer;

an optical system in operable communication with the computer, comprising;

a movable stage;

an automated loading and unloading member for loading and unloading of a slide;

an identification member;

an optical sensing array in optical communication with the stage configured to acquire an image at each location in a scan area;

an image processor in electrical communication with the sensing array and operable to process each image to detect candidate objects of interest in the image through an automated process;

a storage member for storing the location of a candidate object of interest; and

a storage device for storing each image.

2. The apparatus of claim 1, wherein the system processor comprises a high performance processor of at least 90 MHz.

3. The apparatus of claim 1, wherein the input device is selected from keyboard, trackball, mouse, or touch screen monitor.

4. The apparatus of claim 1, wherein the identification member detects a bar code.

5. The apparatus of claim 1, wherein the optical sensing array is a CCD array.

6. The apparatus of claim 1, wherein the image is a digital image.

7. The apparatus of claim 1, wherein the storage device is a magnetic disk.

8. The apparatus of claim 1, wherein the storage device is an optical disk.

9. A method for automatic image analysis of a slide having a biological specimen, comprising:

positioning the slide having a biological specimen on a stage which is optically coupled to an optical sensing array;

acquiring an image of the biological specimen;

processing the image;

identifying a candidate object of interest by means of a computer processor;

storing the coordinates of the candidate object of interest;

acquiring a higher magnification image at the coordinates of the object of interest;

processing the higher magnification image; and

storing the higher magnification image, wherein the step of identifying the candidate object of interest is by an automated process.

10. The method of claim 9, wherein pixels of the image are transformed from a first color space to a second color space to differentiate candidate object of interest pixels from background pixels.

11. The method of claim 10, wherein the first color space includes red, green, and blue components for each pixel and the transforming step includes forming a ratio between two components of the red, blue and green components for each pixel in the first color space to transform the pixels to the second color space.

12. The method of claim 11, further comprising selecting a gray scale value for each pixel in the second color space which corresponds to the ratio of components in the first color space.

13. The method of claim 10, wherein the first color space includes red, green, and blue components for each pixel and the transforming step includes converting components of the red, blue and green components for each pixel in the first color space to pixel values in a hue, saturation, and intensity space.

14. The method of claim 10, wherein the first color space includes red, green, and blue components for each pixel and the transforming step includes comparing pixel values for a single component for each pixel to a threshold to identify pixels having a components value equal to or greater than said threshold as candidate object of interest pixels and pixels having a component value less than the threshold as background pixels.

15. The method of claim 9, further comprising reconstructing the sample from a plurality of images.

16. The method of claim 9 or 15, further comprising, morphologically processing candidate object of interest pixels to identify artifact pixels and identifying the candidate object of interest from the remaining candidate object of interest pixels not identified as artifact pixels.

17. The method of claim 16, further comprising filtering said candidate object of interest pixels with a low pass filter prior to morphologically processing the low pass filtered candidate object of interest pixels.

18. The method of claim 17, further comprising comparing said low pass filtered candidate object of interest pixels to a threshold prior to morphologically processing the candidate object of interest pixels which have values greater than or equal to the threshold value.

19. The method of claim 18, further comprising

computing a mean value of said candidate object of interest pixels;

specifying a threshold factor;

computing a standard deviation for the candidate object of interest pixels; and

setting the threshold to the sum of the mean value and the product of the threshold factor and the standard deviation prior to comparing the candidate object of interest pixels to the threshold.

20. The method of claim 16, further comprising:

grouping said morphologically processed candidate object of interest pixels into regions of connected candidate object of interest pixels to identify objects of interest;

comparing said objects of interest to blob analysis parameters; and

storing location coordinates of the candidate objects of interest having an area corresponding to the blob analysis parameters.

21. The method of claim 20, wherein the method is performed on images acquired at a low magnification and the method further comprises:

adjusting an optical system viewing the slide from which the objects of interest were identified to a higher magnification;

acquiring a higher magnification image of the slide at the corresponding location coordinates for each candidate object of interest;

transforming pixels of the higher magnification image in the fist color space to a second color space to differentiation higher magnification candidate objects of interest pixels from background pixels; and

identifying higher magnification objects of interest form the candidate object of interest pixels in the second color space.

22. The method of claim 21, further comprising morphologically processing the higher magnification candidate object of interest pixels to identify artifact pixels and identifying the higher magnification objects of interest form the remaining higher magnification candidate object of interest pixels not identified as artifact pixels.

23. The method of claim 22, further comprising filtering said higher magnification candidate object of interest pixels with a low pass filter prior to morphologically processing the low pass filtered higher magnification candidate object of interest pixels.

24. The method of claim 23, further comprising, comparing said low pass filtered higher magnification candidate object of interest pixels to a threshold prior to morphologically processing the higher magnification candidate object of interest pixels which have values greater than or equal to the threshold value.

25. The method of claim 24, further comprising:

computing a mean value of said higher magnification candidate object of interest pixels;

specifying a threshold factor;

computing a standard deviation for the higher magnification candidate object of interest pixels; and

setting the threshold to the sum of the mean value and the product of the threshold factor and the standard deviation prior to comparing the higher magnification candidate object of interest pixels to the threshold.

26. The method of claim 14, further comprising:

grouping said low pass filtered higher magnification candidate object of interest pixels into regions of connected higher magnification candidate object of interest pixels to identify higher magnification object of interest;

comparing said higher magnification objects of interest to blob analysis parameters; and

storing the location coordinates of the higher magnification objects of interest corresponding to the blob analysis parameters.

27. The method of claim 21, wherein the optical system is initially focused prior to performing the low magnification processing.

28. The method of claim 27, wherein the initial focusing of the optical system further comprises:

a) positioning the optical system at an initial Z stage position;

b) acquiring at low magnification an image of a slide having a stained biological specimen thereon and calculating a pixel variance about a pixel mean for the acquired image;

c) incrementing the position of the Z stage;

d) repeating steps (b) and (c) for a fixed number of course iterations to form a first set of variance data;

e) performing a least squares fit of the first set of variance data to a first function;

f) position the Z stage at a position near the peak of the first iteration;

g) repeating steps (b) and (c) for a fixed number of fine iterations to form a second set of variance data;

h) performing a least squared fit of the second set of variance data to a second function;

i) selecting the peak value of the least squares fit curve as an estimate of the best focal position; and

j) performing the above steps for an array of stage positions to form an array of focal positions and performing a least squares fit of the array of focal positions to yield a least squares for focal plane.

29. The method of claim 27, wherein the initial focusing of the optical system further comprises the steps of:

a) positioning the optical system at an initial Z stage position;

b) acquiring an image and calculating a pixel variance about a pixel mean for the acquired image;

c) incrementing the position of the Z stage;

d) repeating steps (b) and (c) or a fixed number of iterations;

e) performing a least squares fit of the variance data to a known function; and

f) selecting the peak value of the least squares fit curve as an estimate of the best focal position.

30. The method of claim 21, wherein adjusting the optical system further comprises the steps of:

a) positioning the optical system at an initial Z stage position;

b) acquiring an image and selecting a center pixel of a candidate object of interest;

c) defining a region of interest centered about the selected center pixel;

d) performing a fast fourier transform of said region of interest to identify frequency components for the region of interest and complex magnitudes for the frequency components;

e) computing a power value by summing the square of the complex magnitudes for the frequency components that are within the rage of frequencies of 25% to 75% of a maximum frequency component of the fast fourier transform of the region of interest;

f) incrementing the position of the Z stage;

g) repeating steps (b) to (e) for a fixed number of iterations; and

h) selecting the Z stage position corresponding to the largest power value as the best focal position.

31. The method of claim 21, wherein adjusting the optical system further comprises the steps of:

a) positioning the optical system at an initial Z stage position;

b) acquiring an image and selecting a center pixel of a candidate object of interest;

c) defining a region of interest centered about the selected center pixel;

d) applying a Hanning window function to the region of interest;

e) performing a fast fourier transform of said region of interest following the application of the Hanning window function to identify frequency components of the region of interest and complex magnitudes for the frequency components;

f) computing a power value by summing the square of the complex magnitudes for the frequency components for the fast fourier transform of the region of interest;

g) incrementing the position of the Z stage;

h) repeating steps (b) to (e) for a fixed number of iterations; and

i) selecting the Z stage position corresponding to the largest power value as the best focal position.