Automated Detection of Cell Colonies and Coverslip Detection Using Hough Transforms

• US 20090129660A1
• Filed: 10/16/2008
• Published: 05/21/2009
• Est. Priority Date: 10/22/2007
• Status: Active Grant

• Current Assignee: Leica Biosystems Imaging Inc. (Danaher Corporation)
• Original Assignee: Applied Imaging Corporation (Danaher Corporation)
• Inventors: Gregson, Mark
• Granted Patent: US 8,064,678 B2
• Time in Patent Office: Days
• US Class Current: 382/133
• CPC Class Codes: G01N 33/5005 involving human or animal cells immunoassay G01N33/56966; immunoassays of protozoa G01N33/56905; protozoa in screening assays C12Q1/025
• CPC Class Codes: G02B 21/34 Microscope slides, e.g. mounting specimens on microscope slides
• CPC Class Codes: G06T 2207/10056 Microscopic image
• CPC Class Codes: G06T 2207/20036 Morphological image processing
• CPC Class Codes: G06T 2207/20061 Hough transform
• CPC Class Codes: G06T 2207/20224 Image subtraction
• CPC Class Codes: G06T 2207/30024 Cell structures in vitro; Tissue sections in vitro
• CPC Class Codes: G06T 7/0012 Biomedical image inspection
• CPC Class Codes: G06T 7/12 Edge-based segmentation
• CPC Class Codes: G06T 7/194 involving foreground-background segmentation
• CPC Class Codes: G06V 20/69 Microscopic objects, e.g. biological cells or cellular parts

Abstract

The present invention provides methods and systems for automatic detection of the location of cell colonies on a specimen slide, in particular under the coverslip of a specimen slide. Slide scanning can be performed using an automated microscope with motorized axes. The location of the colonies can be determined by image analysis, which is followed by automatically finding metaphase cells and associating them with each colony. The invention also provides an automated, Hough-transform-based method for identifying the location of the slide coverslip and, if desired, analyzing only the image area contained within the coverslip.

Claims

1. A method for an automatic detection of cell colonies, comprising:

acquiring digital images of sub-areas of a slide;

generating background intensity images;

forming background-subtracted sub-area images by subtracting the minimum of a morphologically processed image and the original digital image from the original image;

generating shrunk sub-area images by shrinking the background-subtracted sub-area images by a predefined scale factor;

stitching the shrunk sub-area images into a composite image representing the entire slide;

converting the composite image into a binary edge image by the application of an automatic thresholding technique;

passing the binary edge image through a Hough transform to produce a Hough space transformed image;

analyzing the Hough space transformed image to identify peaks representing a coverslip edge location;

subjecting the Hough space transformed image to an inverse Hough transform to convert said peaks back to the edges of the coverslip in the binary edge image;

setting to zero all pixels of the composite image outside of the coverslip area;

morphologically processing the composite binary edge image;

identifying colonies in the composite binary edge image;

eliminating unwanted colonies from the composite binary edge image based on their size or morphology;

recording the location, size, or binary image masks of the remaining colonies on the slide;

generating a map of automatic focusing values over the remaining colonies on the slide using a high power objective;

acquiring a plurality of high power images over the remaining colonies on the slide using a high power objective and the map of automatic focusing values;

performing metaphase detection; and

assigning each detected metaphase to a colony.

2. The method of claim 1 wherein said digital images are acquired by a digital camera with a low power or a high power objective at a fixed calibrated focus.

3. The method of claim 2 wherein said digital camera is a CCD camera.

4. The method of claim 2 wherein said low power objective is chosen from the range of 1.25 to 5.

5. The method of claim 2 wherein said high power objective is chosen from the range of 10 to 20.

6. The method of claim 1 wherein said morphological processing comprises:

morphologically closing the sub-area image with a small structuring element; and

morphologically opening the resulting image with a large structuring element.

7. The method of claim 1 wherein the composite image stitching is enhanced by closing with a small structuring element, followed by opening with a small structuring element and subtracting the result from the original composite image.

8. The method of claim 1 wherein said automatic thresholding technique is based on an analysis of the grey level gradients around a modal grey level of the composite image.

9. The method of claim 1 wherein said Hough transform is a linear Hough transform configured for a rectilinear coverslip, the Hough transform is based on polar coordinates of a line, said transformed image is analyzed to identify peaks close to 90°

and 180°

representing horizontal and vertical lines in the binary edge image, and said transformed image is subjected to an inverse Hough transform to convert said peaks close to 90° and

180°

back to horizontal and vertical lines representing the edges of the coverslip in the binary edge image.

10. The method of claim 1 wherein said Hough transform is configured for a curvilinear coverslip, having a substantially circular shape with a known radius R, the Hough transform is based on a weighted circular Hough transform, said transformed image is analyzed to find the most probable locations of coverslip centers, and said transformed image is subjected to an inverse circular Hough transform to convert said most probable locations of coverslip centers back to the locations in the binary edge image.

11. The method of claim 10 wherein said transformed image is analyzed to find the most probable locations of coverslip centers by:

(a) treating all non-zero pixels in the binary image as the centers of a circle C1 of radius R;

(b) treating each point on circle C1 as the center P,Q of a circle C2 of radius R;

(c) for each point X,Y on circle C2 stepping back about 5° and

about 10° and

stepping forward about 5° and

about 10°

starting from the point X,Y along the circle C2, and checking when the pixel at that location is non-zero in the binary image, and when it is, then incrementing the value of pixel at P,Q in the Hough space image;

(d) moving to the next point on circle C1 in steps of about 1°

, and repeating steps (b) and (c);

treating the highest value of pixel at P,Q in the Hough image as the most probable location of the center of the circular shape coverslip.

12. The method of claim 1 wherein the composite image outside of the coverslip area is not used in generating a map of automatic focusing values.

13. The method of claim 1 wherein the composite image outside of the coverslip area is not used in colony detection.

14. The method of claim 1 wherein the composite image outside of the coverslip area is cropped to reduce the image to the area within the coverslip.

15. The method of claim 1 wherein assigning each detected metaphase to a colony is based on the metaphase proximity to the colony center of gravity, radius, and binary mask.

16. An apparatus for an automatic detection of cell colonies, comprising:

an optical system comprising a digital camera for acquiring digital images of the slide, a computing unit for storing and processing the digital images of the slide, said computing unit executing a method so as to cause;

images of sub-areas of a slide to be acquired by digital camera with a low power objective at a fixed calibrated focus;

background intensity images to be generated;

a background-subtracted sub-area images to be formed by subtracting the minimum of a morphologically processed image and the original digital image from the original image;

shrunk sub-area images to be generated by shrinking the background-subtracted sub-area images by a predefined scale factor;

the shrunk sub-area images to be stitched into a composite image representing the entire slide;

the composite image to be converted into a binary edge image by the application of an automatic thresholding technique;

the binary edge image to be passed through a Hough transform to produce a Hough space transformed image;

the Hough space transformed image to be analyzed to identify peaks, representing coverslip edge location;

the Hough space transformed image to be subjected to the inverse Hough transform to convert said peaks back to the edges of the coverslip in the binary edge image;

all pixels of the composite image to be set to zero outside of the coverslip area;

the composite binary edge image to be morphologically processed;

colonies in the composite binary edge image to be identified;

unwanted colonies from the composite binary edge image to be eliminated based on their size or morphology;

the location, size, or binary image masks of the remaining colonies on the slide to be recorded;

a map of automatic focusing values over the remaining colonies on the slide to be generated using a high power objective;

a plurality of high power images to be acquired over the remaining colonies on the slide using a high power objective and the map of automatic focusing values;

metaphase detection to be performed; and

each detected metaphase to be assigned to a colony.

17. The apparatus of claim 16 wherein said digital camera is a CCD camera.

18. The apparatus of claim 16 wherein said low power objective is chosen from the range of 1.25 to 5.

19. The apparatus of claim 16 wherein the background subtraction is done by causing:

the sub-area image to be morphologically closed with a small structuring element;

the resulting noise image to be morphologically opened with a large structuring element;

the resulting noise image minimum of the sub-area and the original digital image of that sub-area to be found.

20. The apparatus of claim 16 wherein the composite image stitches enhancement is done by closing with a small structuring element, followed by opening with a small structuring element and subtracting the result from the composite image.

21. The apparatus of claim 16 wherein said automatic thresholding technique is based on a predetermined grey level value.

22. The apparatus of claim 16 wherein said computing unit causes said Hough transform to be a linear Hough transform configured for a rectilinear coverslip, the linear Hough transform to be based on polar coordinates of a line, said transformed image to be analyzed to identify peaks close to 90°

and 180°

representing horizontal and vertical lines in the binary edge image, and said transformed image to be subjected to an inverse Hough transform to convert said peaks close to 90° and

180°

back to horizontal and vertical lines representing the edges of the coverslip in the binary edge image.

23. The apparatus of claim 16 wherein said computing unit causes said Hough transform to be configured for a curvilinear coverslip having approximately circular shape with a known radius R, the transform to be based on a weighted circular Hough transform, said transformed image to be analyzed to find the most probable locations of coverslip centers, and said transformed image to be subjected to an inverse circular Hough transform to convert said most probable locations of coverslip centers back to the locations in the binary edge image.

24. The apparatus of claim 23 wherein said computing unit causes said transformed image to be analyzed to find the most probable locations of coverslip centers by:

(a) treating all non-zero pixels in the binary image as the center of a circle C1 of radius R;

(b) treating each point on circle C1 as the center P,Q of a circle C2 of radius R;

(c) for each point X,Y on circle C2 stepping back about 5° and

about 10° and

stepping forward about 5° and

about 10°

starting from the point X,Y along the circle C2, and checking if the pixel at that location is non-zero in the binary image, and if it is then incrementing the value of pixel at P,Q in the Hough image;

(d) moving to a next point on circle C1 in steps of about 1°

, and repeating steps (b) and (c);

treating the highest value of pixel at P,Q in the Hough image as the most probable location of the coverslip.

25. The apparatus of claim 16 wherein the composite image outside of the coverslip area is not used in generating a map of automatic focusing values.

26. The apparatus of claim 16 the composite image outside of the coverslip area is not used in colony detection.

27. The apparatus of claim 16 wherein the composite image outside of the coverslip area is cropped to reduce the image to the area within the coverslip.

28. The apparatus of claim 16 wherein assigning each detected metaphase to a colony is based on the metaphase proximity to the colony center of gravity, radius, and binary mask.

29. A method for an automatic detection of slide coverslips, comprising:

acquiring digital images of sub-areas of the slide;

generating background intensity images;

forming background-subtracted sub-area images by subtracting the minimum of a morphologically processed image and the original digital image from the original image;

generating shrunk sub-area images by shrinking the background-subtracted sub-area images by a predefined scale factor;

stitching the shrunk sub-area images into a composite image representing the entire slide;

converting the composite image into a binary edge image by the application of an automatic thresholding technique;

passing the binary edge image through a Hough transform to produce a Hough space transformed image;

analyzing the Hough space transformed image to identify peaks representing a coverslip edge location;

subjecting the Hough space transformed image to an inverse Hough transform to convert said peaks back to the edges of the coverslip in the binary edge image; and

setting to zero all pixels of the composite image outside of the coverslip area.

30. The method of claim 29 wherein said digital images are acquired by a digital camera with a low power or a high power objective at a fixed calibrated focus.

31. The method of claim 30 wherein said digital camera is a CCD camera.

32. The method of claim 30 wherein said low power objective is chosen from the range of 1.25 to 5.

33. The method of claim 29 wherein the composite image stitching is enhanced by closing with a small structuring element, followed by opening with a small structuring element and subtracting the result from the composite image.

34. The method of claim 29 wherein said morphological processing comprises:

morphologically closing the sub-area image with a small structuring element; and

morphologically opening the resulting noise image with a large structuring element.

35. The method of claim 29 wherein said automatic thresholding technique is based on an analysis of the grey level gradients around a modal grey level of the composite image.

36. The method of claim 29 wherein said Hough transform is a linear Hough transform configured for a rectilinear coverslip, the linear Hough transform is based on polar coordinates of a line, said transformed image is analyzed to identify peaks close to 90°

and 180°

representing horizontal and vertical lines in the binary edge image, and said transformed image is subjected to an inverse Hough transform to convert said peaks close to 90° and

180°

back to horizontal and vertical lines representing the edges of the coverslip in the binary edge image.

37. The method of claim 29 wherein said Hough transform is configured for a curvilinear coverslip, having a substantially circular shape with a known radius R, the Hough transform is based on a weighted circular Hough transform, said transformed image is analyzed to find the most probable locations of coverslip centers, and said transformed image is subjected to an inverse circular Hough transform to convert said most probable locations of coverslip centers back to the locations in the binary edge image.

38. The method of claim 37 wherein said transformed image is analyzed to find the most probable locations of coverslip centers by:

(a) treating all non-zero pixels in the binary image as the centers of a circle C1 of radius R;

(b) treating each point on circle C1 as the center P,Q of a circle C2 of radius R;

(c) for each point X,Y on circle C2 stepping back about 5° and

about 10° and

stepping forward about 5° and

about 10°

starting from the point X,Y along the circle C2, and checking when the pixel at that location is non-zero in the binary image, and when it is, then incrementing the value of pixel at P,Q in the Hough space image;

(d) moving to the next point on circle C1 in steps of about 1°

, and repeating steps (b) and (c);

treating the highest value of pixel at P,Q in the Hough image as the most probable location of the circular shape coverslip.

39. The method of claim 29 further comprising:

morphologically processing the composite binary edge image;

identifying colonies in the composite binary edge image;

eliminating unwanted colonies from the composite binary edge image based on their size or morphology;

recording the location, size, or binary image masks of the remaining colonies on the slide;

generating a map of automatic focusing values over the remaining colonies on the slide using a high power objective;

acquiring a plurality of high power images over the remaining colonies on the slide using a high power objective and the map of automatic focusing values;

performing metaphase detection; and

assigning each detected metaphase to a colony.

40. The method of claim 39 wherein said high power objective is chosen from the range of 10 to 20.

41. The method of claim 39 wherein the composite image outside of the coverslip area is not used in generating a map of automatic focusing values.

42. The method of claim 39 wherein the composite image outside of the coverslip area is not used in colony detection

43. The method of claim 39 wherein the composite image outside of the coverslip area is cropped to reduce the image to the area within the coverslip

44. The method of claim 39 wherein assigning each detected metaphase to a colony is based on the metaphase proximity to the colony center of gravity, radius, and binary mask.

45. An apparatus for an automatic detection of slide coverslips, comprising:

an optical system comprising a digital camera for acquiring digital images of the slide, a computing unit for storing and processing the digital images of the slide, said computing unit executing a method so as to cause;

images of sub-areas of a slide to be acquired by digital camera with a low power objective at a fixed calibrated focus;

background intensity images to be generated;

a background-subtracted sub-area images to be formed by subtracting the minimum of a morphologically processed image and the original digital image from the original image;

shrunk sub-area images to be generated by shrinking the background-subtracted sub-area images by a predefined scale factor;

the shrunk sub-area images to be stitched into a composite image representing the entire slide;

the composite image to be converted into a binary edge image by the application of an automatic thresholding technique;

the binary edge image to be passed through a Hough transform to produce a Hough space transformed image;

the Hough space transformed image to be analyzed to identify peaks, representing coverslip edge location;

the Hough space transformed image to be subjected to the inverse Hough transform to convert said peaks back to the edges of the coverslip in the binary edge image; and

all pixels of the composite image to be set to zero outside of the coverslip area.

46. The apparatus of claim 45 wherein said digital camera is a CCD camera.

47. The apparatus of claim 45 wherein said low power objective is chosen from the range of 1.25 to 5.

48. The apparatus of claim 45 wherein the background subtraction is done by causing:

the sub-area image to be morphologically closed with a small structuring element;

the resulting noise image to be morphologically opened with a large structuring element;

the resulting noise image minimum of the sub-area and the original digital image of that sub-area to be found.

49. The apparatus of claim 45 wherein the composite image stitches enhancement is done by closing with a small structuring element, followed by opening with a small structuring element and subtracting the result from the composite image.

50. The apparatus of claim 45 wherein said automatic thresholding technique is based on a predetermined grey level value.

51. The apparatus of claim 45 wherein said computing unit causes said Hough transform to be a linear Hough transform configured for a rectilinear coverslip, the linear Hough transform to be based on polar coordinates of a line, said transformed image to be analyzed to identify peaks close to 90°

and 180°

representing horizontal and vertical lines in the binary edge image, and said transformed image to be subjected to an inverse Hough transform to convert said peaks close to 90° and

180°

back to horizontal and vertical lines representing the edges of the coverslip in the binary edge image.

52. The apparatus of claim 45 wherein said computing unit causes said Hough transform to be configured for a curvilinear coverslip having approximately circular shape with a known radius R, the linear Hough transform to be based on a weighted circular Hough transform, said transformed image to be analyzed to find the most probable locations of coverslip centers, and said transformed image to be subjected to an inverse circular Hough transform to convert said most probable locations of coverslip centers back to the locations in the binary edge image.

53. The apparatus of claim 52 wherein said computing unit causes said transformed image to be analyzed to find the most probable locations of coverslip centers by:

(a) treating all non-zero pixels in the binary image as the center of a circle C1 of radius R;

(b) treating each point on circle C1 as the center P,Q of a circle C2 of radius R;

(c) for each point X,Y on circle C2 stepping back about 5° and

about 10° and

stepping forward about 5° and

about 10°

starting from the point X,Y along the circle C2, and checking if the pixel at that location is non-zero in the binary image, and if it is then incrementing the value of pixel at P,Q in the Hough image;

(d) moving to a next point on circle C1 in steps of about 1°

, and repeating steps (b) and (c);

treating the highest value of pixel at P,Q in the Hough image as the most probable location of the center of the coverslip.

54. The apparatus of claim 45 wherein said computing unit further causes:

the composite binary edge image to be morphologically processed;

colonies in the composite binary edge image to be identified;

unwanted colonies from the composite binary edge image to be eliminated based on their size or morphology;

the location, size, or binary image masks of the remaining colonies on the slide to be recorded;

a map of automatic focusing values over the remaining colonies on the slide to be generated using a high power objective;

a plurality of high power images to be acquired over the remaining colonies on the slide using a high power objective and the map of automatic focusing values;

metaphase detection to be performed; and

each detected metaphase to be assigned to a colony.

55. The apparatus of claim 54 wherein the composite image outside of the coverslip area is not used in generating a map of automatic focusing values.

56. The apparatus of claim 54 wherein the composite image outside of the coverslip is not used in colony detection.

57. The apparatus of claim 54 wherein the composite image outside of the coverslip area is cropped to reduce the image to the area within the coverslip

58. The apparatus of claim 54 wherein assigning each detected metaphase to a colony is based on the metaphase proximity to the colony center of gravity, radius, and binary mask.