Measurement of mitotic activity

• US 7,577,280 B2
• Filed: 11/13/2003
• Issued: 08/18/2009
• Est. Priority Date: 11/18/2002
• Status: Active Grant

• Current Assignee: Qinetiq Limited (QinetiQ Group PLC)
• Original Assignee: Qinetiq Limited (QinetiQ Group PLC)
• Inventors: Kesidis, Anastasios, Petrou, Maria, Cipolla, Roberto, Varga, Margaret Jai, Ducksbury, Paul Gerard, Guittet, Christelle Marie
• Primary Examiner(s): AZARIAN, SEYED H
• Application Number:
  US10/535,140
• Publication Number:
  US 20060098858A1
• Time in Patent Office: 2,105 Days
• Field of Search: 382/100, 382/128, 382/129, 382/130, 382/131, 382/132, 382/133, 382/134, 382162-165, 382/168, 382/181, 382/192, 382/194, 382/203, 382/219, 382/232, 382/254, 382/274, 382/276, 382/291, 382/298, 382/305, 382/213, 702/19, 536/23.1, 435/3
• US Class Current: 382/128
• CPC Class Codes: G01N 15/1433 using image recognition
• CPC Class Codes: G06T 2207/30024 Cell structures in vitro; Tissue sections in vitro
• CPC Class Codes: G06T 7/0012 Biomedical image inspection
• CPC Class Codes: G06V 20/69 Microscopic objects, e.g. biological cells or cellular parts

Abstract

A method of measurement of mitotic activity from histopathological specimen images initially identifies image pixels with luminances corresponding to mitotic figures and selects from them a reference pixel to provide a reference color. Pixels similar to the reference color are located; image regions are grown on located pixels by adding pixels satisfying thresholds of differences to background and image region luminances. Grown regions are thresholded in area, compactness, width/height ratio, luminance ratio to background and difference between areas grown with perturbed thresholds. Grown regions are counted as indicating mitotic figures by thresholding region number, area and luminance. An alternative method of measuring mitotic activity measures a profile of an image region and counts the image region as corresponding to a mitotic figure if its profile is above a threshold at an intensity associated with mitotic figures. A mitotic figure is also indicated if the profile does not meet the previous criterion but has three other values satisfying respective threshold criteria.

Claims

1. A method of measurement of mitotic activity from histopathological specimen image data, the method comprising the steps of:

a) identifying pixels in the image data having luminances associated with mitotic figures;

b) selecting from among the identified pixels a reference pixel which is sufficiently close in position and luminance to another identified pixel to provide a reference colour;

c) locating pixels in the image data with luminances sufficiently close to that of the reference colour to indicate potentially mitotic figures;

d) incrementing image regions corresponding to potentially mitotic figures from the located pixels by adding pixels thereto, potential increments to image regions being implemented or rejected by according to whether or not their luminances are sufficiently close to respective image region luminances and sufficiently far from an image data background luminance;

e) selecting grown image regions on the basis of thresholds for image region area, compactness and width/height ratio; and

f) counting selected grown image regions as actually indicating mitotic figures on the basis of thresholds for number of such regions;

wherein the method is implemented by a processor.

2. A method according to claim 1 wherein the step of selecting grown image regions also involves thresholds for ratio of image region luminance to background luminance and area difference between areas derived by growing each image region with multiple thresholds.

3. A method according to claim 2 wherein the thresholds for image region area, compactness, width/height ratio, luminance and area difference are:

355 pixels<

area<

1700 pixels, 0.17<

compactness<

0.77, width/height ratio<

2.7, luminance percentage<

44%, area difference<

23 area/100.

4. A method according to claim 1 wherein the step of counting selected grown image regions as actually indicating mitotic figures also involves thresholds for region area and luminance.

5. A method according to claim 1 wherein successive potential increments to image regions are individual pixels each of which is an immediate row or column neighbor of an existing image region pixel.

6. A method according to claim 1 wherein step b) is implemented with a reference pixel having a luminance differing by less than 8% compared to another identified pixel distant from it by not more than two percent of a smaller of two image dimensions.

7. A method according to claim 1 wherein step a) includes white balancing and median filtering the image data prior to identifying pixels having luminances corresponding to mitotic figures.

8. A method according to claim 1 wherein in step c) pixels are cued for acceptance or rejection as regards indicating mitotic figures by:

a) thresholding colour image data to remove pixels lacking intensities associated with mitotic figure imagery,

b) removal pixels not present in all colors, and

c) thresholding image region areas to remove those too small and too large to be potential mitotic figures.

9. A method according to claim 1 wherein in step c) pixels are cued for acceptance or rejection as regards indicating mitotic figures by:

a) segmenting to identify pixels with intensities associated with mitotic figure imagery,

b) thresholding image region areas to remove those too small and too large to be potential mitotic figures,

c) cluster analysis to determine whether or not a pixel'"'"'s image region is in a sufficiently large cluster, and

d) necrotic and hairy edge filtering.

10. A method of measuring mitotic activity from histopathological specimen color image data, the method having the steps of:

a) staining a histopathological specimen with a staining agent to color and delineate tissue and cellular structure appropriately for assessment of mitotic activity,

b) obtaining color image data from the staining of the stained histopathological specimen,

Wherein the steps c-g are implemented by a processor;

c) measuring an intensity profile of an image region in the color image data, the image region corresponding to a potentially mitotic figure,

d) counting the image region as indicating a mitotic figure if its intensity profile has a non-zero number of pixels with intensity associated with mitotic figure imagery, and that pixel number is greater than a prearranged threshold value,

e) if one or more other image regions corresponding to potentially mitotic figures are available in the specimen image, repeating steps c) and d) for such region or regions,

f) repeating steps c), d) and e) for additional histopathological specimen image data in order to obtain mitotic figure counts for a plurality of specimen images, and

g) summing the mitotic figure counts obtained in steps c) to f) to provide an indication of degree of mitotic activity.

11. A method of measuring mitotic activity from histopathological specimen image data, the method having the steps of:

a) measuring an intensity profile of an image region corresponding to a potentially mitotic figure, and

b) counting the image region as indicating a mitotic figure if its profile has a first value not greater than the prearranged threshold at a position in the profile having intensity associated with mitotic figure imagery, a second value greater than a prearranged second threshold, a third value greater than a prearranged third threshold, and a minimum value less than a prearranged fourth threshold;

wherein the method is implemented by a processor.

12. A method according to claim 11 wherein the first value is at one end of the profile, the first and second values adjoin one another in the profile and the third value does not adjoin the second value.

13. A method according to claim 11 wherein the image data comprise a first Principal Component obtained by Principal Component Analysis (PCA) of colored image data.

14. A method according to claim 11 wherein step a) includes preprocessing image data by:

i) decomposing the image data into overlapping sub-images,

ii) applying PCA to the sub-images to derive a first Principal Component image,

iii) thresholding the first Principal Component image to produce a binary image of blobs and background

iv) rejecting blobs adjacent to or intersecting sub-image boundaries,

v) filling holes in blobs,

vi) rejecting blobs too small to correspond to potential mitotic figures, and

vii) reassembling the sub-images into a single image for image region profile measurement as aforesaid in step a).

15. A method according to claim 14 wherein after step g) pixels are cued for acceptance or rejection as regards indicating mitotic figures by:

a) thresholding colour image data to remove pixels lacking intensities associated with mitotic figure imagery,

b) removal pixels not present in all colors, and

c) thresholding image region areas to remove those too small and too large to be potential mitotic figures.

16. A method according to claim 14 wherein after step g) pixels are cued for acceptance or rejection as regards indicating mitotic figures by:

a) segmenting to identify pixels with intensities associated with mitotic figure imagery,

b) thresholding image region areas to remove those too small and too large to be potential mitotic figures,

c) cluster analysis to determine whether or not a pixel'"'"'s image region is in a sufficiently large cluster, and

d) necrotic and hairy edge filtering.

17. Computer apparatus for measuring mitotic activity from histopathological specimen image data, the apparatus being programmed to execute the steps of:

a) identifying pixels in the image data having luminances associated with mitotic figures;

b) selecting from among the identified pixels a reference pixel which is sufficiently close in position and luminance to another identified pixel to provide a reference colour;

c) locating pixels in the image data with luminances sufficiently close to that of the reference colour to indicate potentially mitotic figures;

d) incrementing image regions corresponding to potentially mitotic figures from the located pixels by adding pixels thereto, potential increments to image regions being implemented or rejected by according to whether or not their luminances are sufficiently close to respective image region luminances and sufficiently far from an image data background luminance;

e) selecting grown image regions on the basis of thresholds for image region area, compactness and width/height ratio; and

f) counting selected grown image regions as actually indicating mitotic figures on the basis of thresholds for number of such regions.

18. Computer apparatus according to claim 17 programmed to execute the step of selecting grown image regions by also using thresholds for ratio of image region luminance to background luminance and area difference between areas derived by growing each image region with multiple thresholds.

19. Computer apparatus according to claim 18 wherein the thresholds for image region area, compactness, width/height ratio, luminance and area difference are:

355 pixels<

area<

1700 pixels, 0.17<

compactness<

0.77, width/height ratio<

2.7, luminance percentage<

44%, area difference<

23 area/100.

20. Computer apparatus according to claim 17 programmed to execute the step of counting selected grown image regions as actually indicating mitotic figures by also using thresholds for region area and luminance.

21. Computer apparatus according to claim 17 wherein successive potential increments to image regions are individual pixels each of which is an immediate row or column neighbor of an existing image region pixel.

22. Computer apparatus according to claim 17 programmed to execute step b) with a reference pixel having a luminance differing by less than 8% compared to another identified pixel distant from it by not more than two percent of a smaller of two image dimensions.

23. Computer apparatus for measuring mitotic activity from histopathological specimen color image data obtained from a histopathological specimen stained with a staining agent to color and delineate tissue and cellular structure appropriately for assessment of mitotic activity, the apparatus being programmed to execute the steps of:

a) measuring an intensity profile of an image region in the color image data, the image region corresponding to a potentially mitotic figure,

b) counting the image region as indicating a mitotic figure if its intensity profile has a non-zero number of pixels with intensity associated with mitotic figure imagery, and that pixel number is greater than a prearranged threshold value,

c) if one or more other image regions corresponding to potentially mitotic figures are available in the specimen image, repeating steps a) and b) for such region or regions,

d) repeating steps a), b) and c) for additional histopathological specimen image data in order to obtain mitotic figure counts for a plurality of specimen images, and

e) summing the mitotic figure counts obtained in steps a) to d) to provide an indication of degree of mitotic activity.

24. Computer apparatus for measuring mitotic activity from histopathological specimen image data, the apparatus being programmed to execute the steps of:

a) measuring an intensity profile of an image region corresponding to a potentially mitotic figure, and

b) counting an image region as indicating a mitotic figure if its profile has a first value not greater than the prearranged threshold at a position in the profile having intensity associated with mitotic figure imagery, a second value greater than a prearranged second threshold, a third value greater than a prearranged third threshold, and a minimum value less than a prearranged fourth threshold.

25. Apparatus according to claim 24 wherein the first value is at one end of the profile, the first and second values adjoin one another in the profile and the third value does not adjoin the second value.

26. Apparatus according to claim 24 wherein the image data comprise a first Principal Component obtained by Principal Component Analysis (PCA) of colored image data.

27. A computer software product comprising a computer readable medium encoded with computer readable instructions and for use in measuring mitotic activity from histopathological specimen image data, the computer readable instructions being for controlling computer apparatus to implement the steps of:

a) identifying pixels in the image data having luminances associated with mitotic figures;

b) selecting from among the identified pixels a reference pixel which is sufficiently close in position and luminance to another identified pixel to provide a reference colour;

c) locating pixels in the image data with luminances sufficiently close to that of the reference colour to indicate potentially mitotic figures;

d) incrementing image regions corresponding to potentially mitotic figures from the located pixels by adding pixels thereto, potential increments to image regions being implemented or rejected by according to whether or not their luminances are sufficiently close to respective image region luminances and sufficiently far from an image data background luminance;

e) selecting grown image regions on the basis of thresholds for image region area, compactness and width/height ratio; and

f) counting selected grown image regions as actually indicating mitotic figures on the basis of thresholds for number of such regions.

28. A computer software product according to claim 27 wherein the computer readable instructions provide for implementing the step of selecting grown image regions by also using thresholds for ratio of image region luminance to background luminance and area difference between areas derived by growing each image region with multiple thresholds.

29. A computer software product according to claim 28 wherein the thresholds for image region area, compactness, width/height ratio, luminance and area difference are:

355 pixels<

area<

1700 pixels, 0.17<

compactness<

0.77, width/height ratio<

2.7, luminance percentage<

44%, area difference<

23 area/100.

30. A computer software product according to claim 27 wherein the computer readable instructions to provide for implementing the step of counting selected grown image regions as actually indicating mitotic figures using also thresholds for region area and luminance.

31. A computer software product comprising a computer readable medium encoded with computer readable instructions for use in measuring mitotic activity from histopathological specimen color image data obtained from a histopathological specimen stained with a staining agent to color and delineate tissue and cellular structure appropriately for assessment of mitotic activity, the computer readable instructions being for controlling computer apparatus to implement the steps of:

a) measuring an intensity profile of an image region in the color image data, the image region corresponding to a potentially mitotic figure,

b) counting the image region as indicating a mitotic figure if its intensity profile has a non-zero number of pixels with intensity associated with mitotic figure imagery, and that pixel number is greater than a prearranged threshold value,

c) if one or more other image regions corresponding to potentially mitotic figures are available in the specimen image, repeating steps a) and b) for such region or regions,

d) repeating steps a), b) and c) for additional histopathological specimen image data in order to obtain mitotic figure counts for a plurality of specimen images, and

e) summing the mitotic figure counts obtained in steps a) to d) to provide an indication of degree of mitotic activity.

32. A computer software product comprising a computer readable medium encoded with computer readable instructions and for use in measuring mitotic activity from histopathological specimen image data, the computer readable instructions being for controlling computer apparatus to implement the steps of:

a) measuring an intensity profile of an image region corresponding to a potentially mitotic figure, and

b) counting the image region as indicating a mitotic figure if its profile has a first value not greater than the prearranged threshold at a position in the profile having intensity associated with mitotic figure imagery, a second value greater than a prearranged second threshold, a third value greater than a prearranged third threshold, and a minimum value less than a prearranged fourth threshold.

33. A computer software product according to claim 32 wherein the first value is at one end of the profile, the first and second values adjoin one another in the profile and the third value does not adjoin the second value.

34. A computer software product according to claim 32 wherein the computer readable instructions provide for step a) to include preprocessing image data by:

i) decomposing the image data into overlapping sub-images,

ii) applying PCA to the sub-images to derive a first Principal Component image,

iii) thresholding the first Principal Component image to produce a binary image of blobs and background

iv) rejecting blobs adjacent to or intersecting sub-image boundaries,

v) filling holes in blobs,

vi) rejecting blobs too small to correspond to potential mitotic figures, and

vii) reassembling the sub-images into a single image for image region profile measurement as aforesaid in step a).