Automatic detection of red lesions in digital color fundus photographs

• US 7,474,775 B2
• Filed: 03/29/2006
• Issued: 01/06/2009
• Est. Priority Date: 03/31/2005
• Status: Active Grant

• Current Assignee: UMC Utrecht Holding B.V., University of Iowa Research Foundation
• Original Assignee: University of Iowa Research Foundation
• Inventors: van Ginneken, Bram, Abràmoff, Michael D., Niemeijer, Meindert
• Primary Examiner(s): Mehta; Bhavesh M
• Assistant Examiner(s): Seth; Manav
• Application Number:
  US11/392,268
• Publication Number:
  US 20060257031A1
• Time in Patent Office: 1,014 Days
• Field of Search: 382/100, 382/115, 382/117, 382/128, 382173-180, 382190-195, 382224-227, 600/300, 600/318, 600/356, 600/357, 600/400, 600/558, 600/162, 600/236, 351/200, 351/206, 351/211
• US Class Current: 382/128
• CPC Class Codes: G06F 18/2415 based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate
• CPC Class Codes: G06T 2207/30041 Eye; Retina; Ophthalmic
• CPC Class Codes: G06T 7/0012 Biomedical image inspection
• CPC Class Codes: G06V 2201/03 Recognition of patterns in medical or anatomical images

Abstract

Disclosed is an automated method which can detect images containing red lesions. First, each image is preprocessed; next, candidate objects that may represent red-lesions are extracted; and in the final stage the probability for each candidate to represent a red-lesion is estimated using a classifier and a large set of specifically designed features.

Claims

1. A method of identifying a subject with a disease of the retina comprising automatically detecting in a digital color fundus photograph of the subject abnormal objects in an Image_ORG, comprising:

a. reducing intensity variation in the Image_ORG producing a shade-corrected image, Image_SC and a background image, Image_BG;

b. automatically detecting candidate objects in the Image_SC; and

c. automatically classifying the candidate objects as abnormal or normal, wherein classification of an object as abnormal identifies a subject with a disease of the retina, wherein step b comprises,

removing bright lesions in the Image_SC to produce an Image_PP,

training a classifier with a supervised procedure using example pixels extracted from a labeled reference standard training set,

obtaining pixel feature vectors from the Image_PP comprising applying filters for determining pixel intensities or other characteristics of pixels at each pixel location or pixel region in the Image_PP,

automatically classifying each of a plurality of the Image_PP pixels as foreground or background pixels,

assigning a posterior probability of being a foreground pixel to each Image _PP pixel according to the equation p=n/k, producing an Image_PROB,

removing the Image_PROB pixels where the posterior probability is less than a probability threshold, producing an Image_BIN-PC,

removing any object in the Image_BIN-PC with a size that is above a size threshold, generating a plurality of one or more seed coordinates,

applying a region-growing procedure using the plurality of one or more seed coordinates to the Image_SC to grow a set of candidate objects, and

filling empty pixels within the grown objects of the set of candidate objects.

2. The method of claim 1, wherein the classifier is selected from the group consisting of:

a. a k-Nearest Neighbors classifier;

b. a linear discriminant classifier;

c. a quadratic discriminant classifier; and

d. a support vector machine.

3. The method of claim 1, wherein the probability threshold is from about 0 to about 1.

4. The method of claim 3, wherein the probability threshold is about 0.4.

5. The method of claim 1, wherein automatically classifying each of a plurality of the Image_PP pixels as foreground or background pixels comprises a supervised procedure.

6. The method of claim 1, wherein the size threshold is from about 30 pixels to about 600 pixels.

7. The method of claim 6, wherein the size threshold is about 300 pixels.

8. The method of claim 1, wherein the set of candidate objects comprises image coordinates.

9. The method of claim 1, wherein the set of candidate objects comprises an image.

21. The method of claim 1, wherein automatically classifying the candidate objects as abnormal or normal, wherein classification of an object as abnormal identifies a subject with a disease of the retina comprises:

a. training a classifier with a supervised procedure using example objects extracted from a labeled reference standard training set;

b. comparing each of the candidate objects using a set of features, wherein objects recognized by the classifier as abnormal are classified as abnormal with a certain probability and objects recognized by the classifier as normal are classified as normal with a certain probability; and

c. identifying a subject whose candidate objects are identified as abnormal with a certain probability as having a disease of the retina.

22. The method of claim 21, wherein the classifier is selected from the group consisting of:

a. a k-Nearest Neighbors classifier;

b. a linear discriminant classifier;

c. a quadratic discriminant classifier; and

d. a support vector machine.

23. The method of claim 21 wherein the set of features is selected from the group consisting of:

a. the area α

=Σ

<sub>jε

Ω</sub>

1 where Ω

is the set of pixels in the candidate;

b. the perimeter p which is approximated using the chaincodes of the object;

c. the number of odd n_o and even n_e chaincodes are counted then p=n_o√

{square root over (2)}+n_e;

d. the aspect ratio r=l/w, wherein l is the length of the largest and w of the second largest eigenvector of the covariance matrix of the object;

e. the circularity

$c=\frac{p^2}{4<br><br>\mathrm{\pi <br><br>}<br><br><br><br>a};<br><br>$

f. the total intensity of the object in the original green plane image i_green=Σ

<sub>jε

Ω</sub>

g_j where g_j represents the j-th pixel in the original green plane image;

g. the total intensity of the object in the shade corrected image i_sc=Σ

<sub>jε

Ω</sub>

s_j where s_j represents the j-th pixel in the shade corrected image;

h. the mean intensity under the object in the original green plane image m_green=i_green/α

;

i. the mean intensity under the object in the shade corrected image m_sc=i_sc/α

;

j. the normalized intensity in the original green plane image

${\mathrm{NI}}_{\mathrm{green}}=\frac{1}{\mathrm{\sigma <br><br>}}<br><br>\left(i_{\mathrm{green}}-x\right)$

where σ

and x are the standard deviation and average pixel value of I_bg;

k. the normalized intensity in the shade corrected image

${\mathrm{NI}}_{\mathrm{sc}}=\frac{1}{\mathrm{\sigma <br><br>}}<br><br>i;<br><br>$

l. the normalized mean intensity in the original image

${\mathrm{NM}}_{\mathrm{green}}=\frac{1}{\mathrm{\sigma <br><br>}}<br><br>\left(m_{\mathrm{green}}-x\right);<br><br>$

m. the normalized mean intensity in the shade corrected image

${\mathrm{NM}}_{\mathrm{sc}}=\frac{1}{\mathrm{\sigma <br><br>}}<br><br>m_{\mathrm{green}};<br><br>$

n. the intensity of the region growing seed in the match filtered image I_match;

o. the compactness

$v=\sqrt{\frac{\underset{j=1}{\overset{n}{\sum <br><br>}}<br><br>d_j-\stackrel{\_}{d}}{n}}$

wherein d_j is the distance from the centroid of the object to its j-th boundary pixel and d is the mean of all the distances from the centroid to all the edge pixels and n is the number of edge pixels;

p. the difference between the mean pixel values inside the object and mean values in a circular region centered on the object in the red plane from the RGB color space

$\mathrm{rd}<br><br>\left(j\right)=\frac{1}{n_{\mathrm{\Omega <br><br>}}}<br><br>{\mathrm{\Sigma <br><br>}}_{j<br><br><br><br>\mathrm{\epsilon <br><br>}<br><br><br><br>\mathrm{\Omega <br><br>}}<br><br>R_j-\frac{1}{n_{\mathrm{\Theta <br><br>}}}<br><br>{\mathrm{\Sigma <br><br>}}_{j<br><br><br><br>\mathrm{\epsilon <br><br>}<br><br><br><br>\mathrm{\Theta <br><br>}}<br><br>R_j,$

wherein Ω

is the set of pixels in the candidate and Θ

is the set of pixels included in the circular region excluding the pixels in Ω

. R_j is the j-th pixel in the red image plane and the diameter of the circular region is determined by taking the maximum distance from the object centroid to any of the edges of an axis aligned bounding box and adding three. n_Ω

and n_Θ

are the number of pixels in Ω and

Θ

respectively;

q. as feature p but extracted from the green image plane;

r. as feature p but extracted from the blue image plane;

s. as feature p but extracted from the hue image plane taken from the HSI color space;

t. the mean and standard deviation of filter outputs under the object wherein the filters comprise the Gaussian and its derivatives up to second order at scales σ

=1,2,4,8 pixels;

u. the average value under the object of the absolute difference of the two largest eigenvalues of the Hessian tensor wherein the scale σ

=2 is used for the Gaussian partial derivatives that make up the Hessian;

v. the average output under the object of an iris filter used on I_sc with a minimum circle radius of 4, maximum radius of 12 and 8 directions; and

w. adding the determinant and eigenvalues of the Hessian matrix of a pixel.

24. The method of claim 1 wherein the objects classified as abnormal comprise at least one of:

microaneurysms, dot hemorrhages, flame-shaped hemorrhages, sub-intimal hemorrhages, sub-retinal hemorrhages, pre-retinal hemorrhages, micro-infarctions, cotton-wool spots, and yellow exudates.

10. A method of identifying a subject with a disease of the retina comprising automatically detecting in a digital color fundus photograph of the subject abnormal objects in an Image_ORG, comprising:

a. reducing intensity variation in the Image_ORG producing a shade-corrected image, Image_SC and a background image, Image_BG;

b. automatically detecting candidate objects in the Image_SC; and

c. automatically classifying the candidate objects as abnormal or normal, wherein classification of an object as abnormal identifies a subject with a disease of the retina, wherein step b comprises,

applying mathematical morphology based candidate object detection to produce a first set of candidate objects,

applying a supervised pixel classification based candidate object detection to produce a second set of candidate objects,

combining the first set of candidate objects and the second set of candidate objects to form a third set of candidate objects,

identifying in the third set of candidate objects a plurality of pairs of overlapping objects, and

removing one object from each pair of overlapping objects in the third set of candidate objects.

11. A method of identifying a subject with a disease of the retina comprising automatically detecting in a digital color fundus photograph of the subject abnormal objects in an Image_ORG, comprising:

a. reducing intensity variation in the Image_ORG producing a shade-corrected image, Image_SC and a background image, Image_BG;

b. automatically detecting candidate objects in the Image_SC; and

c. automatically classifying the candidate objects as abnormal or normal, wherein classification of an object as abnormal identifies a subject with a disease of the retina, wherein step b comprises,

applying a morphological transformation to the Image_SC to discriminate circular from elongated regions producing a vasculature map;

subtracting the vasculature map from the Image_SC to produce an Image_LESION;

applying a matched filter to the Image_LESION to enhance the contrast between background regions and abnormal regions producing an Image_MATCH;

applying a brightness threshold to the Image_MATCH to produce a binary digital fundus photograph, Image_BIN-MM, wherein the threshold is above the modal value of the Image_MATCH, generating a first plurality of one or more seed coordinates;

applying a region-growing procedure using the first plurality of one or more seed coordinates to the Image_SC to grow a first set of candidate objects;

removing bright lesions in the Image_SC to produce an Image_PP;

training a classifier with a supervised procedure using example pixels extracted from a labeled reference standard training set;

obtaining pixel feature vectors from the Image_PP comprising applying filters for determining pixel intensities or other characteristics of pixels at each pixel location or pixel region in the Image_PP;

automatically classifying each of a plurality of the Image_PP pixels as foreground or background pixels;

assigning a posterior probability of being a foreground pixel to each Image_PP pixel according to the equation p=n/k, producing an Image_PROB;

removing the Image_PROB pixels where the posterior probability is less than a probability threshold, producing an Image_BIN-PC;

removing any object in the Image_BIN-PC with a size that is above a size threshold, generating a second plurality of one or more seed coordinates;

applying a region-growing procedure using the second plurality of one or more seed coordinates to the Image_SC to grow a second set of candidate objects;

filling empty pixels in the grown objects of the second set of candidate objects;

combining the first set of candidate objects and the second set of candidate objects to form a third set of candidate objects;

identifying in the third set of candidate objects a plurality of pairs of overlapping objects; and

removing one object from each pair of overlapping objects in the third set of candidate objects.

12. The method of claim 11, wherein the classifier is selected from the group consisting of:

a. a k-Nearest Neighbors classifier;

b. a linear discriminant classifier;

c. a quadratic discriminant classifier; and

d. a support vector machine.

13. The method of claim 11, wherein the brightness threshold is from about 2 to about 6.

14. The method of claim 13, wherein the brightness threshold is about 4.

15. The method of claim 11, wherein the probability threshold is from about 0 to about 1.

16. The method of claim 15, wherein the probability threshold is about 0.4.

17. The method of claim 11, wherein automatically classifying each of a plurality of the Image_PP pixels as foreground or background pixels comprises a supervised procedure.

18. The method of claim 11, wherein the size threshold is from about 30 pixels to about 600 pixels.

19. The method of claim 18, wherein the size threshold is about 300 pixels.

20. The method of claim 11, wherein the removed overlapping object is the larger object of each pair.

25. A system that identifies a subject with a disease of the retina by detecting in a digital color fundus photograph of the subject abnormal objects in an Image_ORG, comprising:

a processor; and

a computer readable medium coupled to said processor for storing a computer program comprising,

computer code that reduces intensity variation in the Image_ORG producing a shade-corrected image, Image_SC and a background image, Image_BG,

computer code that automatically detects candidate objects in the Image_SC, and

computer code that automatically classifies the candidate objects as abnormal or normal, wherein classification of an object as abnormal identifies a subject with a disease of the retina, wherein computer code that automatically detects candidate objects in the Image_SC comprises,

removing bright lesions in the Image_SC to produce an Image_PP,

training a classifier with a supervised procedure using example pixels extracted from a labeled reference standard training set,

obtaining pixel feature vectors from the Image_PP comprising applying filters for determining pixel intensities or other characteristics of pixels at each pixel location or pixel region in the Image_PP,

automatically classifying each of a plurality of the Image_PP pixels as foreground or background pixels,

assigning a posterior probability of being a foreground pixel to each Image_PP pixel according to the equation p=n/k, producing an Image_PROB,

removing the Image_PROB pixels where the posterior probability is less than a probability threshold, producing an Image_BIN-PC,

removing any object in the Image_BIN-PC with a size that is above a size threshold, generating a plurality of one or more seed coordinates,

applying a region-growing procedure using the plurality of one or more seed coordinates to the Image_SC to grow a set of candidate objects, and

filling empty pixels within the grown objects of the set of candidate objects.

26. The system of claim 25, wherein the computer code that automatically classifies the candidate objects as abnormal or normal, wherein classification of an object as abnormal identifies a subject with a disease of the retina further comprises:

computer code that trains a classifier with a supervised procedure using example objects extracted from a labeled reference standard training set;

computer code that compares each of the candidate objects using a set of features, wherein objects recognized by the classifier as abnormal are classified as abnormal with a certain probability and objects recognized by the classifier as normal are classified as normal with a certain probability; and

computer code that identifies a subject whose candidate objects are identified as abnormal with a certain probability as having a disease of the retina.

27. A system that identifies a subject with a disease of the retina by detecting in a digital color fundus photograph of the subject abnormal objects in an Image_ORG, comprising.

a storage device for storing digital fundus image data;

and

a processor operably coupled to the storage device for performing the steps of;

reducing intensity variation in the Image_ORG producing a shade-corrected image, Image_SC and a background image, Image_BG,

automatically detecting candidate objects in the Image_SC, and

automatically classifying the candidate objects as abnormal or normal, wherein classification of an object as abnormal identifies a subject with a disease of the retina, wherein automatically detecting candidate objects in the Image_SC comprises,

applying mathematical morphology based candidate object detection to produce a first set of candidate objects,

applying a supervised pixel classification based candidate object detection to produce a second set of candidate objects,

combining the first set of candidate objects and the second set of candidate objects to form a third set of candidate objects,

identifying in the third set of candidate objects a plurality of pairs of overlapping objects, and

removing one object from each pair of overlapping objects in the third set of candidate objects.

28. The system of claim 27, wherein automatically classifying the candidate objects as abnormal or normal, wherein classification of an object as abnormal identifies a subject with a disease of the retina further comprises:

training a classifier with a supervised procedure using example objects extracted from a labeled reference standard training set;

comparing each of the candidate objects using a set of features, wherein objects recognized by the classifier as abnormal are classified as abnormal with a certain probability and objects recognized by the classifier as normal are classified as normal with a certain probability; and

identifying a subject whose candidate objects are identified as abnormal with a certain probability as having a disease of the retina.

29. A computer readable storage medium having computer executable instructions embodied thereon for performing a method of identifying a subject with a disease of the retina comprising automatically detecting in a digital color fundus photograph of the subject abnormal objects in an Image_ORG, comprising:

a. reducing intensity variation in the Image_ORG producing a shade-corrected image, Image_SC and a background image, Image_BG;

b. automatically detecting candidate objects in the Image_SC; and

c. automatically classifying the candidate objects as abnormal or normal, wherein classification of an object as abnormal identifies a subject with a disease of the retina, wherein step b comprises,

removing bright lesions in the Image_SC to produce an Image_PP,

training a classifier with a supervised procedure using example pixels extracted from a labeled reference standard training set,

obtaining pixel feature vectors from the Image_PP comprising applying filters for determining pixel intensities or other characteristics of pixels at each pixel location or pixel region in the Image_PP,

automatically classifying each of a plurality of the Image_PP pixels as foreground or background pixels,

assigning a posterior probability of being a foreground pixel to each Image_PP pixel according to the equation p=n/k, producing an Image_PROB,

removing the Image_PROB pixels where the posterior probability is less than a probability threshold, producing an Image_BIN-PC,

removing any object in the Image_BIN-PC with a size that is above a size threshold, generating a plurality of one or more seed coordinates,

applying a region-growing procedure using the plurality of one or more seed coordinates to the Image_SC to grow a set of candidate objects, and

filling empty pixels within the grown objects of the set of candidate objects.

30. A computer readable storage medium having computer executable instructions embodied thereon for performing a method of identifying a subject with a disease of the retina comprising automatically detecting in a digital color fundus photograph of the subject abnormal objects in an Image_ORG, comprising

a. reducing intensity variation in the Image_ORG producing a shade-corrected image, Image_SC and a background image, Image_BG;

b. automatically detecting candidate objects in the Image_SC; and

c. automatically classifying the candidate objects as abnormal or normal, wherein classification of an object as abnormal identifies a subject with a disease of the retina, wherein step b comprises,

applying mathematical morphology based candidate object detection to produce a first set of candidate objects,

applying a supervised pixel classification based candidate object detection to produce a second set of candidate objects,

combining the first set of candidate objects and the second set of candidate objects to form a third set of candidate objects,

identifying in the third set of candidate objects a plurality of pairs of overlapping objects, and

removing one object from each pair of overlapping objects in the third set of candidate objects.