Systems and methods for automated interest region detection in retinal images

• US 8,879,813 B1
• Filed: 04/30/2014
• Issued: 11/04/2014
• Est. Priority Date: 10/22/2013
• Status: Active Grant

• Current Assignee: Eyenuk, Inc.
• Original Assignee: Eyenuk, Inc.
• Inventors: Solanki, Kaushal Mohanlal, Amai Ramachandra, Chaithanya, Bhat Krupakar, Sandeep
• Primary Examiner(s): Chen, Wenpeng
• Application Number:
  US14/266,749
• Time in Patent Office: 188 Days
• Field of Search: None
• US Class Current: 382/128
• CPC Class Codes: A61B 3/0025 characterised by electronic signal processing, e.g. eye models
• CPC Class Codes: A61B 3/12 for looking at the eye fundus, e.g. ophthalmoscopes A61B3/13 takes precedence
• CPC Class Codes: A61B 3/14 Arrangements specially adapted for eye photography apparatus or arrangements for taking photographs per se G03B
• CPC Class Codes: G06F 16/51 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/583 using metadata automatically derived from the content
• CPC Class Codes: G06F 16/5866 using information manually generated, e.g. tags, keywords, comments, manually generated location and time information
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/20016 Hierarchical, coarse-to-fine, multiscale or multiresolution image processing; Pyramid transform
• CPC Class Codes: G06T 2207/20032 Median filtering
• CPC Class Codes: G06T 2207/20036 Morphological image processing
• CPC Class Codes: G06T 2207/30041 Eye; Retina; Ophthalmic
• CPC Class Codes: G06T 2207/30096 Tumor; Lesion
• CPC Class Codes: G06T 2207/30104 Vascular flow; Blood flow; Perfusion
• CPC Class Codes: G06T 2207/30168 Image quality inspection
• CPC Class Codes: G06T 3/14 Transformations for image registration, e.g. adjusting or mapping for alignment of images
• CPC Class Codes: G06T 3/18 Image warping, e.g. rearranging pixels individually
• CPC Class Codes: G06T 3/40 Scaling of whole images or parts thereof, e.g. expanding or contracting
• CPC Class Codes: G06T 5/20 using local operators
• CPC Class Codes: G06T 5/94 based on local image properties, e.g. for local contrast enhancement
• CPC Class Codes: G06T 7/0012 Biomedical image inspection
• CPC Class Codes: G06T 7/0014 : using an image reference approach
• CPC Class Codes: G06T 7/0016 : involving temporal comparison
• CPC Class Codes: G06V 10/267 : by performing operations on regions, e.g. growing, shrinking or watersheds
• CPC Class Codes: G06V 10/44 : Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
• CPC Class Codes: G06V 10/50 : by performing operations within image blocks; by using histograms, e.g. histogram of oriented gradients [HoG]; by summing image-intensity values; Projection analysis
• CPC Class Codes: G06V 10/758 : Involving statistics of pixels or of feature values, e.g. histogram matching
• CPC Class Codes: G06V 2201/03 : Recognition of patterns in medical or anatomical images
• CPC Class Codes: G06V 40/14 : Vascular patterns
• CPC Class Codes: G06V 40/18 : Eye characteristics, e.g. of the iris
• CPC Class Codes: G06V 40/193 : Preprocessing; Feature extraction
• CPC Class Codes: G16H 30/20 : for handling medical images, e.g. DICOM, HL7 or PACS
• CPC Class Codes: G16H 30/40 : for processing medical images, e.g. editing
• CPC Class Codes: G16H 50/20 : for computer-aided diagnosis, e.g. based on medical expert systems
• CPC Class Codes: G16Z 99/00 : Subject matter not provided for in other main groups of this subclass

Abstract

Embodiments disclose systems and methods that aid in screening, diagnosis and/or monitoring of medical conditions. The systems and methods may allow, for example, for automated identification and localization of lesions and other anatomical structures from medical data obtained from medical imaging devices, computation of image-based biomarkers including quantification of dynamics of lesions, and/or integration with telemedicine services, programs, or software.

Claims

1. A computing system for automated detection of active pixels in retinal images, the computing system comprising:

one or more hardware computer processors; and

one or more storage devices configured to store software instructions configured for execution by the one or more hardware computer processors in order to cause the computing system to;

access a retinal image;

generate a first median normalized image using the retinal image with a median computed over a first geometric shape of a first size;

generate a second median normalized image using the retinal image with a median computed over the first geometric shape of a second size, the second size different from the first size;

automatically generate a difference image by computing a difference between the first median normalized image and the second median normalized image;

generate a binary image by computing a hysteresis threshold of the difference image using at least two thresholds to detect dark and bright structures in the difference image;

and

store the binary image in a memory of the computing system.

2. The computing system of claim 1, wherein the difference image comprises a plurality of pixels and the computing system is further configured to:

compute a determinant of a Hessian matrix at each pixel in the difference image, and for each pixel of the difference image whose determinant of the Hessian matrix indicates a local maximum, assigning said each pixel a maximum value of the difference image.

3. The computing system of claim 1, wherein the computing system is further configured to resize the retinal image to a standard size such that dimensions of the resized image are determined using field of view information provided or inferred from a structure in the retinal image or using a diameter of a fundus mask.

4. The computing system of claim 1, wherein the computing system is further configured to:

scale the retinal image progressively up or down using a fixed scaling factor;

designate groups of neighboring pixels within a retinal image as active areas; and

include the active areas from each scale as interest regions across multiple scales.

5. The computing system of claim 1, wherein the computing system is further configured to determine an eccentricity of each of a plurality of local regions and to remove regions from the plurality of local regions which have an eccentricity greater than or smaller than a predetermined threshold, so as to meet a pre-defined requirement or query for designating interesting regions.

6. The computing system of claim 1, wherein the computing system is further configured to select a first set of local regions from a plurality of local regions by:

computing a sharpness number for each local region by computing one or more of;

mean, minimum, maximum, or median of one or more of;

determinant of a Hessian matrix, or ratio of Eigen values of a Hessian matrix corresponding to each pixel within the local region; and

ranking each local region based on the corresponding sharpness number;

selecting a predetermined number of local regions with the highest sharpness numbers; and

adding the selected local regions to the first set of local regions.

7. The computing system of claim 1, wherein the computing system is further configured to select a first set of local regions from a plurality of local regions by:

computing a contrast number for each local region by computing one or more of;

mean, maximum, or median of intensity of each pixel within the local region, or by using a contrast measure;

ranking each local region based on the corresponding contrast number; and

selecting a predetermined number of local regions with the highest contrast numbers; and

adding the selected local regions to the first set of local regions.

8. The computing system of claim 6, wherein the computing system is further configured to select a second set of local regions from a plurality of local regions by:

computing a contrast number for each local region by computing one or more of;

mean, maximum, or median of intensity of each pixel within the local region, or by using a contrast measure;

ranking each local region based on the corresponding contrast number; and

selecting a predetermined number of local regions with the highest contrast numbers; and

adding the selected local regions to the second set of local regions; and

the computing system is further configured to generate a set of interesting regions by combining the first set of local regions and the second set of local regions.

9. The computing system of claim 1, wherein the computing system is further configured to:

remove regions from a plurality of local regions with an area above a maximum threshold, and remove regions from the plurality of local regions with an area below a minimum threshold.

10. The computing system of claim 1, wherein the computing system is further configured to automatically perform at least one of lesion localization, screening, and analysis of a vascular network.

11. A computer-implemented method for automated detection of active pixels in retinal images, the method comprising:

as implemented by one or more computing devices configured with specific executable instructions;

accessing a retinal image;

generating a first median normalized image using the retinal image with a median computed over a first geometric shape of a first size;

generating a second median normalized image using the retinal image with a median computed over the first geometric shape of a second size, the second size different from the first size;

automatically generating a difference image by computing a difference between the first median normalized image and the second median normalized image;

generating a binary image by computing a hysteresis threshold of the difference image using at least two thresholds to detect dark and bright structures in the difference image;

and

storing the binary image in a memory.

12. The computer implemented method of claim 11, wherein the difference image comprises a plurality of pixels and the method further comprises:

determining computing a determinant of a Hessian matrix at each pixel in the difference image, and for each pixel of the difference image whose determinant of the Hessian matrix indicates a local maximum, assigning said each pixel a maximum value of the difference image.

13. The computer implemented method of claim 11, further comprising resizing the retinal image to a standard size such that dimensions of the resized image are determined using field of view information provided or inferred from a structure in the retinal image or using a diameter of a fundus mask.

14. The computer implemented method of claim 11, further comprising:

scaling the retinal image progressively up or down using a fixed scaling factor;

designating groups of neighboring pixels within a retinal image as active areas; and

including the active areas from each scale as interest regions across multiple scales.

15. The computer implemented method of claim 11, further comprising:

determining an eccentricity of each of a plurality of local regions and removing regions from the plurality of local regions which have an eccentricity greater than or smaller than a predetermined threshold, so as to meet a pre-defined requirement or query for designating interesting regions.

16. The computer implemented method of claim 11, further comprising:

selecting a first set of local regions from a plurality of local regions by;

computing a sharpness number for each local region by computing one or more of;

mean, minimum, maximum, or median of one or more of;

determinant of a Hessian matrix, or ratio of Eigen values of a Hessian matrix corresponding to each pixel within the local region; and

ranking each local region based on the corresponding sharpness number;

selecting a predetermined number of local regions with the highest sharpness numbers; and

adding the selected local regions to the first set of local regions.

17. The computer implemented method of claim 11, further comprising:

selecting a first set of local regions from a plurality of local regions by;

computing a contrast number for each local region by computing one or more of;

mean, maximum, or median of intensity of each pixel within the local region, or by using a contrast measure;

ranking each local region based on the corresponding contrast number; and

selecting a predetermined number of local regions with the highest contrast numbers; and

adding the selected local regions to the first set of local regions.

18. The computer implemented method of claim 11, further comprising:

selecting a second set of local regions from a plurality of local regions by;

computing a contrast number for each local region by computing one or more of;

mean, maximum, or median of intensity each pixel within the local region, or by using a contrast measure;

ranking each local region based on the corresponding contrast number; and

selecting a predetermined number of local regions with the highest contrast numbers; and

adding the selected local regions to the second set of local regions; and

generating a set of interesting regions by combining the first set of local regions and the second set of local regions.

19. The computer implemented method of claim 11, further comprising:

removing regions from a plurality of local regions with an area above a maximum threshold, and removing regions from the plurality of local regions with an area below a minimum threshold.

20. The computer implemented method of claim 11, further comprising:

automatically performing at least one of lesion localization, screening, and analysis of a vascular network.

21. Non-transitory computer storage that stores executable program instructions that, when executed by one or more computing devices, configure the one or more computing devices to perform operations comprising:

accessing a retinal image;

generating a first median normalized image using the retinal image with a median computed over a first geometric shape of a first size;

generating a second median normalized image using the retinal image with a median computed over the first geometric shape of a second size, the second size different from the first size;

automatically generating a difference image by computing a difference between the first median normalized image and the second median normalized image;

generating a binary image by computing a hysteresis threshold of the difference image using at least two thresholds to detect dark and bright structures in the difference image;

and

storing the binary image in a memory.

22. The non-transitory computer storage of claim 21, wherein the difference image comprises a plurality of pixels and the method further comprises:

computing a determinant of a Hessian matrix at each pixel in the difference image, and for each pixel of the difference image whose determinant of the Hessian matrix indicates a local maximum, assigning said each pixel a maximum value of the difference image.

23. The non-transitory computer storage of claim 21, further comprising resizing the retinal image to a standard size such that dimensions of the resized image are determined using field of view information provided or inferred from a structure in the retinal image or using a diameter of a fundus mask.

24. The non-transitory computer storage of claim 21, further comprising:

scaling the retinal image progressively UP or down using a fixed scaling factor;

designating groups of neighboring pixels within a retinal image as active areas; and

including the active areas from each scale as interest regions across multiple scales.

25. The non-transitory computer storage of claim 21, further comprising:

determining an eccentricity of each of a plurality of local regions and removing regions from the plurality of local regions which have an eccentricity greater than or smaller than a predetermined threshold, so as to meet a pre-defined requirement or query for designating interesting regions.

26. The non-transitory computer storage of claim 21, further comprising:

selecting a first set of local regions from a plurality of local regions by;

computing a sharpness number for each local region by computing one or more of;

mean, minimum, maximum, or median of one or more of;

determinant of a Hessian matrix, or ratio of Eigen values of a Hessian matrix corresponding to each pixel within the local region; and

ranking each local region based on the corresponding sharpness number;

selecting a predetermined number of local regions with the highest sharpness numbers; and

adding the selected local regions to the first set of local regions.

27. The non-transitory computer storage of claim 21, further comprising:

selecting a first set of local regions from a plurality of local regions by;

computing a contrast number for each local region by computing one or more of;

mean, maximum, or median of intensity of each pixel within the local region, or by using a contrast measure;

ranking each local region based on the corresponding contrast number; and

selecting a predetermined number of local regions with the highest contrast numbers; and

adding the selected local regions to the first set of local regions.

28. The non-transitory computer storage of claim 21, further comprising:

selecting a second set of local regions from a plurality of local regions by;

computing a contrast number for each local region by computing one or more of;

mean, maximum, or median of intensity each pixel within the local region, or by using a contrast measure;

ranking each local region based on the corresponding contrast number; and

selecting a predetermined number of local regions with the highest contrast numbers; and

adding the selected local regions to the second set of local regions; and

generating a set of interesting regions by combining the first set of local regions and the second set of local regions.

29. The non-transitory computer storage of claim 21, further comprising:

removing regions from a plurality of local regions with an area above a maximum threshold, and removing regions from the plurality of local regions with an area below a minimum threshold.

30. The non-transitory computer storage of claim 21, further comprising:

automatically performing at least one of lesion localization, screening, and analysis of a vascular network.