Architecture and method for maximally stable extremal regions (MSERs)-based exudates detection in fundus images for diabetic retinopathy

• US 10,456,027 B2
• Filed: 04/26/2017
• Issued: 10/29/2019
• Est. Priority Date: 04/26/2017
• Status: Active Grant

• Current Assignee: Khalifa University Of Science, Technology & Research
• Original Assignee: Khalifa University of Science and Technology
• Inventors: Salahat, Ehab Najeh, Saleh, Hani Hasan Mustafa, Sluzek, Andrzej Stefan, Al-Qutayri, Mahmoud, Mohammad, Baker, Elnaggar, Mohammed Ismail
• Primary Examiner(s): Bali, Vikkram
• Application Number:
  US15/497,302
• Publication Number:
  US 20180310817A1
• Time in Patent Office: 916 Days
• Field of Search: None
• CPC Class Codes: A61B 3/0025 characterised by electronic signal processing, e.g. eye models
• CPC Class Codes: A61B 3/0041 characterised by display arrangements
• CPC Class Codes: A61B 3/12 for looking at the eye fundus, e.g. ophthalmoscopes A61B3/13 takes precedence
• CPC Class Codes: G06T 2207/30041 Eye; Retina; Ophthalmic
• CPC Class Codes: G06T 2207/30101 Blood vessel; Artery; Vein; Vascular
• CPC Class Codes: G06T 7/0012 Biomedical image inspection
• CPC Class Codes: G06T 7/60 Analysis of geometric attributes
• CPC Class Codes: G06T 7/62 of area, perimeter, diameter or volume
• CPC Class Codes: G06V 2201/03 Recognition of patterns in medical or anatomical images

Abstract

Architecture and a method for maximally stable extremal regions (MSERs)-based detection of exudates in an ocular fundus is disclosed. The architecture includes a communication interface to receive pixels of an ocular fundus image. The architecture further includes processing circuitry that is coupled to the communication interface. The processing circuitry is configured to automatically provide labels for light image regions and dark image regions within the ocular fundus image for a given intensity threshold and find MSERs within the ocular fundus image based on the labels. The architecture also determines MSER regions based on the MSER criteria and then highlights the pixels of the ocular fundus image that are located within MSER regions to indicate the exudates in the ocular fundus. The architecture is further configured to determine MSER ellipses parameters based on MSER regions and MSER criteria and then highlight the locations of the exudates in the ocular fundus.

Claims

1. Architecture for maximally stable extremal regions (MSERs)-based detection of exudates in an ocular fundus comprising:

a communication interface configured to receive pixels of an ocular fundus image; and

processing circuitry having a memory coupled to the communication interface, wherein the processing circuitry is configured to automatically;

store the pixels of the ocular fundus image within the memory;

provide labels within the memory for light image regions and dark image regions within the ocular fundus image for a given intensity threshold;

find MSERs within the stored pixels of the ocular fundus image based upon the labels;

generate an MSER pixel list based on the MSERs and MSER criteria that include a minimum MSER size in pixels and a maximum MSER size in pixels based upon an eye diameter size in the stored pixels of the ocular fundus image and store the MSER pixel list within the memory; and

provide for display through the communication interface the pixels of the ocular fundus image that are located within the MSER pixel list to indicate the exudates in the ocular fundus.

2. The architecture of claim 1 wherein the processing circuitry is configured to substantially simultaneously provide the labels for the light image regions and the dark image regions within the ocular fundus image for the given intensity threshold.

3. The architecture of claim 1 wherein the processing circuitry is configured to provide the labels for the light image regions and the dark image regions within the ocular fundus image during a single processing pass for the given intensity threshold.

4. The architecture of claim 1 wherein the MSER criteria include a nested MSER tolerance value.

5. The architecture of claim 4 wherein the MSER criteria further include a minimum MSER area, a maximum MSER area, and an acceptable growth rate value for MSER areas.

6. The architecture of claim 1 wherein the processing circuitry is further configured to determine MSER ellipses parameters that include a center of gravity, a major axis length, a minor axis length, and an angle of the major axis length with respect to a horizontal axis.

7. The architecture of claim 1 wherein the processing circuitry includes MSER moments calculator hardware configured to calculate MSER moments.

8. The architecture of claim 7 wherein the processing circuitry further includes elliptical fit approximator hardware configured to receive the MSER moments from the MSER moments calculator hardware and fit an MSER ellipse to an extremal region based upon the MSER moments.

9. The architecture of claim 1 wherein the processing circuitry includes union-find hardware configured to provide the labels for the light image regions and the dark image regions within the ocular fundus image for the given intensity threshold t.

10. The architecture of claim 9 wherein the processing circuitry includes extremal region find hardware that is configured to receive the labels for the light image regions and the dark image regions and find extremal regions based upon the labels for the light image regions and the dark image regions.

11. The architecture of claim 10 wherein the extremal region find hardware is configured to find extremal regions using a mathematical relationship q(t)=|Q(t +Δ

)\Q(t−

Δ

)|\|Q(t)| where each extremal region'"'"'s cardinality,|Q(t)| is a function of the given intensity threshold t and an intensity threshold increment Δ

.

12. The architecture of claim 1 wherein the processing circuitry includes MSER selector hardware configured to automatically select MSERs based on the MSER criteria.

13. Architecture for maximally stable extremal regions (MSERs)-based detection of exudates in an ocular fundus comprising:

a communication interface;

memory;

intensity image process hardware coupled to the memory and configured to receive a data stream through the communication interface of an ocular fundus image and store the pixels of the ocular fundus image within the memory and output labels to a first matrix within the memory for light image regions and dark image regions within the ocular fundus image for a given intensity threshold;

extremal regions find hardware configured to receive the labels for the ocular fundus image and find MSERs within the ocular fundus image;

MSER process hardware configured to receive MSER criteria that include a minimum MSER size in pixels and a maximum MSER size in pixels based upon an eye diameter size in pixels of the ocular fundus image and output through the communication interface a list of pixels for display based on the MSERs; and

display hardware coupled to the communication interface for displaying highlighted pixels from the list of pixels for display.

14. The architecture of claim 13 wherein the intensity image process hardware is configured to substantially simultaneously provide the labels for the light image regions and the dark image regions within the ocular fundus image for the given intensity threshold.

15. The architecture of claim 13 wherein the intensity image process hardware is configured to provide the labels for the light image regions and the dark image regions within the ocular fundus image during a single processing pass for the given intensity threshold.

16. The architecture of claim 13 wherein the MSER criteria include a nested MSER tolerance value.

17. The architecture of claim 16 wherein the MSER criteria further include a minimum MSER area value, a maximum MSER area value, and an acceptable growth rate value for MSER areas.

18. The architecture of claim 13 wherein the MSER process hardware is further configured to determine MSER ellipses parameters that include a center of gravity, a major axis length, a minor axis length, and an angle of the major axis length with respect to a horizontal axis.

19. The architecture of claim 13 wherein the extremal regions find hardware is configured to find extremal regions using a mathematical relationship q(t)=|Q(t +Δ

) \Q(t −

Δ

)|\|Q(t)| where each extremal region'"'"'s cardinality, |Q(t)| is a function of an intensity threshold t and an intensity threshold increment Δ

.

20. The architecture of claim 13 wherein the intensity image process hardware includes union-find hardware configured to provide the labels for the light image regions and the dark image regions within the ocular fundus image for a given intensity threshold t.

21. The architecture of claim 20 wherein the intensity image process hardware further includes labeled region seeds updater/unifier hardware configured to prevent a seed that is a first pixel location within the ocular fundus image from being stored in a seed list if the seed is presently stored in the seed list.

22. The architecture of claim 21 further including region map updater hardware configured to store a value of Q(t +Δ

), Q(t), and Q(t −

Δ

) for each seed, where t is the intensity threshold and Δ

is an increment of the intensity threshold t.

23. The architecture of claim 13 wherein the intensity image process hardware, the extremal regions find hardware, and the MSER process hardware are fabricated on a single application-specific integrated circuit.

24. The architecture of claim 13 wherein the intensity image process hardware, the extremal regions find hardware, and the MSER process hardware are implemented on a single field-programmable gate array.

25. A method for maximally stable extremal regions (MSERs)-based detection of exudates in an ocular fundus via processing circuitry with memory comprising:

receiving a data stream of an ocular fundus image via a communication interface in communication with the processing circuitry;

storing the ocular fundus image within the memory via the processing circuitry;

generating labels for light image regions and dark image regions within the ocular fundus image for a given intensity threshold t in real time via the processing circuitry;

storing the labels within the memory via the processing circuitry;

finding MSERs within the ocular fundus image based on the labels and MSER criteria in real time via the processing circuitry; and

generating an MSER pixel list based on the MSERs and MSER criteria that include a minimum MSER size in pixels and a maximum MSER size in pixels based upon an eye diameter size in pixels of the ocular fundus image in real time via the processing circuitry;

providing for display through the communication interface via the processing circuitry the pixels of the ocular fundus image that are located within the MSER pixel list to indicate the exudates in the ocular fundus.

26. The method of claim 25 wherein generating the labels for the light image regions and generating labels for the dark image regions occurs substantially simultaneously.

27. The method of claim 25 wherein generating the labels for the light image regions and generating labels for the dark image regions occurs during a single processing pass for the given intensity threshold t.

28. The method of claim 25 wherein the MSER criteria include a nested MSER tolerance value.

29. The method of claim 28 wherein the MSER criteria further include a minimum MSER area, a maximum MSER area, and an acceptable growth rate value for MSER areas.

30. The method of claim 25 further including determining MSER ellipses parameters that include a center of gravity, a major axis length, a minor axis length, and an angle of the major axis length with respect to a horizontal axis via the processing circuitry.

31. The method of claim 25 wherein the processing circuitry includes MSER moments calculator hardware configured to calculate MSER moments.

32. The method of claim 31 wherein the processing circuitry further includes elliptical fit approximator hardware configured to receive the MSER moments from the MSER moments calculator hardware and fit an MSER ellipse to an extremal region based on the MSER moments.

33. The method of claim 25 wherein the processing circuitry includes union-find hardware configured to provide the labels for the light image regions and the dark image regions within the ocular fundus image that match the given intensity threshold t.

34. The method of claim 31 wherein the processing circuitry includes extremal region find hardware that is configured to receive the labels for the light image regions and the dark image regions and find extremal regions based on the labels for the light image regions and the dark image regions.

35. The method of claim 34 wherein the extremal region find hardware is configured to find extremal regions using a mathematical relationship q(t)=|Q(t +Δ

)\Q(t −

Δ

)|/|Q(t)| where each extremal region'"'"'s cardinality |Q(t)| is a function of an intensity threshold t and an intensity threshold increment Δ

.

36. The method of claim 25 wherein the processing circuitry includes MSER selector hardware configured to automatically select the MSERs based on the MSER criteria.