HARDWARE ARCHITECTURE FOR REAL-TIME EXTRACTION OF MAXIMALLY STABLE EXTREMAL REGIONS (MSERs)

• US 20160071280A1
• Filed: 09/10/2014
• Published: 03/10/2016
• Est. Priority Date: 09/10/2014
• Status: Active Grant

• Current Assignee: Khalifa University Of Science, Technology & Research
• Original Assignee: Khalifa University Of Science, Technology & Research
• Inventors: Salahat, Ehab Najeh, Saleh, Hani Hasan Mustafa, Sluzek, Andrzej Stefan, Al-Qutayri, Mahmoud, Mohammad, Baker, Elnaggar, Mohammed Ismail
• Granted Patent: US 9,489,578 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06T 2207/30232 Surveillance
• CPC Class Codes: G06V 10/267 by performing operations on regions, e.g. growing, shrinking or watersheds
• CPC Class Codes: G06V 10/955 using specific electronic processors
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69

Abstract

Hardware architecture for real-time extraction of maximally stable extremal regions (MSERs) is disclosed. The architecture includes a communication interface and processing circuitry that are configured in hardware to receive a data stream of an intensity image in real-time and provide labels for image regions within the intensity image that match a given intensity threshold. The communication interface and processing circuitry are also configured in hardware to find extremal regions within the intensity image based upon the labels and to determine MSER ellipses parameters based upon the extremal regions and MSER criteria. In at least one embodiment, the MSER criteria include minimum and maximum MSER areas, and an acceptable growth rate value for MSER area. In another embodiment, the MSER criteria include a nested MSER tolerance value.

Claims

1. An architecture for real-time extraction of maximally stable extremal regions (MSERs) comprising a communication interface and processing circuitry configured in hardware to in real-time:

receive a data stream of an intensity image via the communication interface;

provide labels for image regions within the intensity image that match a given intensity threshold;

find extremal regions within the intensity image based upon the labels; and

determine MSER ellipses parameters based on the extremal regions and MSER criteria.

2. The architecture of claim 1 wherein the MSER criteria include a nested MSER tolerance value.

3. The architecture of claim 2 wherein the MSER criteria further include a minimum MSER area, a maximum MSER area, and an acceptable growth rate value for MSER area.

4. The architecture of claim 1 wherein the MSER ellipses parameters include a center of gravity, a major axis length, a minor axis length, and an angle of the major axis length with respect to a horizontal axis.

5. The architecture of claim 1 wherein the processing circuitry includes MSER moments calculator hardware configured to calculate MSER moments.

6. The architecture of claim 5 wherein the processing circuitry further includes elliptical fit approximator hardware configured to receive MSER moments from the MSER moments calculator hardware and fit an MSER ellipse to an extremal region based upon the MSER moments.

7. The architecture of claim 1 wherein the processing circuitry includes union-find hardware configured to provide the labels for the image regions within the intensity image that match the given intensity threshold.

8. The architecture of claim 7 wherein the processing circuitry includes extremal region find hardware that is configured to receive the labels for the image regions and find extremal regions based upon the labels for the image regions.

9. The architecture of claim 8 wherein the extremal region find hardware is configured to find extremal regions using a mathematical relationship q(t)=|Q(t+Δ

)\Q(t−

Δ

)|/|Q(t)|, where each extremal region'"'"'s cardinality, |Q(t)| is a function of an intensity threshold t.

10. The architecture of claim 1 wherein the processing circuitry includes MSER selector hardware configured to automatically select MSERs based upon the MSER criteria.

14. The architecture of claim 1 wherein the MSER ellipses parameters include a center of gravity, a major axis length, a minor axis length, and an angle of the major axis length with respect to a horizontal axis.

11. An architecture for real-time extraction of maximally stable extremal regions (MSERs) comprising:

intensity image process hardware configured to receive a data stream of an intensity image and output labels for image regions within the intensity image that match a given intensity threshold;

extremal regions find hardware configured to receive the labels for the intensity image and find extremal regions within the intensity image; and

MSER process hardware configured to receive MSER criteria and output MSER ellipses parameters based upon the extremal regions.

12. The architecture of claim 11 wherein the MSER criteria include a nested MSER tolerance value.

13. The architecture of claim 12 wherein the MSER criteria further include a minimum MSER area value, a maximum MSER area value, and an acceptable growth rate value for MSER areas.

15. The architecture of claim 11 wherein the intensity image process hardware includes union-find hardware configured to label region seeds.

16. The architecture of claim 11 wherein the extremal regions find hardware is configured to find extremal regions using a mathematical relationship q(t)=|Q(t+Δ

)\Q(t−

Δ

)|/|Q(t)|, where each extremal region'"'"'s cardinality, |Q(t)| is a function of an intensity threshold t.

17. The architecture of claim 11 wherein the intensity image process hardware includes union-find hardware configured to provide the labels for the image regions within the intensity image that match a given intensity threshold.

18. The architecture of claim 17 wherein the intensity image process hardware further includes labeled region seeds updater/unifier hardware configured to prevent a seed that is a first pixel location within the intensity image from being stored in a seed list, if the seed is presently stored in the seed list.

19. The architecture of claim 18 further including region map updater hardware configured to store a value of Q(t+Δ

), Q(t), and Q(t−

Δ

) for each seed, where t is an intensity threshold and Δ

is an increment of the intensity threshold t.

20. The architecture of claim 11 wherein the intensity image process hardware, the extremal regions find hardware, and the MSER process hardware are fabricated on a single application specific integrated circuit (ASIC).

21. The architecture of claim 11 wherein the intensity image process hardware, the extremal regions find hardware and the MSER process hardware are implemented on a single field programmable gate array (FPGA).

22. A method for real-time extraction of maximally stable extremal regions (MSERs) via processing circuitry comprising:

receiving a data stream of an intensity image via a communication interface in communication with the processing circuitry;

generating labels for image regions within the intensity image that match a given intensity threshold in real-time via the processing circuitry;

finding extremal regions within the intensity image based upon the labels in real-time via the processing circuitry; and

determining MSER ellipses parameters based on the extremal regions and MSER criteria in real-time via the processing circuitry.

23. The method for real-time extraction of maximally stable extremal regions (MSERs) via the processing circuitry of claim 22 wherein the MSER criteria include a nested MSER tolerance value.

24. The method for real-time extraction of maximally stable extremal regions (MSERs) via the processing circuitry of claim 23 wherein the MSER criteria further include a minimum MSER area, a maximum MSER area, and an acceptable growth rate value for MSER areas.

25. The method for real-time extraction of maximally stable extremal regions (MSERs) via the processing circuitry of claim 22 wherein the MSER ellipses parameters include a center of gravity, a major axis length, a minor axis length, and an angle of the major axis length with respect to a horizontal axis.

26. The method for real-time extraction of maximally stable extremal regions (MSERs) via the processing circuitry of claim 22 wherein the processing circuitry includes MSER moments calculator hardware configured to calculate MSER moments.

27. The method for real-time extraction of maximally stable extremal regions (MSERs) via the processing circuitry of claim 26 wherein the processing circuitry further includes elliptical fit approximator hardware configured to receive MSER moments from the MSER moments calculator hardware and fit an MSER ellipse to an extremal region based upon the MSER moments.

28. The method for real-time extraction of maximally stable extremal regions (MSERs) via the processing circuitry of claim 22 wherein the processing circuitry includes union-find hardware configured to provide the labels for the image regions within the intensity image that match a given intensity threshold.

29. The method for real-time extraction of maximally stable extremal regions (MSERs) via the processing circuitry of claim 28 wherein the processing circuitry includes extremal region find hardware that is configured to receive the labels for the image regions and find extremal regions based upon the labels for the image regions.

30. The method for real-time extraction of maximally stable extremal regions (MSERs) via the processing circuitry of claim 29 wherein the extremal region find hardware is configured to find extremal regions using a mathematical relationship q(t)=|Q(t+Δ

)\Q(t−

Δ

)|/|Q(t)|, where each extremal region'"'"'s cardinality, |Q(t)| is a function of an intensity threshold t.

31. The method for real-time extraction of maximally stable extremal regions (MSERs) via the processing circuitry of claim 22 wherein the processing circuitry includes MSER selector hardware configured to automatically select MSERs based upon the MSER criteria.