Method of object recognition in image data using combined edge magnitude and edge direction analysis techniques

• US 8,160,382 B2
• Filed: 10/15/2007
• Issued: 04/17/2012
• Est. Priority Date: 10/15/2007
• Status: Expired due to Fees

• Current Assignee: Lockheed Martin Corporation (Martin Marietta Corporation)
• Original Assignee: Lockheed Martin Corporation (Martin Marietta Corporation)
• Inventors: Sefcik, Jason A.
• Primary Examiner(s): Allison, Andrae S
• Application Number:
  US11/907,596
• Publication Number:
  US 20090226033A1
• Time in Patent Office: 1,646 Days
• Field of Search: 382/103, 382/104, 382/118, 382/141, 382/157, 382/190, 382/199, 382/224, 382/225, 382/226, 382/284, 382/291, 250/330, 250/334, 250/342, 345/7, 345/449, 348/149, 348/458
• US Class Current: 382/266
• CPC Class Codes: G06T 2207/10048 Infrared image
• CPC Class Codes: G06T 2207/30212 Military
• CPC Class Codes: G06T 2207/30232 Surveillance
• CPC Class Codes: G06T 7/12 Edge-based segmentation
• CPC Class Codes: G06T 7/194 involving foreground-background segmentation
• CPC Class Codes: G06V 10/25 Determination of region of interest [ROI] or a volume of interest [VOI]

Abstract

Methods for detecting areas of interest in an image using combined edge magnitude and edge direction analysis techniques are presented. One embodiment features using thermal imaging data to detect hotspots in maritime settings that may be potential targets for tracking or weapons systems. The edge magnitude and edge direction data are derived from the intensity image and then combined with the intensity image and analyzed morphologically to remove noise and background elements. The combined image data is then selectively filtered to remove horizontal non-target elements and then analyzed further against target size information to determine which detected and analyzed hotspots are valid targets. Another embodiment features receiving as input an intensity image along with its associated edge magnitude and edge direction images, which have both been created by a means outside the detection method. Yet another embodiment features a detection method that does not selectively filter out horizontal image elements.

Claims

1. A method for identifying targets by detecting and analyzing hotspots in 2-dimensional infrared (IR) images through the combined use of Sobel magnitude and Sobel direction images comprising:

receiving as input an infra-red (IR) intensity image, a thresholded Sobel magnitude image, a thresholded Sobel direction image, and target size parameters;

detecting potential targets by morphologically processing said Sobel magnitude image and combining it with said intensity image to create a masked image including detected hotspots;

filtering out incorrectly detected hotspots by checking each hotspot for a minimum number of distinct directions in said Sobel direction image and removing potential distortions from horizon or littoral lines; and

determining if the filtered hotspots are valid targets by evaluating the filtered hotspots against at least one of intensity, size, or width thresholds contained in said target size parameters, and discarding as non-targets the filtered hotspots that are not within a certain percentage of said thresholds.

2. A method for detecting areas of interest in an image by combining edge magnitude and direction information comprising:

receiving input image data and detection parameters;

detecting potential areas of interest using edge magnitude by

morphologically processing said magnitude image to fill gaps and eliminate noise from the image data;

combining said morphologically processed image and original intensity image into a masked image; and

identifying and counting said areas of interest in said masked image;

filtering out incorrectly identified areas of interest using edge direction and edge magnitude; and

determining if said areas of interest meet said detection parameters.

3. The method of claim 2, receiving input image data comprising detecting magnitude and direction of edges in said image to generate a thresholded magnitude image and a thresholded direction image with a threshold value contained in said detection parameters.

4. The method of claim 2, said detection parameters comprising target size information.

5. The method of claim 2, said input image data comprising an IR intensity image.

6. The method of claim 2, said morphologically processing said magnitude image to fill gaps and eliminate noise from image data comprising:

binarizing said magnitude image to create a binarized image; and

using a neighborhood of zeros and ones to dilate and erode said binarized image a predetermined number of times.

7. The method of claim 6, said predetermined number of dilation and erosion operations being determined by number of targets known to be present based on said detection parameters.

8. The method of claim 6, said morphologically processing said magnitude image to fill gaps and eliminate noise from image data further comprising removing singleton pixels from said binarized image after said predetermined number of dilation and erosion operations.

9. The method of claim 2, said combining said morphologically processed image and original intensity image into a masked image comprising multiplying said morphologically processed and original intensity images together.

10. The method of claim 2, said identifying and counting said areas of interest in said masked image comprising morphological image analysis on said masked image to identify contiguous groups of pixels in said masked image.

11. The method of claim 2, said determining if said areas of interest meet said detection parameters comprising:

applying threshold parameters to remaining detected areas of interest;

calculating centroid, size, and width of said remaining detected areas of interest; and

discarding said remaining detected areas of interest which do not meet size a threshold.

12. The method of claim 11, said discarding said remaining detected areas of interest which do not meet size thresholds comprising comparing said calculated centroid, size, and width of said remaining detected area of interest against a predefined size value and discarding said remaining detected area of interest if said remaining detected area of interest is not within a certain percentage of an acceptable size value.

13. A method for detecting areas of interest in an image by combining edge magnitude and direction information comprising:

receiving input image data and detection parameters;

detecting potential areas of interest using edge magnitude;

filtering out incorrectly identified areas of interest using edge direction and edge magnitude by

evaluating a detected area of interest with a directional constraints filter;

removing background elements from said detected area of interest;

enhancing said detected area of interest with a weighting factor; and

performing a convolution on said detected area of interest with a predefined kernel; and

determining if said areas of interest meet said detection parameters.

14. The method of claim 13, said directional constraints filter comprising:

extracting pixels of said detected area of interest from said direction image;

counting how many separate, distinct directions said pixels in said detected area of interest have in said direction image;

comparing said count of separate, distinct directions to a minimum threshold value; and

removing from further consideration as an area of interest any said detected area are of interest with a count of said separate, distinct directions less than said minimum threshold value.

15. The method of claim 13, said removing background elements from said detected area of interest comprising removing pixels with horizontal gradient direction from said detected area of interest if said detected area of interest occupies more than a predetermined percentage of the horizontal field of view of said input image data and more than a certain percentage of pixels in said detected area of interest have horizontal gradient direction.

16. The method of claim 15, said removing background elements from said detected area of interest further comprising dilating remaining pixels in said detected area of interest a predetermined number of times after said removal of pixels with horizontal gradient direction from said detected area of interest.

17. The method of claim 13, said removing background elements from said detected area of interest comprising removing pixels with a specific horizontal gradient direction from said detected area of interest if said detected area of interest touches a vertical edge of said image input data and said detected area of interest occupies a certain percentage of the horizontal field of view of said image and a predetermined number of leftmost pixels of said detected area of interest have a specific horizontal gradient direction.

18. The method of claim 17, said removing background elements from said detected area of interest further comprising dilating remaining pixels in said detected area of interest a predetermined number of times after said removal of pixels with said specific horizontal gradient direction from said detected area of interest.

19. The method of claim 13, said enhancing said detected area of interest with a weighting factor comprising:

binarizing said detected area of interest;

calculating said weighting factor from mean value of pixels analogous to said pixels in said detected area of interest in said magnitude image; and

multiplying said analogous pixels from said masked image with said binarized detected area of interest and with said weighting factor.

20. The method of claim 13, said determining if said areas of interest meet said detection parameters comprising:

applying threshold parameters to remaining detected areas of interest;

calculating centroid, size, and width of said remaining detected areas of interest; and

discarding said remaining detected areas of interest which do not meet size a threshold.

21. A method for detecting areas of interest in an image by combining edge magnitude and direction information comprising:

receiving input image data and detection parameters;

detecting potential areas of interest using edge magnitude;

filtering out incorrectly identified areas of interest using edge direction and edge magnitude; and

determining if said areas of interest meet said detection parameters by

applying threshold parameters to remaining detected areas of interest;

calculating centroid, size, and width of said remaining detected areas of interest; and

discarding said remaining detected areas of interest which do not meet size a threshold;

where said applying threshold parameters to remaining detected areas of interest includes

identifying and counting said remaining detected areas of interest using morphological image analysis to identify contiguous groups of pixels; and

generating and applying a threshold value to said remaining detected areas of interest if;

more than one said detected area of interest remains,

horizontal pixels were removed from said remaining detected area of interest,

said removing background elements from said detected area of interest process indicated the presence of removable background elements in said image data, or

said detection parameters indicate the presence of known areas of interest separated in said image data by less than the size of one of said known areas of interest.

22. The method of claim 21, said generating and applying a threshold value to said remaining detected areas of interest comprising:

generating a threshold value;

applying said threshold value; and

iterating said steps of threshold value generation and application as determined by a threshold factor.

23. The method of claim 22, said generating a threshold value comprising:

calculating image statistics based on said remaining detected areas of interest;

setting said threshold factor value based on said calculated image statistics and said detection parameters; and

calculating a threshold value based on said calculated image statistics and said threshold factor value.

24. The method of claim 22, said applying said threshold value comprising:

removing pixels from said remaining detected areas of interest if said pixels have a value less than said threshold value;

comparing number of said remaining detected areas of interest after pixel removal against a predefined number of areas of interest and checking is said threshold factor value exceeds a defined minimum value if said number of areas of interest remaining does not meet or exceed said predefined number of areas after pixel removal

iterating said steps of threshold value generation if said threshold value exceeds said defined minimum value and said number of said remaining detected areas of interest after pixel removal is less than said predefined number of areas of interest.

25. A non-transitory computer readable medium having stored thereon a computer executable program for detecting areas of interest in an image, comprising:

receiving input image data and detection parameters;

detecting magnitude and direction of edges in said image to generate a magnitude image and a direction image;

detecting potential areas of interest using edge magnitude by

morphologically processing said magnitude image to fill gaps and eliminate noise from the image data;

combining said morphologically processed image and original intensity image into a masked image; and

identifying and counting said areas of interest in said masked image;

filtering out incorrectly identified areas of interest using edge direction and edge magnitude; and

determining if said areas of interest meet said detection parameters.