Target detection method and system

• US 7,430,303 B2
• Filed: 03/28/2003
• Issued: 09/30/2008
• Est. Priority Date: 03/29/2002
• Status: Expired due to Fees

• Current Assignee: Lockheed Martin Corporation (Martin Marietta Corporation)
• Original Assignee: Lockheed Martin Corporation (Martin Marietta Corporation)
• Inventors: Lee, Harry C., Olson, Teresa L., Sefcik, Jason
• Primary Examiner(s): AZARIAN, SEYED H
• Application Number:
  US10/400,536
• Publication Number:
  US 20030185420A1
• Time in Patent Office: 2,013 Days
• Field of Search: 382/103, 382/106, 382/107, 382/168, 382/169, 382/172, 382/178, 382/193, 382/194, 382/199, 382/201, 382/203, 382/209, 382/236, 382/243, 382/259, 382/274, 382/276, 382/237, 382/149, 382/174, 382/181, 382/305, 345/419, 342/64, 348/94, 348/97
• US Class Current: 382/103
• CPC Class Codes: G06V 10/255 Detecting or recognising potential candidate objects based on visual cues, e.g. shapes

Abstract

A method and system detects candidate targets or objects from a viewed scene by simplifying the data, converting the data to gradient magnitude and direction data which is thresholded to simplify the data. Horizontal edges within the data are softened to reduce their masking of adjacent non-horizontal features. One or more target boxes are stepped across the image data and the number of directions of gradient direction data within the box is used to determine the presence of a target. Atmospheric attenuation is compensated. The thresholding used in one embodiment compares the gradient magnitude data to a localized threshold calculated from the local variance of the image gradient magnitude data. Imagery subsets are containing the candidate targets may then be used to detect and identify features and apply a classifier function to screen candidate detections and determine a likely target.

Claims

1. A method for detecting targets within an image comprising:

receiving image data;

applying a gradient operator to determine gradient magnitude and direction of the pixels of said image data to produce gradient image data;

skeletonizing the gradient image data to remove unnecessary detail to produce skeletonized image data representing the image primarily by the edges of the objects contained therein;

applying atmospheric compensation to said gradient image data to adjust the skeletonized image data magnitudes in relationship to range;

defining at least one target size box related to the expected target size in the image data;

establishing a gradient threshold level under which the image data is determined to be unrelated to a target edge;

thresholding the skeletonized image data by comparing the skeletonized image data magnitudes to said gradient threshold and producing simplified image data representative of edges of image elements when said skeletonized image data magnitude exceeds said gradient threshold determined in said step of establishing; and

identifying candidate targets within said skeletonized image data by examining the number of different edge directions within the data defined by the target box size surrounding each pixel of each image data element of said simplified image data based on the number of directions of edge image data present in a region of said image.

2. The method of claim 1 wherein said step of applying said gradient operator applies a Sobel operator to said image data.

3. The method of claim 1 further comprising downsampling said image data to reduce data volume prior to processing of the data.

4. The method of claim 1 further comprising attenuating horizontally oriented gradients to attenuate any undue masking effect the horizontal gradients may have on adjacent non-horizontal pixels.

5. The method of claim 4 wherein said step of attenuating is performed before said step of thresholding.

6. The method of claim 1 further comprising the step of removing artifacts within said gradient image data caused by edge effects by padding or clearing edges of the image so that edges of the image have zero data values.

7. The method of claim 1 further comprising simplifying the skeletonized image data by removing rows defining image data beyond a target interest range.

8. The method of claim 1 wherein the target box size defined in said step of defining is one of plural target box sizes related to the expected minimum, average and maximum target box sizes of expected targets.

9. The method of claim 1 further comprising:

processing the image data of said candidate targets within said image subsets to detect and identify features; and

applying a classifier function to screen candidate detections and determine a likely target.

10. The method of claim 1 wherein said step of identifying candidate targets determines the number of edge directions in a region using a logical operator.

11. The method of claim 1 wherein, prior to the step of thresholding, single pixels having direction data non adjacent to at least the direction data of an adjacent pixel are zeroed.

12. The method of claim 1 wherein said step of skeletonizing includes thinning detected edges by zeroing out the magnitude and direction data of pixels adjacent to detected edges.

13. The method of claim 1 wherein said step of skeletonizing determines that an edge is present when the gradient magnitude has a localized maximum value in the gradient direction.

14. The method of claim 10 wherein the number of edge directions within a target size box used to identify a candidate target varies with range.

15. The method of claim 1 wherein said vertical and horizontal box size is determined based on an estimate of target data size for each row of the image data and is based on target range.

16. The method of claim 12 wherein the step establishing a gradient threshold level uses a histogram of the gradient magnitude of a box size to determine threshold.

17. The method of claim 12 wherein said percentage threshold varies across the image.

18. The method of claim 17 wherein said percentage threshold is determined as a relationship to localized variance of the gradient image data.

19. The method of claim 1 wherein said gradient threshold is varied based on local gradient variance within the image data of the target size box about each image pixel of said skeletonized image data.

20. A system for detecting targets within an image from image data supplied thereto comprising:

a gradient processor applying a gradient operator to determine gradient magnitude and direction of the pixels of said image data to produce gradient image data;

a skeletonizer removing unnecessary detail from the gradient image data to produce skeletonized image data representing the image primarily by the edges of the objects contained therein;

atmospheric compensator compensating for range related atmospheric attenuation in said gradient image data by adjusting the skeletonized image data magnitudes in an amount related to range;

a thresholder eliminating data that falls below a predetermined threshold within the skeletonized image data by comparing the skeletonized image data magnitudes to a gradient threshold and producing simplified image data representative of edges of image elements when said skeletonized image data magnitude exceeds said gradient threshold, the gradient threshold being determined to eliminate data having gradient values lower than that expected from a target edge; and

a candidate target identifier identifying candidate targets by examining the number of edge directions within the data defined by the target box size surrounding each pixel of said simplified image data based on the number of directions of edge image data present in a region of said image.

21. The system of claim 20 wherein said gradient processor applies a Sobel operator to said image data.

22. The system of claim 20 further comprising a data downsampler downsampling said image data to reduce data volume prior to processing of the data.

23. The system of claim 20 further comprising a horizontal edge attenuator attenuating horizontally oriented gradients to attenuate any undue masking effect the horizontal gradients may have on adjacent non-horizontal pixels.

24. The system of claim 23 wherein said horizontal edge attenuator operates on said data prior to supply to said thresholder.

25. The system of claim 20 further comprising a edge zeroer providing zeros at the edge of said image data to prevent within said gradient image data caused by edge effects by padding or clearing edges of the image so that edges of the image have zero data values.

26. The system of claim 20 further comprising a data simplifier removing rows of the skeletonized image data defining image data beyond a target interest range.

27. The system of claim 20 wherein the target box size used by said candidate target identifier is one of plural target box sizes related to the expected minimum, average and maximum target box sizes of expected targets.

28. The system of claim 20 further comprising:

a feature identifier processing the image data of said candidate targets within said image subsets to detect and identify features; and

a classifier applying a classifier function to screen candidate detections and determine a likely target.

29. The system of claim 20 further comprising a single pixel data eliminator, provided before said thresholder, for zeroing single pixels having direction data non adjacent to at least the direction data of one adjacent pixel.

30. The system of claim 20 wherein said skeletonizer thins image edges by zeroing out the magnitude and direction data of pixels adjacent to detected edges.

31. The system of claim 20 wherein said threshold used by said thresholded varies across the image.

32. The system of claim 31 wherein said thresholder determines the threshold at each location in the image based on localized variance of the gradient image data.

33. The method of claim 32 wherein said gradient threshold used n said thresholder is varied based on local gradient variance within the image data of the target size box about each image pixel of said skeletonized image data.

34. In a computer processing environment, a computer data storing medium storing a computer program, the computer program, when controlling the computer processing environment, performing the steps of:

receiving image data;

applying a gradient operator to determine gradient magnitude and direction of the pixels of said image data to produce gradient image data;

skeletonizing the gradient image data to remove unnecessary detail to produce skeletonized image data representing the image primarily by the edges of the objects contained therein;

applying atmospheric compensation to said gradient image data to adjust the skeletonized image data magnitudes in relationship to range;

defining at least one target size box related to the expected target size in the image data;

establishing a gradient threshold level under which the image data is determined to be unrelated to a target edge;

thresholding the skeletonized image data by comparing the skeletonized image data magnitudes to said gradient threshold and producing simplified image data representative of edges of image elements when said skeletonized image data magnitude exceeds said gradient threshold determined in said step of establishing; and

identifying candidate targets within said skeletonized image data by examining the number of different edge directions within the data defined by the target box size surrounding each pixel of each image data element of said simplified image data based on the number of directions of edge image data present in a region of said image.

35. A method for detecting targets within an image comprising:

receiving image data;

applying a gradient operator to determine gradient magnitude and direction of the pixels of said image data;

establishing a gradient threshold level under which the image data is determined to be unrelated to a target edge, said gradient threshold being varied based on local gradient variance within the image data of a single image view;

comparing the gradient magnitude of said image data to said gradient threshold to produce simplified image data representative of edges of image elements; and

identifying candidate targets within said image based on the number of directions of edge image data present in a region of said image, wherein

said step of comparing eliminates the gradient magnitude and direction data of pixels having gradient magnitude data below the gradient threshold.

36. The method of claim 35 further comprising:

identifying targets from said identified candidate targets.

37. The method of claim 35 wherein said step of identifying candidate targets includes the substeps of,

determining a target box size related to expected size of the targets to be detected,

determining the number of directions of edge image data present within the gradient direction data in said target box, and

identifying a candidate target if the number of directions of edge image data exceeds a threshold.

38. The method of claim 37 wherein said step of identifying includes the substep of determining a target box size less than the entirety of the image data, wherein said local threshold is determined from a histogram of gradient magnitude with the target box.

39. The method of claim 35 wherein the step of establishing a gradient threshold level includes the substeps of,

i) determining a portion of pixels within the subset of the image to be retained to determine a threshold value for each subset of said image, the threshold value being related to local conditions of the pixel magnitude data,

ii) repeating said steps i) and ii) for all subsets of the image to establish threshold values for each subset of the image,

iii) associating the threshold values of each subset with a pixel at the center of the subset to produce a threshold value table for pixels in said image data,

iv) performing two dimensional bi-linear interpolation on the threshold values in said substep iii) to associate a threshold value with each of the pixels.

40. The method of claim 39 wherein said step i) of determining determines a percentage value of pixels to be retained in the subset to produce a percentage threshold value, the method including the substep of v) converting each percentage threshold value to an image magnitude threshold value that will retain the desired percentage of pixels when applied.

41. The method of claim 39 wherein said step of establishing a gradient threshold level includes the substep vi) of determining a localized gradient magnitude variance within a subset of the image to determine a percentage threshold value for that subset of the image data.

42. A system for detecting targets within an image from received image data comprising:

a gradient processor applying a gradient operator to determine gradient magnitude and direction of the pixels of said image data to produce gradient image data;

a threshold determiner establishing a gradient threshold level under which the image data is determined to be unrelated to a target edge, said gradient threshold being varied based on local gradient variance within the image data of a single image view;

a thresholder comparing the gradient magnitude of said image data to said gradient threshold produced by said threshold determiner to produce simplified image data representative of edges of image elements; and

a candidate target identifier identifying candidate targets by examining the number of edge directions within the data present in a region of said image, wherein

said thresholder eliminates the gradient magnitude and direction data of pixels having gradient magnitude data below the gradient threshold.

43. The system of claim 42 further comprising:

a target identifier identifying targets from said candidate targets identified by said candidate target identifier.

44. The system of claim 42 wherein said candidate target identifier includes,

a target box size definer defining a target box size related to expected size of the targets to be detected,

a summer counting the number of directions of edge image data present within the gradient direction data in said target box, and

said candidate target identifier identifying a candidate target if the number of directions of edge image data exceeds a threshold.

45. The system of claim 44 wherein the target box size definer defines a target box size less than the entirety of the image data, wherein said gradient threshold is determined from a histogram of gradient magnitude with the target box.

46. The method of claim 42 wherein said threshold determiner includes,

a region thresholder determining a threshold value for each subset of said image, the threshold value being related to local conditions of the pixel magnitude data, said region value thresholder associating the threshold values of each subset with a pixel at the center of the subset to produce a threshold value table for pixels in said image data,

a two dimensional bi-linear interpolator performing interpolation on the threshold values in said substep iii) to associate a threshold value with each of the pixels.

47. The system of claim 45 wherein said threshold determiner determines the threshold for a subset of the image data based on a localized gradient magnitude variance within that subset of the image.

48. In a computer processing environment, a computer data storing medium storing a computer program, the computer program, when controlling the computer processing environment, performing the steps of:

receiving image data;

applying a gradient operator to determine gradient magnitude and direction of the pixels of said image data;

establishing a gradient threshold level under which the image data is determined to be unrelated to a target edge, said gradient threshold being varied based on local gradient variance within the image data of a single image view;

comparing the gradient magnitude of said image data to said gradient threshold to produce simplified image data representative of edges of image elements; and

identifying candidate targets within said image based on the number of directions of edge image data present in a region of said image, wherein

said step of comparing eliminates the gradient magnitude and direction data of pixels having gradient magnitude data below the gradient threshold.

49. A method for detecting targets within an image comprising:

receiving image data;

simplifying the image data to remove unnecessary detail to produce simplified image data representing the image primarily by the edges of the objects contained therein;

applying atmospheric compensation to said simplified image data to adjust the simplified image data magnitudes in an amount related to range, and

examining the simplified image data to investigate the presence of candidate targets therein, wherein

the step of applying the atmospheric compensation varies the amount of compensation in relationship to the amount of moisture in the atmosphere.

50. The method of claim 49 further comprising the step of

attenuating horizontally oriented edges in said simplified image data to attenuate any undue masking effect they may have on adjacent non-horizontal pixels.

51. The method of claim 50 wherein the step of simplifying the image data includes the substep of applying a gradient operator to determine gradient magnitude and direction of the pixels of said image data to produce gradient image data;

said method further comprising the step of attenuating horizontal elements of the image by attenuating the magnitude of gradient magnitude data where the direction data indicates a vertical gradient representing a generally horizontal edge in said image data.

55. The system of claim 49 wherein the atmospheric compensator varies the amount of compensation in relationship to the amount of moisture in the atmosphere.

52. A system for detecting targets within an image from received image data comprising:

a simplifier simplifying the image data by removing unnecessary detail to produce simplified image data representing the image primarily by the edges of the objects contained therein;

an atmospheric compensator applying atmospheric compensation to said simplified image data to adjust the simplified image data magnitudes in an amount related to range, and

a candidate target identifier examining the simplified image data to investigate the presence of candidate targets therein, wherein

the atmospheric compensator varies the amount of compensation in relationship to the amount of moisture in the atmosphere.

53. The system of claim 52 further comprising:

a horizontal edge compensator attenuating horizontally oriented edges in said simplified image data to attenuate any undue masking effect they may have on adjacent non-horizontal pixels.

54. The system of claim 52 further comprising a gradient processor applying a gradient operator to determine gradient magnitude and direction of the pixels of said image data to produce gradient image data.

56. A method for detecting targets within an image comprising:

receiving image data;

producing a target box sized in relation to the expected target size within the image data;

examining the image data defined by the target box surrounding a pixel of said image data to investigate the presence of a target within the target box surrounding that pixel by processing the image data within said target box to determine the presence of a candidate target; and

repeating the step of examining for pixels within the image view, wherein

said target box is somewhat greater than the expected target size in the image at that pixel, said step of producing produces plural target boxes of sizes differing in at least one dimension, said target boxes vary in size with varying range, and said image data is divided into zone of similar range to the pixels thereof, the target box size of a given target box being constant within a zone but varying between zones.

57. The method of claim 56 wherein said plural target boxes have box sizes which differ in relation to the change of target shape when viewed from differing dimensions, the step of examining being performed with each of the plural target boxes of differing sizes.

58. The method of claim 56 wherein said step of examining includes the substeps of,

applying a gradient operator to determine gradient magnitude and direction of the pixels of said image data to produce gradient image data,

skeletonizing the gradient image data to remove unnecessary detail to produce skeletonized image data representing the image primarily by the edges of the objects contained therein,

comparing said gradient magnitude data to a threshold and eliminating data falling below the threshold,

determining the number of directions of edge image data present within the skeletonized data in said target box, and

identifying a candidate target if the number of directions of edge image data exceeds a threshold.

59. A system for detecting targets within an image from image data thereof comprising:

a target box generator producing a target box sized in relation to the expected target size within the image data;

a candidate target detector examining the image data defined by the target box surrounding a pixel of said image data to investigate the presence of a target within the target box surrounding that pixel by processing the image data within said target box to determine the presence of a candidate target, the examination of the image data being performed for each pixel of the data, wherein

said target box generator produces plural target boxes of sizes differing in at least one dimension, and said target box generator produces target boxes vary in size with varying range.

60. The system of claim 59 wherein said target box is somewhat greater than the expected target size in the image at that pixel.

61. The system of claim 59 wherein said plural target boxes have box sizes which differ in relation to the change of target shape when viewed from differing dimensions, said candidate target detector examining the image data with each of the plural target boxes of differing sizes.

62. The system of claim 59 wherein said candidate target detector includes,

a gradient processor applying a gradient operator to determine gradient magnitude and direction of the pixels of said image data to produce gradient image data,

a skeletonizer skeletonizing the gradient image data to remove unnecessary detail to produce skeletonized image data representing the image primarily by the edges of the objects contained therein,

a thresholder comparing said gradient magnitude data to a threshold and eliminating data falling below the threshold,

a logical processor determining the number of directions of edge image data present within the skeletonized data in said target box, and

said candidate target detector identifying a candidate target if the number of directions of edge image data determined by said logical processor exceeds a threshold.

63. The system of claim 59 wherein said target box generator varies target box size by dividing image data into zones of similar range to the pixels thereof, the target box size of a given target box being constant within a zone but varying between zones.

64. In a computer processing environment, a computer data storing medium storing a computer program, the computer program, when controlling the computer processing environment, performing the steps of:

receiving image data;

producing a target box sized in relation to the expected target size within the image data;

examining the image data defined by the target box surrounding a pixel of said image data to investigate the presence of a target within the target box surrounding that pixel by processing the image data within said target box to determine the presence of a candidate target; and

repeating the step of examining for pixels within the image view, wherein

said target box is somewhat greater than the expected target size in the image at that pixel, said step of producing produces plural target boxes of sizes differing in at least one dimension, said target boxes vary in size with varying range, and said image data is divided into zone of similar range to the pixels thereof, the target box size of a given target box being constant within a zone but varying between zones.

65. A method for detecting targets within an image comprising:

receiving image data;

simplifying the image data to remove unnecessary detail to produce simplified image data representing the image primarily by the edges of the objects contained therein;

attenuating horizontally oriented edges in said simplified image data to attenuate any undue masking effect they may have on adjacent non-horizontal pixels;

examining the simplified image data to investigate the presence of candidate targets therein, and

applying atmospheric compensation to said simplified image data to attenuate skeletonized image data magnitudes in an amount inversely related to range.

66. The method of claim 65 further comprising the step of applying a gradient operator to determine gradient magnitude and direction of the pixels of said image data to produce gradient image data;

said step of attenuating the magnitude of gradient magnitude data where the direction data indicates a vertical gradient representing a generally horizontal edge in said image data.

67. The method of claim 65 wherein the step of examining includes the sub steps of,

applying a gradient operator to determine gradient magnitude and direction of the pixels of said image data to produce gradient image data,

defining at least one target size box related to the expected target size in the image data,

establishing a gradient threshold level under which the image data is determined to be unrelated to a target edge,

thresholding the skeletonized image data by comparing the skeletonized image data magnitudes to said gradient threshold and producing simplified image data representative of edges of image elements when said skeletonized image data magnitude exceeds said gradient threshold,

identifying candidate targets within said skeletonized image data by examining the number of different edge directions within the data defined by the target box size surrounding each pixel of each image data element of said simplified image data based on the number of directions of edge image data present in a region of said image, and

repeating the step of examining for pixels within the image view.

68. A system for detecting targets within an image from image data thereof comprising:

a simplifier simplifying the image data to remove unnecessary detail to produce simplified image data representing the image primarily by the edges of the objects contained therein;

a horizontal edge attenuator attenuating horizontally oriented edges in said simplified image data to attenuate any undue masking effect they may have on adjacent non-horizontal pixels;

a candidate target examiner examining the simplified image data to investigate the presence of candidate targets therein, and

an atmospheric compensator applying atmospheric compensation to said simplified image data to attenuate skeletonized image data magnitudes in an amount inversely related to range.

69. The system of claim 68 further comprising a gradient processor applying a gradient operator to determine gradient magnitude and direction of the pixels of said image data to produce gradient image data;

said horizontal edge attenuator attenuating the magnitude of gradient magnitude data where the direction data indicates a vertical gradient representing a generally horizontal edge in said image data.

70. The system of claim 68 wherein said candidate target examiner includes,

a gradient processor applying a gradient operator to determine gradient magnitude and direction of the pixels of said image data to produce gradient image data,

a target box definer defining at least one target size box related to the expected target size in the image data,

a threshold determiner establishing a gradient threshold level under which the image data is determined to be unrelated to a target edge,

a thresholder thresholding the skeletonized image data by comparing the skeletonized image data magnitudes to said gradient threshold and producing simplified image data representative of edges of image elements when said skeletonized image data magnitude exceeds said gradient threshold,

said candidate target identifier identifying candidate targets within said skeletonized image data by examining the number of different edge directions within the data defined by the target box size surrounding each pixel of each image data element of said simplified image data based on the number of directions of edge image data present in a region of said image.

71. In a computer processing environment, a computer data storing medium storing a computer program, the computer program, when controlling the computer processing environment, performing the steps of:

receiving image data;

simplifying the image data to remove unnecessary detail to produce simplified image data representing the image primarily by the edges of the objects contained therein;

attenuating horizontally oriented edges in said simplified image data to attenuate any undue masking effect they may have on adjacent non-horizontal pixels; and

examining the simplified image data to investigate the presence of candidate targets therein and

applying atmospheric compensation to said simplified image data to attenuate skeletonized image data magnitudes in an amount inversely related to range.

72. In a computer processing environment, a computer data storing medium storing a computer program, the computer program, when controlling the computer processing environment, performing the steps of:

receiving data;

simplifying the image data to remove unnecessary detail to produce simplified mage data representing the image primarily by the edges of the objects contained therein;

applying atmospheric compensation to said simplified image data to adjust the simplified image data magnitude in an amount related to range, and

examining the simplified image data to investigate the presence of candidate targets therein, wherein the step of applying the atmospheric compensation varies the amount of compensation in relationship to the amount of moisture in the atmosphere.