Target detection improvements using temporal integrations and spatial fusion

• US 7,742,620 B2
• Filed: 03/18/2004
• Issued: 06/22/2010
• Est. Priority Date: 03/21/2003
• Status: Expired due to Fees

• Current Assignee: Lockheed Martin Corporation (Martin Marietta Corporation)
• Original Assignee: Lockheed Martin Corporation (Martin Marietta Corporation)
• Inventors: Olson, Teresa L., Sutha, Surachai, Chen, Hai-Wen
• Primary Examiner(s): Do; Anh Hong
• Application Number:
  US10/555,104
• Publication Number:
  US 20070076917A1
• Time in Patent Office: 2,287 Days
• Field of Search: 382/103, 382/147, 382/149, 382/312, 342/52, 342/195, 342/53, 455/226.3, 702/116, 702/189, 702/190, 702/193, 324/164, 514/12, 375/240.12, 348/E5.042, 348/126, 348/87, 378/21, 378/22, 378/4, 378/137, 250/358.1
• US Class Current: 382/103
• CPC Class Codes: G06F 18/251 of input or preprocessed data
• CPC Class Codes: G06F 18/256 of results relating to different input data, e.g. multimodal recognition
• CPC Class Codes: G06V 10/255 Detecting or recognising potential candidate objects based on visual cues, e.g. shapes
• CPC Class Codes: G06V 10/803 of input or preprocessed data
• CPC Class Codes: G06V 10/811 the classifiers operating on different input data, e.g. multi-modal recognition

Abstract

A method for identifying potential targets as far away as possible is disclosed. In a simple background scene such as a blue sky, a target may be recognized from a relatively long distance, but for some high clutter situations such as mountains and cities, the detection range is severely reduced. The background clutter may also be non-stationary further complicating the detection of a target. To solve these problems, target detection (recognition) of the present invention is based upon temporal fusion (integration) of sensor data using pre-detection or post-detection integration techniques, instead of using the prior art technique of fusing data from only a single time frame. Also disclosed are double-thresholding and reversed-thresholding techniques which further enhance target detection and avoid the shortcomings of the traditional constant false alarm rate (CFAR) thresholding technique. The present invention further discloses improved spatial fusion techniques for target detection (recognition) employing multiple sensors instead of employing the more conventional single sensor techniques. If spatial fusion is implemented with more than three sensors, then target detection can be enhanced by also using post-detection techniques. Moreover, since the pre-detection and the post-detection technique are complementary to each other, a combination of these two integration techniques will further improve target detection (recognition) performance.

Claims

1. A method to identify a potential target from image data representing a scene, comprising:

receiving at least two frames of image data from at least one imaging sensor;

performing at least one of a pre-detection temporal fusion and a pre-detection spatial fusion of the frames of image data;

thresholding the fused image data after performing the step of performing; and

identifying candidate targets from the thresholded image data, wherein

the pre-detection temporal fusion comprises temporally integrating image data from a said imaging sensor across a plurality of time frames and the at least two frames of image data are from the sensor; and

the pre-detection spatial fusion comprises fusing the image data from a plurality of said imaging sensors across a single time frame and the at least two frames of image data include at least one frame of image data from two different sensors.

2. The method of claim 1, wherein the pre-detection temporal fusion and the pre-detection spatial fusion includes at least one of an additive fusion, multiplicative fusion, minimum fusion, and maximum fusion.

3. The method of claim 1, wherein the thresholding includes at least one of:

a double-thresholding wherein an upper and a lower bound thresholds are set to identify the potential target; and

a reverse-thresholding wherein a potential target identification level is set to be below a particular threshold.

4. A method to identify a potential target from image data of a scene, comprising:

receiving at least two frames of image data from at least one imaging sensor;

thresholding the frames of image data, wherein said frames of image data are frames of image data from across multiple time frames of said at least one sensor or frames of image data from a plurality of said sensors;

fusing the frames of image data after thresholding, by spatial fusion if the frames of image data are frames of image data from said plurality of sensors or by temporal fusion if the frames of image data are frames of image data from across multiple time frames of said at least one sensor; and

identifying candidate targets from the fused image data.

5. The method of claim 4, wherein thresholding includes at least one of:

double-thresholding the image data, where an upper and a lower bound threshold are set to identify the potential target; and

reverse-thresholding wherein a potential target identification level is set to be below a particular threshold.

6. A device to identify potential targets from at least two frames of image data generated by at least one imaging sensor and representative of a scene, comprising:

a fusion module configured to perform at least one of a temporal fusion and a spatial fusion of the generated frames of image data; and

a threshold module configured to apply thresholding techniques on the fused image data, wherein

the temporal fusion includes temporally fusing the frames of image data across a plurality of time frames and the at least two frames of image data are from the same sensor; and

the spatial fusion includes fusing the frames of image data across a single time frame and the at least two frames of image data are from different sensors.

7. The device of claim 6 wherein said device further includes at least one imaging sensor.

8. The device of claim 6, wherein fusion module is configured to perform at least one of an additive fusion, multiplicative fusion, minimum fusion, and maximum fusion.

9. The device of claim 6, wherein the fusion module is configured to perform at least one of a pre-detection fusion and a persistence test.

10. The device of claim 6, wherein the threshold module is configured to perform at least one of a double-thresholding technique and a reverse-thresholding technique.

11. A method to identify a potential target from data, comprising the steps of:

receiving as input data, a plurality of time frames of data from at least one sensor;

extracting, from said time frames of data, at least one feature;

performing a pre-detection technique on the least one extracted feature, where said pre-detection technique includes either a double threshold technique or a reverse threshold technique; and

determining whether said extracted feature is a potential target.

12. A method according to claim 11 wherein said determining includes performing a post-detection technique to determine whether a certain criteria has been met.

13. A method according to claim 11 wherein said pre-detection technique includes said double threshold technique, and further wherein said double-threshold technique includes setting a detection criteria having a lower bound threshold value and an upper bound threshold for determining whether an object corresponding to the extracted feature has a feature value between the lower bound threshold value and upper bound threshold value.

14. A method according to claim 11 wherein said pre-detection technique includes said reverse threshold technique, and further wherein said reverse threshold technique includes setting a detection criteria for a non-stationary object such that a mean value of an extracted feature is compared to a target mean value, and the extracted feature is determined to be a non-stationary object when its mean value is greater or lesser than the target mean value.

15. A method to identify a potential target from data, comprising the steps of:

receiving, as input, data from a plurality of sensors;

performing a pre-detection fusion technique on data corresponding to at least one extracted feature from each sensor;

wherein said pre-detection fusion technique includes at least one technique that is selected from a group comprised of additive fusion, multiplicative fusion, minimum fusion and maximum fusion; and

determining whether the pre-detection fused data is a potential target.

16. A method according to claim 15 which further includes the step of:

performing a post-detection technique when there is data from at least three sensors, said post-detection technique selected from a group consisting of a double threshold technique and a reverse threshold technique.

17. A method according to claim 16 wherein said double threshold includes setting a detection criteria having a lower bound threshold value and an upper bound threshold for determining whether an object corresponding to the extracted feature has a feature value between the lower bound threshold value and upper bound threshold value.

18. A method according to claim 16 wherein said reverse threshold technique includes setting a detection criteria for a non-stationary object such that a mean value of an extracted feature is compared to a target mean value, and the extracted feature is determined to be a non-stationary object when its mean value is greater or lesser than the target mean value.

19. A target detection apparatus, comprising:

a plurality of sensors for outputting data related to a target, said data from each sensor having a plurality of time frames;

temporal processing means for integrating the data supplied from each of said plurality of sensors;

spatial processing means for fusing the temporally integrated sensor data from said temporal processing means, wherein said spatial processing means detects the target from the spatially fused data and provides an indication corresponding to the detected target; and

means for utilizing the indication of the detected target.

20. The target detection apparatus of claim 19, the apparatus further comprising:

a threshold module that performs a pre-detection technique on at least one extracted feature, where said feature is extracted from said plurality of time frames of data of a given sensor; and

wherein said pre-detection technique includes either a double threshold technique or a reverse threshold technique.