Method and system for data fusion using spatial and temporal diversity between sensors

• US 20030184468A1
• Filed: 03/25/2003
• Published: 10/02/2003
• Est. Priority Date: 03/26/2002
• Status: Active Grant

• Current Assignee: Lockheed Martin Corporation (Martin Marietta Corporation)
• Original Assignee: Lockheed Martin Corporation (Martin Marietta Corporation)
• Inventors: Chen, Hai-Wen, Olson, Teresa L.
• Granted Patent: US 6,909,997 B2
• Time in Patent Office: Days
• US Class Current: 342/52
• CPC Class Codes: G06F 18/25 Fusion techniques
• CPC Class Codes: G06F 18/251 of input or preprocessed data
• CPC Class Codes: G06F 18/256 of results relating to different input data, e.g. multimodal recognition
• CPC Class Codes: G06T 2207/10048 Infrared image
• CPC Class Codes: G06T 5/009 Global, i.e. based on properties of the image as a whole applied in cameras using an electronic image sensor H04N23/80, H04N23/70
• CPC Class Codes: G06T 5/40 using histogram techniques
• CPC Class Codes: G06T 5/92 based on global image properties

Abstract

A method and system provide a multi-sensor data fusion system capable of adaptively weighting the contributions from each one of a plurality of sensors using a plurality of data fusion methods. During a predetermined tracking period, the system receives data from each individual sensor and each data fusion method is performed to determine a plurality of reliability functions for the system based on combining each sensor reliability function which are individually weighted based on the S/N (signal-to-noise) ratio for the received data from each sensor, and a comparison of predetermined sensor operation characteristics for each sensor and a best performing (most reliable) sensor. The system may dynamically select to use one or a predetermined combination of the generated reliability functions as the current (best) reliability function which provides a confidence level for the multi-sensor system relating to the correct classification (recognition) of targets and decoys.

Claims

1. A method for integrating data received from a plurality of sensors, comprising:

receiving data from a plurality of sensors;

determining a S/N (signal-to-noise) ratio for each sensor based on signal measurements of the received data;

determining a plurality of reliability functions for the plurality of sensors from using a plurality of predetermined calculations combining each sensor reliability function which are determined and individually weighted based on said S/N ratio for each sensor and differences between predetermined operating characteristics for each sensor; and

selecting one or a predetermined combination of said plurality of reliability functions as a current reliability function based on the selected one or predetermined combination satisfying predetermined thresholds.

2. The method of claim 1, further comprising:

determining to delay said selecting for a predetermined time period based on said selected at least one failing to satisfy said predetermined threshold.

3. The method of claim 1, wherein said determining a plurality of reliability functions includes determining said plurality of reliability functions from using one of a predetermined additive, multiplicative, and fuzzy logic calculation combing each sensor reliability function.

4. The method of claim 3, wherein said determining a plurality of reliability functions includes determining empty sets and ignorance sets for the received data from the plurality of sensors based on said predetermined multiplicative calculation.

5. The method of claim 4, wherein said predetermined multiplicative calculation includes at least a modified portion of a Dempster'"'"'s data fusion algorithm.

6. The method of claim 1, wherein said determining a plurality of reliability functions includes determining said individual weighting based on a predetermined parameter for the plurality of sensors satisfying a predetermined threshold, and determining said plurality of reliability functions, based on using said plurality of predetermined calculations for a single sensor reliability function, when said predetermined parameter fails to satisfy said predetermined threshold.

7. The method of claim 6, wherein said predetermined parameter is a false alarm rate for the plurality of sensors.

8. The method of claim 1, wherein said determining a plurality of reliability functions includes determining said plurality of reliability functions based on differences between probabilities of classification for each sensor as compared to a single sensor having the highest probability of classification.

9. The method of claim 1, wherein said plurality of sensors includes at least one of a laser, IR (infrared), and RF (radio frequency) sensor.

10. The method of claim 1, wherein said determining a plurality of reliability functions includes determining said plurality of reliability functions based on determining a single sensor, as compared to the plurality of sensors, as having the best performance for at least one predetermined sensor parameter.

11. The method of claim 10, wherein said at least one predetermined sensor parameter is one of operating characteristics and S/N ratio for said single sensor.

12. The method of claim 11, wherein said single sensor is one of a laser, IR, and RF sensor.

13. The method of claim 1, wherein said selecting includes selecting said current reliability function to increase the probability of classifying one of a target and decoy above a predetermined threshold.

14. A multi-sensor system, comprising:

a plurality of sensors for receiving data; and

at least one controller for performing the steps of;

determining a S/N (signal-to-noise) ratio for each sensor based on signal measurements of the received data;

determining a plurality of reliability functions for the plurality of sensors from using a plurality of predetermined calculations combining each sensor reliability function which are determined and individually weighted based on said S/N ratio for each sensor and differences between predetermined operating characteristics for each sensor; and

selecting one or a predetermined combination of said plurality of reliability functions as a current reliability function based on the selected at least one satisfying a predetermined threshold.

15. The system of claim 14, wherein said controller to determine said individual weighting based on a predetermined parameter for the plurality of sensors satisfying a predetermined threshold, and said controller to determine said plurality of reliability functions, based on using said plurality of predetermined calculations for a single sensor reliability function, when said predetermined parameter fails to satisfy said predetermined threshold.

16. The system of claim 15, wherein said predetermined parameter is a false alarm rate for the plurality of sensors.

17. The system of claim 14, wherein said plurality of sensors includes at least one of a laser, IR (infrared) sensor, and RF (radio frequency) sensor.

18. A machine-readable medium having stored thereon a plurality of executable instructions, the plurality of instructions comprising instructions to:

receive data from a plurality of sensors;

determine a S/N (signal-to-noise) ratio for each sensor based on signal measurements of the received data;

determine a plurality of reliability functions for the plurality of sensors from using a plurality of predetermined calculations combining each sensor reliability function which are determined and individually weighted based on said S/N ratio for each sensor and differences between predetermined operating characteristics for each sensor; and

select at one or a predetermined combination of said plurality of reliability functions to use as a current reliability function based on the selected at least one satisfying a predetermined threshold.

19. The medium of claim 18, wherein said instructions to determine a plurality of reliability functions include instructions to determine said individual weighting based on a predetermined parameter for the plurality of sensors satisfying a predetermined threshold.

20. The medium of claim 18, wherein said instructions to determine a plurality of reliability functions include instructions to determine said plurality of reliability functions based on differences between probabilities of classification for each sensor as compared to a single sensor having the highest probability of classification.

21. The medium of claim 18, wherein said instructions to determine a plurality of reliability functions include instructions to determine said plurality of reliability functions from using one of a predetermined additive, multiplicative, and fuzzy logic calculation combining each sensor reliability function.

22. The medium of claim 18, wherein said instructions to select include instructions to select said current reliability function to increase the probability of classifying one of a target and a decoy above a predetermined threshold.