Method and system for data fusion using spatial and temporal diversity between sensors

• US 6,909,997 B2
• Filed: 03/25/2003
• Issued: 06/21/2005
• Est. Priority Date: 03/26/2002
• Status: Expired due to Term

• Current Assignee: Lockheed Martin Corporation (Martin Marietta Corporation)
• Original Assignee: Lockheed Martin Corporation (Martin Marietta Corporation)
• Inventors: Chen, Hai-Wen, Olson, Teresa L.
• Primary Examiner(s): Hoff, Marc S.
• Assistant Examiner(s): SUAREZ, FELIX E
• Application Number:
  US10/395,269
• Publication Number:
  US 20030184468A1
• Time in Patent Office: 819 Days
• Field of Search: 702/190, 702/192, 702 51- 53, 702/187, 702/189, 702/191, 702/196, 700/31, 345/173, 701/214
• US Class Current: 702/189
• CPC Class Codes: G06F 18/25 Fusion techniques
• CPC Class Codes: G06F 18/251 of input or preprocessed data
• CPC Class Codes: G06F 18/256 of results relating to different input data, e.g. multimodal recognition
• CPC Class Codes: G06T 2207/10048 Infrared image
• CPC Class Codes: G06T 5/40 using histogram techniques
• CPC Class Codes: G06T 5/92 based on global image properties

Abstract

A method and system provide a multi-sensor data fusion system capable of adaptively weighting the contributions from each one of a plurality of sensors using a plurality of data fusion methods. During a predetermined tracking period, the system receives data from each individual sensor and each data fusion method is performed to determine a plurality of reliability functions for the system based on combining each sensor reliability function which are individually weighted based on the S/N (signal-to-noise) ratio for the received data from each sensor, and a comparison of predetermined sensor operation characteristics for each sensor and a best performing (most reliable) sensor. The system may dynamically select to use one or a predetermined combination of the generated reliability functions as the current (best) reliability function which provides a confidence level for the multi-sensor system relating to the correct classification (recognition) of targets and decoys.

Claims

1. A method for integrating data received from a plurality of sensors, comprising:

receiving data from a plurality of sensors having overlapping scanning areas;

determining a S/N (signal-to-noise) ratio for each sensor based on signal measurements of the received data;

determining a plurality of reliability functions for the plurality of sensors from using a plurality of predetermined calculations combining each sensor reliability function which are determined and individually weighted based on said S/N ratio for each sensor and differences between predetermined operating characteristics for each sensor;

selecting one or a predetermined combination of said plurality of reliability functions as a current reliability function based on the selected one or predetermined combination satisfying predetermined threshold; and

determining the presence or absence of an object of interest within the overlapping scanning areas, based upon data from the sensors and at least one selected reliability function.

2. The method of claim 1, further comprising:

determining to delay said selecting for a predetermined time period based on said selected at least one failing to satisfy said predetermined threshold.

3. The method of claim 1, wherein said determining a plurality of reliability functions includes determining said plurality of reliability functions from using one of a predetermined additive, multiplicative, and fuzzy logic calculation combing each sensor reliability function.

4. A method according to claim 1, wherein the overlapping scanning areas form a search area within which the object of interest is located.

5. The method of claim 1, wherein said determining a plurality of reliability functions includes determining said individual weighting based on a predetermined parameter for the plurality of sensors satisfying a predetermined threshold, and determining said plurality of reliability functions, based on using said plurality of predetermined calculations for a single sensor reliability function, when said predetermined parameter fails to satisfy said predetermined threshold.

6. The method of claim 1, wherein said determining a plurality of reliability functions includes determining said plurality of reliability functions based on differences between probabilities of classification for each sensor as compared to a single sensor having the highest probability of classification.

7. The method of claim 1, wherein said plurality of sensors includes at least one of a laser, IR (infrared), and RF (radio frequency) sensor.

8. The method of claim 1, wherein said determining a plurality of reliability functions includes determining said plurality of reliability functions based on determining a single sensor, as compared to the plurality of sensors, as having the best performance for at least one predetermined sensor parameter.

9. The method of claim 8, wherein said at least one predetermined sensor parameter is one of operating characteristics and S/N ratio for said single sensor.

10. The method of claim 9, wherein said single sensor is one of a laser, IR, and RF sensor.

11. The method of claim 1, wherein said selecting includes selecting said current reliability function to increase the probability of classifying one of a target and decoy above a predetermined threshold.

12. A method for integrating data received from a plurality of sensors, comprising:

receiving data from a plurality of sensors;

determining a S/N (signal-to-noise) ratio for each sensor based on signal measurements of the received data;

determining a plurality of reliability functions for the plurality of sensors from using a plurality of predetermined calculations combining each sensor reliability function which are determined and individually weighted based on said S/N ratio for each sensor and differences between predetermined operating characteristics for each sensor; and

selecting one or a predetermined combination of said plurality of reliability functions as a current reliability function based on the selected one or predetermined combination satisfying predetermined thresholds;

wherein said determining a plurality of reliability functions includes determining said plurality of reliability functions from using one of a predetermined additive, multiplicative, and fuzzy logic calculation combing each sensor reliability function; and

wherein said determining a plurality of reliability functions includes determining empty sets and ignorance sets for the received data from the plurality of sensors based on said predetermined multiplicative calculation.

13. The method of claim 12, wherein said predetermined multiplicative calculation includes at least a modified portion of a Dempster'"'"'s data fusion algorithm.

14. A method according to claim 12, wherein each sensor includes an overlapping scanning area and the overlapping scanning areas form a search area within which the object of interest is located.

15. A method for integrating data received from a plurality of sensors, comprising:

receiving data from a plurality of sensors;

determining a S/N (signal-to-noise) ratio for each sensor based on signal measurements of the received data;

determining a plurality of reliability functions for the plurality of sensors from using a plurality of predetermined calculations combining each sensor reliability function which are determined and individually weighted based on said S/N ratio for each sensor and differences between predetermined operating characteristics for each sensor; and

selecting one or a predetermined combination of said plurality of reliability functions as a current reliability function based on the selected one or predetermined combination satisfying predetermined thresholds;

wherein said determining a plurality of reliability functions includes determining said individual weighting based on a predetermined parameter for the plurality of sensors satisfying a predetermined threshold, and determining said plurality of reliability functions, based on using said plurality of predetermined calculations for a single sensor reliability function, when said predetermined parameter fails to satisfy said predetermined threshold;

wherein said predetermined parameter is a false alarm rate for the plurality of sensors.

16. A method according to claim 15, wherein each sensor includes an overlapping scanning area and the overlapping scanning areas form a search area within which the object of interest is located.

17. A multi-sensor system, comprising:

a plurality of sensors for receiving data having overlapping scanning areas; and

at least one controller for performing the steps of;

determining a S/N (signal-to-noise) ratio for each sensor based on signal measurements of the received data;

determining a plurality of reliability functions for the plurality of sensors from using a plurality of predetermined calculations combining each sensor reliability function which are determined and individually weighted based on said S/N ratio for each sensor and differences between predetermined operating characteristics for each sensor;

selecting one or a predetermined combination of said plurality of reliability functions as a current reliability function based on the selected at least one satisfying a predetermined threshold; and

determining the presence or absence of an object of interest within the overlapping scanning areas, based upon data from the sensors and at least one selected reliability function.

18. The system of claim 17, wherein said controller to determine said individual weighting based on a predetermined parameter for the plurality of sensors satisfying a predetermined threshold, and said controller to determine said plurality of reliability functions, based on using said plurality of predetermined calculations for a single sensor reliability function, when said predetermined parameter fails to satisfy said predetermined threshold.

19. A system according to claim 17, wherein the overlapping scanning areas form a search area within which the object of interest is located.

20. The system of claim 17, wherein said plurality of sensors includes at least one of a laser, IR (infrared) sensor, and RF (radio frequency) sensor.

21. A method for integrating data received from a plurality of sensors, comprising:

receiving data from a plurality of sensors;

determining a S/N (signal-to-noise) ratio for each sensor based on signal measurements of the received data;

determining a plurality of reliability functions for the plurality of sensors from using a plurality of predetermined calculations combining each sensor reliability function which are determined and individually weighted based on said S/N ratio for each sensor and differences between predetermined operating characteristics for each sensor; and

selecting one or a predetermined combination of said plurality of reliability functions as a current reliability function based on the selected one or predetermined combination satisfying predetermined thresholds;

wherein said controller to determine said individual weighting based on a predetermined parameter for the plurality of sensors satisfying a predetermined threshold, and said controller to determine said plurality of reliability functions, based on using said plurality of predetermined calculations for a single sensor reliability function, when said predetermined parameter fails to satisfy said predetermined threshold;

wherein said predetermined parameter is a false alarm rate for the plurality of sensors.

22. A method according to claim 21, wherein each sensor includes an overlapping scanning area and the overlapping scanning areas form a search area within which the object of interest is located.

23. A machine-readable medium having stored thereon a plurality of executable instructions, the plurality of instructions comprising instructions to:

receive data from a plurality of sensors having overlapping scanning areas;

determine a S/N (signal-to-noise) ratio for each sensor based on signal measurements of the received data;

determine a plurality of reliability functions for the plurality of sensors from using a plurality of predetermined calculations combining each sensor reliability function which are determined and individually weighted based on said S/N ratio for each sensor and differences between predetermined operating characteristics for each sensor; and

select at one or a predetermined combination of said plurality of reliability functions to use as a current reliability function based on the selected at least one satisfying a predetermined threshold; and

determine the presence or absence of an object of interest within the overlapping scanning areas, based upon data from the sensors and at least one selected reliability function.

24. The medium of claim 23, wherein said instructions to determine a plurality of reliability functions include instructions to determine said individual weighting based on a predetermined parameter for the plurality of sensors satisfying a predetermined threshold.

25. The medium of claim 23, wherein said instructions to determine a plurality of reliability functions include instructions to determine said plurality of reliability functions based on differences between probabilities of classification for each sensor as compared to a single sensor having the highest probability of classification.

26. The medium of claim 23, wherein said instructions to determine a plurality of reliability functions include instructions to determine said plurality of reliability functions from using one of a predetermined additive, multiplicative, and fuzzy logic calculation combining each sensor reliability function.

27. The medium of claim 23, wherein said instructions to select include instructions to select said current reliability function to increase the probability of classifying the object of interest, which corresponds to either a target or a decoy.

28. A medium according to claim 23, wherein the overlapping scanning areas form a search area within which the object of interest is located.