Method and system for data fusion using spatial and temporal diversity between sensors

• US 7,576,681 B2
• Filed: 05/16/2005
• Issued: 08/18/2009
• Est. Priority Date: 03/26/2002
• Status: Expired due to Fees

• Current Assignee: Lockheed Martin Corporation (Martin Marietta Corporation)
• Original Assignee: Lockheed Martin Corporation (Martin Marietta Corporation)
• Inventors: Chen, Hai-Wen, Olson, Teresa L.
• Primary Examiner(s): Gregory; Bernarr E
• Application Number:
  US11/129,395
• Publication Number:
  US 20060082490A1
• Time in Patent Office: 1,555 Days
• Field of Search: 700/31, 700/90, 345/173, 701/214, 702 51- 53, 702/187, 702189-199, 702/1, 702/85, 702/92, 702/93, 702/127, 702/150, 702/182, 342/22, 342/27, 342/28, 342 52- 59, 342 89- 97, 342165-175, 342/195, 342450-465, 342/64, 342/13, 342/159, 342176-186, 367 13- 15, 367 20- 24, 367 36- 38, 367 59- 63, 367/87, 367/93, 367/94, 367/117, 703/13, 348/25, 706/15, 706/25, 382/100, 382/103
• US Class Current: 342/52
• CPC Class Codes: G06F 18/25 Fusion techniques
• CPC Class Codes: G06F 18/251 of input or preprocessed data
• CPC Class Codes: G06F 18/256 of results relating to different input data, e.g. multimodal recognition
• CPC Class Codes: G06T 2207/10048 Infrared image
• CPC Class Codes: G06T 5/40 using histogram techniques
• CPC Class Codes: G06T 5/92 based on global image properties

Abstract

A method and system provide a multi-sensor data fusion system capable of adaptively weighting the contributions from each one of a plurality of sensors using a plurality of data fusion methods. During a predetermined tracking period, the system receives data from each individual sensor and each data fusion method is performed to determine a plurality of reliability functions for the system based on combining each sensor reliability function which are individually weighted based on the S/N (signal-to-noise) ratio for the received data from each sensor, and a comparison of predetermined sensor operation characteristics for each sensor and a best performing (most reliable) sensor. The system may dynamically select to use one or a predetermined combination of the generated reliability functions as the current (best) reliability function which provides a confidence level for the multi-sensor system relating to the correct classification (recognition) of targets and decoys.

Claims

1. A method for integrating data received from a plurality of sensors, comprising:

receiving data from a plurality of sensors having overlapping scanning areas;

determining a plurality of reliability functions for the plurality of sensors, the plurality of reliability functions being based upon individually weighted reliability functions for each sensor;

selecting one or a predetermined combination of said plurality of reliability functions as a current reliability function based on the selected one or predetermined combination satisfying predetermined thresholds; and

based upon data from the sensors and at least one selected reliability function, determining whether or not there is an object of interest within the overlapping scanning areas.

2. The method according to claim 1, wherein the individually weighted reliability functions are based upon the relative received operating characteristics between each sensor.

3. The method of claim 1, further comprising:

determining to delay said selecting for a predetermined time period based on said selected at least one failing to satisfy said predetermined threshold.

4. The method of claim 1, wherein said determining a plurality of reliability functions includes determining said plurality of reliability functions from using one of a predetermined additive, multiplicative, and fuzzy logic calculation combing each sensor reliability function.

5. The method of claim 1, wherein said determining a plurality of reliability functions includes determining said individual weighting based on a predetermined parameter for the plurality of sensors satisfying a predetermined threshold, and determining said plurality of reliability functions, based on using said plurality of predetermined calculations for a single sensor reliability function, when said predetermined parameter fails to satisfy said predetermined threshold.

6. The method of claim 1, wherein said determining a plurality of reliability functions includes determining said plurality of reliability functions based on differences between probabilities of classification for each sensor as compared to a single sensor having the highest probability of classification.

7. The method of claim 1, wherein said plurality of sensors includes at least one of a laser, IR (infrared), and RF (radio frequency) sensor.

8. The method of claim 1, wherein said determining a plurality of reliability functions includes determining said plurality of reliability functions based on determining a single sensor, as compared to the plurality of sensors, as having the best performance for at least one predetermined sensor parameter.

9. The method of claim 8, wherein said at least one predetermined sensor parameter is selected from the group comprising operating characteristics of said single sensor and S/N ratio for said single sensor.

10. The method of claim 9, wherein said single sensor is one of a laser, IR, and RF sensor.

11. The method of claim 1, wherein said selecting includes selecting said current reliability function to increase the probability of classifying one of a target and decoy above a predetermined threshold.

12. A method according to claim 1, wherein the overlapping scanning areas form a search area within which the object of interest is located.

13. A method for integrating data received from a plurality of sensors, comprising:

receiving data from a plurality of sensors;

determining a plurality of reliability functions for the plurality of sensors, the plurality of reliability functions being based upon individually weighted reliability functions for each sensor; and

selecting one or a predetermined combination of said plurality of reliability functions as a current reliability function based on the selected one or predetermined combination satisfying predetermined thresholds;

wherein said determining a plurality of reliability functions includes determining said plurality of reliability functions from using one of a predetermined additive, multiplicative, and fuzzy logic calculation combing each sensor reliability function; and

wherein said determining a plurality of reliability functions includes determining empty sets and ignorance sets for the received data from the plurality of sensors based on said predetermined multiplicative calculation.

14. The method of claim 13, wherein said predetermined multiplicative calculation includes a Dempster-Shafer data fusion method.

15. A method according to claim 13, wherein each sensor includes an overlapping scanning area and the overlapping scanning areas form a search area within which the object of interest is located.

16. A method for integrating data received from a plurality of sensors, comprising:

receiving data from a plurality of sensors;

determining a plurality of reliability functions for the plurality of sensors, the plurality of reliability functions being based upon individually weighted reliability functions for each sensor; and

selecting one or a predetermined combination of said plurality of reliability functions as a current reliability function based on the selected one or predetermined combination satisfying predetermined thresholds;

wherein said determining a plurality of reliability functions includes determining said individual weighting based on a predetermined parameter for the plurality of sensors satisfying a predetermined threshold, and determining said plurality of reliability functions, based on using said plurality of predetermined calculations for a single sensor reliability function, when said predetermined parameter fails to satisfy said predetermined threshold;

wherein said predetermined parameter is a false alarm rate for the plurality of sensors.

17. A method according to claim 16, wherein each sensor includes an overlapping scanning area and the overlapping scanning areas form a search area within which the object of interest is located.

18. A multi-sensor system, comprising:

a plurality of sensors for receiving data having overlapping scanning areas; and

at least one controller for performing the steps of;

determining a plurality of reliability functions for the plurality of sensors, the plurality of reliability functions being based upon individually weighted reliability functions for each sensor;

selecting one or a predetermined combination of said plurality of reliability functions as a current reliability function based on the selected at least one satisfying a predetermined threshold; and

based upon data from the sensors and at least one selected reliability function, determining whether or not there is an object of interest within the overlapping scanning areas.

19. The system of claim 18, wherein said controller to determine said individual weighting based on a predetermined parameter for the plurality of sensors satisfying a predetermined threshold, and said controller to determine said plurality of reliability functions, based on using said plurality of predetermined calculations for a single sensor reliability function, when said predetermined parameter fails to satisfy said predetermined threshold.

20. The system of claim 18, wherein said plurality of sensors includes at least one of a laser, IR (infrared) sensor, and RF (radio frequency) sensor.

21. A system according to claim 18, wherein the overlapping scanning areas form a search area within which the object of interest is located.

22. A method for integrating data received from a plurality of sensors, comprising:

receiving data from a plurality of sensors;

determining a S/N (signal-to-noise) ratio for each sensor based on signal measurements of the received data;

determining a plurality of reliability functions for the plurality of sensors, the plurality of reliability functions being based upon individually weighted reliability functions for each sensor; and

selecting one or a predetermined combination of said plurality of reliability functions as a current reliability function based on the selected one or predetermined combination satisfying predetermined thresholds;

wherein said controller to determine said individual weighting based on a predetermined parameter for the plurality of sensors satisfying a predetermined threshold, and said controller to determine said plurality of reliability functions, based on using said plurality of predetermined calculations for a single sensor reliability function, when said predetermined parameter fails to satisfy said predetermined threshold;

wherein said predetermined parameter is a false alarm rate for the plurality of sensors.

23. A method according to claim 22, wherein each sensor includes an overlapping scanning area and the overlapping scanning areas form a search area within which the object of interest is located.

24. A computer program product comprising a machine-readable medium having stored thereon a plurality of executable instructions for causing a processor to carry out the operation of:

receiving data from a plurality of sensors having overlapping scanning areas;

determining a plurality of reliability functions for the plurality of sensors, the plurality of reliability functions being based upon individually weighted reliability functions for each sensor; and

selecting at least one or a predetermined combination of said plurality of reliability functions to use as a current reliability function based on the selected at least one satisfying a predetermined threshold; and

based upon data from the sensors and at least one selected reliability function, determining whether or not there is an object of interest within the overlapping scanning areas.

25. The computer program product of claim 24, wherein determining a plurality of reliability functions includes determining said individual weighting based on a predetermined parameter for the plurality of sensors satisfying a predetermined threshold.

26. The computer program product of claim 24, wherein determining a plurality of reliability functions includes determining said plurality of reliability functions based on differences between probabilities of classification for each sensor as compared to a single sensor having the highest probability of classification.

27. The computer program product of claim 24, wherein determining a plurality of reliability functions includes determining said plurality of reliability functions from using one of a predetermined additive, multiplicative, and fuzzy logic calculation combining each sensor reliability function.

28. The computer program product of claim 24, wherein selecting at least one or a predetermined combination of said plurality of reliability functions includes selecting said current reliability function to increase the probability of classifying the object of interest, which corresponds to either a target or a decoy.

29. The computer program product according to claim 24, wherein the overlapping scanning areas form a search area within which the object of interest is located.