Associating signal intelligence to objects via residual reduction

• US 9,297,654 B2
• Filed: 02/14/2014
• Issued: 03/29/2016
• Est. Priority Date: 03/15/2013
• Status: Active Grant

• Current Assignee: Raytheon Company (Rtx Corporation)
• Original Assignee: Raytheon Company (Rtx Corporation)
• Inventors: Guisewite, Geoffrey M., Cole, Robert J., Deardorff, Thomas P.
• Primary Examiner(s): Chu, Randolph I
• Application Number:
  US14/180,439
• Publication Number:
  US 20140270368A1
• Time in Patent Office: 774 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G01C 11/04 Interpretation of pictures
• CPC Class Codes: G01S 3/7864 T.V. type tracking systems
• CPC Class Codes: G01S 5/0278 involving statistical or probabilistic considerations G01S5/0252, G01S5/0294 take precedence
• CPC Class Codes: G01S 5/14 Determining absolute distances from a plurality of spaced points of known location
• CPC Class Codes: G06F 16/29 Geographical information databases
• CPC Class Codes: G06F 16/786 using motion, e.g. object motion or camera motion
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51

Abstract

Generally discussed herein are systems and apparatuses that are configured to and techniques for associating a SIGnal INTelligence (SIGINT) signal with an object or tracklet. According to an example a technique can include estimating Times of Arrival (ToAs) at each of a plurality of collectors of a first signal from each of a plurality of moving transmitters, each first signal transmitted from a transmitter on a tracklet extracted from video data and received at the plurality of collectors, wherein a location of each of the plurality of collectors is known, comparing each estimated ToA to a respective actual ToA of a SIGINT signal received at each of the collectors, or determining a likelihood that the signal corresponds to the SIGINT signal to determine whether the SIGINT signal was transmitted from a transmitter on the corresponding tracklet.

Claims

1. A method for associating tracklets with SIGnal INTelligence (SIGINT) signals, the method comprising:

receiving, at a processing unit, the SIGINT signals and tracklets extracted from video data, the SIGINT signals including a respective actual Time of Arrival (ToA) indicating a time at which the SIGINT signal arrived at a collector of a plurality of collectors at known locations, the tracklets associated with respective objects in the video data, each tracklet identifying geolocations of a moving object of the respective objects;

estimating, using the processing unit, an amount of time it takes for a first signal to travel from a geolocation of a tracklet of the tracklets to travel to each collector of the plurality of collectors and adding the estimated time to a video capture time corresponding to a time at which the object was at the geolocation of the tracklet to create estimated ToAs;

comparing each estimated ToA of the estimated ToAs to the respective actual ToA of a SIGINT signal of the SIGINT signals received at each of the collectors;

determining, based on the comparison, a likelihood that the first signal corresponds to the SIGINT signal to determine whether the SIGINT signal was transmitted from a transmitter on the corresponding tracklet; and

associating the SIGINT signal to a tracklet of the tracklets corresponding to the greatest determined likelihood.

2. The method of claim 1, further comprising:

generating a bounding area so as to constrain tracklets to tracklets within the bounding area, wherein the bounding area includes a geographical region in which a SIGINT event is estimated to have originated from as a function of an estimated location and corresponding covariance defining a confidence that the estimated location is the actual location the SIGINT event originated from, wherein each of the plurality of SIGINT events includes a SIGINT signal and the actual ToA of the SIGINT signal.

3. The method of claim 2, further comprising:

determining a plurality of residual errors, one residual error for each tracklet in a plurality of tracklets per SIGINT event, wherein each residual error represents a likelihood that a SIGINT event originated from a respective object on the tracklet, wherein the residual error is determined as a function of (1) an interpolated location of the transmitter at a specific time, the interpolated location determined based on tracklet data from full motion video, (2) the actual ToAs at each collector of the SIGINT event, and (3) the location of each collector.

4. The method of claim 3, further comprising:

determining the SIGINT signal originated from an object on the tracklet that corresponds to a lowest residual error of the plurality of residual errors.

5. The method of claim 4, further comprising:

determining if it is more likely that the SIGINT signal originated from a moving transmitter or a stationary transmitter.

6. The method of claim 5, wherein estimating the ToA at the plurality of collectors includes:

estimating a first time, the first time indicating how long it would take the first signal to travel from a point on the tracklet to a collector of the plurality of collectors;

determining a second time, the second time indicating the time at which the transmitter was at the point on the tracklet; and

determining an estimated ToA at the collector as a function of the first time and the second time.

7. The method of claim 6, wherein the plurality of tracklets were each active in the bounding area in a time window, the time window determined as a function of the actual ToAs of SIGINT signals at the plurality of collectors.

8. A non-transitory computer readable storage device including instructions stored thereon, the instructions, which when executed by a machine, cause the machine to perform operations for associating tracklets with SIGnal INTelligence (SIGINT) signals, the operations comprising:

receiving the SIGINT signals and tracklets extracted from video data, the SIGINT signals including a respective actual Time of Arrival (ToA) indicating a time at which the SIGINT signal arrived at a collector of a plurality of collectors at known locations, the tracklets associated with respective objects in the video data, each tracklet identifying geolocations of a moving object of the respective objects;

estimating an amount of time it takes for a first signal to travel from a geolocation of a tracklet of the tracklets to travel to each collector of the plurality of collectors and adding the estimated time to a video capture time corresponding to a time at which the object was at the geolocation of the tracklet to create estimated ToAs;

comparing each estimated ToA of the estimated ToAs to the respective actual ToA of a SIGINT signal of the SIGINT signals received at each of the collectors;

determining, based on the comparison, a likelihood that the first signal corresponds to the SIGINT signal to determine whether the SIGINT signal was transmitted from a transmitter on the corresponding tracklet; and

associating the SIGINT signal to a tracklet of the tracklets corresponding to the greatest determined likelihood.

9. The storage device of claim 8, further comprising instructions, which when executed by the machine, cause the machine to perform operations comprising:

generating a bounding area so as to constrain tracklets to tracklets within the bounding area, wherein the bounding area includes a geographical region in which a SIGINT event is estimated to have originated from as a function of an estimated location and corresponding covariance defining a confidence that the estimated location is the actual location the SIGINT event originated from, wherein each of the plurality of SIGINT events includes a SIGINT signal and the actual ToA of the SIGINT signal.

10. The storage device of claim 9, further comprising instructions, which when executed by the machine, cause the machine to perform operations comprising:

determining a plurality of residual errors, one residual error for each tracklet in a plurality of tracklets per SIGINT event, wherein each residual error represents a likelihood that a respective SIGINT event originated from an object on a respective tracklet, wherein the residual error is determined as a function of (1) an interpolated location of the transmitter at a specific time, the interpolated location determined based on tracklet data from full motion video, (2) the actual TOA at each collector, and (3) the location of each collector.

11. The storage device of claim 10, further comprising instructions, which when executed by the machine, cause the machine to perform operations comprising:

determining the SIGINT signal originated from the tracklet that corresponds to the lowest residual error of the plurality of residual errors.

12. The storage device of claim 11, further comprising:

determining if it is more likely that the SIGINT signal originated from a moving transmitter or a stationary transmitter.

13. The storage device of claim 12, wherein the instructions for estimating the ToA at the plurality of collectors include instructions, which when executed by the machine, cause the machine to perform operations comprising:

estimating a first time, the first time indicating how long it would take the first signal to travel from a point on the tracklet to a collector of the plurality of collectors;

determining a second time, the second time indicating the time at which the transmitter was at the point on the tracklet; and

determining an estimated ToA at the collector as a function of the first time and the second time.

14. The storage device of claim 13, wherein the plurality of tracklets were each active in the bounding area in a time window, the time window determined as a function of the actual ToAs of SIGINT events at the plurality of collectors.

15. A device configured to associate tracklets with SIGnal INTelligence (SIGINT) signals, the device comprising:

a processor configured to;

receive the SIGINT signals and tracklets extracted from video data, the SIGINT signals including a respective actual Time of Arrival (ToA) indicating a time at which the SIGINT signal arrived at a collector of a plurality of collectors at known locations, the tracklets associated with respective objects in the video data, each tracklet identifying geolocations of a moving object of the respective objects;

estimate an amount of time it takes for a first signal to travel from a geolocation of a tracklet of the tracklets to travel to each collector of the plurality of collectors and adding the estimated time to a video capture time corresponding to a time at which the object was at the geolocation of the tracklet to create estimated ToAs;

compare each estimated ToA of the estimated ToAs to the respective actual ToA of a SIGINT signal of the SIGINT signals received at each of the collectors;

determine, based on the comparison, a likelihood that the first signal corresponds to the SIGINT signal to determine whether the SIGINT signal was transmitted from a transmitter on the corresponding tracklet; and

associate the SIGINT signal to a tracklet of the tracklets corresponding to the greatest determined likelihood.

16. The device of claim 15, wherein the processor is further configured to:

determine a plurality of residual errors, one residual error for each tracklet in a plurality of tracklets per SIGINT signal, wherein each residual error represents a likelihood that a SIGINT signal originated on a respective tracklet, wherein the residual error is determined as a function of (1) an interpolated location of the transmitter at a specific time, the interpolated location determined based on tracklet data from full motion video, (2) the actual TOA at each collector, and (3) the location of each collector.

17. The device of claim 16, wherein the processor is further configured to:

determine the SIGINT signal originated from the tracklet that corresponds to the lowest residual error of the plurality of residual errors.

18. The device of claim 17, wherein the processor is further configured to:

determine if it is more likely that the SIGINT signal originated from a moving transmitter or a stationary transmitter.

19. The device of claim 18, wherein the processor is further configured to:

generate a bounding area so as to constrain tracklets to tracklets within the bounding area, wherein the bounding area includes a geographical region in which a SIGINT event is estimated to have originated from as a function of an estimated location and corresponding covariance defining a confidence that the estimated location is the actual location the SIGINT event originated from, wherein each of the plurality of SIGINT events includes a SIGINT signal and the actual ToA of the SIGINT signal.

20. The device of claim 19, wherein the processor configured to estimate the ToA at the plurality of collectors includes the processor further configured to:

estimate a first time, the first time indicating how long it would take the first signal to travel from a point on the tracklet to a collector of the plurality of collectors;

determine a second time, the second time indicating the time at which the transmitter was at the point on the tracklet; and

determine an estimated ToA at the collector as a function of the first time and the second time.