Associating signal intelligence to objects via residual reduction

• US 10,024,661 B2
• Filed: 03/25/2016
• Issued: 07/17/2018
• Est. Priority Date: 03/15/2013
• Status: Active Grant

• Current Assignee: Raytheon Company (Rtx Corporation)
• Original Assignee: Raytheon Company (Rtx Corporation)
• Inventors: Guisewite, Geoffrey M., Cole, Robert J., Deardorff, Thomas P.
• Primary Examiner(s): Patel, Nirav G
• Application Number:
  US15/081,208
• Publication Number:
  US 20160209212A1
• Time in Patent Office: 844 Days
• Field of Search: None
• CPC Class Codes: G01C 11/04 Interpretation of pictures
• CPC Class Codes: G01S 3/7864 T.V. type tracking systems
• CPC Class Codes: G01S 5/0278 involving statistical or probabilistic considerations G01S5/0252, G01S5/0294 take precedence
• CPC Class Codes: G01S 5/14 Determining absolute distances from a plurality of spaced points of known location
• CPC Class Codes: G06F 16/29 Geographical information databases
• CPC Class Codes: G06F 16/786 using motion, e.g. object motion or camera motion
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51

Abstract

Generally discussed herein are systems and apparatuses that are configured to and techniques for associating a SIGnal INTelligence (SIGINT) signal with an object or tracklet. According to an example a technique can include estimating Times of Arrival (ToAs) at each of a plurality of collectors of a first signal from each of a plurality of moving transmitters, each first signal transmitted from a transmitter on a tracklet extracted from video data and received at the plurality of collectors, wherein a location of each of the plurality of collectors is known, comparing each estimated ToA to a respective actual ToA of a SIGINT signal received at each of the collectors, or determining a likelihood that the signal corresponds to the SIGINT signal to determine whether the SIGINT signal was transmitted from a transmitter on the corresponding tracklet.

Claims

1. A method for associating tracklets extracted from video data with SIGnal INTelligence (SIGINT) signals, each tracklet identifying geolocations of a respective moving object in the video data, the method comprising:

determining, using a processing unit, a plurality of time estimates, each of the time estimates representing an estimated amount of time it takes for a first signal to travel from a geolocation of a respective tracklet of the tracklets to travel to a respective collector of a plurality of collectors;

adding, using the processing unit, each of the time estimates to a video capture time, the video capture time indicating a time at which an object was at the geolocation of the respective tracklet to create estimated time of arrivals (ToAs);

comparing, using the processing unit, each of the estimated ToAs to actual ToAs to determine which estimated ToAs are closest to the actual ToAs, the actual ToAs indicating a time at which a SIGINT signal of the SIGINT signals was received at each of the collectors; and

associating the WANT signal with a tracklet of the tracklets corresponding to the estimated ToAs determined to be closest to the actual ToAs to associate the object with the tracklet.

2. The method of claim 1, further comprising generating a bounding area so as to constrain tracklets to tracklets within the bounding area, wherein the bounding area includes a geographical region in which a SIGINT event of a plurality of SIGINT events is estimated to have originated from as a function of an estimated location and corresponding covariance defining a confidence that the estimated location is the actual location the SIGINT event originated from, wherein each of the plurality of SIGINT events includes a SIGINT signal and the actual ToA of the SIGINT signal.

3. The method of claim 2, further comprising determining a plurality of residual errors, one residual error for each tracklet of the tracklets per SIGINT event, wherein each residual error represents a likelihood that a SIGINT event originated from a transmitter at the geolocation on the tracklet, wherein the residual error is determined as a function of (1) an interpolated location of the transmitter at a specific time, the interpolated location determined based on the video data, the video data from full motion video, (2) the actual ToAs at each collector of the SIGINT signal of the SIGINT event, and (3) the location of each collector.

4. The method of claim 3, further comprising determining the SIGINT signal originated from the tracklet that corresponds to a lowest residual error of the plurality of residual errors.

5. The method of claim 4, further comprising determining if it is more likely that the SIGINT signal originated from a moving transmitter or a stationary transmitter.

6. The method of claim 5, wherein:

determining the plurality of time estimates includes, for each time estimate estimating a first time, the first time indicating how long it would take the first signal to travel from a point corresponding to a geolocation on the tracklet to a collector of the plurality of collectors; and

the video capture time indicates a time at which the transmitter was at the point on the tracklet.

7. The method of claim 6, wherein the plurality of tracklets were each active in the bounding area in a time window, the time window determined as a function of the actual ToAs.

8. A non-transitory machine-readable storage device including instructions stored thereon, the instructions, which when executed by a machine, cause the machine to perform operations for associating tracklets extracted from video data with SIGnal INTelligence (SIGINT) signals, each tracklet identifying geolocations of a moving object in the video data, the operations comprising:

determining a plurality of time estimates, each of the plurality of time estimates estimating an amount of time it takes for a first signal to travel from a geolocation of a respective tracklet of the tracklets to travel to a respective collector of a plurality of collectors;

adding each of the time estimates to a video capture time, the video capture time indicating a time at which an object was at the geolocation of the respective tracklet to create estimated time of arrivals (ToAs);

comparing each of the estimated ToAs to actual ToAs to determine which estimated ToAs are closest to the actual ToAs, the actual ToAs indicating a time at which a SIGINT signal of the SIGINT signals was received at each of the plurality of collectors; and

associating the SIGINT signal with a tracklet of the tracklets corresponding to the estimated ToAs determined to be closest to the actual ToAs.

9. The storage device of claim 8, further comprising instructions which, when executed by the machine, cause the machine to perform further operations comprising generating a bounding area so as to constrain tracklets to tracklets within the bounding area, wherein the bounding area includes a geographical region in which a WANT event of a plurality of SIGINT events is estimated to have originated from, the bounding area determined as a function of an estimated location and corresponding covariance defining a confidence that the estimated location is the actual location the SIGINT event originated from, wherein each of the plurality of SIGINT events includes a SIGINT signal and the actual ToA of the SIGINT signal.

10. The storage device of claim 9, further comprising instructions which, when executed by the machine, cause the machine to perform further operations comprising determining a plurality of residual errors, one residual error for each tracklet of the tracklets per SIGINT event, wherein each residual error represents a likelihood that a SIGINT event originated from a transmitter at the geolocation on the tracklet, wherein the residual error is determined as a function of (1) an interpolated location of the transmitter at a specific time, the interpolated location determined based on the video data, the video data from full motion video, (2) the actual ToAs at each collector of the SIGINT signal of the SIGINT event, and (3) the location of each collector.

11. The storage device of claim 10, further comprising instructions which, when executed by the machine, cause the machine to perform further operations comprising determining the SIGINT signal originated from the tracklet that corresponds to a lowest residual error of the plurality of residual errors.

12. The storage device of claim 11, further comprising instructions which, when executed by the machine, cause the machine to perform further operations comprising determining whether it is more likely that the SIGINT signal originated from a moving transmitter or a stationary transmitter.

13. The storage device of claim 12, wherein:

the instructions for determining the plurality of time estimates include instructions which, when executed by the machine, cause the machine to perform operations comprising for each time estimate estimating a first time, the first time indicating how long it would take the first signal to travel from a point corresponding to a geolocation on the tracklet to a collector of the plurality of collectors; and

the video capture time indicate a time at which the transmitter was at the point on the tracklet.

14. The storage device of claim 13, wherein the plurality of tracklets were each active in the bounding area in a time window, the time window determined as a function of the actual ToAs.

15. A device configured to associate tracklets extracted from video data with SIGnal INTelligence (SIGINT) signals, each tracklet identifying geolocations of a moving object in the video data, the device comprising:

a processor configured to;

determine a plurality of time estimates, each of the plurality of time estimates estimating an amount of time it takes for a first signal to travel from a geolocation of a respective tracklet of the tracklets to travel to a respective collector of a plurality of collectors;

add each of the time estimates to a video capture time, the video capture time indicating a time at which an object was at the geolocation of the respective tracklet to create estimated time of arrivals (ToAs);

compare each of the estimated ToAs to actual ToAs to determine which estimated ToAs are closest to the actual ToAs, the actual ToAs indicating a time at which a SIGINT signal of the SIGINT signals was received at each of the plurality of collectors; and

associate the SIGINT signal with a tracklet of the tracklets corresponding to the estimated ToAs determined to be closest to the actual ToAs.

16. The device of claim 15, wherein the processor is further configured to generate a bounding area so as to constrain tracklets to tracklets within the bounding area, wherein the bounding area includes a geographical region in which a SIGINT event of a plurality of SIGINT events is estimated to have originated from as a function of an estimated location and corresponding covariance defining a confidence that the estimated location is the actual location the SIGINT event originated from, wherein each of the plurality of SIGINT events includes a SIGINT signal and the actual ToA of the SIGINT signal.

17. The device of claim 16, wherein the processor is further configured to determine a plurality of residual errors, one residual error for each tracklet of the tracklets per SIGINT event, wherein each residual error represents a likelihood that a SIGINT event originated from a transmitter at the geolocation on the tracklet, wherein the residual error is determined as a function of (1) an interpolated location of the transmitter at a specific time, the interpolated location determined based on the video data, the video data from full motion video, (2) the actual ToAs at each collector of the SIGINT signal of the SIGINT event, and (3) the location of each collector.

18. The device of claim 17, wherein the processor is further to determine the SIGINT signal originated from the tracklet that corresponds to a lowest residual error of the plurality of residual errors.

19. The device of claim 18, wherein the processor is further configured to determine if it is more likely that the WANT signal originated from a moving transmitter or a stationary transmitter.

20. The device of claim 19, herein the plurality of tracklets were each active in the bounding area in a time window, the time window determined as a function of the actual ToAs.