Unmanned vehicle recognition and threat management

• US 10,529,241 B2
• Filed: 02/14/2019
• Issued: 01/07/2020
• Est. Priority Date: 01/23/2017
• Status: Active Grant

• Current Assignee: Digital Global Systems, Inc.
• Original Assignee: Digital Global Systems, Inc.
• Inventors: Kleinbeck, David William
• Primary Examiner(s): Vo, Nguyen T
• Application Number:
  US16/275,575
• Publication Number:
  US 20190180630A1
• Time in Patent Office: 327 Days
• CPC Class Codes: G01S 3/043 Receivers
• CPC Class Codes: G01S 3/046 Displays or indicators
• CPC Class Codes: G01S 3/46 using antennas spaced apart and measuring phase or time difference between signals therefrom, i.e. path-difference systems
• CPC Class Codes: G06N 3/044 Recurrent networks, e.g. Hopfield networks
• CPC Class Codes: G06N 3/045 Combinations of networks
• CPC Class Codes: G06N 3/08 Learning methods
• CPC Class Codes: G06N 3/084 Backpropagation, e.g. using gradient descent
• CPC Class Codes: G06N 7/01 Probabilistic graphical models, e.g. probabilistic networks
• CPC Class Codes: G08G 5/0026 located on the ground
• CPC Class Codes: G08G 5/0069 specially adapted for an unmanned aircraft
• CPC Class Codes: G08G 5/0082 for monitoring traffic from a ground station radar or analogous systems specially adapted for traffic control G01S13/91

Abstract

Systems and methods for automated unmanned aerial vehicle recognition. A multiplicity of receivers captures RF data and transmits the RF data to at least one node device. The at least one node device comprises a signal processing engine, a detection engine, a classification engine, and a direction finding engine. The at least one node device is configured with an artificial intelligence algorithm. The detection engine and classification engine are trained to detect and classify signals from unmanned vehicles and their controllers based on processed data from the signal processing engine. The direction finding engine is operable to provide lines of bearing for detected unmanned vehicles.

Claims

1. A system for unmanned vehicle (UV) recognition in a radio frequency (RF) environment, comprising:

a multiplicity of RF receivers and a displaying device in network communication with at least one node device;

wherein the multiplicity of RF receivers is operable to capture RF data in the RF environment, convert the RF data to fast Fourier transform (FFT) data, and transmit the FFT data to the at least one node device;

wherein the at least one node device comprises a signal processing engine, a detection engine, and a direction-finding engine, and wherein the at least one node device is configured with an artificial intelligence (AI) algorithm;

wherein the signal processing engine is operable to average the FFT data into at least one tile;

wherein the detection engine is operable to detect at least one signal related to at least one UV and corresponding at least one UV controller in the least one tile based on the AI algorithm;

wherein the direction-finding engine is operable to calculate a line of bearing for the at least one UV; and

wherein the displaying device is operable to display the line of bearing of the at least one UV.

2. The system of claim 1, wherein the AI algorithm comprises an inception-based convolutional neural network operable to generate probabilities that UVs and UV controllers are detected.

3. The system of claim 1, wherein the AI algorithm comprises a You Only Look Once (YOLO) algorithm operable to receive the at least one tile, generate an output for each of the at least one tile to identify the at least one UV with a probability, and calculate an average probability based on the output for each of the at least one tile.

4. The system of claim 1, wherein each of the at least one tile is a 256 by 256 array representing 125 MHz of bandwidth and 80 ms of time.

5. The system of claim 4, wherein each of the at least one tile is visually represented in a waterfall image via a graphical user interface on the displaying device.

6. The system of claim 1, wherein the at least one node device further comprises a classification engine operable to classify the at least one UV and the corresponding at least one UV controller by comparing the at least one signal to classification data in real time or near real time.

7. The system of claim 1, wherein the displaying device is operable to display an angle and a frequency of the at least one signal.

8. The system of claim 1, wherein the RF data is from a spectrum between 20 MHz and 6 GHz.

9. The system of claim 1, wherein the detection engine is operable to detect the at least one UV and the corresponding at least one UV controller by radio communication protocols.

10. The system of claim 1, wherein the at least one node device further comprises a learning engine operable to update a classification library with emerging protocols.

11. The system of claim 1, wherein the at least one UV is aerial, terrestrial or water borne.

12. The system of claim 1, where the at least one node device is operable to transmit an alert related to the at least one UV to a counter UV system via email, short message service (SMS), or a third-party integration.

13. The system of claim 12, wherein the counter UV system is operable to intercept communications between the at least one UV and the corresponding at least one UV controller.

14. The system of claim 1, wherein the at least one node device is operable to train the AI algorithm for UV recognition by capturing and recording the RF data from a multiplicity of UVs and corresponding UV controllers, respectively over different channels and different RF bandwidths.

15. A system for unmanned vehicle (UV) recognition in a radio frequency (RF) environment, comprising:

a multiplicity of RF receivers and a displaying device in network communication with a multiplicity of node devices;

wherein the multiplicity of RF receivers is operable to capture RF data in the RF environment, convert the RF data to fast Fourier transform (FFT) data, and transmit the FFT data to the multiplicity of node devices;

wherein the multiplicity of node devices each comprises a signal processing engine, a detection engine, a classification engine, a direction-finding engine, and at least one artificial intelligence (AI) engine;

wherein the signal processing engine is operable to average the FFT data into at least one tile;

wherein the detection engine is operable to group the FFT data into discrete FFT bins over time, calculate average and standard deviation of power for the discrete FFT bins, and identify at least one signal related to at least one UV and/or corresponding at least one UV controller;

wherein the at least one AI engine is operable to generate an output for each of the at least one tile to identify at least one UV and corresponding at least one UV controller with a probability;

wherein the classification engine is operable to classify the at least one UV and/or the at least one UV controller by comparing the at least one signal to classification data in real time or near real time;

wherein the direction-finding engine is operable to calculate a line of bearing for the at least one UV; and

wherein the displaying device is operable to display the line of bearing of the at least one UV.

16. The system of claim 15, wherein each of the multiplicity of node devices further comprises a state engine operable to control a flow of the at least one tile into the at least one AI engine.

17. The system of claim 15, wherein the multiplicity of node devices is operable to estimate a geographical location for the at least one UV and/or the corresponding UV controller based on the line of bearing from each of the multiplicity of node devices.

18. A method for unmanned vehicle (UV) recognition in a radio frequency (RF) environment, comprising:

providing a system comprising a multiplicity of RF receivers and a displaying device in network communication with at least one node device, wherein the at least one node device each comprises a signal processing engine, a detection engine, a direction-finding engine, and at least one artificial intelligence (AI) engine;

the multiplicity of RF receivers capturing RF data in the RF environment, converting the RF data to fast Fourier transform (FFT) data, and transmitting the FFT data to the at least one node device;

the signal processing engine averaging the FFT data into at least one tile;

the detection engine grouping the FFT data into discrete FFT bins over time, calculating average and standard deviations of power for the discrete FFT bins, and identifying at least one signal related to at least one UV and/or corresponding at least one UV controller;

the at least one AI engine generating an output for each of the at least one tile to identify the at least one UV and/or the corresponding at least one UV controller with a probability;

the direction-finding engine calculating a line of bearing for the at least one UV; and

the displaying device displaying and the line of bearing of the at least one UV.

19. The method of claim 18, further comprising the at least one node device training the at least one AI engine for UV recognition by capturing and recording signals from a multiplicity of UVs and corresponding UV controllers, respectively over different channels and different RF bandwidths.

20. The method of claim 18, further comprising tuning the multiplicity of receivers to a different frequency span.