Systems, methods, and devices for unmanned vehicle detection

• US 10,498,951 B2
• Filed: 02/13/2019
• Issued: 12/03/2019
• Est. Priority Date: 01/23/2017
• Status: Active Grant

• Current Assignee: Digital Global Systems, Inc.
• Original Assignee: Digital Global Systems, Inc.
• Inventors: Kleinbeck, David William
• Primary Examiner(s): Vo, Nguyen T
• Application Number:
  US16/274,933
• Publication Number:
  US 20190208112A1
• Time in Patent Office: 293 Days
• CPC Class Codes: G01R 29/0892 Details related to signal analysis or treatment; presenting results, e.g. displays; measuring specific signal features other than field strength, e.g. polarisation, field modes, phase, envelope, maximum value
• CPC Class Codes: G01S 5/0221 Receivers
• CPC Class Codes: G01S 5/02585 at least one of the measurements being a non-radio measurement
• CPC Class Codes: G01S 5/12 by co-ordinating position lines of different shape, e.g. hyperbolic, circular, elliptical or radial
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20056 Discrete and fast Fourier transform, [DFT, FFT]
• CPC Class Codes: G06T 2207/30212 Military
• CPC Class Codes: G06T 2207/30232 Surveillance
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G08B 25/08 using communication transmission lines G08B13/19658, G08B21/0286, G08B25/016 take precedence
• CPC Class Codes: G08B 25/10 using wireless transmission systems G08B25/009 takes precedence
• CPC Class Codes: G08B 29/185 Signal analysis techniques for reducing or preventing false alarms or for enhancing the reliability of the system
• CPC Class Codes: G08B 5/36 using visible light sources
• CPC Class Codes: H04N 23/60 Control of cameras or camera modules
• CPC Class Codes: H04N 23/61 based on recognised objects
• CPC Class Codes: H04N 23/69 Control of means for changing angle of the field of view, e.g. optical zoom objectives or electronic zooming
• CPC Class Codes: H04N 23/695 Control of camera direction for changing a field of view, e.g. pan, tilt or based on tracking of objects
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

Systems, methods, and apparatus for detecting UAVs in an RF environment are disclosed. An apparatus is constructed and configured for network communication with at least one camera. The at least one camera captures images of the RF environment and transmits video data to the apparatus. The apparatus receives RF data and generates FFT data based on the RF data, identifies at least one signal based on a first derivative and a second derivative of the FFT data, measures a direction from which the at least one signal is transmitted, analyzes the video data. The apparatus then identifies at least one UAV to which the at least one signal is related based on the analyzed video data, the RF data, and the direction from which the at least one signal is transmitted, and controls the at least one camera based on the analyzed video data.

Claims

1. An apparatus for detecting unmanned aerial vehicles in a radio frequency (RF) environment, comprising:

at least one RF receiver, an RF analytics module, a direction finding (DF) module, and a video analytics module;

wherein the apparatus is in network communication with at least one video sensor;

wherein the at least one video sensor is configured to capture images of the RF environment and stream video data to the apparatus;

wherein the at least one RF receiver is configured to receive RF data and generate fast Fourier transform (FFT) data based on the RF data;

wherein the RF analytics module is configured to identify at least one signal based on a first derivative and a second derivative of the FFT data;

wherein the DF module is configured to measure a direction from which the at least one signal is transmitted;

wherein the video analytics module is configured to analyze the video data, thereby creating analyzed video data;

wherein the video analytics module is configured to identify at least one unmanned aerial vehicle to which the at least one signal is related based on the analyzed video data, the RF data, and the direction from which the at least one signal is transmitted; and

wherein the apparatus is configured to control the at least one video sensor based on the direction from which the at least one signal is transmitted.

2. The apparatus of claim 1, wherein the video analytics module is configured to determine a distance of the at least one unmanned aerial vehicle and the inclination or declination of the at least one unmanned aerial vehicle based on the analyzed video data.

3. The apparatus of claim 1, wherein the video analytics module is configured to determine a type of the at least one unmanned aerial vehicle based on the analyzed video data.

4. The apparatus of claim 1, wherein the video analytics module is configured to detect if the at least one unmanned aerial vehicle has at least one future payload.

5. The apparatus of claim 1, wherein the video analytics module is further operable to determine if the at least one unmanned aerial vehicle is utilizing more than one camera based on the analyzed video data and the RF data.

6. The apparatus of claim 1, wherein the at least one signal comprises at least one drone radio signal.

7. The apparatus of claim 1, wherein the at least one signal comprises at least one drone controller signal.

8. The apparatus of claim 1, wherein the video analytics module is further operable to determine if there are more than one unmanned aerial vehicle when the at least one signal comprises multiple drone radio signals.

9. The apparatus of claim 1, wherein the at least one signal comprises a multiplicity of drone controller signals, and wherein the video analytics module is further operable to detect one or more unmanned aerial vehicles associated with the multiplicity of drone controller signals.

10. The apparatus of claim 1, wherein the video analytics module is operable to track a formation of the at least one unmanned aerial vehicle.

11. The apparatus of claim 1, wherein the video analytics module is operable to detect humans, animals, and land-based vehicles based on the analyzed video data.

12. The apparatus of claim 1, wherein the apparatus is operable to measure a deterministic approximation of a flight path for the at least one unmanned aerial vehicle based on the RF data and the analyzed video data.

13. The apparatus of claim 1, wherein the apparatus is operable to provide points of reference to the video sensor.

14. A system for detecting unmanned aerial vehicles in a radio frequency (RF) environment, comprising:

at least one apparatus constructed and configured for network communication with at least one camera;

wherein the at least one apparatus comprises at least one RF receiver, an RF analytics module, a direction finding (DF) module, and a video analytics module;

wherein the at least one camera is configured to capture images of the RF environment and stream video data to the apparatus;

wherein the at least one RF receiver is configured to receive RF data and generate fast Fourier transform (FFT) data based on the RF data;

wherein the RF analytics module is configured to identify at least one signal based on a first derivative and a second derivative of the FFT data;

wherein the DF module is configured to measure a direction from which the at least one signal is transmitted;

wherein the video analytics module is configured to analyze the video data, thereby creating analyzed video data;

wherein the video analytics module is configured to identify at least one unmanned aerial vehicle to which the at least one signal is related based on the analyzed video data, the RF data, and the direction from which the at least one signal is transmitted; and

wherein the at least one apparatus is configured to control the at least one camera based on the direction from which the at least one signal is transmitted.

15. The system of claim 14, wherein each of the at least one camera comprises a multiplicity of lenses.

16. The system of claim 15, wherein the multiplicity of lenses includes a synthesized aperture between 70 degrees and 150 degrees.

17. The system of claim 14, wherein the at least one camera is stationary.

18. The system of claim 14, wherein the at least one camera has pan, tilt, and zoom features.

19. A method for detecting unmanned aerial vehicles in a radio frequency (RF) environment, comprising:

providing at least one apparatus constructed and configured for network communication with at least one camera, wherein the at least one apparatus comprises at least one RF receiver, an RF analytics module, a direction finding (DF) module, and a video analytics module;

the at least one RF receiver receiving RF data in the RF environment and generating fast Fourier transform (FFT) data based on the RF data;

the RF analytics module identifying at least one signal based on a first derivative and a second derivative of the FFT data;

the DF module measuring a direction from which the at least one signal is transmitted;

the at least one apparatus providing a point of reference for the at least one camera based on the direction from which the at least one signal is transmitted;

the at least one camera capturing images of the RF environment and streaming video data to the at least one apparatus;

the video analytics module analyzing the video data, thereby creating analyzed video data; and

the at least one apparatus identifying at least one unmanned aerial vehicle to which the at least one signal is related based on the analyzed video data, the RF data, and the direction from which the at least one signal is transmitted.

20. The method of claim 19, wherein the at least one apparatus comprises a multiplicity of apparatus and the at least one camera comprises a multiplicity of cameras, and wherein the multiplicity of apparatus and the multiplicity of cameras are in a cluster layout connecting to an analytics and control platform.