Automated drone systems

• US 10,303,167 B2
• Filed: 12/05/2017
• Issued: 05/28/2019
• Est. Priority Date: 03/12/2015
• Status: Active Grant

• Current Assignee: Nightingale Intelligent Systems
• Original Assignee: Nightingale Intelligent Systems
• Inventors: Chen, Ian, Boyer, Hugo, Hsu, John Ming-Jey, Gaugel, Frank Berthold Alfons, Bauk, Davor, Schoter, Arnd, Damm, Cyril, Wu, Jack
• Primary Examiner(s): Wong, Yuen
• Application Number:
  US15/832,648
• Publication Number:
  US 20180095460A1
• Time in Patent Office: 539 Days
• CPC Class Codes: B64U 2101/31 for surveillance
• CPC Class Codes: B64U 2201/20 Remote controls
• CPC Class Codes: B64U 50/38 by wireless transmission
• CPC Class Codes: B64U 70/90 Launching from or landing on platforms
• CPC Class Codes: B64U 80/70 in containers B64U80/60 takes precedence
• CPC Class Codes: G05D 1/0038 by providing the operator with simple or augmented images from one or more cameras located onboard the vehicle, e.g. tele-operation images analyzed by a computer and used for automatic navigation G05D1/0246
• CPC Class Codes: G05D 1/0044 by providing the operator with a computer generated representation of the environment of the vehicle, e.g. virtual reality, maps maps used for automatic navigation G05D1/0274; flight directors G01C23/005
• CPC Class Codes: G06V 20/10 Terrestrial scenes scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59
• CPC Class Codes: G06V 20/17 taken from planes or by drones
• CPC Class Codes: G06V 20/176 Urban or other man-made structures
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G07C 5/008 communicating information to a remotely located station transmission systems for measured values G08C
• CPC Class Codes: G08C 17/02 using a radio link
• CPC Class Codes: G08G 5/0013 with a ground station
• CPC Class Codes: G08G 5/0026 located on the ground
• CPC Class Codes: G08G 5/0052 for cruising combined instruments indicating more than one navigational value G01C23/00
• CPC Class Codes: G08G 5/006 in accordance with predefined flight zones, e.g. to avoid prohibited zones
• CPC Class Codes: G08G 5/0069 specially adapted for an unmanned aircraft
• CPC Class Codes: G08G 5/0078 for monitoring traffic from the aircraft radar or analogous systems specially adapted for traffic control G01S13/91
• CPC Class Codes: G08G 5/0086 for monitoring terrain radar or analogous systems specially adapted for terrain avoidance G01S13/935
• CPC Class Codes: G08G 5/045 : Navigation or guidance aids, e.g. determination of anti-collision manoeuvers radar or analogous systems specially adapted for anti-collision between aircraft G01S13/933
• CPC Class Codes: H04L 65/613 : for the control of the source by the destination control signals issued by the client directed to the server or network components specially adapted for selective content distribution H04N21/637
• CPC Class Codes: H04L 67/12 : specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
• CPC Class Codes: H04L 67/125 : involving control of end-device applications over a network

Abstract

An automated drone security system for surveilling a location includes one or more drones with onboard sensors and an imaging device for measuring surveillance data. The surveillance data may include images, telemetry data, infrared data, or other detectable information of the location. Drones may be capable of executing one or multiple flight operations as well as storing and transmitting the surveillance data to a server assembly operable for coordinating the drone and receiving the surveillance data. A drone dock may be included for drone launching, landing, and/or storing the drones. A user computing device may be in communication with the server assembly and the drone(s), the user computing device being capable of receiving user input and displaying surveillance data from the drone. Flight operations associated with surveilling the location may be automatically and/or manually controlled by the user computing device and/or or the server assembly in connection with the location.

Claims

1. A security system for surveilling a location, comprising:

a drone comprising one or more onboard sensors and an imaging device for measuring surveillance data at least comprising images and telemetry data of the location, the drone being capable of executing a flight operation for surveilling the location for a period of time,

storing and transmitting the surveillance data to a server assembly;

the server assembly for coordinating the drone and receiving the surveillance data measured from the drone;

a drone dock operable for drone launching, landing, and/or storing the drone; and

a user computing device in communication with the server assembly and the drone, the user computing device having a non-transitory storage medium, a processor for processing the surveillance data between the server assembly and the drone, and a user interface for receiving user input and displaying surveillance data from the drone, wherein the user computing device is configured for;

continuously monitoring the location by forming a continuously updated surveillance map and one or more live streams of the location;

configuring alert message parameters for transmitting one or more alert messages and recipients of the one or more alert messages;

selectively viewing the one or more live streams;

deactivating and/or reactivating the drone;

controlling one or more parameters of the onboard sensors of the drone including minimum thresholds for a predetermined alert or pan, zoom, tilt, direction, and frames per second for the imaging device; and

activating one or more alarm conditions on one of the onboard sensors or the imaging device;

wherein the flight operation is automatically and/or manually controlled by the user computing device or the server assembly in connection with the location.

2. The system of claim 1, wherein a battery of the drone is charged by the drone dock through a conduction pad or through an inductive charging device.

3. The system of claim 2, wherein when the battery of the drone has a minimum charge, the drone is automatically returned to the drone dock for charging.

4. The system of claim 2, the system further comprising flight and security control logic comprising one or more of the following processes:

executive logic defined by one or more predefined flight and operation procedures triggered by sensor states associated with the one or more onboard sensors of the drone including velocity, a timer, an inertial measurement unit, and/or a global positioning system (GPS);

flight range prediction logic defined by a status of the battery of the drone and environmental conditions including wind speed and direction, humidity, altitude, temperature, and air pressure, and a flight trajectory planner associated with one or more flight operations; and

autonomous logic associated with the flight range prediction logic including collision avoidance logic and control and encryption for information transmitted between the drone, the server assembly and the user computing device.

5. The system of claim 4, the collision avoidance logic including a detect and track module associated with static and dynamic structures and obstacles defined by a predetermined location, a predetermined flight path, and/or static and dynamic structures and obstacles sensed by the onboard sensors of the drone.

6. The system of claim 4, the system further comprising:

one or more additional drones and associated onboard computing systems with flight and security control logic, imaging devices, onboard sensors and associated sensor states;

wherein the user computing device or the server assembly partially or automatically manage a flight operation associated with each of the one or more additional drones for continuous dynamic aerial coverage of the location.

7. The system of claim 6, the system further comprising a process comprising:

forming or monitoring a continuously-updated two-dimensional and/or three-dimensional map of the location with image stitching logic defined by two- and/or three-dimensional blending of previously and live recorded two- and/or three-dimension data gathered from the one or more onboard sensors and imaging devices of each drone.

8. The system of claim 6, wherein one or more of the drones of the system maintain connectivity with the server assembly or the user computing device through 3G/4G, RF, and/or a local wireless network.

9. The system of claim 8, wherein one or more of the drones or the user computing device is automatically configured based on data detected from onboard sensors of the one or more drones, alert messages are automatically configured based on one or more predetermined detected parameters of the location or one or more of the drones, and causes alert messages to be manually or automatically transmitted to the user computing device or other user computing devices securely connected therewith.

10. The system of claim 6, further comprising:

a plurality of drone docks positioned on or throughout the location, wherein each drone is capable of being housed, landing upon, or receiving a charge to its battery in connection with any one of the plurality of drone docks,

wherein each drone dock is wirelessly connectable to the server assembly and the user computing device to store and transmit data from one or more of the drones docked thereon.

11. The system of claim 6, the user computing device further comprising an event scheduling manager for manually or automatically selecting one of the drones to execute one of a plurality of flight operations.

12. The system of claim 6, the user computing device further comprising an event viewer for live viewing the flight operation being executed by drones of the system and for viewing previously captured data by the drones of the system.

13. The system of claim 2, wherein the drone operates according to one or more of the following states:

a non-operational state defined by being docked on the drone dock;

a charging state defined by receiving a charge on the drone dock;

an on-duty state when the drone is executing the flight operation; and

a malfunction state when the drone is malfunctioned and needs to be repaired.

14. The system of claim 1, the system further comprising a video feed system comprising one or more of the following processes:

stream quarter video graphics array (QVGA) video from an onboard computing system of the drone to a cloud server associated with the server assembly with a predetermined latency;

relay QVGA video from the cloud server to the user computing device with a predetermined latency; and

stream or upload video in a predetermined format from the onboard computing system of the drone to the cloud server, wherein the predetermined format is determined pending network bandwidth and encryption.

15. The system of claim 1, wherein the user interface displays operational status of the drone.

16. The system of claim 1, wherein the server assembly comprises:

a database server operatively connected to one or more web servers across one or more networks, each server operable to permanently store and continuously update a database of master surveillance data, telemetry information, mission data, collision mapping, drone coordination, air traffic control, captured images, and video streams;

a stream server for live video and image streaming; and

a notify server for automatically transmitting alert messages between the drones, the user computing device, and the server assembly;

wherein a web service is dynamically provisioned pending network bandwidth associated with the one or more networks, encryption, and the video and image streaming.

17. An automated drone security system comprising:

a drone comprising one or more onboard sensors and an imaging device for measuring surveillance data comprising images and telemetry data of the location, the drone being capable of executing a flight operation for a period of time, storing and transmitting the surveillance data to a server assembly;

the server assembly for coordinating the drone and receiving data measured from the drone;

a drone dock operable for drone launching, landing, and/or storing the drone; and

a mobile device being in communication with the server assembly and the drone, the mobile device comprising a non-transitory storage medium, a processor for processing data transmitted between the server assembly and the drone, and a user interface for receiving user input and displaying data transmitted from the drone, wherein the mobile device is configured for;

continuously monitoring the location by forming a continuously updated surveillance map and one or more live streams of the location;

configuring alert message parameters for transmitting one or more alert messages and recipients of the one or more alert messages;

selectively viewing the one or more live streams;

deactivating and/or reactivating the drone;

controlling one or more parameters of the onboard sensors of the drone including minimum thresholds for a predetermined alert or pan, zoom, tilt, direction, and frames per second for the imaging device; and

activating one or more alarm conditions on one of the onboard sensors or the imaging device.

18. The mobile device of claim 17, the server assembly further comprising flight and security control logic comprising one or more of the following processes:

executive logic defined by one or more predefined flight and operation procedures triggered by sensor states associated with onboard sensors of the drone including velocity, a timer, an inertial measurement unit, or a global positioning system (GPS);

flight range prediction logic defined by a status of the battery of the drone and environmental conditions including wind speed and direction, humidity, altitude, temperature, and air pressure, and a flight trajectory planner associated with one or more flight operations; and

autonomous logic associated with the flight range prediction logic including collision avoidance logic and control and encryption for information transmitted between the drone, the server assembly and the mobile device.

19. The mobile device of claim 17, the automated drone security system further comprising one or more additional drones and associated onboard computing systems with flight and security control logic, imaging devices, and sensor state;

wherein the mobile device or the server assembly partially or automatically manage a flight operation associated with each drone for continuous dynamic aerial coverage of the location; and

wherein a continuously-updated two-dimensional and three-dimensional map is formed on the user interface of the mobile device of the location with image stitching logic defined by two- and/or three-dimensional blending of previously and live recorded two- and/or three-dimension data gathered from the one or more onboard sensors and imaging devices of each drone.