Drone tours in security systems

• US 11,157,021 B2
• Filed: 10/17/2014
• Issued: 10/26/2021
• Est. Priority Date: 10/17/2014
• Status: Active Grant

• Current Assignee: Johnson Controls Tyco IP Holdings LLP (Johnson Controls International plc)
• Original Assignee: Tyco Fire & Security GmbH (Johnson Controls International plc)
• Inventors: Mucci, Anthony
• Primary Examiner(s): Wiltey, Nicholas K
• Application Number:
  US14/516,651
• Publication Number:
  US 20160116914A1
• Time in Patent Office: 2,566 Days
• Field of Search: None
• CPC Class Codes: B64C 39/024 of the remote controlled vehicle type, i.e. RPV
• CPC Class Codes: B64U 10/13 Flying platforms
• CPC Class Codes: B64U 2101/30 for imaging, photography or videography
• CPC Class Codes: B64U 2101/31 for surveillance
• CPC Class Codes: B64U 2201/20 Remote controls
• CPC Class Codes: G05D 1/0011 associated with a remote control arrangement
• CPC Class Codes: G05D 1/0088 characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours using knowledge based models G06N5/00
• CPC Class Codes: G05D 1/101 specially adapted for aircraft
• CPC Class Codes: G08B 13/19604 involving reference image or background adaptation with time to compensate for changing conditions, e.g. reference image update on detection of light level change
• CPC Class Codes: G08B 13/1965 the vehicle being an aircraft
• CPC Class Codes: G08G 5/003 Flight plan management

Abstract

An unmanned aerial vehicle is described and includes a computer carried by the unmanned aerial vehicle to control flight of the unmanned aerial vehicle and at least one sensor. The unmanned aerial vehicle is caused to fly a specific pattern within a facility, receive sensor data from a sensor carried by the vehicle, apply processing to the sensor data to detect an unacceptable level of detected feature differences in features contained in the sensor data, determine a new flight instruction for the vehicle based on the processing; and send the new flight instruction for the vehicle to a system for controlling flight of the vehicle.

Claims

1. A method, comprising:

sending, by a computer that assists in controlling flight of a drone, to the drone, a first graph data structure defining a first specific flight route within a facility, the first graph data structure comprising a set of nodes corresponding to a first set of waypoints in the facility and a set of edges that define relationships between the nodes, the drone using the first set of waypoints in the facility to avoid first obstacles in the first specific flight route and to recharge a battery on the drone or refuel based on a remaining power of the battery or fuel supply;

receiving, by the computer, sensor data from a sensor device carried by the drone, the sensor data corresponding to a first location in the facility along the first specific flight route;

processing, by the computer, the sensor data to detect feature differences in the sensor data by comparing the sensor data to stored sensor data previously captured at the first location in the facility;

generating, by the computer and responsive to detecting the feature differences, a first alarm condition based at least in part on the detected feature differences;

receiving, by the computer, an indication of a second alarm condition from a security system of the facility, the second alarm corresponding to a second location in the facility;

determining, by the computer, a second, different flight route based on the first or second alarm condition;

sending, by the computer and to the drone, a second graph data structure defining the second, different flight route, the second graph data structure comprising a set of nodes corresponding to a second set of waypoints in the facility and a set of edges that define relationships between the nodes, the drone using the second set of waypoints to fly in the facility to avoid second obstacles in the second, different flight route and to recharge the battery on the drone or refuel based on the remaining power of the battery or fuel supply, wherein the drone is configured to fly to the second location in the facility if the second, different flight route is determined based on the second alarm condition;

causing, by the computer, the drone to check in at pre-programmed location stations associated one or more of the first or second set of waypoints within the facility to indicate its presence; and

receiving, by the computer and in response to the drone checking in at the pre-programmed location stations, records that detail time, identity of the pre-programmed stations, and identity of the drone.

2. The method of claim 1, wherein the first specific route is a scheduled tour, and the method further comprises:

causing the drone to make an initial tour of the facility;

receiving by the computer second sensor data collected by the sensor of the drone on the initial tour; and

processing by the computer the second sensor data to provide the stored sensor data, wherein the initial tour occurs prior to the scheduled tour and the scheduled tour is one of a plurality of scheduled tours of the facility.

3. The method of claim 2, further comprising:

retrieving from a database sensor data captured from a previous day'"'"'s tour; and

processing the sensor data captured during the initial tour to detect feature differences by comparing the sensor data captured during the initial tour to the sensor data captured from the previous day'"'"'s tour to determine a baseline for subsequent comparisons for subsequent tours of the plurality of scheduled tours.

4. The method of claim 1,

wherein the check in is performed by transmitting, by the drone and to a beacon associated with each of the pre-programmed locations, a low power signal.

5. The method of claim 1, further comprising:

causing the drone to return to one of the of pre-programmed location stations,

wherein each of the pre-programmed location stations include a device that sends a signal to the drone to identify a corresponding location station to the drone.

6. The method of claim 1 wherein the system for controlling flight of the drone includes the computer, with the computer being carried by the drone.

7. The method of claim 1 wherein the computer is a first computer, and system for controlling flight of the drone includes a second computer, with the first computer being a server system that is remote from the drone.

8. The method of claim 1, wherein the computer is a first computer and the system to apply feature recognition is a second computer system that is remote from the drone.

9. An unmanned aerial vehicle, comprising:

a body supporting one or more electric motors and one or more propellers, a computer to control flight of the unmanned aerial vehicle, and at least one sensor device carried by the unmanned aerial vehicle, with the computer configured to;

cause the unmanned aerial vehicle to fly a first specific flight route within a facility while using a first graph data structure corresponding to a first set of waypoints in the facility to avoid obstacles in the first specific flight route and to recharge a battery on the unmanned aerial vehicle or refuel based on a remaining power of the battery or fuel supply, wherein the unmanned aerial vehicle is configured to check-in at each of the first set of waypoints by transmitting a low power signal to a detector associated with each of the first set of waypoints;

receive sensor data from at least one sensor device carried by the unmanned aerial vehicle and corresponding to a first location in the facility along the first specific flight route;

send the sensor data to a remote system that is configured to;

apply feature detection processing to the sensor data to detect feature differences contained in the sensor data with respect to baseline data stored by a remote system, the baseline data associated with the first location;

generate, responsive to detecting a difference between features in the sensor data and the baseline, an alarm condition based at least in part on the detected feature differences;

receive an indication of a second alarm condition from a security system of the facility, the second alarm corresponding to a second location in the facility;

determine a second specific route that is a different flight route based on the first or the second alarm condition;

program the unmanned aerial vehicle with the second specific that is the different flight route;

control flight of the unmanned aerial vehicle over the second specific route that is the different flight route, using a second graph data structure corresponding to a second set of waypoints in the facility to avoid second obstacles in the second specific route, different flight route and to recharge the battery on the unmanned aerial vehicle or refuel based on the remaining power of the battery or fuel supply, wherein the unmanned aerial vehicle is configured to fly to the second location in the facility if the second specific route that is the different flight route is determined based on the second alarm condition, and wherein the unmanned aerial vehicle is configured to check-in at each of the second set of waypoints by transmitting a low power signal to a detector associated with each of the second set of waypoints; and

in response to the unmanned aerial vehicle checking in at a waypoint of the second set of waypoints, recording a time of the check in, an identity of waypoint, and identity of the unmanned aerial vehicle.

10. The unmanned aerial vehicle of claim 9, wherein the second, specific route includes flight instructions that include flight instructions to hover over an area.

11. The unmanned aerial vehicle of claim 9, further comprising:

outputting by the computer an indication of features differences to cause the drone to fly the second specific route.

12. The unmanned aerial vehicle of claim 9, wherein the first specific route is a scheduled tour.

13. The unmanned aerial vehicle of claim 12, further configured to:

cause the unmanned aerial vehicle to make an initial tour of the facility prior to the scheduled tour;

retrieve from a database sensor data captured during a previous day'"'"'s tour; and

process the sensor data captured during the initial tour to detect feature differences by comparing the sensor data captured during the initial tour to the sensor data captured from the previous day'"'"'s tour to determine a baseline.

14. The unmanned aerial vehicle of claim 9, wherein the system to apply feature recognition is the computer.