Fixed drone visualization in security systems

• US 10,301,018 B2
• Filed: 10/17/2014
• Issued: 05/28/2019
• Est. Priority Date: 10/17/2014
• Status: Active Grant

• Current Assignee: Johnson Controls Tyco IP Holdings LLP (Johnson Controls International plc)
• Original Assignee: Tyco Fire & Security GmbH (Johnson Controls International plc)
• Inventors: Mucci, Anthony
• Primary Examiner(s): Wiltey, Nicholas K
• Application Number:
  US14/516,671
• Publication Number:
  US 20160107749A1
• Time in Patent Office: 1,684 Days
• Field of Search: None
• CPC Class Codes: B64C 39/024 of the remote controlled vehicle type, i.e. RPV
• CPC Class Codes: B64U 2101/30 for imaging, photography or videography
• CPC Class Codes: B64U 2101/31 for surveillance
• CPC Class Codes: B64U 2201/10 autonomous, i.e. by navigating independently from ground or air stations, e.g. by using inertial navigation systems [INS]
• CPC Class Codes: G05D 1/0094 involving pointing a payload, e.g. camera, weapon, sensor, towards a fixed or moving target
• CPC Class Codes: G06Q 50/26 Government or public services business processes related to the transportation industry G06Q50/40
• CPC Class Codes: G06T 2207/30232 Surveillance
• CPC Class Codes: G08B 13/19695 Arrangements wherein non-video detectors start video recording or forwarding but do not generate an alarm themselves
• CPC Class Codes: G08B 25/001 Alarm cancelling procedures or alarm forwarding decisions, e.g. based on absence of alarm confirmation
• CPC Class Codes: G08B 25/08 using communication transmission lines G08B13/19658, G08B21/0286, G08B25/016 take precedence
• CPC Class Codes: G08B 29/185 Signal analysis techniques for reducing or preventing false alarms or for enhancing the reliability of the system

Abstract

An unmanned aerial vehicle is described and includes a computer carried by the unmanned aerial vehicle to control flight of the unmanned aerial vehicle and at least one sensor. The unmanned aerial vehicle is caused to fly to a specific location within a facility, where the unmanned aerial vehicle enters a hover mode, where the unmanned aerial vehicle remains in a substantially fixed location hovering over the specific location within the facility and sends raw or processing results of sensor data from the sensor to a remote server system.

Claims

1. A method comprises:

receiving by a computer system from a security system an assertion of an alarm condition;

establishing by the computer system, a graph data structure comprising a plurality of nodes and plurality of individual edges;

receiving, from a plurality of drones, navigation data associated with physical objects;

associating each individual edge in the plurality of edges of the graph data structure with the received navigation data associated with the physical objects;

determining, by the computer system a route for a drone of the plurality of drones, from an initial location of the drone to a radio frequency (RF) waypoint emitter where the alarm condition was asserted based on the graph data structure;

sending, by the computer system, a portion of the graph data structure based on the route to the drone causing the drone to fly to the RF waypoint emitter, using the portion of the graph data structure;

determining when the drone has reached the RF waypoint emitter;

causing the drone at the RF waypoint emitter to hover over a substantially fixed area about the RF waypoint emitter;

receiving, by the computer system, sensor data including image data captured by at least an image capture sensor device carried by the drone, with the image data obtained at the RF waypoint emitter;

retrieving a stored image previously obtained at a location;

applying, by the computer system, analytical processing to the received sensor data including the image data to detect feature differences in features contained in the sensor data obtained at the RF waypoint emitter in relation to the retrieved stored image; and

when a difference is detected,

outputting, by the computer system, an indication to the security system.

2. The method of claim 1, further comprising:

autonomously launching the drone from a drone station on demand by reception of the portion of the data structure by the drone.

3. The method of claim 1, further comprising:

determining a level of feature differences; and

determining an action based on the determined level of feature differences.

4. The method of claim 1 wherein the sensor data is video frames data and the method, further comprises:

processing by the server computer system, the sensor data captured by the drone by applying feature recognition processing that compares images previously captured with images currently captured to determine the detected feature differences, where the detected features are features different from features expected between the previously captured images and the images currently captured by the drone.

5. The method of claim 1, wherein the method further comprises:

sending the sensor data to a central monitoring station that processes the data to determine the existence of the feature differences; and

receiving a modified flight pattern from the central monitoring station;

producing by the central monitoring station, the modified flight pattern based results of analytic processing; and

reprogramming the drone with the new flight pattern to fly to a specific location within the indoor facility.

6. The method of claim 1, wherein the RF waypoint emitters are configured as at least one of an Asset to Protect (ATP) waypoint that is placed near an item or an area, a Home Asset to Protect (HATP) waypoint that is the home for the drone where the drone docks, recharges, and exchanges data with the server, or a Perimeter Point to Protect (PPP) waypoints that are placed along a route.

7. The method of claim 1, further comprising:

determining, by the computer system, the route based on navigation data of the data structure.

8. The method of claim 1, wherein the sensor data is a first set of sensor data, the method further comprising:

receiving by the computer system a second set of sensor data from the sensor carried by the drone as the drone flies to the specific location within a indoor facility.

9. The method of claim 1, wherein the drone comprises:

at least one sensor capable of collecting proximity data, making relative distance measurements and distinguishing individual objects; and

a computer capable of establishing a graph data structure, associating sensor data with the graph data structure, and sending the graph data structure to the computer system.

10. The method of claim 1, wherein the RF waypoint emitter comprises at least one of an Asset to Protect (ATP) waypoint that is placed near an item or an area, a Home Asset to Protect (HATP) waypoint that is a home for a drone where the drone docks, recharges, and exchanges data with the computer system, or a Perimeter Point to Protect (PPP) waypoint that is placed along a route.

11. An unmanned aerial vehicle comprises:

a body, having a configuration that permits the unmanned aerial vehicle to operate within confines of an indoor facility;

at least one sensor carried by the body of the unmanned aerial vehicle;

a computer carried by the body of the unmanned aerial vehicle to control flight of the unmanned aerial vehicle, wherein the computer is configured to;

establish a graph data structure comprising a plurality of individual nodes and a plurality of individual edges;

associate each individual node of the plurality of nodes of the graph data structure with an individual RF waypoint of a plurality of RF waypoint emitters;

receive, from a plurality of other drones, navigation data associated with physical objects;

associate each individual edge in the plurality of edges of the graph data structure with the received navigation data associated with the physical objects;

receive, from a computer system, a portion of the graph data structure based on a route from a current position of the unmanned aerial vehicle to a RF waypoint emitter, wherein the RF waypoint emitter is associated with a location where an assertion of an alarm condition in the indoor facility has occurred;

send, to the computer system, a message, in response to reaching the RF waypoint emitter;

cause the unmanned vehicle to fly to the RF waypoint emitter based on the portion of the graph data structure;

determine when the unmanned aerial vehicle has reached the RF waypoint emitter;

cause the unmanned aerial vehicle at the RF waypoint emitter to hover over a substantially fixed area about the RF waypoint emitter;

receive, from the at least one sensor, sensor data including image data captured about the RF waypoint emitter;

send, to the computer system, the sensor data.

12. The unmanned aerial vehicle of claim 11 wherein the computer is further configured to:

send the sensor data to the computer system that processes the data to determine the existence of feature differences; and

receive a modified flight pattern from the central monitoring station that produces the modified flight pattern that takes into consideration results of analytics processing to reprogram the unmanned aerial vehicle with the new flight pattern to fly to a specific location within the indoor facility.

13. The unmanned aerial vehicle of claim 11 wherein the computer is further configured to:

apply a waypoint based navigation process to cause the unmanned aerial vehicle to navigate to the specific location;

wherein a first type of waypoint is an Asset to Protect (ATP)

waypoint that is placed near an item or an area;

a second type of waypoint is Home Asset to Protect (HATP) waypoint where the unmanned aerial vehicle docks and recharges, and Perimeter Point to Protect (PPP) waypoints placed along a route.

14. The unmanned aerial vehicle of claim 11 wherein the sensor data is a first set of sensor data, the computer further configured to:

receive a second set of sensor data from the sensor carried by the unmanned aerial vehicle as the unmanned aerial vehicle flies to the RF waypoint emitter.

15. The unmanned aerial vehicle of claim 11, wherein the unmanned aerial vehicle is further configured to:

receive proximity data from the at least one sensor;

establish a second graph data structure comprising a plurality of nodes and a plurality of edges;

associate each individual node in the plurality of nodes of the second graph data structure with an individual physical object;

associate each individual edge in the plurality of edges of the second graph data structure with a relative distance;

send the second graph data structure to the computer system as navigation data.

16. The unmanned aerial vehicle of claim 11, wherein the at least one sensor is further configured to:

collect proximity data;

make relative distance measurements; and

distinguish individual objects.