Unmanned aerial vehicle based surveillance as a service

• US 10,322,801 B1
• Filed: 06/12/2015
• Issued: 06/18/2019
• Est. Priority Date: 06/12/2015
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Yeturu, Kalidas
• Primary Examiner(s): Kim, Hee-Yong
• Application Number:
  US14/738,444
• Time in Patent Office: 1,467 Days
• Field of Search: 348144
• CPC Class Codes: B64U 2101/31 for surveillance
• CPC Class Codes: B64U 2101/64 for parcel delivery or retrieval
• CPC Class Codes: G06Q 10/083 Shipping
• CPC Class Codes: G06T 2207/30181 Earth observation
• CPC Class Codes: G06T 2207/30232 Surveillance
• CPC Class Codes: G06V 20/13 Satellite images
• CPC Class Codes: G06V 20/17 taken from planes or by drones
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

The delivery of a package using an unmanned aerial vehicle (UAV) may include performing a surveillance action at a property of an authorized party. The surveillance action may include imaging the property, and may be performed before or after the package is delivered. Surveillance data such as image data gathered in a surveillance action may be modified to remove or obscure adjacent properties not associated with the property of the authorized party. Based on surveillance data, the probability or confidence value of a surveillance event may be determined and a surveillance alert may be generated. Surveillance actions may be scheduled by a central controller to ensure that the surveillance action may be performed without exhausting the resources, such as battery power or time, of the UAV. Surveillance actions may be received as an interrupt and may have an associated priority level.

Claims

1. A method to facilitate a delivery of a package using an unmanned aerial vehicle (UAV), the method comprising:

dispatching the UAV to travel from an origination location to deliver the package at a destination location;

determining one or more remaining resources of the UAV after the delivery of the package at the destination location;

receiving a list of surveillance locations and one or more surveillance actions to be performed at each surveillance location in the list of surveillance locations;

determining one or more anticipated resources expected to be used to perform at least one surveillance action of the one or more surveillance actions;

determining that the one or more remaining resources is greater than or equal to the one or more anticipated resources;

scheduling the at least one surveillance action to be performed by the UAV based at least in part on the one or more remaining resources being greater than or equal to the one or more anticipated resources;

determining, based at least in part on scheduling the at least one surveillance action, a first surveillance location from the list of surveillance locations and one or more first surveillance actions at the first surveillance location, the one or more first surveillance actions including causing the UAV to deviate from a direct return path between the destination location and the origination location and to capture imagery, using a camera, of a property at the first surveillance location for an authorized party that is verified as owning or controlling the property;

assigning the at least one surveillance action to be performed by the UAV;

receiving the imagery captured by the UAV, the imagery depicting the property and an adjacent property;

modifying the imagery to at least obscure or remove depictions of the adjacent property while maintaining the imagery of the property;

detecting a surveillance event at the property;

creating a surveillance alert; and

sending the surveillance alert to an electronic device associated with an owner of the first surveillance location.

2. The method of claim 1, the modifying further comprising:

determining a boundary defining the property of the authorized party; and

obscuring or removing a portion of the imagery corresponding to an area outside of the boundary defining the property of the authorized party to generate geo-clipped surveillance data.

3. The method of claim 1, wherein the determining the one or more remaining resources of the UAV further comprises determining at least one of a remaining power resource, a time resource, or a sensor resource of the UAV;

and

wherein scheduling the at least one surveillance action to be performed by the UAV further comprises selecting the at least one surveillance action from the one or more surveillance actions based in part on a priority of each surveillance action and an amount of resources required by each surveillance action.

4. The method of claim 1, further comprising:

analyzing the imagery captured by the UAV;

determining a confidence value associated with the surveillance event based in part on the analyzing the imagery captured by the UAV; and

generating the surveillance alert for the surveillance event in response to the confidence value being above a threshold.

5. The method of claim 1, further comprising:

receiving one or more surveillance parameters from a user interface, the one or more surveillance parameters including at least one of;

a surveillance tier indicating a level of surveillance;

a monitoring type indicating a type of surveillance data to be gathered;

a frequency indicating a frequency of conducting surveillance;

or

an alert indicating a method of presenting surveillance data to a user; and

wherein the assigning the at least one surveillance action to be performed by the UAV is based in part on the one or more surveillance parameters.

6. A system comprising:

one or more processors; and

memory coupled to the one or more processors, the memory including one or more instructions that when executed by the one or more processors, cause the one or more processors to perform acts comprising;

dispatching a UAV to travel from an origination location to deliver a package at a destination location;

determining one or more remaining resources of the UAV after delivery of the package at the destination location;

receiving a list of surveillance locations and one or more surveillance actions to be performed at an individual surveillance location in the list of surveillance locations;

determining one or more anticipated resources associated with at least one surveillance action of the one or more surveillance actions;

determining that the one or more remaining resources is greater than or equal to the one or more anticipated resources;

scheduling the at least one surveillance action to be performed by the UAV based at least in part on the one or more remaining resources being greater than or equal to the one or more anticipated resources;

determining, based at least in part on scheduling the at least one surveillance action, a first surveillance location in the list of surveillance locations and one or more first surveillance actions at the first surveillance location, the one or more first surveillance actions comprising;

causing the UAV to deviate from a direct return path between the destination location and the origination location; and

causing the UAV to capture imagery of a property at the first surveillance location for an authorized party that is verified as owning or controlling the property;

assigning the at least one surveillance action to be performed by the UAV;

receiving the imagery captured by the UAV, the imagery depicting the property and an adjacent property;

modifying the imagery to at least one of obscure or remove depictions of the adjacent property;

detecting a surveillance event at the property;

creating a surveillance alert; and

sending the surveillance alert to an electronic device associated with an owner of the first surveillance location.

7. The system of claim 6, wherein the list of surveillance locations and the one or more surveillance actions is received from the owner of the first surveillance location.

8. The system of claim 6, wherein modifying the imagery comprises:

determining a boundary defining the property of the authorized party; and

obscuring or removing a portion of the imagery corresponding to an area outside of the boundary defining the property of the authorized party to generate geo-clipped surveillance data.

9. The system of claim 6, wherein the acts further comprise:

generating surveillance data from the at least one surveillance action; and

transmitting the surveillance data.

10. The system of claim 6, wherein at least one of the one or more remaining resources comprises at least one of:

a power resource indicating at least one of a remaining amount of a battery power or fuel of the UAV;

or

a sensor resource indicating at an availability of at least one sensor of the UAV.

11. The system of claim 6, wherein the acts further comprise:

determining a priority level of the at least one surveillance action; and

wherein assigning the at least one surveillance action is based at least in part on the priority level.

12. The system of claim 6, wherein the determining the one or more remaining resources of the UAV further comprises:

determining at least one of a remaining power resource, a time resource, or a sensor resource of the UAV; and

wherein scheduling the at least one surveillance action to be performed by the UAV further comprises selecting the at least one surveillance action from the one or more surveillance actions based in part on a priority of each surveillance action and an amount of resources required by each surveillance action.

13. The system of claim 6, wherein the acts further comprise:

analyzing the imagery captured by the UAV;

determining a confidence value associated with the surveillance event based in part on the analyzing the imagery captured by the UAV; and

generating the surveillance alert for the surveillance event in response to the confidence value being above a threshold.

14. A non-transitory computer-readable medium storing instructions executable by one or more processors, wherein the instructions, when executed, cause the one or more processors to perform operations comprising:

dispatching a UAV to travel from an origination location to deliver a package at a destination location;

determining one or more remaining resources of the UAV after delivery of the package at the destination location;

receiving a list of surveillance locations and one or more surveillance actions to be performed at an individual surveillance location in the list of surveillance locations;

determining one or more anticipated resources associated with at least one surveillance action of the one or more surveillance actions;

determining that the one or more remaining resources is greater than or equal to the one or more anticipated resources;

scheduling the at least one surveillance action to be performed by the UAV based at least in part on the one or more remaining resources being greater than or equal to the one or more anticipated resources;

determining, based at least in part on scheduling the at least one surveillance action, a first surveillance location in the list of surveillance locations and one or more first surveillance actions at the first surveillance location, the one or more first surveillance actions comprising;

causing the UAV to deviate from a direct return path between the destination location and the origination location; and

causing the UAV to capture imagery of a property at the first surveillance location for an authorized party that is verified as owning or controlling the property;

assigning the at least one surveillance action to be performed by the UAV;

receiving the imagery captured by the UAV, the imagery depicting the property and an adjacent property;

modifying the imagery to at least one of obscure or remove depictions of the adjacent property;

detecting a surveillance event at the property;

creating a surveillance alert; and

sending the surveillance alert to an electronic device associated with an owner of the first surveillance location.

15. The non-transitory computer-readable medium of claim 14, wherein modifying the imagery comprises:

determining a boundary defining the property of the authorized party; and

obscuring or removing a portion of the imagery corresponding to an area outside of the boundary defining the property of the authorized party to generate geo-clipped surveillance data.

16. The non-transitory computer-readable medium of claim 14, wherein at least one of the one or more remaining resources comprises at least one of:

a power resource indicating at least one of a remaining amount of a battery power or fuel of the UAV;

or

a sensor resource indicating at an availability of at least one sensor of the UAV.

17. The non-transitory computer-readable medium of claim 14, wherein the operations further comprise:

determining a priority level of the at least one surveillance action; and

wherein assigning the at least one surveillance action is based at least in part on the priority level.

18. The non-transitory computer-readable medium of claim 14, wherein the determining the one or more remaining resources of the UAV further comprises:

determining at least one of a remaining power resource, a time resource, or a sensor resource of the UAV; and

wherein scheduling the at least one surveillance action to be performed by the UAV further comprises selecting the at least one surveillance action from the one or more surveillance actions based in part on a priority of each surveillance action and an amount of resources required by each surveillance action.

19. The non-transitory computer-readable medium of claim 14, wherein the operations further comprise:

analyzing the imagery captured by the UAV;

determining a confidence value associated with the surveillance event based in part on the analyzing the imagery captured by the UAV; and

generating the surveillance alert for the surveillance event in response to the confidence value being above a threshold.

20. The non-transitory computer-readable medium of claim 14, wherein the operations further comprise:

receiving one or more surveillance parameters from a user interface, the one or more surveillance parameters including at least one of;

a surveillance tier indicating a level of surveillance;

a monitoring type indicating a type of surveillance data to be gathered;

a frequency indicating a frequency of conducting surveillance;

or

an alert indicating a method of presenting surveillance data to a user; and

wherein the assigning the at least one surveillance action to be performed by the UAV is based in part on the one or more surveillance parameters.