Obstacle awareness based guidance to clear landing space

• US 10,049,589 B1
• Filed: 09/08/2016
• Issued: 08/14/2018
• Est. Priority Date: 09/08/2016
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Boyd, Scott Patrick, Cui, Chengwu, Graber, Sarah, O'Brien, Barry James, Watson, Joshua John, Wilcox, Scott Michael
• Primary Examiner(s): Goldman, Richard A
• Application Number:
  US15/260,113
• Time in Patent Office: 705 Days
• Field of Search: 701 3, 701 4, 701 11, 701 16, 701 26, 2441374, 382103
• CPC Class Codes: B64U 2101/30 for imaging, photography or videography
• CPC Class Codes: B64U 2101/64 for parcel delivery or retrieval
• CPC Class Codes: B64U 2201/10 autonomous, i.e. by navigating independently from ground or air stations, e.g. by using inertial navigation systems [INS]
• CPC Class Codes: B64U 2201/20 Remote controls
• CPC Class Codes: B64U 30/20 Rotors; Rotor supports
• CPC Class Codes: G05D 1/0676 specially adapted for landing
• CPC Class Codes: G05D 1/106 Change initiated in response to external conditions, e.g. avoidance of elevated terrain or of no-fly zones
• CPC Class Codes: G06Q 10/083 Shipping
• CPC Class Codes: G06T 7/13 Edge detection
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06V 20/13 Satellite images
• CPC Class Codes: G06V 20/17 taken from planes or by drones
• CPC Class Codes: G08G 5/0034 Assembly of a flight plan
• CPC Class Codes: G08G 5/0069 specially adapted for an unmanned aircraft
• CPC Class Codes: G08G 5/025 Navigation or guidance aids radar or analogous systems specially adapted for landing purposes G01S13/913
• CPC Class Codes: H04N 13/204 using stereoscopic image cameras stereoscopic photography G03B35/00
• CPC Class Codes: H04N 13/207 using a single 2D image sensor
• CPC Class Codes: H04N 13/271 wherein the generated image signals comprise depth maps or disparity maps

Abstract

Techniques for providing an object awareness guidance to clear a landing space may be provided. For example, during delivery an unmanned aerial vehicle (UAV) may capture an image of a potential landing zone and identify one or more objects in the image that may impede or obstruct delivery of the item in the potential landing zone. The UAV may be configured to generate and provide instructions to a user device to move or remove the identified one or more objects from the potential landing zone thereby creating a safe and unobstructed landing zone to deliver the item.

Claims

1. A computer-implemented method, comprising:

receiving, by a computer system, an order to deliver a package associated with an electronic marketplace;

generating, by the computer system, a flight plan that instructs an unmanned aerial vehicle (UAV) to deliver the package to a location based in part on information associated with the order;

receiving, by the computer system and from the UAV upon arriving at the location, a first image of an area associated with the location, the area including a landing marker utilized to guide the UAV to deliver the package;

identifying, by the computer system, one or more objects in the area utilizing the first image;

determining a time period in which to attempt to communicate with a user device associated with the location upon identifying the one or more objects in the area based at least in part on a distance between the location and an origination location for the UAV;

transmitting, by the computer system and to the user device, instructions for moving the one or more objects in the area based at least in part on the time period; and

instructing, by the computer system, the UAV to land and deliver the package on the landing marker in response to receiving a second image of the area that does not include the one or more objects.

2. The computer-implemented method of claim 1, wherein identifying the one or more objects in the area further includes using the first image as a data set for edge detection algorithms.

3. The computer-implemented method of claim 1, wherein transmitting the instructions for moving the one or more objects in the area includes transmitting the instructions to a speaker component associated with the UAV.

4. The computer-implemented method of claim 1, wherein the UAV captures the first image utilizing one or more stereoscopic cameras associated with the UAV.

5. An unmanned aerial vehicle (UAV), comprising:

an image capture component;

a set of propellers associated with a propulsion system configured to provide propulsion for the UAV; and

a computing system configured to manage the propulsion system during a flight associated with a delivery of a payload and further configured to instruct the UAV to;

determine, by the computing system, arrival at a location associated with a delivery order for an item;

obtain, via the image capture component, a first image of an area of the location, the area including a landing marker;

identify, by the computing system, an object in the area utilizing the obtained first image;

transmit, by the computing system and to a user device associated with the location, a notification that identifies the object in the area; and

identify, by the computing system, an alternate area associated with the location based at least in part on the first image, the alternate area including another landing marker.

6. The UAV of claim 5, wherein the notification that identifies the object in the area includes the first image obtained via the image capture component.

7. The UAV of claim 6, wherein the first image is altered to highlight the object in the area prior to providing the first image to the user device.

8. The UAV of claim 5, further comprising an auditory component and a visual component, the auditory component and the visual component configured to transmit instructions for moving the object in the area.

9. The UAV of claim 5, wherein transmitting the notification includes providing the notification to an application of the user device for notifying a user about the identified object in the area.

10. The UAV of claim 5, wherein the computing system is further configured to instruct the UAV to transmit a subsequent notification about the identified object in the area to another user device of the alternate area.

11. The UAV of claim 5, wherein the computer system is further configured to instruct the UAV to deliver the item to the alternate area using the other landing marker in response to failing to identify one or more objects in the alternate area from a second image of the alternate area.

12. A computer-implemented method, comprising:

generating, by a computer system, a flight plan that instructs an unmanned aerial vehicle (UAV) to deliver a package to a location based in part on information associated with an order for the package;

receiving, by the computer system and from the UAV upon arriving at the location, a first image of an area associated with the location, the area including a landing marker utilized to guide the UAV to deliver the package;

identifying, by the computer system, one or more objects in the area utilizing the first image;

determining, by the computer system, a time period in which to attempt to communicate with a user device associated with the location upon identifying the one or more objects in the area based at least in part on a distance between the location and an origination location for the UAV;

transmitting, by the computer system and to the user device, instructions for moving the one or more objects in the area based at least in part on the time period;

in response to receiving, by the computer system and from the UAV, a second image of the area;

identifying, by the computer system, that the area does not include the one or more objects; and

instructing, by the computer system, the UAV to land and deliver the package on the landing marker in response to identifying that the area does not include the one or more objects.

13. The computer-implemented method of claim 12, wherein identifying the one or more objects in the area further includes using the first image as a data set for edge detection algorithms.

14. The computer-implemented method of claim 12, wherein transmitting the instructions for moving the one or more objects in the area includes transmitting the instructions to a speaker component associated with the UAV.

15. The computer-implemented method of claim 12, wherein the UAV captures the first image utilizing one or more stereoscopic cameras associated with the UAV.

16. The computer-implemented method of claim 12, further comprising determining a time period in which to attempt to communicate with a user associated with the location upon identifying the one or more objects in the area based at least in part on a distance between the location and an origination location for the UAV.

17. The computer-implemented method of claim 12, wherein the UAV further comprises a visual component that is configured to transmit the instructions for moving the one or more objects in the area.