Drone marker and landing zone verification

• US 10,198,955 B1
• Filed: 09/08/2016
• Issued: 02/05/2019
• Est. Priority Date: 09/08/2016
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Boyd, Scott Patrick, Cui, Chengwu, Graber, Sarah, O'Brien, Barry James, Watson, Joshua John, Wilcox, Scott Michael
• Primary Examiner(s): Melton, Todd M
• Assistant Examiner(s): Roberson, Jason R
• Application Number:
  US15/260,162
• Time in Patent Office: 880 Days
• Field of Search: None
• CPC Class Codes: B64C 39/022 Tethered aircraft
• CPC Class Codes: B64C 39/024 of the remote controlled vehicle type, i.e. RPV
• CPC Class Codes: B64D 45/04 Landing aids; Safety measures to prevent collision with earth's surface
• CPC Class Codes: B64D 47/08 Arrangements of cameras
• CPC Class Codes: B64U 2101/30 for imaging, photography or videography
• CPC Class Codes: B64U 2101/60 for transporting passengers; for transporting goods other than weapons
• CPC Class Codes: B64U 2101/64 for parcel delivery or retrieval
• CPC Class Codes: B64U 2101/67 the UAVs comprising tethers for lowering the goods
• CPC Class Codes: B64U 2201/10 autonomous, i.e. by navigating independently from ground or air stations, e.g. by using inertial navigation systems [INS]
• CPC Class Codes: B64U 2201/20 Remote controls
• CPC Class Codes: B64U 30/20 Rotors; Rotor supports
• CPC Class Codes: B64U 70/97 Means for guiding the UAV to a specific location on the platform, e.g. platform structures preventing landing off-centre
• CPC Class Codes: G05D 1/0676 specially adapted for landing
• CPC Class Codes: G05D 1/101 specially adapted for aircraft
• CPC Class Codes: G05D 1/106 Change initiated in response to external conditions, e.g. avoidance of elevated terrain or of no-fly zones
• CPC Class Codes: G06F 16/29 Geographical information databases
• CPC Class Codes: G06K 7/10861 sensing of data fields affixed to objects or articles, e.g. coded labels postal sorting B07C3/14, conveying articles B65G47/48
• CPC Class Codes: G06Q 10/083 Shipping
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 2207/10032 Satellite or aerial image; Remote sensing
• CPC Class Codes: G06T 2207/30204 : Marker
• CPC Class Codes: G06T 7/11 : Region-based segmentation
• CPC Class Codes: G06T 7/13 : Edge detection
• CPC Class Codes: G06T 7/70 : Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G06V 10/245 : by locating a pattern; Special marks for positioning
• CPC Class Codes: G06V 20/13 : Satellite images
• CPC Class Codes: G06V 20/17 : taken from planes or by drones
• CPC Class Codes: G08G 5/0013 : with a ground station
• CPC Class Codes: G08G 5/0034 : Assembly of a flight plan
• CPC Class Codes: G08G 5/0039 : Modification of a flight plan
• CPC Class Codes: G08G 5/0056 : in an emergency situation, e.g. hijacking
• CPC Class Codes: G08G 5/006 : in accordance with predefined flight zones, e.g. to avoid prohibited zones
• CPC Class Codes: G08G 5/0069 : specially adapted for an unmanned aircraft
• CPC Class Codes: G08G 5/0086 : for monitoring terrain radar or analogous systems specially adapted for terrain avoidance G01S13/935
• CPC Class Codes: G08G 5/025 : Navigation or guidance aids radar or analogous systems specially adapted for landing purposes G01S13/913
• CPC Class Codes: G08G 5/045 : Navigation or guidance aids, e.g. determination of anti-collision manoeuvers radar or analogous systems specially adapted for anti-collision between aircraft G01S13/933
• CPC Class Codes: H04N 13/204 : using stereoscopic image cameras stereoscopic photography G03B35/00
• CPC Class Codes: H04N 13/239 : using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence

Abstract

Techniques for verifying a location and identification of a landing marker to aid an unmanned aerial vehicle (UAV) to deliver a payload to a location may be provided. For example, upon receiving an indication that a UAV has arrived to a delivery location, a server computer may process one or more images of an area that are provided by the UAV and/or a user interacting with a user device. A landing marker may be identified in the image and a representation of the landing marker along with instructions to guide the UAV to deliver the payload to the landing marker may be transmitted to the UAV and implemented by the UAV.

Claims

1. A computer-implemented method, comprising:

receiving, by a computer system of an unmanned aerial vehicle (UAV), first instructions to deliver a package to a location based on an order for the package;

receiving, by the computer system and from a server computer, an image of an area associated with the location, the area including a landing marker, the image generated by the server computer utilizing a plurality of images of the area captured by a user device;

transforming, by the computer system, the image of the area to a two dimensional map of the area that includes the landing marker based at least in part on removing portions of information included in the plurality of the images captured by the user device; and

generating, by the computer system, second instructions to guide the UAV to deliver the package on the landing marker utilizing the two dimensional map of the area.

2. The computer-implemented method of claim 1, wherein the image is further generated by the server computer utilizing a video file of the area captured by the user device.

3. The computer-implemented method of claim 1, wherein delivering the package includes guiding the package, via a tether component associated with the UAV, onto the landing marker.

4. The computer-implemented method of claim 1, wherein the image is generated by the server computer utilizing a particular image of the area captured by a user device utilizing an augmented reality presentation provided by the server computer to the user device.

5. The computer-implemented method of claim 1, wherein the image is generated by the server computer utilizing data that identifies capture conditions for the plurality of images and device capabilities of the user device.

6. The computer-implemented method of claim 5, wherein the capture conditions include one or more of illumination information, lens information, or light exposure information.

7. An unmanned aerial vehicle (UAV) comprising:

a frame;

a propulsion system; and

a computing system configured to manage the propulsion system during a flight associated with a delivery of a package and further configured to instruct the UAV to;

receive first instructions to deliver the package to a location based on an order for the package;

receive, from a server computer, an image of an area associated with the location, the area including a landing marker, the image generated by the server computer utilizing a plurality of images of the area captured by a user device;

transform the image of the area to a two dimensional map of the area that includes the landing marker based at least in part on removing portions of information included in the plurality of the images captured by the user device; and

generate second instructions to guide the UAV to deliver the package on the landing marker utilizing the two dimensional map.

8. The UAV of claim 7, wherein the image is further generated by the server computer utilizing a video file of the area captured by the user device.

9. The UAV of claim 7, wherein the UAV further comprises a tether component, and wherein delivering the package includes guiding the package, via the tether component, onto the landing marker.

10. The UAV of claim 7, wherein the image is generated by the server computer utilizing a particular image of the area captured by a user device utilizing an augmented reality presentation provided by the server computer to the user device.

11. The UAV of claim 7, wherein the image is generated by the server computer utilizing data that identifies capture conditions for the plurality of images and device capabilities of the user device.

12. The UAV of claim 7, wherein the capture conditions include one or more of illumination information, lens information, or light exposure information.

13. A non-transitory computer-readable storage medium storing computer-executable instructions that, when executed by a computer system of an unmanned aerial vehicle (UAV), configure the computer system to perform operations comprising:

receiving first instructions to deliver a package to a location based on an order for the package;

receiving, from a server computer, an image of an area associated with the location, the area including a landing marker, the image generated by the server computer utilizing a plurality of images of the area captured by a user device;

transforming the image of the area to a two dimensional map of the area that includes the landing marker based at least in part on removing portions of information included in the plurality of the images captured by the user device; and

generating second instructions to guide the UAV to deliver the package on the landing marker utilizing the two dimensional map of the area.

14. The non-transitory computer-readable storage medium of claim 13, wherein the image is further generated by the server computer utilizing a video file of the area captured by the user device.

15. The non-transitory computer-readable storage medium of claim 13, wherein delivering the package includes guiding the package, via a tether component associated with the UAV, onto the landing marker.

16. The non-transitory computer-readable storage medium of claim 13, wherein the image is generated by the server computer utilizing a particular image of the area captured by a user device utilizing an augmented reality presentation provided by the server computer to the user device.

17. The non-transitory computer-readable storage medium of claim 13, wherein the image is generated by the server computer utilizing data that identifies capture conditions for the plurality of images and device capabilities of the user device.

18. The non-transitory computer-readable storage medium of claim 17, wherein the capture conditions include one or more of illumination information, lens information, or light exposure information.