Location determination using ground structures

• US 10,642,284 B1
• Filed: 06/08/2018
• Issued: 05/05/2020
• Est. Priority Date: 06/08/2018
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Barazovsky, Gideon
• Primary Examiner(s): Tran, Khoi H
• Assistant Examiner(s): Peche, Jorge O
• Application Number:
  US16/004,174
• Time in Patent Office: 697 Days
• CPC Class Codes: B64U 10/13 Flying platforms
• CPC Class Codes: B64U 2101/20 for use as communications relays, e.g. high-altitude platforms
• CPC Class Codes: B64U 2101/30 for imaging, photography or videography
• CPC Class Codes: B64U 2101/60 for transporting passengers; for transporting goods other than weapons
• CPC Class Codes: B64U 2101/64 for parcel delivery or retrieval
• CPC Class Codes: B64U 2201/10 autonomous, i.e. by navigating independently from ground or air stations, e.g. by using inertial navigation systems [INS]
• CPC Class Codes: B64U 2201/102 adapted for flying in formations
• CPC Class Codes: B64U 2201/104 using satellite radio beacon positioning systems, e.g. GPS
• CPC Class Codes: G01C 21/005 with correlation of navigation data from several sources, e.g. map or contour matching G01C21/30 takes precedence
• CPC Class Codes: G01C 21/20 Instruments for performing navigational calculations G01C21/24, G01C21/26 take precedence
• CPC Class Codes: G01S 19/06 employing an initial estimate of the location of the receiver as aiding data or in generating aiding data
• CPC Class Codes: G01S 19/12 wherein the cooperating elements are telecommunication base stations
• CPC Class Codes: G01S 19/15 Aircraft landing systems
• CPC Class Codes: G01S 19/40 Correcting position, velocity or attitude
• CPC Class Codes: G01S 19/45 by combining measurements of signals from the satellite radio beacon positioning system with a supplementary measurement
• CPC Class Codes: G01S 19/485 whereby the further system is an optical system or imaging system
• CPC Class Codes: G01S 5/0072 Transmission between mobile stations, e.g. anti-collision systems
• CPC Class Codes: G05D 1/101 specially adapted for aircraft
• CPC Class Codes: G05D 1/104 involving a plurality of aircrafts, e.g. formation flying traffic control systems for aircraft G08G5/00
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/10032 : Satellite or aerial image; Remote sensing
• CPC Class Codes: G06T 2207/30181 : Earth observation
• CPC Class Codes: G06T 2207/30184 : Infrastructure
• CPC Class Codes: G06T 2207/30244 : Camera pose
• CPC Class Codes: G06T 2207/30252 : Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 7/70 : Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G06T 7/73 : using feature-based methods

Abstract

An unmanned aerial vehicle (UAV) may navigate to place a ground structure, having a known location, into a field of view of a camera and capture imagery of the ground structure. Algorithms may be used to identify the ground structure to determine its known location. Ground structures may include identifiers that provide the known location and/or other information to enable determination of the known locations. Algorithms may determine an offset distance of the UAV from the ground structure to enable determination of a location of the UAV. For example, the images may be used to determine a distance and angle(s) from the ground structure when the ground structure is shown in a perspective view and has features that indicate orientation. In some embodiments, the UAV may transmit its location to other UAVs, and/or may create and/or transmit a corrected GPS location based on the location determined by the ground structure.

Claims

1. A unmanned aerial vehicle (UAV) comprising:

a propulsion system;

a transceiver;

a camera; and

a controller in communication with at least the propulsion system, the transceiver, and the camera, the controller to execute computer-readable instructions to;

determine a destination for a delivery;

determine a ground structure in route to the destination, the ground structure having a latitude and a longitude accessible by the UAV;

determining a flight plan for the UAV to travel to the destination;

determining a deviation from the flight plan for the UAV to travel directly over the ground structure, the deviation corresponding to a first distance between the flight plan and the ground structure;

transmit, based at least in part on the deviation, signals to the propulsion system to navigate the UAV to place the ground structure within a field of view of the camera;

capture one or more images of the ground structure using the camera;

analyze the one or more images to determine an offset from the latitude and the longitude of the ground structure, the offset corresponding to at least one of a second distance between the UAV and the ground structure, an altitude of the UAV, or an angle associated with the ground structure and the UAV; and

determine a location of the UAV based at least in part on the offset from the latitude and the longitude of the ground structure.

2. The UAV as recited in claim 1, wherein the controller further executes computer-readable instructions to cause the transceiver to:

determine, from the one or more images, an imaged location of the UAV;

receive a global position system (GPS) location of the UAV;

generate a GPS correction value based at least in part on a difference between the imaged location and the GPS location of the UAV; and

send the GPS correction value to another UAV.

3. The UAV as recited in claim 1, wherein analysis of the one or more images includes computing an imaged latitude and longitude of the UAV based at least in part on one or more images.

4. The UAV as recited in claim 1, wherein analysis of the one or more images further includes (i) determining the second distance from the ground structure based at least in part on the one or more images, and (ii) determining a viewed perspective of the ground structure to determine an angular location between the ground structure and the UAV, and wherein the offset is based at least in part on the second distance and the angular location.

5. A computer-implemented method comprising:

determining a ground structure in route to a destination, the ground structure having a latitude and a longitude accessible by an unmanned aerial vehicle (UAV);

determining a flight plan for the UAV to travel to the destination;

determining a deviation from the flight plan for the UAV to travel directly over the ground structure, the deviation corresponding to a first distance between the flight plan and the ground structure;

transmitting, based at least in part on the deviation, one or more signals to a propulsion system of the UAV to navigate the UAV to place the ground structure within a field of view of the UAV;

causing the UAV to capture one or more images of the ground structure;

analyzing the one or more images to determine an offset from the ground structure, the offset corresponding to at least one of a second distance between the UAV and the ground structure, an altitude of the UAV, or an angle associated with the ground structure and the UAV; and

determining a location of the UAV based at least in part on the offset from the latitude and the longitude of the ground structure.

6. The computer-implemented method as recited in claim 5, further comprising broadcasting at least the location of the UAV to other UAVs flying in a common airspace.

7. The computer-implemented method as recited in claim 5, further comprising:

receiving first location and time information from a first UAV;

receiving second location and time information from a second UAV;

receiving third location and time information from a third UAV; and

determining a subsequent location of the UAV based at least in part on triangulation of the first location and time information, the second location and time information, and the third location and time information.

8. The computer-implemented method as recited in claim 5, further comprising:

receiving a first global position system (GPS) location of the UAV; and

generating a GPS correction value based at least in part on a difference between the location and the first GPS location of the UAV.

9. The computer-implemented method as recited in claim 8, further comprising

receiving a second GPS location of the UAV;

and

determining a corrected location of the UAV by applying the GPS correction value to the second GPS location.

10. The computer-implemented method as recited in claim 5, further comprising reading an identifier of the ground structure to determine the latitude and the longitude of the ground structure.

11. The computer-implemented method as recited in claim 5, further comprising using image recognition to map the ground structure with a known ground structure associated with the latitude and the longitude.

12. The computer-implemented method as recited in claim 5, wherein the analyzing the one or more images further includes:

determining the second distance based at least in part on the one or more images, and

determining a viewed perspective of the ground structure to determine an angular location between the ground structure and the UAV,

wherein the offset is based at least in part on the second distance and the angular location.

13. The computer-implemented method as recited in claim 5, further comprising:

receiving a global position system (GPS) location of the UAV; and

determining that the GPS location is illegitimate based at least in part on a difference between the GPS location and the location of the UAV at a time.

14. The computer-implemented method as recited in claim 5, further comprising:

detecting, from the one or more images, a shadow cast by the ground structure;

determining, from the one or more images and based at least in part on the shadow, an orientation of the ground structure; and

determining the offset based at least in part on the shadow and the orientation.

15. The computer-implemented method as recited in claim 5, further comprising:

transmitting, based at least in part on the deviation, one or more second signals to the propulsion system to cause the UAV to navigate to a position directly over the ground structure; and

causing the UAV to capture one or more second images of the ground structure while the UAV is directly over the ground structure.

16. A device comprising:

one or more processors; and

memory storing one or more computer-readable instructions that, when executed by the one or more processors, cause the one or more processors to perform acts including;

identifying a landscape that includes a ground structure;

determining a flight plan for the device to travel to a destination;

determining a deviation from the flight plan for the device to travel directly over the ground structure, the deviation corresponding to a first distance between the flight plan and the ground structure;

causing, based at least in part on the deviation, the device to navigate in a direction such that the ground structure is placed within a field of view of the device;

identifying the ground structure in one or more images captured by the device;

determining a latitude and a longitude of the ground structure;

analyzing the one or more images to determine an offset from the ground structure, the offset corresponding to at least one of a second distance between the device and the ground structure, an altitude of the device, or an angle associated with the ground structure and the device; and

determining a location of the device based at least in part on the offset from the latitude and the longitude of the ground structure.

17. The device as recited in claim 16, wherein the acts further comprise transmitting at least the location of the device to another device.

18. The device as recited in claim 16, wherein the acts further comprise:

receiving a global position system (GPS) location of the device; and

generating a GPS correction value based at least in part on a difference between the location and the GPS location of the device.

19. The device as recited in claim 18, wherein the acts further comprise sending the GPS correction value to another device.

20. The device as recited in claim 16, wherein the acts further comprise determining an identifier of the ground structure based at least in part on the image and a feature recognition algorithm to positively identify the ground structure.