Payload-release device position tracking

• US 9,849,981 B1
• Filed: 12/29/2014
• Issued: 12/26/2017
• Est. Priority Date: 08/28/2014
• Status: Active Grant

• Current Assignee: Wing Aviation LLC (Alphabet Inc.)
• Original Assignee: X Development LLC (Alphabet Inc.)
• Inventors: Burgess, James Ryan, Cohen, Joanna
• Primary Examiner(s): Lin, Abby
• Assistant Examiner(s): Castro, Paul
• Application Number:
  US14/584,189
• Time in Patent Office: 1,093 Days
• Field of Search: None
• CPC Class Codes: B64C 39/024 of the remote controlled vehicle type, i.e. RPV
• CPC Class Codes: B64D 1/12 Releasing
• CPC Class Codes: B64D 1/22 Taking-up articles from earth's surface
• CPC Class Codes: B64U 2101/60 for transporting passengers; for transporting goods other than weapons
• CPC Class Codes: B64U 2101/64 for parcel delivery or retrieval
• CPC Class Codes: B64U 2101/67 the UAVs comprising tethers for lowering the goods
• CPC Class Codes: G05D 1/0094 involving pointing a payload, e.g. camera, weapon, sensor, towards a fixed or moving target
• CPC Class Codes: G05D 1/042 specially adapted for aircraft

Abstract

An unmanned aerial vehicle (UAV) is disclosed that includes a retractable payload delivery system. The payload delivery system can lower a payload to the ground using a delivery device that secures the payload during descent and releases the payload upon reaching the ground. The location of the delivery device can be determined as it is lowered to the ground using image tracking. The UAV can include an imaging system that captures image data of the suspended delivery device and identifies image coordinates of the delivery device, and the image coordinates can then be mapped to a location. The UAV may also be configured to account for any deviations from a planned path of descent in real time to effect accurate delivery locations of released payloads.

Claims

1. A system comprising:

a retractable delivery system comprising;

(i) a delivery device configured to be releasably coupled to a payload, wherein a light source is situated on the delivery device;

(ii) a tether coupled to an unmanned aerial vehicle (UAV) and the delivery device; and

(iii) a retraction system coupled to the tether and operable to use the tether to lower the delivery device and the payload secured thereby from the UAV;

an imaging system mounted on the UAV such that, while the delivery device is suspended from the UAV via the tether, a field of view of the imaging system includes the light source that is situated on the delivery device and that is arranged to emit light toward the UAV; and

a control system configured to;

(i) while the UAV hovers and the delivery device is suspended from the UAV via the tether, receive image data from the imaging system;

(ii) identify, based on the received image data, an image coordinate associated with the light source situated on the delivery device;

(iii) determine a position of the delivery device based at least in part on the identified image coordinate;

(iv) determine, based at least in part on the position of the delivery device, flight-control commands to navigate the UAV, in hover mode, such that movement of the UAV causes the suspended delivery device to become closer to a path of descent associated with a delivery location; and

(v) cause the UAV to fly, in hover mode, in accordance with the determined flight-control commands.

2. The system of claim 1, wherein the control system is further configured to:

(iv) while the UAV hovers over a delivery location, use the retractable delivery system to initiate a delivery operation, wherein initiating the delivery operation comprises lowering the delivery device and the payload secured thereby toward the ground such that the payload descends from the UAV;

(v) determine that the delivery device is at or near the ground; and

(vi) in response to determining that the delivery device is at or near the ground, release the payload from the delivery device.

3. The system of claim 2, wherein the control system is further configured to:

(vii) determine that the determined position of the delivery device is within a threshold distance of the delivery location; and

(viii) cause the delivery device to release the payload in response to both determining that the delivery device is at or near the ground and determining that the determined position of the delivery device is within the threshold distance of the delivery location.

4. The system of claim 1, wherein the control system is further configured to:

(vi) prior to causing the UAV to fly, in hover mode, in accordance with the determined flight-control commands, determine that the determined position of the delivery device is greater than a threshold distance from the path of descent associated with the target delivery location.

5. The system of claim 1, wherein the determination of the flight-control commands comprises the control system:

(a) determining a translational offset of the UAV which compensates for a displacement between the determined position of the delivery device and the path of descent associated with the target delivery location, wherein the translational offset is based on;

the identified image coordinate associated with the light source, an image coordinate associated with the path of descent, and a current elevation of the UAV, and (b) using a navigational module to determine the flight-control commands which cause the UAV to traverse the determined translational offset in hover mode.

6. The system of claim 1, wherein the imaging system comprises a downward-facing camera mounted to the UAV via a gimbal mount so as to substantially maintain an orientation of the point of view of the camera.

7. The system of claim 1, wherein the light source is a first light source and a second light source is also situated on the delivery device such that, while the delivery device is suspended from the UAV via the tether, both the first and second light sources are arranged to emit light toward the UAV, and

wherein the control system is further configured to:

(iv) identify a pattern in the received image data that corresponds to light received from both the first and second light sources;

(v) identify, in the pattern, respective image coordinates associated with the first and second light sources, and

(vi) determine the position of the delivery device based at least in part on both the image coordinate associated with the first light source and the image coordinate associated with the second light source.

8. The system of claim 7, wherein the first light source is configured to emit light having a first wavelength and the second light source is configured to emit light having a second wavelength different from the first wavelength.

9. The system of claim 7, wherein the first and second light sources are situated at respective positions on the delivery device, and wherein the control system identifying the pattern within the received image data comprises the control system identifying a pattern in which light from the first and second light sources are received from respective emission locations that are spatially separated from one another.

10. The system of claim 9, wherein the control system is further configured to determine an orientation of the delivery device while the delivery device is suspended from the UAV based on the image coordinates associated with the first and second light sources.

11. The system of claim 1, wherein the light source is configured to emit light having a characteristic modulation, and wherein the control system is further configured to use the characteristic modulation to distinguish light from the light source from background light.

12. The system of claim 1, wherein the light source comprises a light emitting diode that emits infrared or near-infrared light.

13. A method comprising:

while a delivery device is suspended from an unmanned aerial vehicle (UAV) via a tether, receiving image data from an imaging sensor system, wherein the imaging sensor system comprises an image sensor mounted on the UAV such that, while the delivery device is suspended from the UAV via the tether, a field of view of the image sensor includes a light source that is situated on the delivery device and that is arranged to emit light toward the UAV;

while the UAV is hovering, identifying, by a computing system, based on the received image data, an image coordinate associated with the light source situated on the delivery device;

determining, by the computing system, based at least in part on the identified image coordinate, a position of the delivery device;

determining, by the computing system, based at least in part on the position of the delivery device, flight-control commands to navigate the UAV, in hover mode, such that movement of the UAV causes the suspended delivery device to become closer to a path of descent associated with a delivery location; and

causing, by the computing system the UAV to fly, in hover mode, in accordance with the determined flight-control commands.

14. The method of claim 13, wherein the delivery device is configured to be releasably coupled to a payload, the method further comprising:

while the UAV hovers over a delivery location, using a retractable delivery system to initiate a delivery operation, wherein initiating the delivery operation comprises lowering the delivery device and the payload secured thereby toward the ground such that the payload descends from the UAV;

determining that the delivery device is at or near the ground;

determining that the determined position of the delivery device is within a threshold distance of the delivery location; and

causing the delivery device to release the payload in response to both determining that the delivery device is at or near the ground and determining that the determined position of the delivery device is within the threshold distance of the delivery location.

15. The method of claim 13, wherein determining the flight-control commands comprises:

determining a translational offset of the UAV which compensates for a displacement between the determined position of the delivery device and the path of descent associated with the delivery location, wherein the translational offset is based on;

the identified image coordinate associated with the light source, an image coordinate associated with the path of descent, and a current elevation of the UAV; and

using a navigational module to determine the flight-control commands which cause the UAV to traverse the determined translational offset in hover mode.