Systems and methods for landing a drone on a moving base

• US 10,152,059 B2
• Filed: 10/10/2016
• Issued: 12/11/2018
• Est. Priority Date: 10/10/2016
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Banerjee, Debdeep, Kandhadai, Ananthapadmanabhan Arasanipalai
• Primary Examiner(s): Weber, Tamara L
• Application Number:
  US15/289,882
• Publication Number:
  US 20180101173A1
• Time in Patent Office: 792 Days
• Field of Search: None
• CPC Class Codes: B64C 39/024 of the remote controlled vehicle type, i.e. RPV
• CPC Class Codes: B64D 47/08 Arrangements of cameras
• CPC Class Codes: B64F 1/007 Helicopter portable landing pads
• CPC Class Codes: B64U 2201/10 autonomous, i.e. by navigating independently from ground or air stations, e.g. by using inertial navigation systems [INS]
• CPC Class Codes: B64U 70/00 Launching, take-off or landing arrangements
• CPC Class Codes: B64U 70/92 Portable platforms
• CPC Class Codes: B64U 80/25 for recharging batteries; for refuelling
• CPC Class Codes: B64U 80/86 Land vehicles
• CPC Class Codes: G05D 1/0094 involving pointing a payload, e.g. camera, weapon, sensor, towards a fixed or moving target
• CPC Class Codes: G05D 1/0684 on a moving platform, e.g. aircraft carrier
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G08G 5/045 Navigation or guidance aids, e.g. determination of anti-collision manoeuvers radar or analogous systems specially adapted for anti-collision between aircraft G01S13/933
• CPC Class Codes: H04N 23/68 for stable pick-up of the scene, e.g. compensating for camera body vibrations

Abstract

A drone is described. The drone includes a depth sensor configured to provide information for determining a distance between the drone and a moving base. The drone also includes a processor configured to control a computer vision tracking algorithm based on the distance, and to control drone movement based on the computer vision tracking algorithm. A vehicle is also described. The vehicle includes a depth sensor configured to provide information for determining a distance between a drone and the vehicle. The vehicle also includes a processor configured to control a computer vision tracking algorithm based on the distance and to send information for controlling drone movement based on the computer vision tracking algorithm.

Claims

1. A drone, comprising:

a depth sensor configured to provide information for determining a distance between the drone and a moving base; and

a processor configured to control a computer vision tracking algorithm by increasing tracking when the distance is within a distance threshold and to control drone movement based on the computer vision tracking algorithm.

2. The drone of claim 1, wherein the processor is configured to control jitter of the computer vision tracking algorithm.

3. The drone of claim 2, wherein the processor is configured to control the jitter by converting an increased number of red-green-blue (RGB) frames to hue-saturation-value (HSV) frames.

4. The drone of claim 2, wherein the processor is configured to control the jitter by:

computing a saliency score based on a model and at least a portion of a frame;

determining whether the frame is a valid frame for tracking based on the saliency score; and

providing the frame to the computer vision tracking algorithm in a case that the frame is a valid frame.

5. The drone of claim 1, wherein the processor is configured to perform frame smoothing on a set of frames provided to the computer vision tracking algorithm.

6. The drone of claim 1, wherein the processor is configured to control drone movement by controlling drone acceleration, using one or more movement mechanisms, based on the computer vision tracking algorithm.

7. The drone of claim 1, wherein the processor is configured to perform obstacle detection using at least one of computer vision or depth sensing, and wherein the processor is configured to control drone movement by changing drone trajectory to avoid an obstacle using one or more movement mechanisms.

8. The drone of claim 1, wherein the processor is configured to determine whether one or more slots on the moving base are occupied.

9. The drone of claim 1, wherein the processor is configured to provide information for controlling an adjustable landing pad on the moving base.

10. A method performed by a drone, comprising:

determining a distance between the drone and a moving base;

controlling a computer vision tracking algorithm, comprising increasing tracking when the distance is within a distance threshold; and

controlling drone movement based on the computer vision tracking algorithm.

11. The method of claim 10, further comprising controlling jitter of the computer vision tracking algorithm.

12. The method of claim 10, further comprising determining whether one or more slots on the moving base are occupied.

13. The method of claim 10, further comprising providing information for controlling an adjustable landing pad on the moving base.

14. A vehicle, comprising:

a depth sensor configured to provide information for determining a distance between a drone and the vehicle; and

a processor configured to control a computer vision tracking algorithm by increasing tracking when the distance is within a distance threshold and to send information for controlling drone movement based on the computer vision tracking algorithm.

15. The vehicle of claim 14, wherein the processor is configured to control jitter of the computer vision tracking algorithm.

16. The vehicle of claim 15, wherein the processor is configured to control the jitter by converting an increased number of red-green-blue (RGB) frames to hue-saturation-value (HSV) frames.

17. The vehicle of claim 15, wherein the processor is configured to control the jitter by:

computing a saliency score based on a model and at least a portion of a frame;

determining whether the frame is a valid frame for tracking based on the saliency score; and

providing the frame to the computer vision tracking algorithm in a case that the frame is a valid frame.

18. The vehicle of claim 14, wherein the information includes drone position information and drone movement information determined by the vehicle based on the computer vision tracking algorithm.

19. The vehicle of claim 14, wherein the processor is configured to control drone movement by controlling drone acceleration, using one or more movement mechanisms, based on the computer vision tracking algorithm.

20. The vehicle of claim 14, wherein the processor is configured to perform obstacle detection using at least one of computer vision or depth sensing, and wherein the processor is configured to control drone movement by changing drone trajectory to avoid an obstacle using one or more movement mechanisms.

21. The vehicle of claim 14, wherein the processor is configured to determine whether one or more slots on the vehicle are occupied.

22. The vehicle of claim 14, wherein the processor is configured to control an adjustable landing pad on the vehicle using an adjustable mechanism based on the computer vision tracking algorithm.

23. A method performed by a vehicle, comprising:

determining a distance between a drone and the vehicle;

controlling a computer vision tracking algorithm, comprising increasing tracking when the distance is within a distance threshold; and

controlling drone movement based on the computer vision tracking algorithm.

24. The method of claim 23, wherein controlling the drone movement is based on drone position information and drone movement information determined by the vehicle based on the computer vision tracking algorithm.

25. The method of claim 23, further comprising determining whether one or more slots on the vehicle are occupied.

26. The method of claim 23, further comprising controlling an adjustable landing pad on the vehicle based on the computer vision tracking algorithm.