Unmanned aerial vehicle obstacle detection and avoidance

• US 10,019,907 B2
• Filed: 01/06/2016
• Issued: 07/10/2018
• Est. Priority Date: 09/11/2015
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Kanade, Parag Mohan, Sweet, III, Charles Wheeler, Gehlhaar, Jeffrey Baginsky
• Primary Examiner(s): Ismail, Mahmoud S
• Application Number:
  US14/989,428
• Publication Number:
  US 20170076616A1
• Time in Patent Office: 916 Days
• Field of Search: 701 3, 701 26, 701469, 701 2, 701 23, 340435
• CPC Class Codes: B64U 2201/10 autonomous, i.e. by navigating independently from ground or air stations, e.g. by using inertial navigation systems [INS]
• CPC Class Codes: G06V 10/454 Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]
• CPC Class Codes: G06V 10/56 relating to colour
• CPC Class Codes: G06V 20/17 taken from planes or by drones
• CPC Class Codes: G06V 20/38 Outdoor scenes
• CPC Class Codes: G08G 5/0069 specially adapted for an unmanned aircraft
• CPC Class Codes: G08G 5/045 Navigation or guidance aids, e.g. determination of anti-collision manoeuvers radar or analogous systems specially adapted for anti-collision between aircraft G01S13/933

Abstract

Apparatuses and methods for detecting an obstacle in a path of an Unmanned Aerial Vehicle (UAV) are described herein, including, but not limited to, receiving data from a single image/video capturing device of the UAV, computing a score based on the received data, and performing at least one obstacle avoidance maneuver based on the score.

Claims

1. A method for obstacle avoidance for an Unmanned Aerial Vehicle (UAV), the method comprising:

providing streaming video captured by a first camera of the UAV as a visual feed of the UAV'"'"'s flight path to an operator of the UAV;

analyzing the streaming video captured by the first camera to identify features in one or more image frames of the streaming video by;

detecting a horizon;

determining a presence of an object cluttering the horizon; and

determining a percentage of a size of the object cluttering the horizon with respect to an area of one of the image frames;

determining a score based on the features;

selecting one of a plurality of different obstacle avoidance maneuvers based on the score; and

performing the selected obstacle avoidance maneuver to alter the flight path of the UAV.

2. The method of claim 1, wherein the features include a perceived dimension of an object as it appears in the streaming video.

3. The method of claim 1, wherein the score indicates a presence of an object in the flight path.

4. The method of claim 3, wherein the score indicates a size of the object and a distance between the object and the UAV.

5. The method of claim 1, wherein analyzing the streaming video further comprises:

determining a number of attributes of the object.

6. The method of claim 5, wherein the number of attributes includes at least one of a size of the object, one or more dimensions of the object, and a length of extension of the object along a bottom portion of the one or more image frames.

7. The method of claim 1, wherein the plurality of different obstacle avoidance maneuvers includes one or more of changing a direction of the UAV, changing an altitude of the UAV, adjusting a pitch of the UAV, adjusting a yaw of the UAV, adjusting a roll of the UAV, halting the UAV, or hovering the UAV in place.

8. The method of claim 1, wherein the plurality of different obstacle avoidance maneuvers includes one or more of notifying the operator, landing the UAV, returning the UAV to base, or transmitting data to the operator.

9. The method of claim 1, further comprising:

receiving a trigger to compute a new score;

rotating the first camera based on the trigger;

capturing a new image frame using the rotated first camera; and

computing the new score based on the new image frame.

10. The method of claim 1, further comprising:

receiving a trigger to compute a new score;

capturing a new image frame using the first camera; and

computing the new score based on a selected portion of the new image frame.

11. The method of claim 10, wherein the trigger is based on detecting that the UAV is turning in a first direction, and the selected portion of the new image frame corresponds to the first direction.

12. The method of claim 1, further comprising:

receiving a trigger to compute a new score;

activating a second camera on the UAV based on the trigger;

capturing a new image frame using the second camera; and

computing the new score based at least in part on the new image frame.

13. An Unmanned Aerial Vehicle (UAV) configured for obstacle avoidance, comprising:

a first camera;

one or more processors coupled to the first camera; and

a memory storing instructions that, when executed by the one or more processors, cause the UAV to;

provide streaming video captured by the first camera as a visual feed of the UAV'"'"'s flight path to an operator of the UAV;

analyze the streaming video captured by the first camera to identify features in one or more image frames of the streaming video by;

detecting a horizon;

determining a presence of an object cluttering the horizon; and

determining a percentage of a size of the object cluttering the horizon with respect to an area of one of the image frames;

determine a score based on the features;

select one of a plurality of different obstacle avoidance maneuvers based on the score; and

perform the selected obstacle avoidance maneuver to alter the flight path of the UAV.

14. The UAV of claim 13, wherein the features include a perceived dimension of an object as it appears in the streaming video.

15. The UAV of claim 13, wherein the score indicates a presence of an object in the flight path.

16. The UAV of claim 15, wherein the score indicates a size of the object and a distance between the object and the UAV.

17. The UAV of claim 13, wherein execution of the instructions to analyze the streaming video further causes the UAV to:

determine a number of attributes of the object.

18. The UAV of claim 17, wherein the number of attributes includes at least one of a size of the object, one or more dimensions of the object, and a length of extension of the object along a bottom portion of the one or more image frames.

19. The UAV of claim 13, wherein the plurality of different obstacle avoidance maneuvers includes one or more of changing a direction of the UAV, changing an altitude of the UAV, adjusting a pitch of the UAV, adjusting a yaw of the UAV, adjusting a roll of the UAV, halting the UAV, or hovering the UAV in place.

20. The UAV of claim 19, wherein the plurality of different obstacle avoidance maneuvers includes one or more of notifying the operator, landing the UAV, returning the UAV to base, or transmitting data to the operator.

21. The UAV of claim 13, wherein execution of the instructions causes the UAV to further:

receive a trigger to compute a new score;

rotate the first camera based on the trigger;

capture a new image frame using the rotated first camera; and

compute the new score based on the new image frame.

22. The UAV of claim 13, wherein execution of the instructions causes the UAV to further:

receive a trigger to compute a new score;

capture a new image frame using the first camera; and

compute the new score based on a selected portion of the new image frame.

23. The UAV of claim 22, wherein the trigger is based on detecting that the UAV is turning in a first direction, and the selected portion of the new image frame corresponds to the first direction.

24. The UAV of claim 13, wherein execution of the instructions causes the UAV to further:

receive a trigger to compute a new score;

activate a second camera on the UAV based on the trigger;

capture a new image frame using the second camera; and

compute the new score based at least in part on the new image frame.

25. A non-transitory computer readable medium comprising instructions that, when executed by one or more processors of an Unmanned Aerial Vehicle (UAV), causes the UAV to perform operations comprising:

providing streaming video captured by a first camera of the UAV as a visual feed of the UAV'"'"'s flight path to an operator of the UAV;

analyzing the streaming video captured by the first camera to identify features in one or more image frames of the streaming video by;

detecting a horizon

determining a presence of an object cluttering the horizon; and

determining a percentage of a size of the object cluttering the horizon with respect to an area of one of the image frames;

determining a score based on the features;

selecting one of a plurality of different obstacle avoidance maneuvers based on the score; and

performing the selected obstacle avoidance maneuver to alter the flight path of the UAV.

26. The non-transitory computer readable medium of claim 25, wherein the features include a perceived dimension of an object as it appears in the streaming video.

27. The non-transitory computer readable medium of claim 25, wherein execution of the instructions causes the UAV to perform operations further comprising:

receiving a trigger to compute a new score;

rotating the first camera based on the trigger;

capturing a new image frame using the rotated first camera; and

computing the new score based on the new image frame.

28. The non-transitory computer readable medium of claim 25, wherein execution of the instructions causes the UAV to perform operations further comprising:

receiving a trigger to compute a new score;

capturing a new image frame using the first camera; and

computing the new score based on a selected portion of the new image frame.

29. The method of claim 28, wherein the trigger is based on detecting that the UAV is turning in a first direction, and the selected portion of the new image frame corresponds to the selected direction.

30. The non-transitory computer readable medium of claim 25, wherein execution of the instructions causes the UAV to perform operations further comprising:

receiving a trigger to compute a new score;

activating a second camera on the UAV based on the trigger;

capturing a new image frame using the second camera; and

computing the new score based on the new image frame.