Autonomous landing and control

• US 10,061,328 B2
• Filed: 09/16/2015
• Issued: 08/28/2018
• Est. Priority Date: 08/12/2015
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Canoy, Michael-David Nakayoshi, Liu, Yinyin, Chau, Kiet Tuan
• Primary Examiner(s): Hutchinson, Alan D
• Application Number:
  US14/855,504
• Publication Number:
  US 20170045894A1
• Time in Patent Office: 1,077 Days
• CPC Class Codes: B64U 10/10 Rotorcrafts
• CPC Class Codes: B64U 10/13 Flying platforms
• CPC Class Codes: B64U 10/14 with four distinct rotor axes, e.g. quadcopters
• CPC Class Codes: B64U 2101/30 for imaging, photography or videography
• CPC Class Codes: B64U 2101/31 for surveillance
• CPC Class Codes: B64U 2101/60 for transporting passengers; for transporting goods other than weapons
• CPC Class Codes: B64U 2101/64 for parcel delivery or retrieval
• CPC Class Codes: B64U 2201/10 autonomous, i.e. by navigating independently from ground or air stations, e.g. by using inertial navigation systems [INS]
• CPC Class Codes: B64U 2201/102 adapted for flying in formations
• CPC Class Codes: B64U 30/20 Rotors; Rotor supports
• CPC Class Codes: B64U 60/10 specially adapted for use on water
• CPC Class Codes: B64U 60/50 with landing legs
• CPC Class Codes: B64U 70/00 Launching, take-off or landing arrangements
• CPC Class Codes: B64U 70/95 Means for guiding the landing UAV towards the platform, e.g. lighting means
• CPC Class Codes: G05D 1/0676 specially adapted for landing
• CPC Class Codes: G06V 20/13 Satellite images
• CPC Class Codes: G06V 20/17 taken from planes or by drones
• CPC Class Codes: G08G 5/0008 with other aircraft
• CPC Class Codes: G08G 5/0013 with a ground station
• CPC Class Codes: G08G 5/0039 Modification of a flight plan
• CPC Class Codes: G08G 5/0056 : in an emergency situation, e.g. hijacking
• CPC Class Codes: G08G 5/0069 : specially adapted for an unmanned aircraft
• CPC Class Codes: G08G 5/0078 : for monitoring traffic from the aircraft radar or analogous systems specially adapted for traffic control G01S13/91
• CPC Class Codes: G08G 5/025 : Navigation or guidance aids radar or analogous systems specially adapted for landing purposes G01S13/913

Abstract

Various embodiments provide methods for controlling landings of a UAV in a landing zone including a plurality of landing bays. Various embodiments include a method implemented on a computing device for receiving continuous real-time sensor data from a transceiver and from sensors onboard the UAV, and detecting a target landing bay within the plurality of landing bays within the landing zone that is available for landing based on the continuous real-time sensor data. Orientation and position coordinates for landing in the target landing bay may be calculated based on the continuous real-time sensor data. Information regarding positions and flight vectors of a plurality of autonomous UAVs may be obtained, and a flight plan for landing in the target landing bay may be generated based on the orientation and the position coordinates, positions and flight vectors of the plurality of autonomous UAVs and a current orientation and position of the UAV.

Claims

1. A method performed by a processor of an unmanned aerial vehicle (UAV) for controlling landing in a landing zone including a plurality of landing bays while flying among a plurality of other UAVs, comprising:

receiving continuous real-time sensor data;

detecting a target landing bay within the plurality of landing bays within the landing zone that is available for landing based on the continuous real-time sensor data;

calculating an orientation and position coordinates for landing in the target landing bay based on the continuous real-time sensor data;

obtaining information regarding positions and flight vectors of the plurality of other UAVs, wherein obtaining the information comprises;

receiving position and flight vector reports from the plurality of other UAVs via a transceiver on board the UAV,

wherein the UAV and the plurality of other UAVs are each independently executing a separate flight mission;

generating a flight plan for landing in the target landing bay based on the orientation and the position coordinates, the positions and flight vector reports of the plurality of other UAVs, and a current orientation and position of the UAV;

performing the flight plan for landing in the target landing bay;

determining whether an exception condition is identified based on the continuous real-time sensor data and the flight plan for landing in the target landing bay; and

in response to determining that an exception condition is identified;

halting performance of the flight plan for landing in the target landing bay; and

performing exception-handling operations based on the identified exception condition, wherein performing the exception-handling operations includes detecting an alternative target landing bay from the plurality of landing bays within the landing zone that is available for landing based on the continuous real-time sensor data by;

performing operations to adjust a perspective of the UAV independent of any landing routine, the operations comprising one or more of;

causing the UAV to ascend to a higher altitude above the landing zone, changing a pitch setting of the UAV, changing a roll setting of the UAV, or changing a yaw setting of the UAV; and

obtaining the continuous real-time sensor data in the adjusted perspective.

2. The method of claim 1, wherein the continuous real-time sensor data is received from sensors on board the UAV.

3. The method of claim 2, wherein obtaining information regarding the positions and flight vectors of the plurality of other UAVs further comprises:

obtaining camera imagery via a camera, wherein the camera is one of the sensors on board the UAV;

tracking the plurality of other UAVs using the camera imagery; and

calculating the positions and flight vectors of the plurality of other UAVs based on the camera imagery and the UAV'"'"'s own position and flight vectors.

4. The method of claim 2, wherein detecting the target landing bay from the plurality of landing bays within the landing zone that is available for landing based on the continuous real-time sensor data comprises:

obtaining camera imagery via a camera, wherein the camera is one of the sensors on board the UAV;

determining whether the camera imagery includes imagery of an assigned landing bay; and

detecting within the camera imagery of an open landing bay that is available for landing in response to determining that the camera imagery does not include the imagery of the assigned landing bay.

5. The method of claim 4, wherein the imagery of the assigned landing bay or the imagery of the open landing bay is imagery of a landing pattern that comprises at least a circle circumscribing an asymmetric symbol, wherein each hemisphere of the asymmetric symbol is different than an opposite hemisphere of the asymmetric symbol such that the landing pattern indicates a global orientation.

6. The method of claim 5, wherein calculating the orientation comprises:

calculating the orientation based on a comparison of a current heading of the UAV to the global orientation of the asymmetric symbol in the imagery of the landing pattern.

7. The method of claim 1, wherein the continuous real-time sensor data is received via at least the transceiver on board the UAV.

8. The method of claim 1, further comprising continuously transmitting reports of the UAV'"'"'s own position and flight vector via the transceiver.

9. The method of claim 1, wherein performing the exception-handling operations comprises:

calculating an alternative orientation and alternative position coordinates for landing in the alternative target landing bay based on the continuous real-time sensor data;

adjusting the flight plan for landing in the alternative target landing bay based on the alternative orientation, the alternative position coordinates, and the current orientation and position of the UAV; and

performing the flight plan for landing in the alternative target landing bay.

10. The method of claim 1, wherein performing the exception-handling operations comprises:

adjusting a parameter of a sensor on board the UAV that is configured to obtain the continuous real-time sensor data, wherein the parameter includes one or more of a zoom setting of a camera and a focus setting of the camera.

11. The method of claim 1,

wherein determining whether an exception condition is identified based on the continuous real-time sensor data comprises:

determining whether the target landing bay is obstructed based on the continuous real-time sensor data; and

wherein halting performance of the flight plan for landing in the target landing bay in response to identifying the exception condition comprises;

halting the flight plan for landing in the target landing bay in response to determining that the target landing bay is obstructed.

12. The method of claim 1,

wherein determining whether an exception condition is identified based on the continuous real-time sensor data comprises:

determining whether the UAV has lost track of the target landing bay based on the continuous real-time sensor data; and

wherein halting performance of the flight plan for landing in the target landing bay in response to identifying the exception condition comprises;

halting the flight plan for landing in the target landing bay in response to determining that the UAV has lost track of the target landing bay based on the continuous real-time sensor data.

13. The method of claim 1,

wherein determining whether an exception condition is identified based on the continuous real-time sensor data and the flight plan for landing in the target landing bay comprises:

continuously monitoring positions and flight vectors of the plurality of other UAVs while performing the flight plan for landing in the target landing bay; and

calculating a probability of a mid-air collision occurring with one or more of the plurality of other UAVs while following the flight plan for landing in the target landing bay based on the positions and flight vectors of the plurality of other UAVs; and

wherein performing the exception-handling operations based on the identified exception condition comprises;

adjusting the flight plan for landing in the target landing bay in response to determining that the calculated probability of the mid-air collision occurring exceeds a safety threshold.

14. The method of claim 1, further comprising:

determining whether the UAV is beginning an approach into the landing zone based on the continuous real-time sensor data,

wherein detecting the target landing bay from the plurality of landing bays within the landing zone that is available for landing based on the continuous real-time sensor data comprises;

detecting the target landing bay within the landing zone based on the continuous real-time sensor data in response to determining that the UAV is beginning the approach.

15. The method of claim 14, wherein determining that the UAV is beginning the approach into the landing zone based on the continuous real-time sensor data comprises:

comparing coordinates of the UAV from the continuous real-time sensor data to coordinates for the landing zone.

16. The method of claim 14, wherein determining that the UAV is beginning the approach into the landing zone based on the continuous real-time sensor data comprises:

detecting imagery of the landing zone within the continuous real-time sensor data.

17. The method of claim 1, wherein calculating the position coordinates for landing in the target landing bay based on the continuous real-time sensor data comprises:

calculating a center point of the surface of the target landing bay.

18. The method of claim 1, wherein the continuous real-time sensor data includes one or more of location data received from global positioning system receiver, audio data from a microphone, movement data from an accelerometer, and orientation data from a gyroscope.

19. A computing device, comprising:

a memory;

a transceiver; and

a processor coupled to the memory and configured with processor-executable instructions to;

receive continuous real-time sensor data;

detect a target landing bay within a plurality of landing bays within a landing zone that is available for landing based on the continuous real-time sensor data;

calculate an orientation and position coordinates for landing in the target landing bay based on the continuous real-time sensor data;

obtain information regarding positions and flight vectors of a plurality of other UAVs operating within the landing zone by;

receiving position and flight vector reports from the plurality of other UAVs via the transceiver,

wherein the UAV and the plurality of other UAVs are each independently executing a separate flight mission;

generate a flight plan for landing in the target landing bay based on the orientation and the position coordinates, the positions and flight vector reports of the plurality of other UAVs, and a current orientation and position of the computing device;

perform the flight plan for landing in the target landing bay;

determine whether an exception condition is identified based on the continuous real-time sensor data and the flight plan for landing in the target landing bay; and

in response to determining that an exception condition is identified;

halt performance of the flight plan for landing in the target landing bay; and

perform exception-handling operations based on the identified exception condition, wherein the exception-handling operations include detecting an alternative target landing bay from the plurality of landing bays within the landing zone that is available for landing based on the continuous real-time sensor data by;

performing operations to adjust a perspective of the UAV independent of any landing routine, the operations comprising one or more of;

 

causing the UAV to ascend to a higher altitude above the landing zone, changing a pitch setting of the UAV, changing a roll setting of the UAV, or changing a yaw setting of the UAV; and

obtaining the continuous real-time sensor data in the adjusted perspective.

20. The computing device of claim 19, wherein the computing device is within an unmanned aerial vehicle (UAV).

21. The computing device of claim 19,

wherein the computing device further comprises sensors, and

wherein the continuous real-time sensor data is received via the sensors, the transceiver, or both.

22. The computing device of claim 21, wherein the processor is further configured with processor-executable instructions to:

continuously transmit reports of the computing device'"'"'s own position and flight vector via the transceiver.

23. The computing device of claim 21,

wherein the processor is further configured with processor-executable instructions to obtain information regarding the positions and flight vectors of the plurality of other UAVs by:

obtaining camera imagery via a camera, wherein the camera is one of the sensors;

tracking the plurality of other UAVs using the camera imagery; and

calculating the positions and flight vectors of the plurality of other UAVs based on the camera imagery and the computing device'"'"'s own position and flight vectors.

24. A non-transitory processor-readable storage medium having stored thereon processor-executable instructions configured to cause a processor of a computing device of an unmanned aerial vehicle (UAV) to perform operations comprising:

receiving continuous real-time sensor data;

detecting a target landing bay within a plurality of landing bays within a landing zone that is available for landing based on the continuous real-time sensor data;

calculating an orientation and position coordinates for landing in the target landing bay based on the continuous real-time sensor data;

obtaining information regarding positions and flight vectors of a plurality of other UAVs operating within the landing zone, wherein obtaining the information comprises;

receiving position and flight vector reports from the plurality of other UAVs via a transceiver on board the UAV,

wherein the UAV and the plurality of other UAVs are each independently executing a separate flight mission;

generating a flight plan for landing in the target landing bay based on the orientation and the position coordinates, the positions and flight vector reports of the plurality of other UAVs, and a current orientation and position of the UAV;

performing the flight plan for landing in the target landing bay;

determining whether an exception condition is identified based on the continuous real-time sensor data and the flight plan for landing in the target landing bay; and

in response to determining that an exception condition is identified;

halting performance of the flight plan for landing in the target landing bay; and

performing exception-handling operations based on the identified exception condition, wherein the exception-handling operations include detecting an alternative target landing bay from the plurality of landing bays within the landing zone that is available for landing based on the continuous real-time sensor data by;

performing operations to adjust a perspective of the UAV independent of any landing routine, the operations comprising one or more of;

causing the UAV to ascend to a higher altitude above the landing zone, changing a pitch setting of the UAV, changing a roll setting of the UAV, or changing a yaw setting of the UAV; and

obtaining the continuous real-time sensor data in the adjusted perspective.

25. A computing device, comprising:

means for receiving continuous real-time sensor data;

means for detecting a target landing bay within a plurality of landing bays within a landing zone that is available for landing based on the continuous real-time sensor data;

means for calculating an orientation and position coordinates for landing in the target landing bay based on the continuous real-time sensor data;

means for obtaining information regarding positions and flight vectors of a plurality of other UAVs operating within the landing zone, wherein means for obtaining the information comprises;

means for receiving position and flight vector reports from the plurality of other UAVs,

wherein the UAV and the plurality of other UAVs are each independently executing a separate flight mission;

means for generating a flight plan for landing in the target landing bay based on the orientation and the position coordinates, the positions and flight vector reports of the plurality of other UAVs, and a current orientation and position of the computing device;

means for performing the flight plan for landing in the target landing bay;

means for determining whether an exception condition is identified based on the continuous real-time sensor data and the flight plan for landing in the target landing bay;

means for halting performance of the flight plan for landing in the target landing bay in response to determining that an exception condition is identified; and

means for performing exception-handling operations based on the identified exception condition in response to determining that an exception condition is identified, comprising means for detecting an alternative target landing bay from the plurality of landing bays within the landing zone that is available for landing based on the continuous real-time sensor data, wherein the means for detecting the alternative target landing bay further comprises;

means for performing operations to adjust a perspective of the UAV independent of any landing routine, comprising one or more of;

means for causing the UAV to ascend to a higher altitude above the landing zone, means for changing a pitch setting of the UAV, means for changing a roll setting of the UAV, or means for changing a yaw setting of the UAV; and

means for obtaining the continuous real-time sensor data in the adjusted perspective.