Commercial and general aircraft avoidance using light pattern detection

• US 9,761,147 B2
• Filed: 12/12/2014
• Issued: 09/12/2017
• Est. Priority Date: 12/12/2014
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Buchmueller, Daniel, Paczan, Nathan Michael
• Primary Examiner(s): Paige, Tyler
• Application Number:
  US14/569,125
• Publication Number:
  US 20160171896A1
• Time in Patent Office: 1,005 Days
• Field of Search: 701 3, 455431
• CPC Class Codes: B64C 39/02 characterised by special use
• CPC Class Codes: B64U 2201/10 autonomous, i.e. by navigating independently from ground or air stations, e.g. by using inertial navigation systems [INS]
• CPC Class Codes: G05D 1/0088 characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours using knowledge based models G06N5/00
• CPC Class Codes: G05D 1/106 Change initiated in response to external conditions, e.g. avoidance of elevated terrain or of no-fly zones
• CPC Class Codes: G08G 5/0008 with other aircraft
• CPC Class Codes: G08G 5/0013 with a ground station
• CPC Class Codes: G08G 5/0021 located in the aircraft
• CPC Class Codes: G08G 5/0039 Modification of a flight plan
• CPC Class Codes: G08G 5/0069 specially adapted for an unmanned aircraft
• CPC Class Codes: G08G 5/045 Navigation or guidance aids, e.g. determination of anti-collision manoeuvers radar or analogous systems specially adapted for anti-collision between aircraft G01S13/933

Abstract

This disclosure is directed to a detection and avoidance apparatus for an unmanned aerial vehicle (“UAV”) and systems, devices, and techniques pertaining to automated object detection and avoidance during UAV flight. The system may detect objects within the UAV'"'"'s airspace through acoustic, visual, infrared, multispectral, hyperspectral, or object detectable signal emitted or reflected from an object. The system may identify the source of the object detectable signal by comparing features of the received signal with known sources signals in a database. The features may be, for example, a light arrangement or number of lights associated with the object. Furthermore, a trajectory envelope for the object may be determined based on characteristic performance parameters for the object such as cursing speed, maneuverability, etc. The UAV may determine an optimized flight plan based on the trajectory envelopes of detected objects within the UAV'"'"'s airspace to avoid the detected objects.

Claims

1. A method comprising:

capturing, over a period of time, by one or more optical sensors coupled to an unmanned aerial vehicle (UAV), sensor data representing images of an airspace surrounding the UAV;

detecting, in the sensor data, information representative of a plurality of lights associated with a flying object;

determining an estimated distance between the UAV and the flying object;

determining one or more characteristic features of the plurality of lights based at least in part on the sensor data and the estimated distance between the UAV and the flying object;

identifying, using a characteristic feature database, the flying object based at least in part on the one or more characteristic features of the plurality of lights;

determining an estimated airspeed of the flying object based at least in part on the sensor data;

determining a trajectory envelope of the flying object based at least in part on performance parameters associated with the identified flying object and the estimated airspeed of the flying object; and

updating a flight plan of the UAV based at least in part on the trajectory envelope of the flying object.

2. The method of claim 1, wherein the one or more characteristic features define an object signature and include at least one of exterior aircraft lighting systems, or one or more anti-collision lights, and wherein the object is identified at least partly by determining at least one of:

an estimated distance between at least two of the plurality of detected lights, or

a rotational frequency of the detected anti-collision light.

3. The method of claim 1,

wherein the identifying includes associating the flying object with a class of flying objects based at least in part on the one or more characteristic features;

further comprising associating the flying object with the one or more performance parameters by lookup of the class of flying objects in a database; and

wherein the determining the trajectory envelope is based at least in part on the performance parameters associated with the class of flying objects.

4. The method of claim 1, wherein the characteristic feature database stores at least one of a rate of climb, a rate of decent, or maneuverability parameters for objects, and wherein the trajectory envelope is based at least in part on the rate of climb, the rate of decent, or the maneuverability parameters associated, via the characteristic feature database, with the flying object.

5. A unmanned aerial vehicle (UAV), comprising:

one or more processors;

memory to store computer-readable instructions;

one or more optical sensors to capture signals providing a visual representation of an airspace at least partially surrounding the UAV; and

a flight management component stored in the memory that, when executed, causes the one or more processors to;

detect, from the signals, an object in the airspace at least partially surrounding the UAV;

determine, from the signals, one or more operating characteristics associated with the object based at least in part on lights associated with the object; and

determine a trajectory envelope associated with the object based at least in part on the one or more operating characteristics.

6. The UAV of claim 5, wherein the fight management component, when executed, further cause the one or more processors to:

determine the likelihood of interaction between the UAV and the trajectory envelope associated with the object; and

update a UAV flight plan to avoid interaction between the UAV and the trajectory envelope.

7. The UAV of claim 5, further comprising a communication component that maintains a communication network between the UAV and one or more additional UAVs operating within an airspace of the UAV, wherein the communication component transmits at least one of:

one or more of the signals of the airspace,

the one or more operating characteristics of the object, or

the trajectory envelope associated with the object.

8. The UAV of claim 5, wherein determination, from the signals, of the one or more operating characteristics associated with the object is based at least in part on the lights from a visible light spectrum.

9. The UAV of claim 5, further comprising a communication component that maintains a communication network between the UAV and one or more additional UAVs operating within an airspace of the UAV, wherein the detecting an object in the airspace at least partially surrounding the UAV is further based on at least additional signals received from the one or more additional UAVs.

10. The UAV of claim 5, further comprising an acoustic sensor to capture acoustic energy emitted or reflected by the object, the flight management component causing the one or more processors to:

detect, from an acoustic signal representing the captured acoustic energy, the object in the airspace at least partially surrounding the UAV; and

determine, from the acoustic signal representing the captured acoustic energy, one or more different operating characteristics associated with the object based at least in part on a fingerprint associated with the acoustic signal representing the captured,

wherein the determining the trajectory envelope is further based at least in part on the one or more different operating characteristics.

11. The UAV of claim 5, further comprising a multispectral sensor to capture specific wavelengths of electromagnetic energy reflected or emitted by the object and indicative of unique materials associated with the object, and further comprising identifying the object based at least in part on the specific wavelengths of electromagnetic energy reflected or emitted by the object.

12. The UAV of claim 5, wherein the determining one or more operating characteristics further comprises:

determining a distance between a first light of the associated lights and a second light of the associated lights; and

associating, via a lookup operation, the distance between the first and second light of the associated lights with one or more characteristic features of objects stored in a database.

13. A flight management system comprising:

one or more processors; and

memory to store computer-executable instructions that, when executed, cause the one or more processors to perform acts comprising;

receiving imagery of at least a portion of an airspace surrounding an unmanned aerial vehicle (UAV);

analyzing the imagery to detect one or more characteristic features of illumination sources shown in the imagery;

identifying, by comparing the one or more characteristic features to a database, an object associated with the illumination sources; and

determining, based at least in part on the identification, a trajectory envelope of the object.

14. The flight management system of claim 13, where the acts further comprise updating a flight plan of the UAV based at least in part on a probability of interaction between the UAV and the trajectory envelope of the object.

15. The flight management system of claim 13, wherein the acts further comprise associating one or more performance parameters with the object using a database, and wherein the determining the trajectory envelope is based at least in part on the performance parameters.

16. The flight management system of claim 15, wherein the performance parameters include at least a rate of climb, a rate of decent, and a maneuverability parameter associated with each of various objects.

17. The flight management system of claim 13, wherein the one or more characteristics features of the illumination sources shown in the imagery is a rotational frequency or flash frequency of at least one of the illumination sources shown in the imagery.

18. The flight management system of claim 13, wherein the trajectory envelope is represented by a volume of airspace and reflects the probability that the object will move to a given location within the volume of airspace within a predetermined amount of time.

19. The flight management system of claim 13, wherein the predetermined amount of time is based in part on operating characteristics of the UAV.

20. The object detection and avoidance system of claim 13, wherein the object is a stationary object.