Collision detection and avoidance

• US 10,679,511 B2
• Filed: 12/29/2016
• Issued: 06/09/2020
• Est. Priority Date: 09/30/2016
• Status: Active Grant

• Current Assignee: Sony Interactive Entertainment Inc. (Sony Group Corp.)
• Original Assignee: Sony Interactive Entertainment Inc. (Sony Group Corp.)
• Inventors: Kikuchi, Megumi, Taylor, Michael, Castleman, Dennis Dale, Young, Andrew Stephen
• Primary Examiner(s): McPherson, James M
• Application Number:
  US15/394,267
• Publication Number:
  US 20180096611A1
• Time in Patent Office: 1,258 Days
• Field of Search: 701 2, 701 3, 701 23, 701 24, 701 314, 701120, 701301, 701 4, 701117, 382103, 725105, 384143
• CPC Class Codes: B64D 45/04 Landing aids; Safety measures to prevent collision with earth's surface
• CPC Class Codes: B64U 10/14 with four distinct rotor axes, e.g. quadcopters
• CPC Class Codes: B64U 2101/30 for imaging, photography or videography
• CPC Class Codes: B64U 2201/10 autonomous, i.e. by navigating independently from ground or air stations, e.g. by using inertial navigation systems [INS]
• CPC Class Codes: B64U 50/23 with each propulsion means having an individual motor
• CPC Class Codes: B64U 50/39 Battery swapping
• CPC Class Codes: G01C 21/16 by integrating acceleration or speed, i.e. inertial navigation
• CPC Class Codes: G05D 1/106 Change initiated in response to external conditions, e.g. avoidance of elevated terrain or of no-fly zones
• CPC Class Codes: G08G 5/0021 located in the aircraft
• CPC Class Codes: G08G 5/0069 specially adapted for an unmanned aircraft
• CPC Class Codes: G08G 5/0086 for monitoring terrain radar or analogous systems specially adapted for terrain avoidance G01S13/935
• CPC Class Codes: G08G 5/045 Navigation or guidance aids, e.g. determination of anti-collision manoeuvers radar or analogous systems specially adapted for anti-collision between aircraft G01S13/933

Abstract

A collision avoidance system for an unmanned aerial vehicle (UAV) receives physical space data for a flight area and creates a virtual world model to represent the flight area by mapping the physical space data with a physics engine. The automatic collision avoidance system creates a virtual UAV model to represent the UAV in the virtual world model. The automatic collision avoidance system receives flight data for the UAV and determines a current position of the virtual UAV model within the virtual world model. The automatic collision avoidance system determines a predicted trajectory of the virtual UAV model within the virtual world model, and determines whether the predicted trajectory will result in a collision of the virtual UAV model with the virtual world model. The automatic collision avoidance system performs evasive actions by the UAV, in response to determining that the predicted trajectory will result in a collision.

Claims

1. A method for a collision avoidance system of an unmanned aerial vehicle (UAV), the method comprising:

determining capability parameters regarding operation of the UAV;

receiving physical space data regarding a flight area;

creating a virtual world model to represent the flight area by mapping the physical space data with a physics engine, wherein a virtual UAV model represents future motion of the UAV in the virtual world model;

receiving flight data as the UAV flies along a flight path within the flight area;

determining a current position of the virtual UAV model within the virtual world model based on a current position of the UAV within the flight area as indicated by the received flight data;

determining a predicted trajectory of the virtual UAV model within the virtual world model based on the determined position of the virtual UAV model within the virtual world model and the flight path of the UAV as indicated by the received flight data;

determining that the predicted trajectory will result in a collision between the virtual UAV model and the virtual world model; and

performing evasive actions by the UAV based on the capability parameters regarding operation of the UAV in response to determining that the predicted trajectory will result in a collision, wherein performing the evasive actions includes overriding a user control.

2. The method of claim 1, further comprising:

determining a loss of user control of the UAV; and

performing different evasive actions by the UAV in response to determining the loss of user control.

3. The method of claim 2, wherein the loss of user control is determined in response to the UAV losing communication to a controller device.

4. The method of claim 2, wherein the loss of user control is determined in response to the UAV receiving erratic controller input from a controller device.

5. The method of claim 1, wherein the physical space data comprises pre-mapped data for the flight area.

6. The method of claim 1, wherein receiving the physical space data comprises receiving sensor data in real-time from at least one sensor on the UAV.

7. The method of claim 6, wherein the at least one sensor comprises at least one of a camera, an infra-red (IR) sensor, a Light Detection and Ranging (LiDAR) sensor, a proximity sensor, a radar, or a sonar sensor.

8. The method of claim 1, wherein creating the virtual world model occurs in real-time and uses simultaneous localization and mapping (SLAM).

9. The method of claim 1, wherein the flight data comprises sensor data from at least one sensor on the UAV.

10. The method of claim 9, wherein the sensor data comprises at least one of speed, altitude, direction, wind speed, wind direction.

11. The method of claim 1, wherein the flight data comprises controller inputs from a controller device.

12. The method of claim 1, wherein determining the predicted trajectory of the UAV comprises an inertial navigation system (INS) using a dead reckoning algorithm to calculate the current position of the UAV.

13. The method of claim 1, wherein overriding the user control comprises:

overriding controller inputs from a controller device; and

performing at least one of a braking or a banking maneuver by adjusting at least one of a roll, pitch, yaw, or throttle of the UAV.

14. The method of claim 1, wherein the capability parameters include at least one of maximum speed, maximum acceleration, maximum deceleration, or turning rate.

15. A system for collision avoidance, the system comprising:

a receiver module that receives;

physical space data regarding a flight area; and

flight data regarding operation of an unmanned aerial vehicle (UAV);

a processing module that executes instructions to;

create a virtual world model to represent the flight area by mapping the physical space data with a physics engine, wherein a virtual UAV model represents future motion of the UAV in the virtual world model;

determine a current position of the virtual UAV model within the virtual world model based on a current position of the UAV within the flight area as indicated by the received flight data;

determine a predicted trajectory of the virtual UAV model within the virtual world model based on the determined position of the virtual UAV model within the virtual world model and the flight path of the UAV as indicated by the received flight data; and

determine that the predicted trajectory will result in a collision between the virtual UAV model and the virtual world model;

a control module that controls performance of evasive actions by the UAV based on capability parameters regarding operation of the UAV in response to determining that the predicted trajectory will result in a collision, wherein performing the evasive actions includes overriding a user control; and

a memory coupled to the processing module, the processing module, and the control module for storing data.

16. The system of claim 15, wherein the physical space data comprises pre-mapped data for the flight area.

17. The system of claim 15, wherein the receiver module receives the physical space data by receiving sensor data in real-time from at least one sensor on the UAV.

18. The system of claim 15, wherein the flight data comprises sensor data from at least one sensor on the UAV.

19. The system of claim 15, wherein the processing unit determines the predicted trajectory of the UAV via an inertial navigation system (INS) that uses a dead reckoning algorithm to calculate the current position of the UAV.

20. The system of claim 15, wherein the control module further controls output of an evasion sound effect from at least one of the UAV or a controller device.