AIRCRAFT AND OBSTACLE AVOIDANCE METHOD AND SYSTEM THEREOF

• US 20180136650A1
• Filed: 06/29/2016
• Published: 05/17/2018
• Est. Priority Date: 06/29/2015
• Status: Active Grant

• Current Assignee: Yuneec Technology Co., Limited
• Original Assignee: Yuneec Technology Co., Limited
• Inventors: TIAN, YU, JIANG, Wenyan
• Granted Patent: US 10,634,500 B2
• Time in Patent Office: Days
• CPC Class Codes: B64C 39/024 of the remote controlled vehicle type, i.e. RPV
• CPC Class Codes: B64D 47/08 Arrangements of cameras
• CPC Class Codes: B64U 10/13 Flying platforms
• CPC Class Codes: B64U 20/87 Mounting of imaging devices, e.g. mounting of gimbals
• CPC Class Codes: B64U 2201/10 autonomous, i.e. by navigating independently from ground or air stations, e.g. by using inertial navigation systems [INS]
• CPC Class Codes: B64U 2201/20 Remote controls
• CPC Class Codes: B64U 30/20 Rotors; Rotor supports
• CPC Class Codes: G01C 21/005 with correlation of navigation data from several sources, e.g. map or contour matching G01C21/30 takes precedence
• CPC Class Codes: G01S 17/86 Combinations of lidar systems with systems other than lidar, radar or sonar, e.g. with direction finders
• CPC Class Codes: G01S 17/933 of aircraft or spacecraft
• CPC Class Codes: G01S 7/481 Constructional features, e.g. arrangements of optical elements
• CPC Class Codes: G03B 15/006 Apparatus mounted on flying objects
• CPC Class Codes: G03B 17/12 with means for supporting objectives, supplementary lenses, filters, masks, or turrets
• CPC Class Codes: G05D 1/0038 by providing the operator with simple or augmented images from one or more cameras located onboard the vehicle, e.g. tele-operation images analyzed by a computer and used for automatic navigation G05D1/0246
• CPC Class Codes: G05D 1/0094 involving pointing a payload, e.g. camera, weapon, sensor, towards a fixed or moving target
• CPC Class Codes: G05D 1/0202 specially adapted to aircraft
• CPC Class Codes: G05D 1/106 Change initiated in response to external conditions, e.g. avoidance of elevated terrain or of no-fly zones
• CPC Class Codes: G06T 2207/10032 Satellite or aerial image; Remote sensing
• CPC Class Codes: G06T 7/60 Analysis of geometric attributes
• CPC Class Codes: G06V 2201/07 Target detection
• CPC Class Codes: G08G 5/045 : Navigation or guidance aids, e.g. determination of anti-collision manoeuvers radar or analogous systems specially adapted for anti-collision between aircraft G01S13/933
• CPC Class Codes: H04N 23/685 : performed by mechanical compensation
• CPC Class Codes: H04N 23/69 : Control of means for changing angle of the field of view, e.g. optical zoom objectives or electronic zooming
• CPC Class Codes: H04N 23/698 : for achieving an enlarged field of view, e.g. panoramic image capture
• CPC Class Codes: H04N 23/90 : Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums

Abstract

An aircraft and an obstacle avoidance method and system thereof. The obstacle avoidance system comprises an image capturing apparatus (11), a gimbal stability-enhancement system, and a second controller (14). The gimbal stability-enhancement system comprises a gimbal body (12) and a gimbal control system (13). The image capturing apparatus (11) is arranged on the gimbal body (12), and is used for capturing an image in a flying direction when the aircraft is flying. The gimbal control system (13) is connected to the gimbal body (12). The second controller (14) is used for determining whether an obstacle exists in the image captured by the image capturing apparatus (11), and if yes, changing the flying direction of the aircraft according to a position of the obstacle, and if not, controlling the aircraft to fly in the current flying direction.

Claims

1. An obstacle avoidance system of an aircraft comprising an image capturing apparatus, a gimbal stability-enhancement system, and a second controller;

the gimbal stability-enhancement system comprising a gimbal body and a gimbal control system, the image capturing apparatus being arranged on the gimbal body and being configured for capturing an image in a flying direction when the aircraft is flying, the gimbal control system being connected to the gimbal body;

the second controller being configured for determining whether an obstacle exists in the image captured by the image capturing apparatus, and if yes, changing the flying direction of the aircraft according to a position of the obstacle, and if not, controlling the aircraft to fly in the current flying direction.

2. The obstacle avoidance system of claim 1, wherein the gimbal control system comprises a first controller, a first motor and a second motor, the first motor and the second motor are configured for controlling the rotation of the gimbal body in the axial direction of the Pitch axis and the Roll axis of a three-dimensional coordinate system respectively, the first controller comprises a balance control module, the balance control module is electrically connected to the first motor and the second motor respectively and is configured for controlling operation of the first motor and the second motor;

the gimbal control system further comprises a third motor for controlling the rotation of the gimbal body in the axial direction of the Yaw axis of the three-dimensional coordinate system, the balance control module is further electrically connected to the third motor and is configured for controlling operation of the first motor, the second motor and the third motor.

3. The obstacle avoidance system of claim 1, further comprising a distance measurement module;

wherein the second controller is further configured for invoking the distance measurement module to detect a distance between the aircraft and an obstacle when it is determined that the obstacle exists in an image captured by the image capturing apparatus, changing the flying direction of the aircraft according to a position of the obstacle if the distance is less than or equal to a distance threshold, and controlling the aircraft to fly in the current flying direction if the distance is greater than the distance threshold.

4. The obstacle avoidance system of claim 1, wherein the second controller is further configured for determining whether the altitude of an obstacle is higher than the flying altitude of the aircraft when it is determined that the obstacle exists in an image captured by the image capturing apparatus, and if yes, increasing the flying altitude of the aircraft, and if not, controlling the aircraft to fly in the current flying direction.

5. The obstacle avoidance system of claim 3, wherein the second controller is further configured for invoking the distance measurement module to detect a distance between the aircraft and an obstacle when it is determined that the obstacle exists in an image captured by the image capturing apparatus, and adjusting the flying speed of the aircraft according to a relationship between the distance and a preset threshold.

6. The obstacle avoidance system of claim 5, wherein the adjusting the flying speed of the aircraft according to a relationship between the distance and a preset threshold comprises reducing the flying speed of the aircraft when the distance is less than or equal to the preset threshold.

7. The obstacle avoidance system of claim 1, wherein the second controller is further configured for adjusting the flying speed of the aircraft according to a relationship between the reference scale information of a reference target in a reference image and the temporary scale information a temporary target in a temporary image when it is determined that an obstacle exists in an image captured by the image capturing apparatus;

wherein the temporary image is an image, among images captured by the image capturing apparatus, where an obstacle exists and whose capture time is closest to the current time;

the reference image is an image, among images captured by the image capturing apparatus, where an obstacle exists and whose capture time is prior to the capture time of the temporary image;

the obstacle in the reference image is referred as a reference target; and

the obstacle in the temporary image is referred as a temporary target.

8. The obstacle avoidance system of claim 7, wherein the second controller is further configured for:

extracting a reference image feature of the reference target from the reference image;

extracting a temporary image feature of the temporary target from the temporary image;

extracting the reference scale information of the reference target from the reference image and extracting the temporary scale information of the temporary target from the temporary image when it is determined that the reference image feature and the temporary image feature are consistent; and

calculating a difference between the reference scale information and the temporary scale information when the temporary scale information is greater than the reference scale information, and reducing the flying speed of the aircraft if the absolute value of the difference is greater than a difference threshold.

9. The obstacle avoidance system of claim 1, wherein the obstacle avoidance system comprises a plurality of image capturing apparatuses, which are arranged in different directions of the gimbal body respectively and are configured for capturing images in different directions respectively, wherein the capturing images in different directions comprises capturing an image in a flying direction when the aircraft is flying.

10. The obstacle avoidance system of claim 9, wherein the obstacle avoidance system comprises three image capturing apparatuses arranged on the gimbal body respectively, and optical axes of lenses of the three image capturing apparatuses are located on the same plane and the angle between optical axes of lenses of two adjacent image capturing apparatuses is 120°

.

11. An obstacle avoidance method for an aircraft, the obstacle avoidance method comprising:

a step S₁ of capturing an image in a flying direction when the aircraft is flying;

a step S₂ of determining whether an obstacle exists in the captured image, and if yes, performing a step S₃, and if not, performing a step S₄;

the step S₃ of changing the flying direction of the aircraft according to a position of the obstacle; and

the step S₄ of controlling the aircraft to fly in the current flying direction.

12. The obstacle avoidance method of claim 18, further comprising:

performing a step T₁ when it is determined that an obstacle exists in the captured image in the step S₂; and

the step T₁ of detecting a distance between the aircraft and the obstacle, and performing the step S₃ if the distance is less than or equal to a distance threshold, and performing the step S₄ if the distance is greater than the distance threshold.

13. The obstacle avoidance method of claim 11, further comprising performing the step P₁ when it is determined that an obstacle exists in the captured image in the step S₂;

and

the step P₁ of determining whether the altitude of the obstacle is higher than the flying altitude of the aircraft, and if yes, increasing the flying altitude of the aircraft, and if not, controlling the aircraft to fly in the current flying direction.

14. The obstacle avoidance method of claim 11, further comprising:

transmitting the captured image to a remote controller for controlling the aircraft.

15. The obstacle avoidance method of claim 14, further comprising:

receiving a control signal from the remote controller and controlling the aircraft according to the control signal.

16. The obstacle avoidance method of claim 12, further comprising:

detecting a distance between the aircraft and an obstacle when it is determined that the obstacle exists in the captured image in the step S₂, and adjusting the flying speed of the aircraft according to a relationship between the distance and a preset threshold.

17. The obstacle avoidance method of claim 16, wherein the adjusting the flying speed of the aircraft according to a relationship between the distance and a preset threshold comprises reducing the flying speed of the aircraft when the distance is less than or equal to the preset threshold.

18. The obstacle avoidance method of claim 11, further comprising:

adjusting the flying speed of the aircraft according to a relationship between the reference scale information of a reference target in a reference image and the temporary scale information a temporary target in a temporary image when it is determined that the obstacle exists in the captured image in the step S₂;

wherein the temporary image is an image, among the captured images, where an obstacle exists and whose capture time is closest to the current time;

the reference image is an image, among the captured images, where an obstacle exists and whose capture time is prior to the capture time of the temporary image;

the obstacle in the reference image is referred as a reference target; and

the obstacle in the temporary image is referred as a temporary target.

19. The obstacle avoidance method of claim 18, further comprising:

extracting a reference image feature of the reference target from the reference image;

extracting a temporary image feature of the temporary target from the temporary image;

extracting the reference scale information of the reference target from the reference image and extracting the temporary scale information of the temporary target from the temporary image when it is determined that the reference image feature and the temporary image feature are consistent; and

calculating a difference between the reference scale information and the temporary scale information when the temporary scale information is greater than the reference scale information, and reducing the flying speed of the aircraft if the absolute value of the difference is greater than a difference threshold.

20. The obstacle avoidance method of claim 11, wherein the aircraft comprises a plurality of image capturing apparatuses, which are arranged in different directions of the gimbal body of the aircraft respectively and are configured for capturing images in different directions respectively, wherein the capturing images in different directions comprises capturing an image in a flying direction when the aircraft is flying;

wherein the obstacle avoidance method further comprises;

splicing the images captured by these image capturing apparatuses into a panoramic image.