Geo-location or navigation camera, and aircraft and navigation method therefor

• US 10,386,188 B2
• Filed: 06/29/2016
• Issued: 08/20/2019
• Est. Priority Date: 06/29/2015
• Status: Active Grant

• Current Assignee: Yuneec Technology Co., Limited
• Original Assignee: Yuneec Technology Co., Limited
• Inventors: Tian, Yu, Jiang, Wenyan
• Primary Examiner(s): Navas, Jr., Edemio
• Application Number:
  US15/577,303
• Publication Number:
  US 20180149479A1
• Time in Patent Office: 1,147 Days
• Field of Search: None
• CPC Class Codes: B64C 39/024 of the remote controlled vehicle type, i.e. RPV
• CPC Class Codes: B64D 47/08 Arrangements of cameras
• CPC Class Codes: B64U 10/13 Flying platforms
• CPC Class Codes: B64U 20/87 Mounting of imaging devices, e.g. mounting of gimbals
• CPC Class Codes: B64U 2201/10 autonomous, i.e. by navigating independently from ground or air stations, e.g. by using inertial navigation systems [INS]
• CPC Class Codes: B64U 2201/20 Remote controls
• CPC Class Codes: B64U 30/20 Rotors; Rotor supports
• CPC Class Codes: G01C 21/005 with correlation of navigation data from several sources, e.g. map or contour matching G01C21/30 takes precedence
• CPC Class Codes: G01S 17/86 Combinations of lidar systems with systems other than lidar, radar or sonar, e.g. with direction finders
• CPC Class Codes: G01S 17/933 of aircraft or spacecraft
• CPC Class Codes: G01S 7/481 Constructional features, e.g. arrangements of optical elements
• CPC Class Codes: G03B 15/006 Apparatus mounted on flying objects
• CPC Class Codes: G03B 17/12 with means for supporting objectives, supplementary lenses, filters, masks, or turrets
• CPC Class Codes: G05D 1/0038 by providing the operator with simple or augmented images from one or more cameras located onboard the vehicle, e.g. tele-operation images analyzed by a computer and used for automatic navigation G05D1/0246
• CPC Class Codes: G05D 1/0094 involving pointing a payload, e.g. camera, weapon, sensor, towards a fixed or moving target
• CPC Class Codes: G05D 1/0202 specially adapted to aircraft
• CPC Class Codes: G05D 1/106 Change initiated in response to external conditions, e.g. avoidance of elevated terrain or of no-fly zones
• CPC Class Codes: G06T 2207/10032 Satellite or aerial image; Remote sensing
• CPC Class Codes: G06T 7/60 Analysis of geometric attributes
• CPC Class Codes: G06V 2201/07 Target detection
• CPC Class Codes: G08G 5/045 : Navigation or guidance aids, e.g. determination of anti-collision manoeuvers radar or analogous systems specially adapted for anti-collision between aircraft G01S13/933
• CPC Class Codes: H04N 23/685 : performed by mechanical compensation
• CPC Class Codes: H04N 23/69 : Control of means for changing angle of the field of view, e.g. optical zoom objectives or electronic zooming
• CPC Class Codes: H04N 23/698 : for achieving an enlarged field of view, e.g. panoramic image capture
• CPC Class Codes: H04N 23/90 : Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums

Abstract

Provided are a geo-location or navigation camera, and an aircraft and a navigation method therefor. The geo-location or navigation camera comprises an image capturing apparatus (15), an image capturing direction of the image capturing apparatus (15) being vertically downward. The camera further comprises a gimbal stability-enhancement system comprising a gimbal body and a gimbal control system connected to the gimbal body. The image capturing apparatus (15) is arranged on the gimbal body. By means of the balance control and shock absorption effects of the gimbal stability-enhancement system, the stability of the image capturing apparatus (15) is better, and the image capturing direction of the image capturing apparatus (15) is maintained to be always vertically downward. The aircraft can still be navigated without a GPS. The method has advantages of being highly accurate and widely applicable.

Claims

1. An aircraft, comprising a navigation system, the navigation system comprises a camera and a controller;

wherein an image capturing apparatus of the camera is configured for capturing an image when the aircraft is flying;

the controller comprises an acquisition module and a correction module;

the acquisition module is configured for acquiring a set of reference images for displaying a designated flying route;

the correction module is configured for comparing an image newly captured by the image capturing apparatus with the set of reference images when the aircraft is flying, and correcting the current flying route of the aircraft,

wherein the aircraft flies toward a location corresponding to a GPS positioning signal, and during the flight of the aircraft toward the location, the image capturing apparatus takes pictures of an area where a reference target is located to obtain a reference image, and an aircraft body of the aircraft extracts from the reference image initial position information of the reference target in the reference image,

if the image capturing apparatus arranged on the aircraft body acquires a temporary image showing a temporary target, the aircraft body extracts from the temporary image temporary position information of the temporary target in the temporary image,

the image capturing apparatus searches for a similar target in real time in the temporary image and extract a reference feature of the similar target, and if the reference feature is consistent with one of a plurality of temporary image features of the temporary target, the similar target is regarded as a candidate target in a target group,

the image capturing apparatus acquires and calculates provisional coordinate information, first provisional scale information and second provisional scale information of the similar target, wherein the first and second provisional scale information are a width and height value of the similar target in the temporary image respectively;

if the image capturing apparatus detects the target group and the image capturing apparatus tracks the temporary target, the provisional coordinate information, the first provisional scale information, and the second provisional scale information of all the candidate targets in the target group are weighted with the temporary position information, first temporary scale information and second temporary scale information of the temporary target respectively, and the result of the weighting calculation is output to the image capturing apparatus to display the result of the weighting calculation.

2. The aircraft of claim 1, wherein the acquisition module is configured for acquiring a topographic map of the designated flying route and using the topographic map as a reference image.

3. The aircraft of claim 2, wherein the designated flying route comprises a return route for the aircraft, and a topographic map of the return route is captured by the image capturing apparatus when the aircraft is on the outward voyage.

4. The aircraft of claim 1, wherein the correction module comprises an image processing module, a comparison module, and a flying control module;

the image processing module is configured for selecting a reference image from the set of reference images as a comparing image, and extracting the feature information from a newly captured image and the comparing image respectively;

the comparison module is configured for comparing the offset of the same feature information in the newly captured image and the comparing image;

the flying control module is configured for changing the current flying direction of the aircraft according to the offset.

5. The aircraft of claim 4, wherein the image processing module is configured for extracting the feature information of each of the set of reference images and selecting a reference image having the largest amount of same feature information as the newly captured image as the comparing image.

6. The aircraft of claim 1, wherein the navigation system further comprises a GPS positioning unit configured for receiving a GPS positioning signal at a location;

the aircraft body extracts from the reference image a reference image feature of the reference target;

if the image capturing apparatus arranged on the aircraft body acquires the temporary image showing the temporary target, the aircraft body extracts from the temporary image a temporary image feature of the temporary target;

if the reference image feature and the temporary image feature are consistent, the image capturing apparatus calculates the deviation direction based on the difference between the initial position information and the temporary position information;

the aircraft body tracks based on the deviation direction.

7. The aircraft of claim 6, wherein:

the image capturing apparatus extracts first reference scale information and second reference scale information of the reference target from the reference image, wherein the first reference scale information is a width value of the reference target in the reference image, and the second reference scale information is a height value of the reference target in the reference image;

if the image capturing apparatus determines that the reference image feature is consistent with the temporary image feature, the image capturing apparatus extracts the first temporary scale information and the second temporary scale information of the temporary target from the temporary image, wherein the first temporary scale information is a width value of the temporary target in the temporary image, and the second temporary scale information is a height value of the temporary target in the temporary image;

if the value of the first reference scale information is larger than the value of the first temporary scale information, the image capturing apparatus calculates a difference between the first reference scale information and the first temporary scale information, and if the absolute value of the difference is larger than a preset threshold, the aircraft body increases its flying speed;

if the value of the first reference scale information is smaller than the value of the first temporary scale information, the image capturing apparatus calculates a difference between the first reference scale information and the first temporary scale information, and if the absolute value of the difference is larger than a preset threshold, the aircraft body reduces its flying speed;

if the value of the second reference scale information is larger than the value of the second temporary scale information, the image capturing apparatus calculates a difference between the second reference scale information and the second temporary scale information, and if the absolute value of the difference is larger than a preset threshold, the aircraft body increases its flying speed;

if the value of the second reference scale information is smaller than the value of the second temporary scale information, the image capturing apparatus calculates a difference between the second reference scale information and the second temporary scale information, and if the absolute value of the difference is larger than a preset threshold, the aircraft body reduces its flying speed.

8. The aircraft of claim 6, wherein:

the distance between the aircraft body and the temporary target conforms to the following constraint;

0.9*Z<

X<

1.1*Z, where Z is a preset threshold, and X is the distance between the aircraft body and the temporary target.

9. The aircraft of claim 8, wherein:

if a temporary vertical coordinate of the temporary position information is not consistent with an initial vertical coordinate of the initial position information, the aircraft body calculates a difference between the temporary vertical coordinate and the initial vertical coordinate and generates longitudinal displacement information, wherein the initial vertical coordinate is a position coordinate of the reference target in the coordinate system established based on the reference image, and the temporary vertical coordinate is a position coordinate of the temporary target in the coordinate system established based on the temporary image;

the aircraft body adjusts a vertical angle of the image capturing apparatus for capturing the temporary image.

10. The aircraft of claim 9, wherein:

if a temporary horizontal coordinate of the temporary position information is not consistent with an initial horizontal coordinate of the initial position information, the aircraft body calculates a difference between the temporary horizontal coordinate and the initial horizontal coordinate and generates lateral displacement information, wherein the initial horizontal coordinate is a position coordinate of the reference target in the coordinate system established based on the reference image, the temporary horizontal coordinate is a position coordinate of the temporary target in the coordinate system established based on the temporary image;

the aircraft body adjusts a horizontal angle of the image capturing apparatus for capturing the temporary image.

11. The aircraft of claim 6, wherein:

if the aircraft body detects a change in any of the temporary image feature, the temporary position information, the first temporary scale information, and the second temporary scale information, the temporary image feature, the temporary position information, the first temporary scale information, and the second temporary scale information after the change are used to update the temporary image feature, the temporary location information, the first temporary scale information, and the second temporary scale information before the change respectively.

12. The aircraft of claim 1, wherein:

if the image capturing apparatus does not detect the target group and the image capturing apparatus tracks the temporary target, the temporary position information, the first temporary scale information, and the second temporary scale information of the temporary target are output to the image capturing apparatus to display the temporary position information, the first temporary scale information and the second temporary scale information;

if the image capturing apparatus detects the target group and the image capturing apparatus does not track the temporary target, the provisional coordinate information, the first provisional scale information, and the second provisional scale information of all the candidate targets in the target group are weighted respectively, and the result of the weighting calculation are output to the image capturing apparatus to display the result of the weighting calculation;

if the image capturing apparatus does not detect the target group and the image capturing apparatus does not track the temporary target, no information is output to the image capturing apparatus.

13. A navigation method for an aircraft comprising a camera, an image capturing apparatus of the camera is configured for capturing an image when the aircraft is flying;

wherein the navigation method comprises;

a step S₁ of acquiring a set of reference images for displaying a designated flying route;

a step S₂ of comparing an image newly captured by the image capturing apparatus with the set of reference images when the aircraft is flying, and correcting the current flying route of the aircraft,

wherein the aircraft flies toward a location corresponding to a GPS positioning signal, and the navigation method further comprises;

during the flight of the aircraft toward the location, taking pictures, by the image capturing apparatus, of an area where a reference target is located to obtain a reference image;

extracting, by an aircraft body of the aircraft, from the reference image initial position information of the reference target in the reference image; and

if the image capturing apparatus arranged on the aircraft body acquires a temporary image showing a temporary target, extracting, by the aircraft body, from the temporary image temporary position information of the temporary target in the temporary image;

wherein the navigation method further comprises;

searching for, by the image capturing apparatus, a similar target in real time in the temporary image and extracting a reference feature of the similar target, and if the reference feature is consistent with one of a plurality of temporary image features of the temporary target, regarding the similar target as a candidate target in a target group,

wherein the navigation method further comprises;

acquiring and calculating, by the image capturing apparatus, provisional coordinate information, first provisional scale information and second provisional scale information of the similar target, wherein the first and second provisional scale information are a width and height value of the similar target in the temporary image respectively; and

if the image capturing apparatus detects the target group and the image capturing apparatus tracks the temporary target, weighting the provisional coordinate information, the first provisional scale information, and the second provisional scale information of all the candidate targets in the target group with the temporary position information, first temporary scale information and second temporary scale information of the temporary target respectively, and outputting the result of the weighting calculation to the image capturing apparatus to display the result of the weighting calculation.

14. The navigation method of claim 13, wherein the step S₂ comprises:

a step S₂₁ of selecting a reference image from the set of reference images as a comparing image, and extracting the feature information from a newly captured image and the comparing image respectively;

a step S₂₂ of comparing the offset of the same feature information in the newly captured image and the comparing image;

a step S₂₃ of changing the current flying direction of the aircraft according to the offset.

15. The navigation method of claim 14, wherein the step S₂₁ comprises:

extracting the feature information of each of the set of reference images and selecting a reference image having the largest amount of same feature information as the newly captured image as the comparing image.

16. The navigation method of claim 13, wherein the navigation method further comprises:

receiving a GPS positioning signal at a location;

extracting, by the aircraft body, from the reference image a reference image feature of the reference target;

if the image capturing apparatus arranged on the aircraft body acquires the temporary image showing the temporary target, extracting, by the aircraft body, from the temporary image a temporary image feature of the temporary target;

if the reference image feature and the temporary image feature are consistent, calculating, by the image capturing apparatus, the deviation direction based on the difference between the initial position information and the temporary position information;

tracking, by the aircraft body, based on the deviation direction.

17. The navigation method of claim 16, wherein the navigation method further comprises:

extracting, by the image capturing apparatus, first reference scale information and second reference scale information of the reference target from the reference image, wherein the first reference scale information is a width value of the reference target in the reference image, and the second reference scale information is a height value of the reference target in the reference image;

if the image capturing apparatus determines that the reference image feature is consistent with the temporary image feature, extracting, by the image capturing apparatus, the first temporary scale information and the second temporary scale information of the temporary target from the temporary image, wherein the first temporary scale information is a width value of the temporary target in the temporary image, and the second temporary scale information is a height value of the temporary target in the temporary image;

if the value of the first reference scale information is larger than the value of the first temporary scale information, calculating, by the image capturing apparatus, a difference between the first reference scale information and the first temporary scale information, and if the absolute value of the difference is larger than a preset threshold, increasing, by the aircraft body, its flying speed;

if the value of the first reference scale information is smaller than the value of the first temporary scale information, calculating, by the image capturing apparatus, a difference between the first reference scale information and the first temporary scale information, and if the absolute value of the difference is larger than a preset threshold, reducing, by the aircraft body, its flying speed;

if the value of the second reference scale information is larger than the value of the second temporary scale information, calculating, by the image capturing apparatus, a difference between the second reference scale information and the second temporary scale information, and if the absolute value of the difference is larger than a preset threshold, increasing, by the aircraft body, its flying speed;

if the value of the second reference scale information is smaller than the value of the second temporary scale information, calculating, by the image capturing apparatus, a difference between the second reference scale information and the second temporary scale information, and if the absolute value of the difference is larger than a preset threshold, reducing, by the aircraft body, its flying speed.

18. The navigation method of claim 17, wherein the navigation method further comprises:

the distance between the aircraft body and the temporary target conforming to the following constraint;

0.9*Z<

X<

1.1*Z, where Z is a preset threshold, and X is the distance between the aircraft body and the temporary target.

19. The navigation method of claim 18, wherein the navigation method further comprises:

if a temporary vertical coordinate of the temporary position information is not consistent with an initial vertical coordinate of the initial position information, calculating, by the aircraft body, a difference between the temporary vertical coordinate and the initial vertical coordinate and generates longitudinal displacement information, wherein the initial vertical coordinate is a position coordinate of the reference target in the coordinate system established based on the reference image, and the temporary vertical coordinate is a position coordinate of the temporary target in the coordinate system established based on the temporary image;

adjusting, by the aircraft body, a vertical angle of the image capturing apparatus for capturing the temporary image.

20. The navigation method of claim 19, wherein the navigation method further comprises:

if a temporary horizontal coordinate of the temporary position information is not consistent with an initial horizontal coordinate of the initial position information, calculating, by the aircraft body, a difference between the temporary horizontal coordinate and the initial horizontal coordinate and generates lateral displacement information, wherein the initial horizontal coordinate is a position coordinate of the reference target in the coordinate system established based on the reference image, the temporary horizontal coordinate is a position coordinate of the temporary target in the coordinate system established based on the temporary image;

adjusting, by the aircraft body, a horizontal angle of the image capturing apparatus for capturing the temporary image.

21. The navigation method of claim 16, wherein the navigation method further comprises:

if the aircraft body detects a change in any of the temporary image feature, using the temporary position information, the first temporary scale information, and the second temporary scale information, the temporary image feature, the temporary position information, the first temporary scale information, and the second temporary scale information after the change to update the temporary image feature, the temporary location information, the first temporary scale information, and the second temporary scale information before the change respectively.

22. The navigation method of claim 13, wherein the navigation method further comprises:

if the image capturing apparatus does not detect the target group and the image capturing apparatus tracks the temporary target, outputting the temporary position information, the first temporary scale information, and the second temporary scale information of the temporary target to the image capturing apparatus to display the temporary position information, the first temporary scale information and the second temporary scale information;

if the image capturing apparatus detects the target group and the image capturing apparatus does not track the temporary target, weighting the provisional coordinate information, the first provisional scale information, and the second provisional scale information of all the candidate targets in the target group respectively, and outputting the result of the weighting calculation to the image capturing apparatus to display the result of the weighting calculation;

if the image capturing apparatus does not detect the target group and the image capturing apparatus does not track the temporary target, outputting no information to the image capturing apparatus.