Target tracking method for air vehicle

• US 10,571,934 B2
• Filed: 04/29/2017
• Issued: 02/25/2020
• Est. Priority Date: 04/29/2016
• Status: Active Grant

• Current Assignee: Yuneec International Co. Ltd.
• Original Assignee: Yuneec International Co. Ltd.
• Inventors: Tian, Yu, Jiang, Wenyan
• Primary Examiner(s): Jha, Abdhesh K
• Application Number:
  US15/582,662
• Publication Number:
  US 20180107229A1
• Time in Patent Office: 1,032 Days
• Field of Search: None
• CPC Class Codes: G05D 1/0094 involving pointing a payload, e.g. camera, weapon, sensor, towards a fixed or moving target
• CPC Class Codes: G05D 1/102 specially adapted for vertical take-off of aircraft
• CPC Class Codes: G05D 1/12 Target-seeking control

Abstract

A target tracking method for an air vehicle includes steps of: during a flight of an air vehicle main body to a position corresponding to a global positioning system (GPS) positioning signal, taking pictures of an area where a reference target is located by a camera, and obtaining a reference image; extracting reference image characteristics of the reference target and initial position information of the reference target in the reference image from the reference image; when the camera obtains a temporary image showing a temporary target, extracting temporary image characteristics of the temporary target and temporary position information of the temporary target in the temporary image from the temporary image; if the reference image characteristics are consistent with the temporary image characteristics, calculating a deviation direction according to a difference between the initial position information and the temporary position information; and tracking the temporary target in the deviation direction.

Claims

1. A target tracking method for an air vehicle comprising steps of:

during a flight process of an air vehicle main body to a position corresponding to a positioning signal of a global positioning system (GPS), taking a picture of an area where a reference target is located by a camera, and obtaining a reference image;

extracting reference image characteristics of the reference target and initial position information of the reference target in the reference image from the reference image by the air vehicle main body;

when the camera on the air vehicle main body obtains a temporary image showing a temporary target, extracting temporary image characteristics of the temporary target and temporary position information of the temporary target in the temporary image from the temporary image by the air vehicle main body;

if the reference image characteristics are consistent with the temporary image characteristics, calculating a deviation direction by the camera according to a difference between the initial position information and the temporary position information; and

tracking the temporary target by the air vehicle main body in the deviation direction;

the method further comprises steps of;

extracting first reference scale information and second reference scale information of the reference target from the reference image by the camera, wherein the first reference scale information is a width value of the reference target in the reference image; and

the second reference scale information is a height value of the reference target in the reference image;

when the camera judges that the reference image characteristics are consistent with the temporary image characteristics, extracting first temporary scale information and second temporary scale information of the temporary target from the temporary image by the camera, wherein the first temporary scale information is a width value of the temporary target in the temporary image; and

the second temporary scale information is a height value of the temporary target in the temporary image;

when a value of the first reference scale information is larger than a value of the first temporary scale information, calculating a difference between the first reference scale information and the first temporary scale information by the camera; and

, if an absolute value of the difference is larger than a preset threshold, accelerating a motion of the air vehicle main body;

when the value of the first reference scale information is smaller than the value of the first temporary scale information, calculating the difference between the first reference scale information and the first temporary scale information by the camera; and

, if the absolute value of the difference is larger than the preset threshold, decelerating the motion of the air vehicle main body;

when a value of the second reference scale information is larger than a value of the second temporary scale information, calculating a difference between the second reference scale information and the second temporary scale information by the camera; and

, if an absolute value of the difference is larger than the preset threshold, accelerating the motion of the air vehicle main body; and

when the value of the second reference scale information is smaller than the value of the second temporary scale information, calculating the difference between the second reference scale information and the second temporary scale information; and

, if the absolute value of the difference is larger than the preset threshold, decelerating the motion of the air vehicle main body.

2. The target tracking method for the air vehicle, as recited in claim 1, wherein:

a distance between the air vehicle main body and the temporary target meets a constraint condition of 0.9×

Z<

X<

1.1×

Z, wherein Z is the preset threshold and X is the distance between the air vehicle main body and the temporary target.

3. The target tracking method for the air vehicle, as recited in claim 2, further comprising steps of:

when temporary vertical coordinates of the temporary position information are inconsistent with initial vertical coordinates of the initial position information, calculating a difference between the temporary vertical coordinates and the initial vertical coordinates, and generating longitudinal displacement information by the air vehicle main body, wherein;

the initial vertical coordinates are position coordinates of the reference target in a first coordinate system which is established according to the reference image;

and the temporary vertical coordinates are position coordinates of the temporary target in a second coordinate system which is established according to the temporary image; and

adjusting a vertical angle of the camera, which captures the temporary image, by the air vehicle main body.

4. The target tracking method for the air vehicle, as recited in claim 3, further comprising steps of:

when temporary horizontal coordinates of the temporary position information are inconsistent with initial horizontal coordinates of the initial position information, calculating a difference between the temporary horizontal coordinates and the initial horizontal coordinates, and generating transverse displacement information by the air vehicle main body, wherein;

the initial horizontal coordinates are position coordinates of the reference target in the first coordinate system which is established according to the reference image; and

the temporary horizontal coordinates are position coordinates of the temporary target in the second coordinate system which is established according to the temporary image; and

adjusting a horizontal angle of the camera, which captures the temporary image, by the air vehicle main body.

5. The target tracking method for the air vehicle, as recited in claim 1, further comprising steps of:

finding a similar target in the temporary image in real-time and extracting reference characteristics of the similar target by the camera; and

, if the reference characteristics are consistent with one of the temporary image characteristics, selecting the similar target as a candidate target in a target group.

6. The target tracking method for the air vehicle, as recited in claim 5, further comprising steps of:

when the air vehicle main body detects that any one of the temporary image characteristics, the temporary position information, the first temporary scale information and the second temporary scale information changes, replacing the temporary image characteristics, the temporary position information, the first temporary scale information and the second temporary scale information before changing by the temporary image characteristics, the temporary position information, the first temporary scale information and the second temporary scale information after changing.

7. The target tracking method for the air vehicle, as recited in claim 6, further comprising steps of:

obtaining and calculating temporary coordinate information, first temporary scale information and second temporary scale information of the similar target by the camera, wherein the first temporary scale information is a width value of the similar target in the temporary image; and

the second temporary scale information is a height value of the similar target in the temporary image;

when the camera detects the target group and tracks the temporary target, weighting temporary coordinate information, first temporary scale information and second temporary scale information of each candidate target in the target group respectively with the temporary position information, the first temporary scale information and the second temporary scale information of the temporary target, outputting weighting results thereof to the camera, and showing the weighting results by the camera;

when the camera does not detect the target group but tracks the temporary target, outputting the temporary position information, the first temporary scale information and the second temporary scale information of the temporary target to the camera, and showing the temporary position information, the first temporary scale information and the second temporary scale information of the temporary target by the camera;

when the camera detects the target group but does not track the temporary target, respectively weighting the temporary coordinate information, the first temporary scale information and the second temporary scale information of all the candidate targets in the target group, outputting weighting results thereof to the camera, and showing the weighting results by the camera; and

when the camera does not detect the target group and track the temporary target, not outputting information to the camera.

8. The target tracking method for the air vehicle, as recited in claim 4, wherein the first coordinate system is same as the second coordinate system.

9. The target tracking method for the air vehicle, as recited in claim 5, wherein methods for recording the temporary image characteristics of the temporary target comprise a gradient direction histogram, a local binary pattern histogram, scale invariant feature transformation, and speeded up robust features.