Unmanned aerial vehicle, control systems for unmanned aerial vehicle and control method thereof

• US 10,703,479 B2
• Filed: 03/14/2018
• Issued: 07/07/2020
• Est. Priority Date: 11/30/2017
• Status: Active Grant

• Current Assignee: Industrial Technology Research Institute
• Original Assignee: Industrial Technology Research Institute
• Inventors: Chen, Guo-Ruei, Tsai, Luo-Wei, Chou, Yeh-Kai
• Primary Examiner(s): Sarwar, Babar
• Application Number:
  US15/921,625
• Publication Number:
  US 20190161186A1
• Time in Patent Office: 846 Days
• CPC Class Codes: B64C 39/024 of the remote controlled vehicle type, i.e. RPV
• CPC Class Codes: B64U 20/87 Mounting of imaging devices, e.g. mounting of gimbals
• CPC Class Codes: B64U 2101/30 for imaging, photography or videography
• CPC Class Codes: B64U 2201/20 Remote controls
• CPC Class Codes: B64U 50/13 using external fans or propellers
• CPC Class Codes: B64U 50/19 using electrically powered motors
• CPC Class Codes: G05D 1/0202 specially adapted to aircraft
• CPC Class Codes: G05D 1/101 specially adapted for aircraft
• CPC Class Codes: G06F 18/23213 with fixed number of clusters, e.g. K-means clustering
• CPC Class Codes: G06V 10/25 Determination of region of interest [ROI] or a volume of interest [VOI]
• CPC Class Codes: G06V 10/462 Salient features, e.g. scale invariant feature transforms [SIFT]
• CPC Class Codes: G06V 20/13 Satellite images
• CPC Class Codes: G06V 20/17 taken from planes or by drones
• CPC Class Codes: H04N 23/60 Control of cameras or camera modules
• CPC Class Codes: H04N 23/69 Control of means for changing angle of the field of view, e.g. optical zoom objectives or electronic zooming

Abstract

A UAV (Unmanned Aerial Vehicle) control method comprises: obtaining a first image; obtaining a second image after obtaining the first image; calculating a plurality of feature points respectively according to the first image and the second image by a computing device, extracting a first feature point set and a second feature point set by the computing device; calculating a first-axial displacement and a second-axial displacement according to the first feature point set and the second feature point set by the computing device; calculating a first ROI (Region Of Interest) area, a second ROI and a third-axial displacement according to the first feature point set and the second feature point set by the computing device; adjusting a camera pose or a flight path of the UAV according to the first-axial displacement, the second-axial displacement and the third-axial displacement.

Claims

1. An Unmanned Aerial Vehicle (UAV) control method adapted to a UAV with a camera for shooting a target comprising:

obtaining a first image at a first timing;

obtaining a second image by the camera at a second timing after the first timing;

calculating a plurality of feature points respectively according to the first image and the second image by a computing device;

extracting a first feature point set and a second feature point set by the computing device, wherein the first feature point set comprises at least three feature points from the plurality of the feature points of the first image, and the second feature point set comprises at least three feature points from the plurality of the feature points of the second image;

calculating a first-axial displacement and a second-axial displacement according to the first feature point set and the second feature point set by the computing device;

calculating a first region of interest (ROI) area and a second ROI area respectively according to the first feature point set and the second feature point set by the computing device and calculating a third-axial displacement according to the first ROI area and the second ROI area; and

adjusting a camera pose or a flight path of the UAV according to the first-axial displacement, the second-axial displacement and the third-axial displacement by a control device.

2. The UAV control method according to claim 1, further comprising before the plurality of feature points is calculated by the computing device:

calculating a total difference amount according to the first image and the second image by the computing device; and

obtaining the second image again when the total difference amount is smaller than a threshold value.

3. The UAV control method according to claim 1, wherein the computing device adopts a SIFT algorithm to calculate the plurality of feature points.

4. The UAV control method according to claim 1, wherein the computing device adopts PCA to extract the first feature point set and the second feature point set.

5. The UAV control method according to claim 1, further comprising before the first-axial displacement and the second-axial displacement are calculated:

using K-means clustering algorithm to partition the plurality of feature points for calculating the first-axial displacement and the second-axial displacement by the computing device.

6. The UAV control method according to claim 1, further comprising before the first image is obtained by the camera:

setting the flight path of the UAV by the control device.

7. An Unmanned Aerial Vehicle (UAV) control system comprising:

a camera configured to shoot a first image and a second image during a flight of a UAV, wherein the first image is shot at a first timing, the second image is shot at a second timing, and the first timing is earlier than the second timing;

a computing device electrically connected to the camera, wherein the computing device is configured to calculate a displacement vector of the UAV during the first timing and the second timing based on the first image and the second image; and

a control device electrically connected to the computing device, wherein the computing device is configured to selectively adjust a gimbal for adjusting a camera pose or a flight path of the UAV according to the displacement vector,

wherein the displacement vector comprises a first-axial displacement, a second-axial displacement, and a third-axial displacement, and the computing device further comprises;

a displacement computing device configured to calculate a plurality of feature points respectively according to the first image and the second image, to extract a first feature point set and a second feature point set, wherein the first feature point set comprises at least three feature points from the plurality of feature points of the first image and the second feature point set comprises at least three feature points from the plurality of feature points of the second image, and to calculate the first-axial displacement and the second-axial displacement according to the first feature point set and the second feature point set; and

a depth computing device coupled to the displacement computing device, wherein the depth computing device is configured to calculate a first region of interest £

ROI) area and a second ROI area according to the first feature point set and the second feature point set and to calculate a third-axial displacement according to the first ROI area and the second ROI area.

8. The UAV control system according to claim 7, wherein the control device further comprises a timer to accumulate an interval time during the first timing and the second timing.

9. The UAV control system according to claim 7, wherein the control device further comprises a positioning device configured to get a world coordinate of the UAV, and the control device adjusts the flight path of the UAV according to the world coordinate and the displacement vector.

10. An Unmanned Aerial Vehicle (UAV) comprising:

a flight system comprising a propeller configured to provide an upward buoyancy for the UAV by rotation;

a motor coupled to the propeller with a spindle for providing a rotation power; and

a motor drive electrically connected to the motor and configured to drive the motor according to a flight control instruction;

a control system comprising;

a camera configured to shoot a first image and a second image during a flight of the UAV, wherein the first image is shot at a first timing, the second image is shot at a second timing, and the first timing is earlier than the second timing;

a computing device electrically connected to the camera, wherein the computing device is configured to calculate a displacement vector of the UAV during the first timing and the second timing based on the first image and the second image; and

a control device electrically connected to the computing device, wherein the control device is configured to selectively send a gimbal adjustment instruction for adjusting a camera pose or to send the flight control instruction for adjusting a flight path of the UAV according to the displacement vector;

a gimbal electrically connected to the camera and the control device, wherein the gimbal comprises a servomotor and the gimbal is configured to receive the gimbal adjustment instruction to drive the servomotor for adjusting the camera pose; and

a frame configured to carry the flight system, the control system and the gimbal;

wherein the displacement vector comprises a first-axial displacement, a second-axial displacement, and a third-axial displacement, and the computing device further comprises;

a displacement computing device configured to calculate a plurality of feature points respectively according to the first image and the second image, to extract a first feature point set and a second feature point set, wherein the first feature point set comprises at least three feature points from the plurality of feature points of the first image and the second feature point set comprises at least three feature points from the plurality of feature points of the second image, and to calculate the first-axial displacement and the second-axial displacement according to the first feature point set and the second feature point set; and

a depth computing device coupled to the displacement computing device, wherein the depth computing device is configured to calculate a first region of interest (ROI) area and a second ROI area according to the first feature point set and the second feature point set and to calculate a third-axial displacement according to the first ROI area and the second ROI area.