Systems and methods for UAV path planning and control

• US 10,860,040 B2
• Filed: 04/25/2018
• Issued: 12/08/2020
• Est. Priority Date: 10/30/2015
• Status: Active Grant

• Current Assignee: SZ DJI Technology Co., Ltd. (d/b/a DJI) (iFlight Technology Company Limited)
• Original Assignee: SZ DJI Technology Co., Ltd. (d/b/a DJI) (iFlight Technology Company Limited)
• Inventors: Hu, Xiao, Liu, Ang, Zhang, Litian, Tang, Ketan
• Primary Examiner(s): Elchanti, Hussein
• Application Number:
  US15/962,817
• Publication Number:
  US 20180246529A1
• Time in Patent Office: 958 Days
• CPC Class Codes: B64C 39/024 of the remote controlled vehicle type, i.e. RPV
• CPC Class Codes: B64U 10/13 Flying platforms
• CPC Class Codes: B64U 20/87 Mounting of imaging devices, e.g. mounting of gimbals
• CPC Class Codes: B64U 2101/30 for imaging, photography or videography
• CPC Class Codes: B64U 2101/31 for surveillance
• CPC Class Codes: B64U 2101/60 for transporting passengers; for transporting goods other than weapons
• CPC Class Codes: B64U 2101/64 for parcel delivery or retrieval
• CPC Class Codes: B64U 2201/10 autonomous, i.e. by navigating independently from ground or air stations, e.g. by using inertial navigation systems [INS]
• CPC Class Codes: B64U 2201/20 Remote controls
• CPC Class Codes: B64U 30/20 Rotors; Rotor supports
• CPC Class Codes: G05D 1/0016 characterised by the operator's input device input arrangements for computing systems in general G06F3/00
• CPC Class Codes: G05D 1/0038 by providing the operator with simple or augmented images from one or more cameras located onboard the vehicle, e.g. tele-operation images analyzed by a computer and used for automatic navigation G05D1/0246
• CPC Class Codes: G05D 1/0202 specially adapted to aircraft
• CPC Class Codes: G05D 1/12 Target-seeking control

Abstract

A method for determining a target direction for a movable object includes providing an image on a computer-implemented display, obtaining a position of a selected point on the image in response to a user selecting the point on the image, and determining the target direction based on the position of the selected point on the image.

Claims

1. A method for determining a target direction for a movable object, comprising:

providing an image on a computer-implemented display;

converting a set of screen coordinates of a selected point into a set of image coordinates in an image coordinate system to obtain a position of the selected point on the image in response to a user selecting the point on the computer-implemented display;

determining a vector of the target direction in the image coordinate system based on the position of the selected point on the image;

converting the vector of the target direction in the image coordinate system to a vector of the target direction in a camera coordinate system of an imaging device carried by the movable object and capturing the image;

collecting data from one or more sensors of the movable object to obtain a translation matrix;

converting, through the translation matrix, the vector of the target direction in the camera coordinate system into a vector of the target direction in a world coordinate system in which the movable object moves; and

controlling the movable object based on the vector of the target direction in the world coordinate system.

2. The method of claim 1, further comprising:

effecting movement of the movable object, based on the vector of the target direction in the world coordinate system, from a reference point.

3. The method of claim 2, wherein effecting the movement of the movable object includes instructing the movable object to move, based on the vector of the target direction in the world coordinate system, in a substantially linear manner.

4. The method of claim 2, further comprising:

generating a motion path for the movable object from an initial position to the reference point, based on one or more motion characteristics of the movable object at the initial position and the reference point.

5. The method of claim 4, further comprising:

effecting the movement of the movable object along the motion path between the initial position and the reference point.

6. The method of claim 4, wherein the motion path is represented by a function of the initial position, the reference point, and the one or more motion characteristics of the movable object at the initial position and the reference point.

7. The method of claim 1, wherein the computer-implemented interface is a touch-screen device.

8. The method of claim 1, further comprising:

projecting the vector of the target direction as a graphical element onto the image in the computer-implemented display.

9. An apparatus for determining a target direction for a movable object, the apparatus comprising one or more processors that are, individually or collectively, configured to:

provide an image on a computer-implemented display;

convert a set of screen coordinates of a selected point into a set of image coordinates in an image coordinate system to obtain a position of the selected point on the image in response to a user selecting the point on the computer-implemented display;

determine a vector of the target direction in the image coordinate system based on the position of the selected point on the image;

convert the vector of the target direction in the image coordinate system to a vector of the target direction in a camera coordinate system of an imaging device carried by the movable object and capturing the image;

collect data from one or more sensors of the movable object to obtain a translation matrix;

convert, through the translation matrix, the vector of the target direction in the camera coordinate system into a vector of the target direction in a world coordinate system in which the movable object moves; and

control the movable object based on the vector of the target direction in the world coordinate system.

10. The apparatus of claim 9, wherein the one or more processors are further, individually or collectively, configured to:

effect movement of the movable object, based on the vector of the target direction in the world coordinate system, from a reference point.

11. The apparatus of claim 10, wherein the one or more processors are further, individually or collectively, configured to:

instruct the movable object to move, based on the vector of the target direction in the world coordinate system, in a substantially linear manner.

12. The apparatus of claim 10, wherein the one or more processors are further, individually or collectively, configured to:

generate a motion path for the movable object from an initial position to the reference point, based on one or more motion characteristics of the movable object at the initial position and the reference point.

13. The apparatus of claim 12, wherein the one or more processors are further, individually or collectively, configured to:

effect the movement of the movable object along the motion path between the initial position and the reference point.

14. The apparatus of claim 13, wherein the one or more processors are further, individually or collectively, configured to:

instruct the movable object to move along the motion path in a substantially curvilinear manner.

15. The apparatus of claim 12, wherein the motion path is represented by a function of the initial position, the reference point, and the one or more motion characteristics of the movable object at the initial position and the reference point.

16. The apparatus of claim 9, wherein the computer-implemented interface is a touch-screen device.

17. The apparatus of claim 9, wherein the one or more processors are further, individually or collectively, configured to:

normalizing the set of image coordinates.

18. The apparatus of claim 17, wherein the vector of the target direction is determined based on the normalized set of image coordinates.

19. The apparatus of claim 9, wherein the one or more processors are further, individually or collectively, configured to:

project the vector of the target direction as a graphical element onto the image in the computer-implemented display.

20. An unmanned aerial vehicle (UAV) system comprising:

an apparatus operable to control the UAV, the apparatus comprising one or more processors that are, individually or collectively, configured to;

provide an image on a computer-implemented display;

convert a set of screen coordinates of a selected point into a set of image coordinates in an image coordinate system to obtain a position of the selected point on the image in response to a user selecting the point on the computer-implemented display;

determine a vector of the target direction in the image coordinate system based on the position of the selected point on the image;

convert the vector of the target direction in the image coordinate system to a vector of the target direction in a camera coordinate system of an imaging device carried by the UAV and capturing the image;

collect data from one or more sensors of the UAV to obtain a translation matrix;

convert, through the translation matrix, the vector of the target direction in the camera coordinate system into a vector of the target direction in a world coordinate system in which the UAV moves; and

control the UAV based on the vector of the target direction in the world coordinate system.