Systems and methods for target tracking

• US 9,567,078 B2
• Filed: 09/04/2015
• Issued: 02/14/2017
• Est. Priority Date: 07/30/2014
• Status: Active Grant

• Current Assignee: SZ DJI Technology Co., Ltd. (d/b/a DJI) (iFlight Technology Company Limited)
• Original Assignee: SZ DJI Technology Co., Ltd. (d/b/a DJI) (iFlight Technology Company Limited)
• Inventors: Zang, Bo
• Primary Examiner(s): Black, Thomas G
• Assistant Examiner(s): Nolan, Peter D
• Application Number:
  US14/845,894
• Publication Number:
  US 20160031559A1
• Time in Patent Office: 529 Days
• US Class Current: 1/1
• CPC Class Codes: B64U 10/13 Flying platforms
• CPC Class Codes: B64U 2101/30 for imaging, photography or videography
• CPC Class Codes: B64U 2101/31 for surveillance
• CPC Class Codes: B64U 2201/10 autonomous, i.e. by navigating independently from ground or air stations, e.g. by using inertial navigation systems [INS]
• CPC Class Codes: B64U 2201/20 Remote controls
• CPC Class Codes: B64U 30/20 Rotors; Rotor supports
• CPC Class Codes: G05D 1/0011 associated with a remote control arrangement
• CPC Class Codes: G05D 1/0038 by providing the operator with simple or augmented images from one or more cameras located onboard the vehicle, e.g. tele-operation images analyzed by a computer and used for automatic navigation G05D1/0246
• CPC Class Codes: G05D 1/0088 characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours using knowledge based models G06N5/00
• CPC Class Codes: G05D 1/0094 involving pointing a payload, e.g. camera, weapon, sensor, towards a fixed or moving target
• CPC Class Codes: G05D 1/12 Target-seeking control
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G08G 5/0069 specially adapted for an unmanned aircraft

Abstract

The present invention provides systems, methods, and devices related to target tracking by UAVs. The UAV may be configured to receive target information from a control terminal related to a target to be tracked by an imaging device coupled to the UAV. The target information may be used by the UAV to automatically track the target so as to maintain predetermined position and/or size of the target within one or more images captured by the imaging device. The control terminal may be configured to display images from the imaging device as well as allowing user input related to the target information.

Claims

1. A system for controlling an unmanned aerial vehicle (UAV) comprising:

one or more receivers configured to receive from a remote user (1) one or more navigation commands to move the UAV, and (2) target information of a target to be tracked by an imaging device on the UAV; and

one or more processors configured to dynamically adjust allocation of control over a tracking process between the user and the UAV based on factors including a surrounding of the UAV,

such that the one or more processors to allocate (1) more control over the tracking process to the user when the UAV is navigating in a more complex environment compared to when the UAV is navigating in a less complex environment, and (2) less control over the tracking process to the user when the UAV is navigating in a less complex environment compared to when the UAV is navigating in a more complex environment, and wherein an environmental complexity is based on a number of potential obstructions in the environment.

2. The system of claim 1, wherein the target is tracked by automatically adjusting at least one of the UAV or the imaging device while the UAV moves according to the one or more navigation commands from the user.

3. The system of claim 2, wherein the one or more navigation commands are adapted to control a speed, position, or attitude of the UAV.

4. The system of claim 1, wherein the tracking process utilizes a selection between (1) an automatic tracking mode in which the user can specify a type of target to be tracked, and (2) a manual tracking mode in which the user can specify a specific target to be tracked.

5. The system of claim 1, wherein the type of target to be tracked is specified based on at least one of the following target characteristics:

a color, texture, pattern, size, shape, or dimension of the target.

6. The system of claim 1, further comprising a display configured to display the target with a tracking indicator indicating that the target is selected.

7. The system of claim 6, wherein the tracking indicator is a graphical object on the display.

8. A system for controlling an unmanned aerial vehicle (UAV) comprising:

one or more receivers configured to receive from a remote user (1) one or more navigation commands to move the UAV, and (2) target information of a target to be tracked by an imaging device on the UAV; and

one or more processors configured to dynamically adjust allocation of control over a tracking process between the user and the UAV based on factors including a surrounding of the UAV,

such that the one or more processors allocate (1) more control over the tracking process to the user when the UAV is navigating at a lower altitude compared to when the UAV is navigating at a higher altitude, and (2) less control over the tracking process to the user when the UAV is navigating at a higher altitude compared to when the UAV is navigating in a lower altitude.

9. The system of claim 8, wherein the target is tracked by automatically adjusting at least one of the UAV or the imaging device while the UAV moves according to the one or more navigation commands from the user.

10. The system of claim 9, wherein the one or more navigation commands are adapted to control a speed, position, or attitude of the UAV.

11. The system of claim 8, wherein the tracking process utilizes a selection between (1) an automatic tracking mode in which the user can specify a type of target to be tracked, and (2) a manual tracking mode in which the user can specify a specific target to be tracked.

12. The system of claim 8, wherein the type of target to be tracked is specified based on at least one of the following target characteristics:

a color, texture, pattern, size, shape, or dimension of the target.

13. The system of claim 8, further comprising a display configured to display the target with a tracking indicator indicating that the target is selected.

14. The system of claim 13, wherein the tracking indicator is a graphical object on the display.

15. A system for controlling an unmanned aerial vehicle (UAV) comprising:

one or more receivers configured to receive from a remote user (1) one or more navigation commands to move the UAV, and (2) target information of a target to be tracked by an imaging device on the UAV; and

one or more processors configured to automatically select between (1) an automatic tracking mode, and (2) a manual tracking mode, wherein in the automatic tracking mode, less control is dynamically allocated to the user over a tracking process compared to the manual tracking mode,

wherein the one or more processors are configured to select between the manual tracking mode and the automatic tracking mode based on an environmental complexity, wherein the environmental complexity is based on a number of potential obstructions in an environment in which the UAV navigates, and wherein when the manual tracking mode is selected, the environmental complexity is greater than when the automatic tracking mode is selected.

16. The system of claim 15, wherein the target is tracked by automatically adjusting at least one of the UAV or the imaging device while the UAV moves according to the one or more navigation commands from the user.

17. The system of claim 16, wherein the one or more navigation commands are adapted to control a speed, position, or attitude of the UAV.

18. The system of claim 15, wherein the tracking process utilizes a selection between (1) the automatic tracking mode in which the user can specify a type of target to be tracked, and (2) the manual tracking mode in which the user can specify a specific target to be tracked.

19. The system of claim 15, wherein the type of target to be tracked is specified based on at least one of the following target characteristics:

a color, texture, pattern, size, shape, or dimension of the target.

20. The system of claim 15, further comprising a display configured to display the target with a tracking indicator indicating that the target is selected.

21. The system of claim 20, wherein the tracking indicator is a graphical object on the display.

22. A system for controlling an unmanned aerial vehicle (UAV) comprising:

one or more receivers configured to receive from a remote user (1) one or more navigation commands to move the UAV, and (2) target information of a target to be tracked by an imaging device on the UAV; and

one or more processors configured to automatically select between (1) an automatic tracking mode, and (2) a manual tracking mode, wherein in the automatic tracking mode, less control is dynamically allocated to the user over a tracking process compared to the manual tracking mode, wherein the one or more processors are configured to select between the manual tracking mode and the automatic tracking mode based on an altitude at which the UAV navigates, and wherein when the manual tracking mode is selected, the altitude is lower than when the automatic tracking mode is selected.

23. The system of claim 22, wherein the target is tracked by automatically adjusting at least one of the UAV or the imaging device while the UAV moves according to the one or more navigation commands from the user.

24. The system of claim 23, wherein the one or more navigation commands are adapted to control a speed, position, or attitude of the UAV.

25. The system of claim 22, wherein the tracking process utilizes a selection between (1) the automatic tracking mode in which the user can specify a type of target to be tracked, and (2) the manual tracking mode in which the user can specify a specific target to be tracked.

26. The system of claim 22, wherein the type of target to be tracked is specified based on at least one of the following target characteristics:

a color, texture, pattern, size, shape, or dimension of the target.

27. The system of claim 22, further comprising a display configured to display the target with a tracking indicator indicating that the target is selected.

28. The system of claim 27, wherein the tracking indicator is a graphical object on the display.