Systems and methods for target tracking

• US 9,846,429 B2
• Filed: 06/14/2016
• Issued: 12/19/2017
• Est. Priority Date: 07/30/2014
• Status: Active Grant

• Current Assignee: SZ DJI Technology Co., Ltd. (d/b/a DJI) (iFlight Technology Company Limited)
• Original Assignee: SZ DJI Technology Co., Ltd. (d/b/a DJI) (iFlight Technology Company Limited)
• Inventors: Zang, Bo
• Primary Examiner(s): Black, Thomas G
• Assistant Examiner(s): Nolan, Peter D
• Application Number:
  US15/182,553
• Publication Number:
  US 20170068246A1
• Time in Patent Office: 553 Days
• CPC Class Codes: B64U 10/13 Flying platforms
• CPC Class Codes: B64U 2101/31 for surveillance
• CPC Class Codes: B64U 2201/10 autonomous, i.e. by navigating independently from ground or air stations, e.g. by using inertial navigation systems [INS]
• CPC Class Codes: B64U 2201/20 Remote controls
• CPC Class Codes: B64U 30/20 Rotors; Rotor supports
• CPC Class Codes: G05D 1/0011 associated with a remote control arrangement
• CPC Class Codes: G05D 1/0038 by providing the operator with simple or augmented images from one or more cameras located onboard the vehicle, e.g. tele-operation images analyzed by a computer and used for automatic navigation G05D1/0246
• CPC Class Codes: G05D 1/0088 characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours using knowledge based models G06N5/00
• CPC Class Codes: G05D 1/0094 involving pointing a payload, e.g. camera, weapon, sensor, towards a fixed or moving target
• CPC Class Codes: G05D 1/12 Target-seeking control
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G08G 5/0069 specially adapted for an unmanned aircraft

Abstract

The present invention provides systems, methods, and devices related to target tracking by UAVs. The UAV may be configured to receive target information from a control terminal related to a target to be tracked by an imaging device coupled to the UAV. The target information may be used by the UAV to automatically track the target so as to maintain predetermined position and/or size of the target within one or more images captured by the imaging device. The control terminal may be configured to display images from the imaging device as well as allowing user input related to the target information.

Claims

1. A method of tracking a target using an unmanned aerial vehicle (UAV) having an on-board imaging device, the method comprising:

a) receiving at one or more receivers, target information of the target to be tracked by the UAV, wherein the imaging device is configured to rotate around at least one axis relative to the UAV;

with aid of one or more processors,

b) tracking the target according to the target information while the UAV is in motion, wherein said tracking the target involves;

i. assessing whether to adjust

     1) the UAV,

     2) the imaging device, or

     3) both the UAV and the imaging device to track the target, wherein said assessing is based on a) maximum linear speed or maximum angular speed allowable for the UAV, or b) maximum angular speed allowable for the imaging device, wherein the maximum angular speed of the UAV or the maximum angular speed of the imaging device is configurable by a user of the UAV; and

ii. adjusting, based on said assessing of (i),

     1) the UAV,

     2) the imaging device, or

     3) both the UAV and the imaging device as the target is moving, such that the target stays within an image captured by the imaging device.

2. The method of claim 1, wherein the one or more receivers receive one or more navigation commands that instruct the UAV to move along its pitch, yaw, or roll axis.

3. The method of claim 1, wherein the one or more receivers receive one or more navigation commands provided by a user of the UAV while the UAV is in motion, wherein the one or more navigation commands control spatial disposition, and speed at which the UAV is flying.

4. The method of claim 1, wherein the adjusting of (ii) is performed in real time as the UAV is executing one or more navigation commands received by the one or more receivers on board of the UAV.

5. The method of claim 1, wherein said assessing is further dependent upon a number of rotational axes of the imaging device, and a path of the UAV.

6. The method of claim 1, wherein said assessing is further dependent upon a number of rotational axes of the imaging device.

7. The method of claim 1, wherein said assessing is further dependent upon a path of the UAV.

8. The method of claim 1, wherein the target is an active target configured to wirelessly transmit the target information to the UAV.

9. The method of claim 1, wherein the target is a passive target not configured to transmit the target information wirelessly to the UAV.

10. The method of claim 1, wherein said step of adjusting comprises a change in an orientation of the UAV.

11. The method of claim 1, wherein said step of adjusting comprises a change in an orientation of the imaging device.

12. The method of claim 1, wherein the target information comprises an initial position of the target.

13. The method of claim 1, wherein the target information comprises an initial size of the target within an image captured by the imaging device.

14. The method of claim 1, wherein the target information comprises the target'"'"'s GPS location.

15. The method of claim 1, wherein the target information comprises a color of the target or a texture of the target.

16. The method of claim 1, wherein the one or more receivers receive one or more navigation commands provided by a user of the UAV via a remote control device, and wherein the one or more navigation commands is provided during the tracking of the target.

17. The method of claim 1, wherein the target information is received from a remote control device.

18. The method of claim 1, wherein the one or more receivers receive one or more navigation commands provided by a user of the UAV and the target information provided by the user via a single remote control device, wherein the one or more navigation commands is provided during the tracking of the target.

19. The method of claim 17, wherein the remote control device comprises a touchscreen.

20. The method of claim 1, wherein tracking the target involves detecting a deviation in the target information, and correcting the detected deviation by performing adjustment of (ii), wherein the detected deviation comprises a change in position of the target.

21. The method of claim 1, wherein the maximum angular speed of the imaging device is configurable by the user of the UAV.

22. An unmanned aerial vehicle (UAV) configured to track a target, the UAV comprising:

a) one or more receivers configured to receive, (1) one or more navigation commands to move the UAV, and (2) target information of the target to be tracked by the UAV;

b) an imaging device configured to capture an image of the target, wherein the imaging device is configured to rotate around at least two axes relative to the UAV;

c) one or more processors configured to;

detect a deviation in the target information, wherein the deviation comprises a change in the position of the target; and

track the target according to the target information while the UAV moves according to the one or more navigation commands, wherein said tracking the target involves;

i. assessing whether to adjust

     1) the UAV,

     2) the imaging device, or

     3) both the UAV and the imaging device to track the target, wherein said assessing is based on a) maximum linear speed or maximum angular speed allowable for the UAV, or b) maximum angular speed allowable for the imaging device, wherein the maximum angular speed of the UAV or the maximum angular speed of the imaging device is configurable by a user of the UAV; and

ii. adjusting, based on said assessing of (i),

     1) the UAV,

     2) the imaging device, or

     3) both the UAV and the imaging device as the target moves, such that the target stays within an image captured by the imaging device.

23. The method of claim 22, wherein the maximum angular speed of the imaging device is configurable by the user of the UAV.

24. The UAV of claim 22, wherein the UAV further comprises one or more proximity, sensors, wherein information acquired by the one or more proximity sensors are further utilized by the one or more processors for the adjustment.

25. The UAV of claim 22, wherein the target information comprises the target'"'"'s GPS location.

26. The UAV of claim 22, wherein the target information comprises a color of the target or a texture of the target.

27. The UAV of claim 22, wherein the one or more receivers receive one or more navigation commands provided by a user of the UAV via a remote control device, and wherein the one or more navigation commands is provided during the tracking of the target.

28. The UAV of claim 22, wherein the one or more receivers receive one or more navigation commands provided by a user of the UAV and the target information provided by the user via a single remote control device, wherein the one or more navigation commands is provided during the tracking of the target.

29. The UAV of claim 22, wherein said assessing of (i) is further dependent upon a number of rotational axes of the imaging device, and a path of the UAV.

30. The UAV of claim 22, wherein the one or more processors are configured to track the target according to the deviation in the target information.