Systems and methods for gimbal simulation

• US 9,858,023 B2
• Filed: 09/15/2016
• Issued: 01/02/2018
• Est. Priority Date: 07/10/2015
• Status: Active Grant

• Current Assignee: SZ DJI Technology Co., Ltd. (d/b/a DJI) (iFlight Technology Company Limited)
• Original Assignee: SZ DJI Technology Co., Ltd. (d/b/a DJI) (iFlight Technology Company Limited)
• Inventors: Tao, Ye, Chen, Zihan, Zhang, Zhiyuan, Liu, Weifeng, Chen, Chaobin
• Primary Examiner(s): Melton, Todd
• Assistant Examiner(s): ROBERSON, JASON R
• Application Number:
  US15/266,877
• Publication Number:
  US 20170010611A1
• Time in Patent Office: 474 Days
• Field of Search: None
• CPC Class Codes: B64C 39/024 of the remote controlled vehicle type, i.e. RPV
• CPC Class Codes: B64U 10/13 Flying platforms
• CPC Class Codes: B64U 10/14 with four distinct rotor axes, e.g. quadcopters
• CPC Class Codes: B64U 20/87 Mounting of imaging devices, e.g. mounting of gimbals
• CPC Class Codes: B64U 2101/30 for imaging, photography or videography
• CPC Class Codes: B64U 2101/60 for transporting passengers; for transporting goods other than weapons
• CPC Class Codes: G05D 1/0094 involving pointing a payload, e.g. camera, weapon, sensor, towards a fixed or moving target
• CPC Class Codes: G06F 3/14 Digital output to display device ; Cooperation and interconnection of the display device with other functional units
• CPC Class Codes: G06F 3/147 using display panels
• CPC Class Codes: G09B 19/165 Control of aircraft
• CPC Class Codes: G09B 9/08 for teaching control of aircraft, e.g. Link trainer
• CPC Class Codes: G09B 9/12 Motion systems for aircraft simulators
• CPC Class Codes: G09B 9/48 a model being viewed and manoeuvred from a remote point
• CPC Class Codes: G09G 2354/00 Aspects of interface with display user
• CPC Class Codes: G09G 2380/12 Avionics applications
• CPC Class Codes: H04L 67/125 involving control of end-device applications over a network

Abstract

Systems, devices and methods are provided for training a user to control a gimbal in an environment. The systems and methods provide a simulation environment to control a gimbal in a virtual environment. The virtual environment closely resembles a real control environment. A controller may be used to transmit simulation commands and receive simulated data for visual display.

Claims

1. A gimbal simulation system comprising:

a gimbal that is physically coupled to a vehicle, wherein the gimbal is configured to carry a payload;

a gimbal control system on-board the vehicle configured to (1) receive gimbal control data from a remote control system, wherein the gimbal control data is used to control the gimbal based on a first input from the remote control system, (2) receive position data describing a simulated attitude of the vehicle generated from a vehicle control system on-board the vehicle in response to a second input from the remote control system; and

(3) generate simulated gimbal response data based on (i) the gimbal control data and (ii) the position data describing the simulated attitude of the vehicle, wherein the simulated gimbal response data indicates a simulated attitude of the gimbal that determines an orientation of the payload; and

a transmitter configured to transmit the simulated gimbal response data to the remote control system.

2. The gimbal simulation system of claim 1 wherein the vehicle is an unmanned aerial vehicle (UAV).

3. The gimbal simulation system of claim 1 wherein the simulated gimbal response data is determined according to a gimbal mode, wherein the gimbal mode includes one of a first person view mode, a following mode, or a free gimbal mode.

4. The gimbal simulation system of claim 1 wherein the simulated gimbal response data stabilizes a pitch axis of the gimbal with respect to an environment without stabilizing a yaw axis and a roll axis, when the gimbal is operated in a first person view mode.

5. The gimbal simulation system of claim 1 wherein the simulated gimbal response data stabilizes a pitch axis and a roll axis of the gimbal with respect to an environment without stabilizing a yaw axis, when the gimbal is operated in a following mode.

6. The gimbal simulation system of claim 1 wherein the simulated gimbal response data stabilizes a pitch axis, a yaw axis, and a roll axis of the gimbal with respect to an environment, when the gimbal is operated in a free gimbal mode.

7. The gimbal simulation system of claim 1 wherein the simulated gimbal response data is determined based on a gimbal mode signal generated at the remote control system remote to the vehicle.

8. The gimbal simulation system of claim 7 wherein the gimbal mode signal is generated in response to a user input indicating a selection of a gimbal mode from the plurality of gimbal modes.

9. The gimbal simulation system of claim 1 wherein the remote control system is configured to transmit the simulated gimbal response data to a display device comprising a visual display.

10. The gimbal simulation system of claim 9 wherein the remote control system communicates with the display device via a wireless connection.

11. The gimbal simulation system of claim 9 wherein the remote control system is used to operate the vehicle and the gimbal on-board the vehicle in a real flight operation.

12. The gimbal simulation system of claim 9 wherein the visual display shows simulated gimbal state information of the vehicle.

13. The gimbal simulation system of claim 12 wherein the simulated gimbal state information includes simulated visual data generated based on the simulated attitude of the gimbal.

14. The gimbal simulation system of claim 1 wherein the simulated gimbal response data describes an orientation, and wherein the gimbal simulation system further comprises a display device that receives the simulated gimbal response data and displays a visual illustration of the gimbal in the orientation described by the simulated gimbal response data.

15. The gimbal simulation system of claim 1 wherein the remote control system is configured to control directional heading of the gimbal.

16. The gimbal simulation system of claim 1 wherein the position data is at least partially based on simulated orientation of the vehicle.

17. The gimbal simulation system of claim 1 wherein the position data is at least partially based on simulated weather data.

18. The gimbal simulation system of claim 1 wherein the position data is at least partially based on position data describing the simulated attitude of the vehicle which includes at least one of (1) rotation of the vehicle about a pitch axis, (2) rotation of the vehicle about a yaw axis, or (3) rotation of the vehicle about a roll axis.

20. The gimbal simulation system of claim 1 wherein the gimbal adjusts arrangement based on the gimbal control data.

21. The gimbal simulation system of claim 1 wherein the gimbal control data indicates a real attitude of the gimbal.

22. The gimbal simulation system of claim 1 wherein the gimbal control data and position data of the vehicle is synchronized with each other in a time domain.

23. The gimbal simulation system of claim 1 wherein the payload is carried by the vehicle via the gimbal.

19. A method of simulating controlling a gimbal that is physically coupled to a vehicle, said method comprising:

receiving, (1) gimbal control data from a remote control system configured to communicate with a gimbal control system on-board the vehicle, wherein the gimbal control data is used to control the gimbal based on a first input at the remote control system, and (2) position data describing a simulated attitude of the vehicle generated from a vehicle control system on-board the vehicle in response to a second input from the remote control system;

generating, at the gimbal control system, simulated gimbal response data based on the gimbal control data and the position data describing the simulated attitude of the vehicle, wherein the simulated gimbal response data indicates a simulated attitude of the gimbal that determines an orientation of the payload; and

transmitting, to the remote control system, the simulated gimbal response data from the gimbal control system on-board the vehicle.