Unmanned aerial vehicle control using a gamepad

• US 8,676,406 B2
• Filed: 05/03/2011
• Issued: 03/18/2014
• Est. Priority Date: 05/03/2011
• Status: Active Grant

• Current Assignee: Raytheon Company (Rtx Corporation)
• Original Assignee: Raytheon Company (Rtx Corporation)
• Inventors: Coffman, Jason R., Solomon, Lee
• Primary Examiner(s): Tran, Khoi
• Assistant Examiner(s): MOTT, ADAM R
• Application Number:
  US13/099,955
• Publication Number:
  US 20120280087A1
• Time in Patent Office: 1,050 Days
• Field of Search: 701/2, 701/3, 701/11, 701/23, 701/24, 701/25, 701/26
• US Class Current: 701/3
• CPC Class Codes: G05D 1/0016 characterised by the operator's input device input arrangements for computing systems in general G06F3/00
• CPC Class Codes: G05D 1/0044 by providing the operator with a computer generated representation of the environment of the vehicle, e.g. virtual reality, maps maps used for automatic navigation G05D1/0274; flight directors G01C23/005

Abstract

In accordance with particular embodiments, a system includes a path creation module configured to create future flight paths for unmanned aerial vehicles (UAVs). Each future flight path comprises one or more branch points marking changes in the UAV'"'"'s flight path. The system also includes a display that is configured to present a graphical user interface that may include a 3D view of a geographic area; flight paths; and a menu of commands. The system also includes a gamepad that may include two thumb-sticks to adjust the displayed geographic area and the perspective of the displayed geographic area. The gamepad also includes time buttons configured to scroll through a flight time that begins with a current actual time and ends with the end of the UAV'"'"'s flight. The gamepad further includes buttons configured to select a command from the menu and to select a first future flight path of the one or more future flight paths to be sent to the UAV.

Claims

1. A system for controlling one or more unmanned aerial vehicle, comprising:

a path creation module configured to create one or more future flight paths for the one or more unmanned aerial vehicles (UAVs), each future flight path comprising one or more branch points marking a change in the UAV'"'"'s flight path;

a display configured to present a graphical user interface comprising;

a three-dimensional view of a geographic area;

one or more lines representing each of the one or more future flight paths for the one or more UAVs; and

a menu comprising one or more commands available for selection;

a gamepad comprising;

at least one first thumb-stick configured to adjust the geographic area displayed on the display;

at least one second thumb-stick configured to adjust a camera perspective of the displayed geographic area;

at least two time buttons configured to scroll through a flight time, the flight time beginning with a current actual time and ending with the end of the one or more UAV'"'"'s flight; and

a plurality of buttons configured to select at least one command from the menu, the selected command associated with a current flight time selected by the at least two time buttons, the plurality of buttons further configured to select a first future flight path of the one or more future flight paths to be sent to the UAV to control an actual flight path of the UAV,

wherein at least one of the first and second thumb-sticks is pressure sensitive and is configured to output a control signal to the path creation module configured to create the one or more future flight paths, wherein pressure indicated by the control signals vary the extent and degree of left and right turns of the at one or more flight paths.

2. The system of claim 1, wherein the one or more commands available for selection comprise one or more commands associated with UAV commands known by a UAV navigation system.

3. The system of claim 1, wherein the plurality of buttons are further configured to select a second future flight path of the one or more future flight paths, the selected second future flight path sent to the UAV to replace the first future flight path and to control the actual flight path of the UAV.

4. The system of claim 1, wherein the at least one second thumb-stick configured to adjust the camera perspective of the displayed geographic area is further configured to:

rotate the camera perspective such that the displayed geographic area rotates in response to a left or a right movement of the thumb-stick; and

adjust a point of view of the camera perspective such that the point of view is adjusted between an overhead point of view and an underneath point of view.

5. The system of claim 1, wherein at least one of the at least one first thumb-stick configured to adjust the geographic area displayed on the display, the at least one first thumb-stick configured to adjust the camera perspective of the displayed geographic area, the at least two zoom buttons, and the at least two time buttons are pressure sensitive.

6. The system of claim 1, wherein at least one of the at least one first thumb-stick configured to adjust the geographic area displayed on the display, the at least one second thumb-stick configured to adjust the camera perspective of the displayed geographic area, the at least two zoom buttons, and the at least two time buttons are analog.

7. The system of claim 1, wherein the gamepad further comprises a directional pad configured to switch between different UAVs of the one or more UAVs.

8. The system of claim 1, wherein the at least two time buttons configured to scroll through the flight time comprise:

a right shoulder button configured to advance flight time from the current flight time to the end of the UAV'"'"'s flight; and

a left shoulder button configured to rewind flight time from the current flight time to the current actual time.

9. The system of claim 1, wherein the gamepad further comprises at least two zoom buttons configured to adjust a zoom of the camera perspective.

10. A method for controlling an unmanned aerial vehicle, comprising:

generating an image of a geographic area for display on a display device;

generating a first future flight path for at least one unmanned aerial vehicle (UAV);

receiving a first input command from a gamepad;

advancing a flight time to a first time based on the first input command;

receiving a second input command from the gamepad;

modifying the first future flight path based on the second input command;

generating a second future flight path based on the modification of the first future flight path, the second future flight following the first future flight path for the first time and branching from the first future flight at the first time based on the second input command; and

outputting a control signal indicating a pressure strength to generate the first and second future flight paths, wherein pressure strength indicated by the control signals varies the extent and degree of left and right turns of the first and second future flight paths; and

receiving from the gamepad a third input command selecting a first selected flight path from among at least the first or the second future flight paths, the first selected flight path to be uploaded to control the actual flight path of the UAV.

11. The method of claim 10, further comprising receiving from the gamepad a fourth input command selecting a second selected flight path from among at least the first or the second future flight paths, the second selected flight path uploaded to replace the first selected flight path and to control the actual flight path of the UAV.

12. The method of claim 10, further comprising:

receiving from a first thumb-stick of the gamepad an image pan signal indicating a change in the image of the geographic area displayed on the display device; and

generating an updated image of a different geographic area for display on the display device based on the image pan signal.

13. The method of claim 12, wherein generating an updated image comprises panning from the image of the geographic area to the updated image of the different geographic area wherein a rate at which the image is panned is based on the distance the first thumb-stick is moved from its center position.

14. The method of claim 10, further comprising:

receiving from a second thumb-stick of the gamepad the camera perspective change signal indicating a change in the perspective of the image of the geographic area displayed on the display device; and

generating an updated image of the geographic area for display on the display device based on the camera perspective change signal, wherein generating the updated image comprises;

upon receiving a camera perspective change signal indicating that the second thumb-stick has moved to left or right, rotating the image of the geographic area in the first or second direction; and

upon receiving a camera perspective change signal indicating that the second thumb-stick has moved up or down, tilting the image of the geographic area in a first tilt direction or a second tilt direction.

15. The method of claim 10, wherein receiving a second input command from the gamepad comprises:

receiving a menu call command from the gamepad, the menu call command requesting that a menu be displayed on the display device;

generating the menu for display on the display device; and

receiving a menu selection from the gamepad, the menu selection requesting the second input command.

16. The method of claim 10, wherein receiving the second input command from the gamepad comprises receiving a change of direction input command specifying a change of direction for the UAV that modifies the first future flight path, the change of direction input command selected by the gamepad from a menu of available input commands understood by a UAV controller.

17. The method of claim 10, further comprising receiving from a directional pad of the gamepad a UAV select command, the UAV select command selecting a different UAV from a UAV currently selected before the UAV select command is received.

18. The method of claim 10, further comprising mapping a plurality of input components of the gamepad to a plurality of input commands.

19. The method of claim 10, further comprising, receiving the third input command from a click of a thumb-stick of the gamepad, the third input command resetting the flight time to the current actual time.

20. A system for controlling at least one unmanned aerial vehicle, comprising:

means for mapping a plurality of input commands to a plurality of input components of a gamepad;

means for generating an image of a geographic area for display on a display device;

means for generating a first future flight path for the at least one unmanned aerial vehicle (UAV);

means for receiving a first input command from a gamepad;

means for advancing the flight time to a first time based on the first input command;

means for receiving a second input command from the gamepad;

means for modifying the first future flight path based on the second input command;

means for generating a second future flight path based on the modification of the first future flight path, the second future flight following the first future flight path for the first time and branching from the first future flight at the first time based on the second input command;

a pressure sensing means configured to receive an physical input and to output a control signal to the at least one of first means for generating a first future flight path and the second means for generating a second future flight path, wherein pressure indicated by the control signal varies the extent and degree of left and right turns of at least one the first and second paths; and

means for receiving from the gamepad a third input command selecting a first selected flight path from among at least the first or the second future flight paths, the first selected flight path to be uploaded to control the actual flight path of the UAV.