Remote vehicle control system

• US 7,731,588 B2
• Filed: 09/28/2006
• Issued: 06/08/2010
• Est. Priority Date: 09/28/2005
• Status: Active Grant

• Current Assignee: United States Department Of Health And Human Services
• Original Assignee: the united states of america as represented by the secretary of the navy
• Inventors: Templeman, James N
• Primary Examiner(s): Jones; Scott
• Assistant Examiner(s): Shah; Milap
• Application Number:
  US11/528,646
• Publication Number:
  US 20070072662A1
• Time in Patent Office: 1,349 Days
• Field of Search: 463/7, 463 36- 39, 318/6, 318/9, 318/11, 318/14, 318/16, 446/454, 446/460, 446/468, 345/8
• US Class Current: 463/38
• CPC Class Codes: A63F 13/00 Video games, i.e. games using an electronically generated display having two or more dimensions gaming systems which provide a financial reward G07F17/32
• CPC Class Codes: A63F 13/24 Constructional details thereof, e.g. game controllers with detachable joystick handles
• CPC Class Codes: A63F 13/42 by mapping the input signals into game commands, e.g. mapping the displacement of a stylus on a touch screen to the steering angle of a virtual vehicle
• CPC Class Codes: A63F 2300/1006 having additional degrees of freedom
• CPC Class Codes: A63F 2300/105 using inertial sensors, e.g. accelerometers, gyroscopes
• CPC Class Codes: A63F 2300/30 characterized by output arrangements for receiving control signals generated by the game device
• CPC Class Codes: A63F 2300/6045 for mapping control signals received from the input arrangement into game commands
• CPC Class Codes: A63F 2300/8076 Shooting
• CPC Class Codes: A63H 30/04 using wireless transmission
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/038 Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object

Abstract

An apparatus for remotely controlling the movements of a vehicle includes a user input means, such as a gamepad with a plurality of joystick-controlled and button-controlled outputs, a tracker, and optional sliding foot pedals; a processor for running a control mapping algorithm; and a remote vehicle controller. A control mapping algorithm maps the outputs to the remote-controlled vehicle'"'"'s course, heading, displacement, and camera view, with the joysticks mapped to provide open loop directional control over the vehicle'"'"'s course and heading, the tracker providing open loop control over the camera view, and the optional sliding foot pedals providing open loop control over the vehicle'"'"'s displacement. The remote vehicle controller sends commands to on-board controls to direct the vehicle'"'"'s movement. A video stream from an on-board camera is transmitted back to the operator station for viewing on a computer desktop display or a head mountable display.

Claims

1. An apparatus for controlling the movements of a vehicle having on-board controls for independently controlling the course and heading of the vehicle in response to a remote operator'"'"'s control and an on-board camera/video transmitter for recording and transmitting a video stream of the camera'"'"'s view of the physical environment within which the vehicle is operating, comprising:

a gamepad comprising a first joystick and a second joystick and having a plurality of outputs responsive to user-controlled movements of the first and the second joystick and one or more user-controlled buttons, wherein the first joystick is mapped to provide an open loop directional control over steering of the vehicle, the second joystick is mapped to provide an open loop directional control over offset of the vehicle to thereby provide an open-loop control over the vehicle course and heading;

a tracker configured to control a head-pose of the remote vehicle as an open-loop control by providing yaw, pitch, and roll orientation angles as outputs;

a processor comprising a control mapping algorithm and a remote vehicle controller, wherein the control mapping algorithm is configured to map the gamepad and the tracker outputs to control the course, heading, displacement, and head-pose of the remote-controlled vehicle, and the remote vehicle controller sends commands to the on-board controls to direct the movement of the remote vehicle; and

a display for receiving the video stream and displaying the movement of the vehicle to the operator.

2. An apparatus as in claim 1, wherein the remote control vehicle includes a turret, the remote vehicle controller includes control over the head-pose of the turret, and the processor further comprises control mapping the head-pose of the turret of the remote vehicle as a directional open-loop heading control.

3. An apparatus as in claim 1, wherein the second joystick is mapped to provide a course-offset.

4. An apparatus as in claim 1, wherein the second joystick is mapped to provide a heading-offset.

5. An apparatus as in claim 1, wherein the tracker is attached to the gamepad.

6. An apparatus as in claim 5, wherein the vehicle moves in accordance to a computed situated-heading and the head-pose.

7. An apparatus as in claim 1, wherein the display is a head mountable display and the tracker is attached thereto.

8. An apparatus as in claim 7, wherein the vehicle moves in accordance to a computed situated-heading and the head-pose.

9. An apparatus as in claim 1, wherein the display is a head-mountable display and further comprising a video rendering system for providing a mono or stereo video image to the head-mountable display.

10. An apparatus as in claim 1, wherein the display is a desk-top monitor display and further comprising a video rendering system for providing a mono or stereo video image to the desk-top monitor display.

11. An apparatus as in claim 1, further comprising a pair of sliding foot pedals having an output mapped to provide open-loop translational control of a displacement of the vehicle.

12. An apparatus as in claim 1, wherein the movements of the vehicle are displayed as three dimensional.

13. A control mapping system for mapping a plurality of a user-controlled dual joystick gamepad and tracker outputs to a movement of a vehicle in a physical environment, comprising:

a control mapping algorithm for mapping the user-controlled movements of a first joystick, a second joystick, and a tracker to a vehicle course, heading, displacement, and head-pose, wherein the first joystick is mapped to provide an open loop directional control over steering of the vehicle, the second joystick is mapped to provide an open loop directional control over offset of the vehicle to thereby provide an open-loop control over the vehicle'"'"'s course and heading, and the tracker is mapped to control a head-pose of the vehicle as an open-loop control by providing yaw, pitch, and roll orientation angles.

14. A control mapping system as in claim 13, further comprising a processor for running a virtual environment software program displaying the vehicle in a physical environment in which the vehicle is programmed to move with a course, heading, displacement, and head-pose.

15. A control mapping system as in claim 13, wherein the second joystick is mapped to provide a course-offset.

16. A control mapping system as in claim 13, wherein the second joystick is mapped to provide a heading-offset.

17. A control mapping system as in claim 13, wherein the vehicle moves in accordance to a computed situated-heading and the head-pose.

18. A method for controlling the movements of a vehicle using a tracker and a gamepad comprising a first joystick a second joystick, and having a plurality of outputs responsive to user-controlled movements of each joystick, the tracker, and user-controlled buttons, comprising:

recording and transmitting a video stream of the vehicle'"'"'s camera view of the physical environment within which the vehicle is operating to a processor;

programming the processor with a control mapping algorithm for mapping the user-controlled movements of each joystick and tracker to a vehicle course, heading, displacement, and head-pose, wherein the first joystick is mapped to provide a steering control of the vehicle, the second joystick is mapped to provide an offset control of the vehicle to thereby provide an open-loop control over the vehicle, and the movement of the tracker is mapped to control a head-pose of the vehicle as an open-loop control by providing yaw, pitch, and roll orientation angles;

transmitting a vehicle control signal to on-board vehicle controls; and

displaying the movement of the vehicle in the physical environment.

19. A method as in claim 18, wherein the remote control vehicle includes a turret, the remote vehicle controller includes control over the head-pose of the turret, and the processor further comprises control mapping the head-pose of the turret of the remote vehicle as a directional open-loop heading control.

20. A method as in claim 18, wherein the second joystick is mapped to provide a course-offset.

21. A method as in claim 18, wherein the second joystick is mapped to provide a heading-offset.

22. A method as in claim 18, wherein the tracker is attached to the gamepad.

23. A method as in claim 22, wherein the vehicle moves in accordance to a computed situated-heading.

24. A method as in claim 18, wherein the tracker is attached to a head-mountable display.

25. A method as in claim 24, wherein the vehicle moves in accordance to a computed situated-heading.

26. A method as in claim 18, further comprising providing a mono or stereo video image to a head-mountable display.

27. A method as in claim 18, further comprising providing a mono or stereo video image to a desk-top monitor display.

28. A method as in claim 18, further comprising control mapping an output of a pair of sliding foot pedals to provide open-loop control of a translational displacement of the vehicle.

29. A method as in claim 18, wherein the movements of the vehicle are displayed as three dimensional.

30. An apparatus for controlling the movements of a vehicle having on-board controls for independently controlling the course and heading of the vehicle in response to a remote operator'"'"'s control and an on-board camera/video transmitter for recording and transmitting a video stream of the camera'"'"'s view of the physical environment within which the vehicle is operating, comprising:

a first and a second dual axis directional input control device having a plurality of outputs responsive to user-controlled operations of the directional control axes and user-controlled buttons;

a third directional input control device that is mapped to direct a head-pose of the vehicle as an open-loop control by providing yaw, pitch, and roll orientation angles as outputs;

a processor for running a remote vehicle control software program which sends commands to the vehicle'"'"'s on-board controls to direct the movement of the remote vehicle;

a control mapping algorithm for mapping the outputs of the directional input control devices to the vehicle'"'"'s course and heading, wherein the directional control axes are mapped to provide open loop directional control of the vehicle'"'"'s course and heading; and

a display for receiving the video stream and displaying the view from the vehicle'"'"'s cameras to the operator.

31. An apparatus as in claim 30, wherein the vehicle includes a turret, the remote vehicle controller includes control over the head-pose of the turret, and the processor further comprises control mapping the head-pose of the turret of the remote vehicle as a directional open-loop heading control.

32. An apparatus as in claim 30, wherein a first directional input control device is mapped to provide open loop directional control over steering the vehicle and a second directional input control device is mapped to provide open loop directional control over a heading offset.

33. An apparatus as in claim 30, wherein a first directional input control device is mapped to provide open loop directional control over steering the vehicle and a second directional input control device is mapped to provide open loop directional control over a course offset.

34. An apparatus as in claim 30, wherein a first directional input control device is mapped to provide open loop directional control over the vehicle'"'"'s heading and a second directional input control device is mapped to provide open loop directional control over the vehicle'"'"'s course.

35. An apparatus as in claim 30, wherein the third directional input control device is a tracker attached to the gamepad to direct the head-pose of the vehicle as an open-loop control.

36. An apparatus as in claim 30, wherein the display is a head mountable display and comprises a tracker attached thereto to direct the head-pose of the vehicle as an open-loop control.