Systems and methods for haptics-enabled teleoperation of vehicles and other devices

• US 7,885,732 B2
• Filed: 06/15/2007
• Issued: 02/08/2011
• Est. Priority Date: 10/25/2006
• Status: Active Grant

• Current Assignee: The Boeing Co.
• Original Assignee: The Boeing Co.
• Inventors: Troy, James J., Erignac, Charles A., Murray, Paul
• Primary Examiner(s): Camby; Richard M.
• Application Number:
  US11/764,119
• Publication Number:
  US 20080103639A1
• Time in Patent Office: 1,334 Days
• Field of Search: 701 23- 28, 701/36, 701/2, 446454-468, 345156-163
• US Class Current: 701/2
• CPC Class Codes: G05D 1/0027 involving a plurality of vehicles, e.g. fleet or convoy travelling fleet control of land vehicles from a control room G05D1/0297; traffic control systems for road vehicles G08G1/00; for marine craft G08G3/00; for aircraft G08G5/00
• CPC Class Codes: G05D 1/0033 by having the operator tracking the vehicle either by direct line of sight or via one or more cameras located remotely from the vehicle
• CPC Class Codes: G05D 1/005 by providing the operator with signals other than visual, e.g. acoustic, haptic

Abstract

Systems and methods are disclosed for haptics-enabled teleoperation of vehicles and other devices, including remotely-controlled air, water, and land-based vehicles, manufacturing robots, and other suitable teleoperable devices. In one embodiment, a system for teleoperation of a vehicle comprises a control component configured to provide position and orientation control with haptic force feedback of the vehicle based on a position measurement of the vehicle and configured to function in a closed-loop feedback manner. In a particular embodiment, the position measurement may include six degree-of-freedom position data provided by a motion capture system to the control and/or haptic I/O components of the application. The system may also use differences in position and/or velocity between the vehicle and a haptic I/O device for feedback control.

Claims

1. A system for teleoperation of a vehicle, comprising:

a control component configured to provide position and orientation control to the vehicle and to receive a haptic force feedback in response to a signal from the vehicle, the haptic force feedback based in part on inertia forces of the vehicle, the haptic force feedback configured to function in a closed-loop feedback manner.

2. The system of claim 1, wherein the control component receives six degree-of-freedom haptic force feedback provided by a motion capture system via a haptic input/output (I/O) component, the six degree-of-freedom haptic force feedback including a rotational force.

3. The system of claim 1, wherein differences in at least one of position and velocity between the vehicle and a haptic I/O device are used for the closed-loop feedback.

4. The system of claim 1, wherein differences in at least one of position and velocity between the vehicle and a simulated proxy object provide one or more control requests to the vehicle, and wherein differences in at least one of position and velocity of the simulated proxy object and a haptic I/O device are used for the closed-loop feedback.

5. The system of claim 1, wherein the control component is further configured to simultaneously control a plurality of vehicles.

6. The system of claim 1, wherein the vehicle is capable of at least one of capturing, manipulating, lifting, depositing, and spraying external objects.

7. The system of claim 1, wherein additional force feedback is generated by one or more constraints of an envelope protection system built into the control component, and wherein the one or more constraints include at least one of collision avoidance, attitude limitations, rate limitations, and preventing departure in uncontrolled flight.

8. The system of claim 1, wherein the control component is further configured to enable a plurality of operators to use a plurality of haptic I/O devices to control of a plurality of unmanned aerial vehicles.

9. The system of claim 1, wherein the control component is further configured to provide the position and orientation control using mapping between a position/orientation input device and the vehicle, and wherein the mapping is scaled at least one of linearly and non-linearly.

10. The system of claim 1, wherein the control component is further configured to provide the position and orientation control using at least one rate controlled control input.

11. The system of claim 1, wherein the vehicle is at least one of a holonomic vehicle, a near-holonomic vehicle, an aerial vehicle, an underwater vehicle, and a zero-gravity environment vehicle.

12. A method of operating a teleoperable device by directly controlling least one of position and orientation of the teleoperable device, comprising:

receiving an input to a haptics device linked to the teleoperable device;

providing at least one control signal to the teleoperable device based on the input, the at least one control signal being configured to directly control at least one of position and orientation of the teleoperable device;

measuring an inertial force of the teleoperable device;

receiving a haptics signal from the teleoperable device based on the intertail force; and

providing a haptics output to an operator based on the haptics signal.

13. The method of claim 12, wherein the receiving the input to the haptics device includes receiving a six degree-of-freedom physical input to the haptics device.

14. The method of claim 12, wherein the measuring the inertial force of the teleoperable device includes measuring at least one motion characteristic of the teleoperable device using a motion capture system.

15. The method of claim 14, wherein the haptics output is further based on at least one of a headwind experienced by the teleoperable device, a turbulence experienced by the teleoperable device, a battery level of the teleoperable device, and an increased mass of the teleoperable device due to a lifting of an object by the teleoperable device.

16. The method of claim 14, further comprising:

determining at least one additional control signal based on the at least one motion characteristic; and

providing the at least one additional control signal to the teleoperable device to control the at least one motion characteristic of the teleoperable device in a closed-loop feedback manner.

17. The method of claim 12, further comprising simulating at least one characteristic of an operating environment of the teleoperable device, and wherein providing the haptics output to an operator includes providing the haptics output to the operator based at least partially on the at least one simulated characteristic of the operating environment.

18. The method of claim 17, wherein the simulating the at least one characteristic of the operating environment includes simulating at least one of a solid obstacle, a moveable object, an airflow condition, a movement boundary condition, and a proximity field disposed about the teleoperable device.

19. The method of claim 12, wherein the providing the haptics output to the operator includes providing a rotational force to a handle felt by the operator.

20. A system for operating a teleoperable device by controlling least one of a position and orientation of the teleoperable device, comprising:

a haptics device coupled to the teleoperable device using at least one virtual coupling and configured to receive an operator input, the haptic device configured to provide a haptics output to an operator based on the inertial force of the teleoperable device, wherein the haptics device is further configured to provide at least one control signal to the teleoperable device to directly control at least one of position and orientation of the teleoperable device based on the operator input; and

a sensing system operatively coupled to the haptics device and configured to measure an inertial force of the teleoperable device.

21. The system of claim 20, wherein the haptics device is coupled to the teleoperable device by a bilateral force feedback arrangement to the teleoperable device.

22. The system of claim 20, wherein the sensing system includes a motion capture system configured to measure the inertial force of the teleoperable device.

23. The system of claim 20, wherein the haptics device is further configured to measure an acceleration of the teleoperable device.

24. The system of claim 20, wherein the haptics device is further configured to provide at least one control signal to the teleoperable device based on the inertial device measured by the sensing system in a closed loop feedback manner.

25. The system of claim 20, wherein the teleoperable device comprises at least one of an aerial vehicle, a manufacturing robot, a water-based vehicle, a land-based vehicle, and a zero-gravity vehicle.