Systems and Methods for Haptics-Enabled Teleoperation of Vehicles and Other Devices

• US 20080103639A1
• Filed: 06/15/2007
• Published: 05/01/2008
• Est. Priority Date: 10/25/2006
• Status: Active Grant

• Current Assignee: The Boeing Co.
• Original Assignee: The Boeing Co.
• Inventors: Troy, James J., Erignac, Charles A., Murray, Paul
• Granted Patent: US 7,885,732 B2
• Time in Patent Office: Days
• US Class Current: 701/2
• CPC Class Codes: G05D 1/0027 involving a plurality of vehicles, e.g. fleet or convoy travelling fleet control of land vehicles from a control room G05D1/0297; traffic control systems for road vehicles G08G1/00; for marine craft G08G3/00; for aircraft G08G5/00
• CPC Class Codes: G05D 1/0033 by having the operator tracking the vehicle either by direct line of sight or via one or more cameras located remotely from the vehicle
• CPC Class Codes: G05D 1/005 by providing the operator with signals other than visual, e.g. acoustic, haptic

Abstract

Systems and methods are disclosed for haptics-enabled teleoperation of vehicles and other devices, including remotely-controlled air, water, and land-based vehicles, manufacturing robots, and other suitable teleoperable devices. In one embodiment, a system for teleoperation of a vehicle comprises a control component configured to provide position and orientation control with haptic force feedback of the vehicle based on a position measurement of the vehicle and configured to function in a closed-loop feedback manner. In a particular embodiment, the position measurement may include six degree-of-freedom position data provided by a motion capture system to the control and/or haptic I/O components of the application. The system may also use differences in position and/or velocity between the vehicle and a haptic I/O device for feedback control.

Claims

1. A system for teleoperation of a vehicle, comprising:

a control component configured to provide position and orientation control with haptic force feedback of the vehicle based on a position measurement of the vehicle and configured to function in a closed-loop feedback manner.

2. The system of claim 1 wherein the position measurement includes six degree-of-freedom position data provided by a motion capture system to at least one of the control component and a haptic input/output (I/O) component of the application.

3. The system of claim 1 wherein differences in at least one of position and velocity between the vehicle and a haptic I/O device are used for the closed-loop feedback control.

4. The system of claim 1 wherein differences in at least one of position and velocity between the vehicle and a simulated proxy object provide one or more control requests to the vehicle, and wherein differences in at least one of position and velocity of the simulated proxy object and a haptic I/O device are used for feedback control.

5. The system of claim 1 where the control component includes a haptic input/output (I/O) device configured to simultaneously control a plurality of vehicles.

6. The system of claim 1 wherein the vehicle is capable of at least one of capturing, manipulating, lifting, depositing, and spraying external objects.

7. The system of claim 1 wherein additional force feedback is generated by one or more constraints of an envelope protection system built into the control component, and wherein the one or more constraints include at least one of collision avoidance, attitude limitations, rate limitations, and preventing departure in uncontrolled flight.

8. The system of claim 1 wherein the control component is configured to enable a plurality of operators to use a plurality of haptic I/O devices to control of a plurality of unmanned aerial vehicles.

9. The system of claim 1 wherein the control component is configured to provide the position and orientation control using mapping between a position/orientation input device and the vehicle, and wherein the mapping is scaled at least one of linearly and non-linearly.

10. The system of claim 1 wherein the control component is configured to provide the position and orientation control using at least one rate controlled control input.

11. The system of claim 1 wherein the vehicle is at least one of a holonomic vehicle, a near-holonomic vehicle, an aerial vehicle, an underwater vehicle, and a zero-gravity environment vehicle.

12. A method of operating a teleoperable device by directly controlling least one of position and orientation of the teleoperable device, comprising:

providing an input to a haptics device virtually coupled to the teleoperable device;

providing at least one control signal to the teleoperable device based on the input, the at least one control signal being configured to directly control at least one of position and orientation of the teleoperable device;

measuring at least one state characteristic of the teleoperable device; and

providing a haptics output to an operator based on at least one of a response from the teleoperable device and the at least one state characteristic.

13. The method of claim 12, wherein providing a physical input to a haptics device includes providing a physical input to a haptics device virtually coupled by a bilateral force feedback arrangement to the teleoperable device.

14. The method of claim 12, wherein measuring at least one state characteristic of the teleoperable device includes measuring at least one motion characteristic of the teleoperable device using a motion capture system.

15. The method of claim 14, wherein providing at least one control signal to the teleoperable device includes providing at least one control signal to the teleoperable device based on at least one of a response from the teleoperable device and the at least one motion characteristic.

16. The method of claim 14, further comprising:

determining at least one additional control signal based on the at least one motion characteristic; and

providing the at least one additional control signal to the teleoperable device to control at least one motion characteristic of the teleoperable device in a closed-loop feedback manner.

17. The method of claim 12, further comprising simulating at least one characteristic of an operating environment of the teleoperable device, and wherein providing a haptics output to an operator includes providing a haptics output to an operator based at least partially on the at least one simulated characteristic of the operating environment.

18. The method of claim 17 wherein simulating at least one characteristic of an operating environment includes simulating at least one of a solid obstacle, a moveable object, an airflow condition, a movement boundary condition, and a proximity field disposed about the teleoperable device.

19. The method of claim 12 wherein providing at least one control signal to the teleoperable device includes simultaneously providing at least one control signal to a plurality of teleoperable devices to controllably move the plurality of teleoperable devices in a coordinated manner.

20. A system for operating a teleoperable device by controlling least one of a position and orientation of the teleoperable device, comprising:

a haptics device coupleable to the teleoperable device and configured to receive an operator input; and

a sensing system operatively coupled to the haptics device and configured to measure at least one state characteristic of the teleoperable device, wherein the haptics device is further configured to provide at least one control signal to the teleoperable device, the at least one control signal being configured to directly control at least one of position and orientation of the teleoperable device based on at least one of the operator input and the at least one state characteristic.

21. The system of claim 20, wherein the haptics device is further configured to be virtually coupleable by a bilateral force feedback arrangement to the teleoperable device.

22. The system of claim 20, wherein the sensing system includes a motion capture system configured to measure at least one motion characteristic of the teleoperable device.

23. The system of claim 20, wherein the sensing system is further configured to measure six degree of freedom motion characteristics of the teleoperable device.

24. The system of claim 20, wherein the haptics device is further configured to provide at least one control signal to the teleoperable device based on the at least one state characteristic measured by the sensing system in a closed loop feedback manner.

25. The system of claim 20, wherein the teleoperable device comprises at least one of an aerial vehicle, a manufacturing robot, a water-based vehicle, a land-based vehicle, and a zero-gravity vehicle.