Systems and methods for improved telepresence

• US 6,958,746 B1
• Filed: 04/04/2000
• Issued: 10/25/2005
• Est. Priority Date: 04/05/1999
• Status: Expired due to Fees

• Current Assignee: Battelle Energy Alliance LLC (Battelle Memorial Institute)
• Original Assignee: Bechtel BWXT Idaho, LLC
• Inventors: Willis, W. David, Kinoshita, Robert A., Anderson, Matthew O.
• Primary Examiner(s): Shalwala, Bipin
• Assistant Examiner(s): PIZIALI, JEFFREY J
• Application Number:
  US09/542,973
• Time in Patent Office: 2,030 Days
• Field of Search: 345/156, 345/158, 345161-163, 345/168, 345/172, 345/184, 345700-702, 345/719, 345/740, 345/864, 345/865, 715700-702, 715/719, 715/740, 715/864, 715/865, 725/37, 710/8, 710 62- 65, 710/73, 348/42, 348/47, 348/48, 348/50, 348/77, 348/78, 348/82, 348/83, 348/113, 348/114, 348/143, 348/159
• US Class Current: 345/161
• CPC Class Codes: H04N 13/296 Synchronisation thereof; Control thereof
• CPC Class Codes: H04N 23/66 Remote control of cameras or camera parts, e.g. by remote control devices

Abstract

The present invention provides a modular, flexible system for deploying multiple video perception technologies. The telepresence system of the present invention is capable of allowing an operator to control multiple mono and stereo video inputs in a hands-free manner. The raw data generated by the input devices is processed into a common zone structure that corresponds to the commands of the user, and the commands represented by the zone structure are transmitted to the appropriate device. This modularized approach permits input devices to be easily interfaced with various telepresence devices. Additionally, new input devices and telepresence devices are easily added to the system and are frequently interchangeable. The present invention also provides a modular configuration component that allows an operator to define a plurality of views each of which defines the telepresence devices to be controlled by a particular input device. The present invention provides a modular flexible system for providing telepresence for a wide range of applications. The modularization of the software components combined with the generalized zone concept allows the systems and methods of the present invention to be easily expanded to encompass new devices and new uses.

Claims

1. A telepresence system for allowing an operator to interact with a remote operating environment, the system comprising:

one or more input devices, the one or more input devices configured to produce raw data representative of operator commands;

an input conversion module configured to convert the raw data into a zone structure representative of the operator commands in a telepresence-device independent format, the zone structure being a format independent of any one of the one or more input devices;

one or more device modules corresponding to one or more telepresence devices, the one or more device modules configured to convert the zone structure into telepresence device commands specific to an associated one of the one or more telepresence devices, the zone structure being a format independent of any of the one or more telepresence devices, the telepresence device commands resulting from at least a portion of the operator commands; and

a configuration module configured to associate a specific one of the one or more input devices corresponding to the zone structure with a specific one of the one or more telepresence devices which responds to the telepresence device commands resulting from the zone structure.

2. A system as defined in claim 1, wherein the one or more input devices comprise one or more of:

a headset, a keyboard, a mouse, and a joystick.

3. A system as defined in claim 1, wherein only one of the one or more input devices is permitted to produce raw data at a time.

4. A system as defined in claim 1, wherein one of the one or more input devices is capable of controlling a plurality of the one or more telepresence devices.

5. The system as defined in claim 1, further comprising a communication link, wherein the telepresence device commands are received by the one or more telepresence devices over a communication link such that the one or more input devices are configured to control the one or more telepresence devices, wherein the telepresence devices provide the operator with one or more visual representations of the operating environment.

6. A system as defined in claim 5, wherein the communication link is a wireless communication link.

7. A system as defined in claim 1, wherein the one or more telepresence devices comprise one or more of a stereo camera set, a zoom camera, a pan and tilt device, a slider bar, and a robot.

8. A system as defined in claim 7, wherein the pan and tilt device is connected to the stereo camera set and is capable of orienting the stereo camera set.

9. A system as defined in claim 7, wherein the pan and tilt device is connected to the zoom camera and is capable of orienting the zoom camera.

10. In a system having input devices and telepresence devices, a method for controlling one or more associated telepresence devices with a selected input device, the method comprising the steps of:

receiving raw data representative of movement commands from the selected input device;

converting the raw data into a zone structure, representative of the movement commands in a telepresence-device independent format, the zone structure being a format independent of any one of the input devices and any one of the one or more associated telepresence devices;

when the selected input device is selectively associated with the one or more associated telepresence devices, processing the zone structure with a device module corresponding to each of the one or more associated telepresence devices to obtain telepresence device commands corresponding to at least a portion of the movement commands for each of the associated telepresence devices; and

transmitting the movement commands to the associated telepresence devices.

11. A method as defined in claim 10, wherein the selected input device is one of a headset, a keyboard, a mouse, or a joystick.

12. A method as defined in claim 10, wherein the zone structure is compatible with the telepresence devices.

13. A method as defined in claim 10, wherein the zone structure is capable of representing a plurality of speeds and directions.

14. A method as defined in claim 10, wherein the associated telepresence devices only respond to portions of the zone structure that correspond to the axes of the associated telepresence devices.

15. A method as defined in claim 10, wherein the raw data corresponds to actions of an operator.

16. A method as defined in claim 10, further comprising the step of executing the movement commands by the associated telepresence devices.

17. A computer readable medium having computer-executable instructions for performing the steps recited in claim 10.

18. A telepresence system for allowing an operator to interact with a remote operating environment, the telepresence system comprising:

a plurality of input devices;

a plurality of telepresence devices, wherein one or more of the telepresence devices is configured to be controlled by one of the plurality of input devices and one or more of the telepresence devices is configured to provide a visual representation of the operating environment;

a computer comprising;

an input conversion module configured to receive raw data representative of operator commands from at least one of the plurality of input devices and further configured to convert the raw data to a zone structure representative of the operator commands in a telepresence-device independent format, the zone structure being a format independent of any of the plurality of input devices and any of the plurality of telepresence devices; and

a plurality of device modules corresponding to the plurality of telepresence devices configured to receive the zone structure and convert the zone structure to movement commands corresponding to the operator commands for each respective telepresence device; and

a communication link for transmitting the movement commands to the telepresence devices.

19. A system as defined in claim 18, wherein the telepresence devices comprises one or more stereo camera sets each connected with a different pan and tilt device and a zoom camera connected with another pan and tilt device.

20. A system as defined in claim 19, wherein the zoom camera is capable of providing stereo vision.

21. A system as defined in claim 18, wherein the raw data generated by the input devices correspond to zones, each zone representative of movement in a particular direction and speed.

22. A system as defined in claim 21, wherein the zone structure integrates any of the input devices with one or more of the telepresence devices.

23. A system as defined in claim 18, wherein the computer further comprises a configuration module.

24. A system as defined in claim 23, wherein the configuration module comprises one or more views, wherein each view defines the one or more telepresence devices controlled by a single input device.

25. A system as defined in claim 24, wherein the operator may select a different view.

26. A system as defined in claim 23, wherein the one or more views stored in the configuration module permits a single input device to control different groups of telepresence devices.

27. A system as defined in claim 18, wherein the plurality of telepresence devices provide the operator with a visual representation of the operating environment.

28. A system as defined in claim 27, wherein the visual representation provides depth perception to the operator.

29. A system as defined in claim 18, wherein the communications link is wireless communication.

30. A system as defined in claim 18, wherein the plurality of input devices allow the operator to control the telepresence devices without the use of the operator'"'"'s hands.