Telematic interface with directional translation

• US 9,002,517 B2
• Filed: 09/24/2014
• Issued: 04/07/2015
• Est. Priority Date: 01/19/2011
• Status: Active Grant

• Current Assignee: Harris Corporation (L3Harris Technologies, Inc.)
• Original Assignee: Harris Corporation (L3Harris Technologies, Inc.)
• Inventors: Bosscher, Paul M., Summer, Matthew D., Wilkinson, Loran J., Bowman, William S.
• Primary Examiner(s): Jen, Ian
• Application Number:
  US14/494,635
• Publication Number:
  US 20150057803A1
• Time in Patent Office: 195 Days
• Field of Search: 700/260, 700/245, 700/248
• US Class Current: 700/260
• CPC Class Codes: B25J 13/025 comprising haptic means
• CPC Class Codes: B25J 19/023 including video camera means
• CPC Class Codes: B25J 5/005 mounted on endless tracks or belts
• CPC Class Codes: B25J 9/1689 Teleoperation
• CPC Class Codes: G05B 2219/39065 Calculate workspace for end effector, manipulator
• CPC Class Codes: G05B 2219/40184 Compliant teleoperation, operator controls motion, system controls contact, force
• CPC Class Codes: G05B 2219/40401 Convert workspace of master to workspace of slave
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883

Abstract

Method and system for telematic control of a slave device. Displacement of a user interface control is sensed with respect to a control direction. A first directional translation is performed to convert data specifying the control direction to data specifying a slave direction. The slave direction will generally be different from the control direction and defines a direction that the slave device should move in response to the physical displacement of the user interface. A second directional translation is performed to convert data specifying haptic sensor data to a haptic feedback direction. The haptic feedback direction will generally be different from the sensed direction and can define a direction of force to be generated by at least one component of the user interface. The first and second directional translation are determined based on a point-of-view of an imaging sensor.

Claims

1. A method for providing haptic feedback to a user at a user interface device when the user interface device is operated to control a slave device, comprising:

using at least one sensor disposed on the slave device to sense a force applied to the slave device in a sensed direction defined with respect to a first three-dimensional set of coordinate axes associated with the slave device, and using said at least one sensor to generate a signal representing the force;

using a computer processor device to perform a directional translation to convert data specifying said sensed direction to data specifying a haptic feedback direction defined with respect to a second three-dimensional set of coordinate axes associated with a user interface control, said haptic feedback direction different from said sensed direction and defining a direction of force to be generated by at least one component of said user interface;

wherein said directional translation comprises performing a mathematical operation to determine a force F_hand to be exerted by the user interface control upon a user'"'"'s hand based on a force F_robot exerted on a the slave device and a relative orientation of a first imaging sensor with respect to the chassis of said slave device.

2. The method according to claim 1, wherein said sensed direction of said force comprises one or more of a linear sensed direction and a rotational sensed direction with respect to said first three-dimensional set of coordinate axes.

3. The method according to claim 2, wherein said haptic feedback direction comprises one or more of a linear haptic direction and a rotational haptic direction with respect to said second three-dimensional set of coordinate axes.

4. The method according to claim 3, wherein said directional translation comprises one or more of a linear translation and a rotational translation.

5. The method according to claim 1, further comprising using said computer processor device to automatically dynamically modify said directional translation that is performed when a point-of-view determined by the relative orientation of the first imaging sensor is modified.

6. The method according to claim 5, further comprising determining said point-of-view based on a sensed position of one or more movable joints comprising said slave unit.

7. The method according to claim 5, further comprising using the computer processor device to automatically dynamically modify said directional translation in response to a user selection of an image obtained from a second imaging sensor, where said second imaging sensor has a different point-of-view as compared to said first imaging sensor.

8. The method according to claim 1, further comprising communicating data representing at least one of said sensed direction and said haptic feedback direction to said user interface to generate a force using at least one component of said user interface.

9. A robot system, comprising:

at least one sensor disposed on a slave device which senses a force applied to the slave device in a sensed direction defined with respect to a first three-dimensional set of coordinate axes associated with the slave device;

the slave device responsive to control signals generated by a user interface control to control at least one movement of the slave device when the user interface control is manipulated by a user;

a communication link for communicating motion control information from said user control interface to said slave device;

a control processor device which receives information concerning the sensed direction and performs a directional translation to convert data specifying said sensed direction to data specifying a haptic feedback direction defined with respect to a second three-dimensional set of coordinate axes associated with a user interface control, said haptic feedback direction different from said sensed direction and defining a direction of force to be generated by at least one component of said user interface control;

wherein said directional translation comprises performing a mathematical operation to determines a force F_hand to be exerted by the user interface control upon a user'"'"'s hand based on a force F_robot exerted on the slave device and a relative orientation of a first imaging sensor with respect to the chassis of said slave device.

10. The robot system according to claim 9, wherein said sensed direction comprises one or more of a linear sensed direction and a rotational sensed direction with respect to said first three-dimensional set of coordinate axes.

11. The robot system according to claim 10, wherein said haptic feedback direction comprises one or more of a linear haptic direction and a rotational haptic direction with respect to said second three-dimensional set of coordinate axes.

12. The robot system according to claim 11, wherein said directional translation comprises one or more of a linear translation and a rotational translation.

13. The robot system according to claim 9, wherein said control processor device automatically dynamically modifies said directional translation that is performed when a point-of-view of said first imaging sensor is modified by changing its relative orientation.

14. The robot system according to claim 13, wherein said control processor device determines said point-of-view based on a sensed position of one or more movable joints comprising said slave unit.

15. The robot system according to claim 9, wherein a first point-of-view of said first imaging sensor is determined by at least one of a location and an orientation of the first imaging sensor, and wherein said control processor device automatically dynamically modifies said directional translation in response to a user input selection of an image obtained from a second imaging sensor in place of said first imaging sensor, where said second imaging sensor has a second point-of-view different as compared to said first imaging sensor.