TELEMATIC INTERFACE WITH DIRECTIONAL TRANSLATION

• US 20120185098A1
• Filed: 01/19/2011
• Published: 07/19/2012
• Est. Priority Date: 01/19/2011
• Status: Active Grant

• Current Assignee: Harris Corporation (L3Harris Technologies, Inc.)
• Original Assignee: Harris Corporation (L3Harris Technologies, Inc.)
• Inventors: Bosscher, Paul M., Summer, Matthew D., Wilkinson, Loran J., Bowman, William S.
• Granted Patent: US 8,918,214 B2
• Time in Patent Office: Days
• US Class Current: 700/260
• CPC Class Codes: B25J 13/025 comprising haptic means
• CPC Class Codes: B25J 19/023 including video camera means
• CPC Class Codes: B25J 5/005 mounted on endless tracks or belts
• CPC Class Codes: B25J 9/1689 Teleoperation
• CPC Class Codes: G05B 2219/39065 Calculate workspace for end effector, manipulator
• CPC Class Codes: G05B 2219/40184 Compliant teleoperation, operator controls motion, system controls contact, force
• CPC Class Codes: G05B 2219/40401 Convert workspace of master to workspace of slave
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883

Abstract

Method and system for telematic control of a slave device. Displacement of a user interface control is sensed with respect to a control direction. A first directional translation is performed to convert data specifying the control direction to data specifying a slave direction. The slave direction will generally be different from the control direction and defines a direction that the slave device should move in response to the physical displacement of the user interface. A second directional translation is performed to convert data specifying haptic sensor data to a haptic feedback direction. The haptic feedback direction will generally be different from the sensed direction and can define a direction of force to be generated by at least one component of the user interface. The first and second directional translation are determined based on a point-of-view of an imaging sensor.

Claims

1. A method for telematic control of a slave device, comprising:

sensing a physical displacement of a user interface control in any control direction defined with respect to a three-dimensional set of coordinate axes;

performing a first directional translation to convert data specifying said control direction to data specifying a slave direction defined with respect to said three-dimensional set of coordinate axes, said slave direction different from said control direction and defining a direction that said slave device should move in response to said physical displacement of said user interface;

wherein said first directional translation is determined based on a point-of-view of a first imaging sensor mounted to a portion of said slave device.

2. The method according to claim 1, wherein said control direction comprises one or more of a linear direction and a rotational direction with respect to said three-dimensional set of coordinate axes.

3. The method according to claim 2, wherein said slave direction comprises one or more of a linear direction and a rotational direction with respect to said three-dimensional set of coordinate axes.

4. The method according to claim 3, wherein said first directional translation comprises one or more of a linear translation and a rotational translation.

5. The method according to claim 1, further comprising automatically dynamically modifying said first directional translation that is performed when said point-of-view is modified.

6. The method according to claim 1, further comprising determining said point-of-view based upon at least one condition selected from the group consisting of a position and an orientation of said first imaging sensor.

7. The method according to claim 6, further comprising determining said point-of-view based on a sensed position of one or more movable joints comprising said slave unit.

8. The method according to claim 1, further comprising automatically dynamically modifying said first directional translation in response to a user selection of an image obtained from a second imaging sensor, where said second imaging sensor has a different point-of-view as compared to said first imaging sensor.

9. The method according to claim 1, further comprising communicating data representing at least one of said control direction and said slave direction to said slave device to cause a movement of said slave device.

10. The method according to claim 1, further comprising selecting said slave device to include a robotic arm having a plurality of joints for facilitating movement of a portion of said robotic arm in said slave direction responsive to said user interface control.

11. The method according to claim 10, further comprising selecting a mounting point for said first imaging sensor which moves with said robotic arm, such that said point-of-view of said first imaging sensor changes concurrently with movement of said robotic arm.

12. The method according to claim 1, further comprising generating with at least one sensor associated with said slave device a signal representing a force applied to said slave device in a sensed direction;

performing a second directional translation to convert data specifying said sensed direction to data specifying a haptic feedback direction defined with respect to said three-dimensional set of coordinate axes, said haptic feedback direction different from said sensed direction and defining a direction of force to be generated by at least one component of said user interface;

wherein said second directional translation is determined based on a point-of-view of said first imaging sensor.

13. The method according to claim 12, wherein said sensed direction of said force comprises one or more of a linear sensed direction and a rotational sensed direction with respect to said three-dimensional set of coordinate axes.

14. The method according to claim 13, wherein said haptic feedback direction comprises one or more of a linear haptic direction and a rotational haptic direction with respect to said three-dimensional set of coordinate axes.

15. The method according to claim 14, wherein said second directional translation comprises one or more of a linear translation and a rotational translation.

16. The method according to claim 12, further comprising automatically dynamically modifying said second directional translation that is performed when said point-of-view is modified.

17. The method according to claim 12, further comprising determining said point-of-view based on a sensed position of one or more movable joints comprising said slave unit.

18. The method according to claim 12, further comprising automatically dynamically modifying said second directional translation in response to a user selection of an image obtained from a second imaging sensor, where said second imaging sensor has a different point-of-view as compared to said first imaging sensor.

19. The method according to claim 12, further comprising communicating data representing at least one of said sensed direction and said haptic feedback direction to said user interface to generate a force using at least one component of said user interface.

24. The robot system according to claim 14, wherein said control processor is further configured for automatically dynamically modifying said first directional translation that is performed when said point-of-view is modified.

25. The robot system according to claim 14, wherein said control processor is configured to determine said point-of-view based upon at least one condition selected from the group consisting of a position and an orientation of said first imaging sensor.

26. The robot system according to claim 14, wherein said slave device comprises a plurality of movable joints and associated joint position sensors, said control processor configured to determine said point-of-view based on information obtained from said joint position sensors.

27. The robot system according to claim 14, wherein said control processor is further configured to automatically dynamically modify said first directional translation in response to a user input selecting an image obtained from a second imaging sensor, where said second imaging sensor has a different point-of-view as compared to said first imaging sensor.

28. The robot system according to claim 14, wherein said communication link is configured to communicate to said slave device data presenting at least one of said control direction and said slave direction to cause a movement of said slave device.

29. The robot system according to claim 14, wherein said slave device includes a robotic arm having a plurality of joints which facilitate movement of a portion of said robotic arm in said slave direction responsive to said user interface control.

31. The robot system according to claim 14, further comprising at least one sensor associated with said slave device configured for generating a signal representing a force applied to said slave device in a sensed direction;

wherein said control processor is configured for performing a second directional translation to convert data specifying said sensed direction to data specifying a haptic feedback direction defined with respect to said three-dimensional set of coordinate axes, said directional translation determined based on said point-of-view of said first imaging sensor so that said haptic feedback direction is different from said sensed direction;

at least one haptic force generator at said user interface responsive to said control processor and configured for generating a haptic force aligned with said feedback direction.

32. The robot system according to claim 31, wherein said sensed direction of said force comprises one or more of a linear sensed direction and a rotational sensed direction with respect to said three-dimensional set of coordinate axes.

33. The robot system according to claim 32, wherein said haptic feedback direction comprises one or more of a linear haptic direction and a rotational haptic direction with respect to said three-dimensional set of coordinate axes.

34. The robot system according to claim 33, wherein said second directional translation comprises one or more of a linear translation and a rotational translation.

35. The robot system according to claim 31, wherein said control processor is configured to automatically dynamically modify said second directional translation that is performed when said point-of-view is modified.

36. The robot system according to claim 31, wherein said control processor is configured to determine said point-of-view based on a sensed position of one or more movable joints comprising said slave unit.

37. The robot system according to claim 31, wherein said control processor is configured to automatically dynamically modify said second directional translation in response to a user input selection of an image obtained from a second imaging sensor in place of said first imaging sensor, where said second imaging sensor has a different point-of-view as compared to said first imaging sensor.

20. A robot system, comprising:

a control interface having at least one sensor for sensing a physical displacement of a user interface control in any control direction defined with respect to a three-dimensional set of coordinate axes;

a slave device configured for moving in response to control signals generated at said control interface;

a control processor coupled to said control interface and configured to perform a first directional translation to convert data specifying said control direction to data specifying a slave direction defined with respect to said three-dimensional set of coordinate axes, said slave direction different from said control direction and defining a direction that said slave device should move in response to said physical displacement of said user interface;

a communication link for communicating motion control information between said control interface and said slave device; and

wherein said first directional translation is determined based on a point-of-view of a first imaging sensor mounted to a portion of said slave device.

21. The robot system according to claim 20, wherein said control direction comprises one or more of a linear direction and a rotational direction with respect to said three-dimensional set of coordinate axes.

22. The robot system according to claim 21, wherein said slave direction comprises one or more of a linear direction and a rotational direction with respect to said three-dimensional set of coordinate axes.

23. The robot system according to claim 22, wherein said first directional translation comprises one or more of a linear translation and a rotational translation.

30. The robot system according to claim 20, wherein a mounting point for said first imaging sensor is coupled to said robotic arm, such that said point-of-view of said first imaging sensor changes concurrently with movement of said robotic arm.