VISUAL FORCE FEEDBACK IN A MINIMALLY INVASIVE SURGICAL PROCEDURE

• US 20100169815A1
• Filed: 04/22/2009
• Published: 07/01/2010
• Est. Priority Date: 12/31/2008
• Status: Active Grant

• Current Assignee: Intuitive Surgical Operations, Inc. (Intuitive Surgical Holdings, LLC)
• Original Assignee: Intuitive Surgical Incorporated (Intuitive Surgical Holdings, LLC)
• Inventors: Larkin, David Q., Zhao, Tao, Zhao, Wenyi
• Granted Patent: US 8,594,841 B2
• Time in Patent Office: Days
• US Class Current: 715/771
• CPC Class Codes: A61B 2034/102 Modelling of surgical devices, implants or prosthesis
• CPC Class Codes: A61B 2090/065 for measuring contact or contact pressure
• CPC Class Codes: A61B 2090/365 augmented reality, i.e. correlating a live optical image with another image
• CPC Class Codes: A61B 34/20 Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
• CPC Class Codes: A61B 34/30 Surgical robots
• CPC Class Codes: A61B 34/35 for telesurgery
• CPC Class Codes: A61B 34/37 Master-slave robots A61B34/35 takes precedence
• CPC Class Codes: A61B 34/76 Manipulators having means for providing feel, e.g. force or tactile feedback
• CPC Class Codes: B25J 9/1633 compliant, force, torque control, e.g. combined with position control
• CPC Class Codes: B25J 9/1697 Vision controlled systems
• CPC Class Codes: Y10S 901/47 Optical

Abstract

Methods of and a system for providing a visual representation of force information in a robotic surgical system. A real position of a surgical end effector is determined. A projected position of the surgical end effector if no force were applied against the end effector is also determined. Images representing the real and projected positions are output superimposed on a display. The offset between the two images provides a visual indication of a force applied to the end effector or to the kinematic chain that supports the end effector. In addition, tissue deformation information is determined and displayed.

Claims

1. A method of providing a visual representation of force information in a robotic surgical system, the method comprising:

determining a force applied against a surgical end effector;

displaying a first image showing an actual position of the end effector under the applied force; and

superimposing on the first image a second image representing a projected position of the end effector offset from the first position, wherein the offset is a visual indication of the force.

2. The method of claim 1, wherein said at least some force comprises force applied to the end effector as a result of movement of the end effector between a first position and a second position, wherein another force is applied to a linkage supporting the end effector, and wherein the offset between the projected position and the actual position disregards at least some of the other force.

3. The method of claim 1, wherein the second image represents kinematic position information derived from joint states of a linkage supporting the end effector.

4. The method of claim 1, wherein the first image comprises a visual image of the end effector.

5. The method of claim 1, wherein displaying comprises displaying on a viewer of a surgeon console of the robotic surgical system.

6. The method of claim 1, wherein the second image comprises a synthetic representation of the tool showing a position of the tool, an orientation of the tool, or both.

7. The method of claim 6, wherein the synthetic representation comprises a model of the tool.

8. The method of claim 7, wherein the model comprises a three-dimensional model of the tool.

9. The method of claim 1, wherein the force is derived from force information derived from active force sensors.

10. The method of claim 9, wherein the active force sensors comprise force sensors located at the end effector.

11. The method of claim 10, wherein the end effector is supported by a linkage having joints, and wherein the active force sensors comprise sensors located at the joints.

12. The method of claim 1, wherein the end effector is supported by a linkage having joints, and further comprising:

determining joint errors from positions of the joints with respect to a commanded position of the joints;

correcting the joints toward the command position; and

wherein the force is derived from information derived from the correcting of the joints.

13. A system of providing a visual representation of force information in a robotic system, the system comprising:

an image input for capturing an image of an end effector while a force is applied to the end effector;

a display coupled to the image input so as to present an actual position of the end effector under the applied force; and

a processor coupled to the display, the processor generating a second image representing a projected position of the end effector offset from the first position so as to visually indicate the force, the processor transmitting the second image to the display so that the second image is superimposed with the first image.

14. A method of providing a visual representation of tissue deformation in a robotic surgical system, the method comprising:

applying a force against a tissue;

determining deformation of the tissue, the deformation being due to application of the force; and

displaying information about the deformation.

15. The method of claim 14, wherein determining the deformation comprises utilizing the force to derive the deformation.

16. The method of claim 15, wherein the force is derived from force information derived from active force sensors.

17. The method of claim 16, wherein the active force sensors comprise force sensors located at the end effector.

18. The method of claim 16, wherein the end effector is supported by a linkage having joints, and wherein the active force sensors comprise sensors located at the joints.

19. The method of claim 15, wherein the end effector is supported by a linkage having joints, and further comprising:

determining joint errors from positions of the joints with respect to a commanded position of the joints;

correcting the joints toward the command position; and

wherein the force is derived from information derived from the correcting of the joints.

20. The method of claim 15, wherein the force is derived from force information, the force information being provided by:

storing first kinematic position information and first actual position information for a first position of an end effector;

moving the end effector via the robotic surgical system from the first position a second position;

storing second kinematic position information and second actual position information; and

providing force information regarding force applied to the end effector at the second position utilizing the first actual position information, the second actual position information, the first kinematic position information, and the second kinematic position information.

21. The method of claim 14, wherein displaying information about the deformation comprises displaying the tissue simultaneous with the information about the deformation.

22. The method of claim 21, wherein the information about the deformation is superimposed over the tissue.

23. The method of claim 22, wherein the information about the deformation comprises superimposing a color over the tissue.

24. The method of claim 23, wherein the color changes based upon the amount of force applied to the tissue.

25. The method of claim 23, wherein the color is varied based upon how far the tissue is from contact with the end effector.

26. The method of claim 14, wherein the deformation information is combined with ultrasound imaging to provide elasticity imaging of tissues.