Camera referenced control in a minimally invasive surgical apparatus

• US 6,424,885 B1
• Filed: 08/13/1999
• Issued: 07/23/2002
• Est. Priority Date: 04/07/1999
• Status: Expired due to Term

• Current Assignee: Intuitive Surgical Operations, Inc. (Intuitive Surgical Holdings, LLC)
• Original Assignee: Intuitive Surgical Incorporated (Intuitive Surgical Holdings, LLC)
• Inventors: Guthart, Gary S., Niemeyer, Gunter D., Nowlin, William C., Swarup, Nitish, Toth, Gregory K., Younge, Robert G.
• Primary Examiner(s): Picard, Leo
• Assistant Examiner(s): Garland, Steven R.
• Application Number:
  US09/373,678
• Time in Patent Office: 1,075 Days
• Field of Search: 700/245, 700/247, 700/248, 700/251, 700/257-260, 700/262-264, 348/65, 382/128, 382/276, 382/295, 382/296, 318/568.11, 318/568.21, 901/34, 901/35, 901/46, 901/47, 600/101-104, 600/109, 600/117, 600/118
• US Class Current: 700/245
• CPC Class Codes: A61B 2017/00477 Coupling A61B2017/0046 takes precedence
• CPC Class Codes: A61B 2034/305 Details of wrist mechanisms at distal ends of robotic arms
• CPC Class Codes: A61B 2090/064 for measuring force, pressure or mechanical tension
• CPC Class Codes: A61B 2090/506 using a parallelogram linkage, e.g. panthograph
• CPC Class Codes: A61B 34/30 Surgical robots
• CPC Class Codes: A61B 34/35 for telesurgery
• CPC Class Codes: A61B 34/37 Master-slave robots A61B34/35 takes precedence
• CPC Class Codes: A61B 34/70 Manipulators specially adapted for use in surgery
• CPC Class Codes: A61B 34/77 Manipulators with motion or force scaling
• CPC Class Codes: A61B 90/361 Image-producing devices, e.g. surgical cameras

Abstract

Enhanced telepresence and telesurgery systems automatically update coordinate transformations so as to retain alignment between movement of an input device and movement of an end effector as displayed adjacent the input device. A processor maps a controller workspace with an end effector workspace, and effects movement of the end effector in response to the movement of the input device. This allows the use of kinematically dissimilar master and slave linkages. Gripping an input member near a gimbal point and appropriate input member to end effector mapping points enhance the operator'"'"'s control. Dexterity is enhanced by accurately tracking orientational and/or angles of movement, even if linear movement distances of the end effector do not correspond to those of the input device.

Claims

1. A surgical robotic system comprising:

a master controller having an input device movable in a controller workspace;

a slave having a surgical end effector, a linkage movably supporting the end effector, and at least one actuator operatively coupled to the end effector, the actuator moving the end effector in a surgical workspace in response to slave actuator signals;

an imaging system including an image capture device with a field of view movable in the surgical workspace and a linkage movably supporting the image capture device, the imaging system generating state variable signals indicating the field of view; and

a processor coupling the master controller to the slave arm, the processor generating the slave actuator signals by mapping the input device in the controller workspace with the end effector in the surgical workspace according to a transformation, the processor deriving the transformation in response to the state of variable signals of the imaging system, the linkages comprising joints and the state variable signals comprising joint configuration signals, the linkages coupled so that the processor derives the transformation in response to the joint configuration signals such that movement of an image of the end effector in a display appears substantially connected to the input device in the workspace;

wherein if the image capture device is moved the state variables of the imaging system change and the input device is re-mapped with the end effector according to another transformation, the other transformation derived in response to the changed state variables of the imaging system.

2. The surgical robotic system of claim 1, wherein the image capture device moves independently of the slave.

3. The surgical robotic system of claim 2, wherein the processor derives the transformation so that an image of the end effector in a display appears substantially connected to the input device in the controller workspace.

4. The surgical robotic system of claim 1, wherein the slave generates state variable signals responsive to a position. of the end effector in the surgical workspace, the processor deriving the transformation using the state variable signals of the slave.

5. The surgical robotic system of claim 4, wherein the processor determines a position and orientation of the input device in the master controller space from state variable signals of the master controller, wherein the processor determines a position and orientation of the end effector in the surgical workspace from the state variable signals of the slave, and wherein the processor generates the slave actuator signals by comparing the position and orientation of the input device and the end effector in a mapped space.

6. The surgical robotic system of claim 4, wherein the slave comprises an elongate shaft supporting the end effector, wherein the image capture device comprises an endoscope having a distal end, and wherein the processor derives the transformation when the shaft and endoscope are inserted into the surgical site through arbitrary minimally invasive access sites so that the end effector and the distal end of the endoscope move by pivoting the shaft and the endoscope about the access sites.

7. The surgical robotic system of claim 1, wherein the slave linkage is mounted to a slave base, and wherein the imaging system linkage is mechanically coupled to the slave base.

8. The surgical robotic system of claim 1, wherein the imaging system linkage is mounted to a base of the slave, and wherein the slave base has wheels for transporting the end effector linkage and the image capture device.

9. The surgical robotic system of claim 1, wherein the slave comprises a tool holder and a tool detachably mounted in the tool holder, the tool including the end effector and at least one joint, the surgical robotic system further comprising a plurality of alternative kinematically dissimilar surgical tools having alternative surgical end effectors, the alternative tools mountable to the at least one tool holder in place of the tool, the processor capable of changing the transformation in response to a tool change signal.

10. The surgical robotic system of claim 1, wherein the processor derives the transformation in real time.

11. A surgical robotic system comprising:

a master controller having an input device movable in a controller workspace;

a plurality of slaves having a surgical end effector and at least one actuator operatively coupled to the end effector, the actuator moving the end effector in a surgical workspace in response to slave actuator signals;

an imaging system including an image capture device with a field of view movable in the surgical workspace, the imaging system generating state variable signals indicating the field of view; and

a processor coupling the master controller to the slave arm, the processor generating the slave actuator signals by mapping the input device in the controller workspace with the end effector in the surgical workspace according to a transformation, the processor deriving the transformation in response to the state of variable signals of the imaging system;

wherein if the image capture device is moved the state variables of the imaging system change and the input device is re-mapped with the end effector according to another transformation, the other transformation derived in response to the changed state variables of the imaging system, the master controller selectively associatable with the slaves in response to a slave selection signal, wherein the processor changes the transformation in response to the slave selection signal so that movement of an image of the end effector of the selected slave as shown in a display substantially corresponds to movement of the input device in the workspace.

12. A surgical robotic system comprising:

a master controller having an input device movable in a controller workspace;

a slave having a surgical end effector and at least one actuator operatively coupled to the end effector, the actuator moving the end effector in a surgical workspace in response to slave actuator signals;

an imaging system including an image capture device with a field of view movable in the surgical workspace, the imaging system transmitting an image to a display; and

a processor coupling the master controller to an arm of the slave, the processor generating the slave actuator signals by mapping the input device in the controller workspace with the end effector in the surgical workspace according to a transformation, the processor deriving the transformation so that an image of the end effector in the display appears substantially connected to the input device in the workspace;

wherein if the image capture device is moved the state variables of the imaging system change and the input device is re-mapped with the end effector according to another transformation, the other transformation derived in response to the changed state variables of the imaging system; and

wherein at least one of the slave, the master controller, and the imaging system comprise a linkage having a detachable connection disposed between joints so as to define a first linkage portion attached to a base and a second linkage portion attached to the first linkage portion, the linkage portions defining first and second coordinate reference systems, wherein the processor derives the transformation indirectly from a first relationship of the first coordinate system to the base of the first linkage system and from a second relationship of the second coordinate system to the first coordinate system.

13. The surgical robotic system of claim 12, wherein the master controller comprises a linkage supporting the input device and the slave comprises a linkage supporting the end effector, the master linkage and the slave linkage being kinematically dissimilar.

14. The surgical robotic system of claim 12, wherein joints of the master linkage and joints of the slave linkage have different degrees of freedom.

15. The surgical robotic system of claim 12, wherein joints of the master linkage and joints of the slave linkage define different locations in the mapped space.

16. The surgical robotic system of claim 12, wherein:

the slave comprises a linkage with a plurality of joints supporting the end effector, the actuators coupled to the joints so as to pivot a rigid shaft of the slave about a first arbitrary minimally invasive access and move the end effector in response to slave motor signals; and

the imaging system comprises actuators and a linkage supporting the image capture device having, the linkage comprising a plurality of joints, the actuators coupled to the joints to move the field of view within the surgical workspace when a rigid shaft of the image capture device extends through a second arbitrary minimally invasive access.

17. The surgical robotic system of claim 12, wherein the processor calculates the transformation in response to a signal indicating at least one member of the group consisting of a movement of a camera, a decoupling and repositioning of one of the master and the slave relative to the other, a tool change mounting a different end effector on the slave, a change in scale of the mapping, manual movement of a passive joint of the master or slave, and association of the master with an alternative slave.

18. The surgical robotic system of claim 12, wherein the slave senses non-visual sensory information at the surgical workspace, the master controller presenting the non-visual information to a surgeon manipulating the input device.

19. The surgical robotic system of claim 12, wherein the master controller presents non-visual sensory information to a surgeon manipulating the input device.

20. The surgical robotic system of claim 19, wherein the master controller presents the non-visual information with an orientation correlating to the image.

21. The surgical robotic system of claim 20, wherein the non-visual information indicates forces and torques applied to the slave.

22. The surgical robotic system of claim 21, wherein the non-visual information comprises forces and torques applied via the input device to the hand of the surgeon so that the input device forces and torques substantially correspond to the slave forces and torques according to the image of the slave shown in the display.

23. The surgical robotic system of claim 19, wherein the master controller comprises a plurality of actuators, the actuators providing tactile feedback to the surgeon manipulating the input device in response to master actuator signals generated by the processor.

24. The surgical robotic system of claim 23, wherein the actuators apply loads against the input device in response to the master actuator signals, and wherein the processor generates the master actuator signals in response to a comparison between a position and orientation of the end effector and a position and orientation of the input device in the mapped space.

25. The surgical robotic system of claim 12, wherein the processor generates the slave actuation signals so that at least one angle selected from the group comprising a change in angular orientation and an angle of translation of the end effector remains within 5 degrees of the at least one angle of the input device.

26. A surgical robotic system as claimed in claim 25, wherein an input device movement defines an input movement distance, and wherein the image of the end effector moves an output movement distance in response to the input movement distance, the output movement distance being significantly different than the input movement distance.

27. A surgical robotic system comprising:

a master controller having an input device movable in a controller workspace;

a slave having a surgical end effector and at least one actuator operatively coupled to the end effector, the actuator moving the end effector in a surgical workspace in response to slave actuator signals;

an imaging system including an image capture device with a field of view movable in the surgical workspace, the imaging system transmitting an image to a display; and

a processor coupling the master controller to an arm of the slave, the processor generating the slave actuator signals by mapping the input device in the controller workspace with the end effector in the surgical workspace according to a transformation, the processor deriving the transformation so that an image of the end effector in the display appears substantially connected to the input device in the workspace;

wherein if the image capture device is moved the state variables of the imaging system change and the input device is re-mapped with the end effector according to another transformation, the other transformation derived in response to the changed state variables of the imaging system; and

wherein the slave senses non-visual sensory information at the surgical workspace, the master controller presenting the non-visual information to a surgeon manipulating the input device, the display showing a graphical representation of the non-visual information.

28. The surgical robotic system of claim 27, wherein the non-visual information comprises force applied at the end effector.

29. The surgical robotic system of claim 28, wherein the display represents the force as a member selected from the group consisting of a bar graph, a force vector, and an end effector color.

30. A surgical robotic system comprising:

a master controller having an input device movable in a controller workspace;

a slave having a surgical end effector and at least one actuator operatively coupled to the end effector, the actuator moving the end effector in a surgical workspace in response to slave actuator signals;

an imaging system including an image capture device with a field of view movable in the surgical workspace, the imaging system transmitting an image to a display; and

a processor coupling the master controller to an arm of the slave, the processor generating the slave actuator signals by mapping the input device in the controller workspace with the end effector in the surgical workspace according to a transformation, the processor deriving the transformation so that an image of the end effector in the display appears substantially connected to the input device in the workspace;

wherein if the image capture device is moved the state variables of the imaging system change and the input device is re-mapped with the end effector according to another transformation, the other transformation derived in response to the changed state variables of the imaging system; and

wherein the slave senses non-visual sensory information at the surgical workspace, the master controller presenting the non-visual information to a surgeon manipulating the input device, wherein the non-visual information comprises force applied at the end effector, and wherein the master controller comprises a sound generator, the sound generator producing a sound varying in response to the force.

31. A surgical robotic system comprising:

a master controller having an input device movable in a controller workspace;

a slave having a surgical end effector and at least one actuator operatively coupled to the end effector, the actuator moving the end effector in a surgical workspace in response to slave actuator signals;

an imaging system including an image capture device with a field of view movable in the surgical workspace, the imaging system transmitting an image to a display; and

a processor coupling the master controller to an arm of the slave, the processor generating the slave actuator signals by mapping the input device in the controller workspace with the end effector in the surgical workspace according to a transformation, the processor deriving the transformation so that an image of the end effector in the display appears substantially connected to the input device in the workspace;

wherein if the image capture device is moved the state variables of the imaging system change and the input device is re-mapped with the end effector according to another transformation, the other transformation derived in response to the changed state variables of the imaging system; and

wherein the master controller presents non-visual sensory information to a surgeon manipulating the input device, wherein the master controller presents the non-visual information with an orientation correlating to the image, wherein the non-visual information indicates forces and torques applied to the slave, and wherein a correlation between the non-visual information and the image is revised by the controller when the transformation is revised.

32. A surgical robotic method comprising:

moving a master input device in a controller workspace by articulating a plurality of master joints;

generating master joint signals responsive to the master joint configuration;

moving an end effector in an end effector workspace by articulating a plurality of slave joints in response to slave motor signals;

generating slave joint signals responsive to the slave joint configuration;

calculating a position of the input device in the master controller space from the masterjoint signals;

calculating the end effector position in the end effector workspace from the slave joint signals;

mapping the end effector workspace with the controller workspace according to a transformation;

generating the slave motor signals in response to a difference between the input device position and the end effector position in the mapped space;

moving a field of view of an image capture device in the end effector workspace by articulating a plurality of image device joints;

displaying an image of the field of view adjacent the master controller and generating image device signals responsive to the image device joint configuration; and

revising the mapping in response to the image device signals so that movement of an image of the end effector shown in the display remains aligned with the movement of the input device in the master controller space.

33. A surgical robotic system comprising:

a master controller having an input device movable in a master controller space, the input device including first and second grip members for actuating with first and second digits of a hand of an operator;

a slave having a surgical end effector, the end effector moving in a surgical workspace in response to slave actuator signals and including first and second end effector elements; and

a processor coupling the master to the slave, the processor generating the slave actuator signals so that movement of the first and second grip members substantially map movement of the first and second end effector elements;

wherein if a re-map signal is generated state variables of an imaging system change and the input device is re-mapped with the end effector according to another transformation, the other transformation derived in response to the changed state variables of the imaging system; and

wherein a midpoint disposed between tips of the first and second grip members is substantially connected to a midpoint between tips of the end effector elements.

34. The surgical robotic system of claim 33, wherein a pivotal joint between the first and second grip members is substantially connected to a pivotal joint between the end effector elements.

35. A surgical robotic system comprising:

a master controller having a handle supported by a plurality of joints so that the handle is movable in a master controller space, the joints defining a gimbal point of rotation about a plurality of axes, the handle disposed adjacent the gimbal point; and

a slave having a surgical end effector, the end effector moving in a surgical workspace in response to movement of the handle;

wherein if a re-map signal is generated state variables of an imaging system change and the input device is re-mapped with the end effector according to another transformation, the other transformation derived in response to the changed state variables of the imaging system.

36. The surgical robotic system of claim 35, wherein the end effector is supported by a plurality of joints, a last joint being disposed adjacent the end effector, and further comprising a processor coupling the master to the slave, the processor generating slave actuator signals so that the gimbal point of the master is substantially connected to the last joint of the slave.

37. A surgical robotic system comprising:

a master controller having a handle, the handle moving in a master controller workspace;

a slave supporting a surgical end effector, the slave moving the end effector within a surgical workspace in response to slave actuation signals; and

a processor coupling the master to the slave, the processor generating the slave actuation signals so that movement of a mapping point along the handle of the master controller substantially maps movement of a mapping point along the end effector, the processor capable of changing at least one of the handle mapping point and the end effector mapping point.

38. The surgical robotic system of claim 37, wherein the end effector includes a first end effector releasably mounted to the slave and a second end effector releasably mountable to the slave in place of the first end effector, the first end effector having a first length from a first end effector pivot, the second end effector having a second length, and wherein the processor can change the at least one mapping point to adjust for a difference between the first and second lengths.

39. The surgical robotic system of claim 37, wherein the handle comprises first and second grip members squeezable about a pivot, wherein a grip axis is disposed on a midline between the first and second grip members and the handle mapping point remains disposed between the grip pivot and an end of the handle on the handle axis wherein the end effector comprises jaws with an end effector axis disposed on a midline between first and second jaw elements, and wherein the end effector mapping point remains disposed between the end effector pivot and an end of the jaw elements along the end effector axis.

40. The surgical robotic system of claim 37, further comprising a display disposed adjacent the master, the display showing an image of the end effector with a magnification, wherein the at least one mapping point moves with changes in the magnification of the display.

41. A surgical robotic method comprising:

moving a master input device in a controller workspace by articulating a plurality of master joints;

generating master joint signals responsive to the master joint configuration;

moving an end effector in an end effector workspace by articulating a plurality of slave joints in response to slave motor signals;

generating slave joint signals responsive to the slave joint configuration;

calculating a position of the input device in the master controller space from the masterjoint signals;

calculating the end effector position in the end effector workspace from the slave joint signals;

mapping the end effector workspace with the controller workspace according to a transformation; and

generating the slave motor signals in response to a difference between the input device position and the end effector position in the mapped space.

42. A surgical robotic system comprising:

a master controller having an input device movable in a controller workspace;

a slave having a surgical end effector and at least one actuator operatively coupled to the end effector, the actuator moving the end effector in a surgical workspace in response to slave actuator signals;

an imaging system including an image capture device with a field of view movable in the surgical workspace, the imaging system generating state variable signals indicating the field of view; and

a processor coupling the master controller to the slave arm, the processor generating the slave actuator signals by mapping the input device in the controller workspace with the end effector in the surgical workspace according to a transformation, the processor deriving the transformation in response to the state of variable signals of the imaging system;

wherein the slave generates state variable signals responsive to a position of the end effector in the surgical workspace, the processor deriving the transformation using the state variable signals of the slave;

wherein the slave comprises an elongate shaft supporting the end effector, wherein the image capture device comprises an endoscope having a distal end, and wherein the processor derives the transformation when the shaft and endoscope are inserted into the surgical site through arbitrary minimally invasive access sites so that the end effector and the distal end of the endoscope move by pivoting the shaft and the endoscope about the access sites.

43. A surgical robotic system comprising:

a master controller having an input device movable in a controller workspace;

a slave having a surgical end effector and at least one actuator operatively coupled to the end effector, the actuator moving the end effector in a surgical workspace in response to slave actuator signals;

an imaging system including an image capture device with a field of view movable in the surgical workspace, the imaging system generating state variable signals indicating the field of view; and

a processor coupling the master controller to the slave arm, the processor generating the slave actuator signals by mapping the input device in the controller workspace with the end effector in the surgical workspace according to a transformation, the processor deriving the transformation in response to the state of variable signals of the imaging system;

wherein the slave comprises a tool holder and a tool detachably mounted in the tool holder, the tool including the end effector and at least one joint, the surgical robotic system further comprising a plurality of alternative kinematically dissimilar surgical tools having alternative surgical end effectors, the alternative tools mountable to the at least one tool holder in place of the tool, the processor capable of changing the transformation in response to a tool change signal.

44. A surgical robotic system comprising:

a master controller having an input device supported by a linkage so that the input device can move in a controller workspace with a first number of degrees of freedom;

a slave having a surgical end effector and a plurality of actuators operatively coupled thereto so that the end effector can move in a surgical workspace with a second number of degrees of freedom in response to slave actuator signals, the second number being less than the first number; and

a processor coupling the master controller to the slave, the processor generating the slave actuator signals by mapping the input device in the controller workspace with the end effector in the surgical workspace;

wherein the slave comprises a manipulator arm releasably supporting the end effector and further comprising an alternative end effector mountable to the arm in place of the end effector, the alternative end effector having at least one more degree of freedom than the end effector,

wherein the processor inhibits movement of the input device in the controller workspace when the end effector is in use so that the input device is movable in the second number of degrees of freedom.

45. The surgical robotic system of claim 44, wherein the master has at least one redundant degree of freedom.