Multi-user medical robotic system for collaboration or training in minimally invasive surgical procedures

• US 20060178559A1
• Filed: 12/27/2005
• Published: 08/10/2006
• Est. Priority Date: 11/20/1998
• Status: Active Grant

• Current Assignee: Intuitive Surgical Operations, Inc. (Intuitive Surgical Holdings, LLC)
• Original Assignee: Intuitive Surgical Incorporated (Intuitive Surgical Holdings, LLC)
• Inventors: Kumar, Rajesh, Mukherjee, Ranjan, Moll, Frederic, Wright, James, Prisco, Giuseppe, Niemeyer, Gunter D., Wang, Yulun, Hoffman, Brian, Uecker, Darrin, Mangaser, Amante, Ghodoussi, Modjtaba, Salisbury, J. Kenneth, Larkin, David, Nowlin, William, Blumenkranz, Stephen
• Granted Patent: US 8,527,094 B2
• Time in Patent Office: Days
• US Class Current: 600/109
• CPC Class Codes: A61B 1/00193 adapted for stereoscopic vision
• CPC Class Codes: A61B 2017/00017 Electrical control of surgical instruments
• CPC Class Codes: A61B 34/30 Surgical robots
• CPC Class Codes: A61B 34/37 Master-slave robots A61B34/35 takes precedence
• CPC Class Codes: A61B 34/70 Manipulators specially adapted for use in surgery
• CPC Class Codes: A61B 90/361 Image-producing devices, e.g. surgical cameras
• CPC Class Codes: A61B 90/37 Surgical systems with images on a monitor during operation
• CPC Class Codes: G09B 23/28 for medicine
• CPC Class Codes: G16H 20/40 relating to mechanical, radiation or invasive therapies, e.g. surgery, laser therapy, dialysis or acupuncture
• CPC Class Codes: G16H 40/63 for local operation
• CPC Class Codes: G16H 80/00 ICT specially adapted for facilitating communication between medical practitioners or patients, e.g. for collaborative diagnosis, therapy or health monitoring
• CPC Class Codes: Y10S 901/16 Cartesian, three degrees of freedom

Abstract

A multi-user medical robotic system for collaboration or training in minimally invasive surgical procedures includes first and second master input devices, a first slave robotic mechanism, and at least one processor configured to generate a first slave command for the first slave robotic mechanism by switchably using one or both of a first command indicative of manipulation of the first master input device by a first user and a second command indicative of manipulation of the second master input device by a second user. To facilitate the collaboration or training, both first and second users communicate with each other through an audio system and see the minimally invasive surgery site on first and second displays respectively viewable by the first and second users.

Claims

1. A medical robotic system comprising:

first master input device configured to generate a first command indicative of manipulation of the first master input device by a first user;

second master input device configured to generate a second command indicative of manipulation of the second master input device by a second user;

first slave robotic mechanism configured to manipulate a first surgery-related device according to a first slave command;

at least one processor configured to generate the first slave command by switchably using one or both of the first command and the second command; and

an audio system configured for audio communication between the first user and the second user.

2. The medical robotic system according to claim 1, wherein the first slave robotic mechanism is further configured so as to be manually manipulatable by a third user, and the audio system is further configured for audio communication between the first user and the third user.

3. The medical robotic system according to claim 1, wherein the first slave robotic mechanism is further configured so as to be manually manipulatable by a third user, and the audio system is further configured for audio communication between the first user, the second user, and the third user.

4. The medical robotic system according to claim 1, wherein the first surgery-related device is a surgical tool suitable for use in a minimally invasive surgical procedure.

5. The medical robotic system according to claim 1, wherein the first surgery-related device is a camera system suitable for use in a minimally invasive surgical procedure.

6. The medical robotic system according to claim 5, wherein the camera system includes an endoscope.

7. The medical robotic system according to claim 6, wherein images captured by the endoscope are concurrently provided to a first display viewable by the first user and a second display viewable by the second user.

8. The medical robotic system according to claim 7, wherein the endoscope is a stereoscopic endoscope and the images captured by the stereoscopic endoscope are processed by the at least one processor to provide three-dimensional views of the images on the first display and the second display.

9. The medical robotic system according to claim 5, wherein the camera system includes first and second endoscopes, and the at least one processor is configured to generate a first slave command responsive to the first command so that the first user may control a first view generated by the first endoscope, and a second slave command responsive to the second command so that the second user may control a second view generated by the second endoscope.

10. The medical robotic system according to claim 1, wherein the at least one processor is configured to generate the first slave command using one or both of the first command and the second command according to a first selection command.

11. The medical robotic system according to claim 10, wherein the first selection command is provided by the first user through a switch manipulated by the user.

12. The medical robotic system according to claim 11, wherein the switch is a button manipulatable by a thumb of the first user.

13. The medical robotic system according to claim 11, wherein the switch is a foot pedal manipulatable by a foot of the first user.

14. The medical robotic system according to claim 11, wherein the switch is voice activated by the voice of the first user.

15. The medical robotic system according to claim 1, wherein the at least one processor generates the first slave command using the first command, but not the second command.

16. The medical robotic system according to claim 15, wherein the at least one processor causes the second master input device to track the manipulation of the first master input device by the first user so that the second user may sense such manipulation.

17. The medical robotic system according to claim 6, wherein indications of forces asserted against the first slave robotic mechanism are provided to the first master input device so that the first user may sense such forces.

18. The medical robotic system according to claim 17, wherein the indications of forces are also provided to the second master input device so that the second user may also sense such forces.

19. The medical robotic system according to claim 1, wherein the at least one processor generates the first slave command using both the first command and the second command.

20. The medical robotic system according to claim 19, wherein indications of forces asserted against the first slave robotic mechanism are provided to the first master input device so that the first user may sense such forces.

21. The medical robotic system according to claim 20, wherein the at least one processor causes the second master input device to track the manipulation of the first master input device by the first user so that the second user may sense such manipulation.

22. The medical robotic system according to claim 21, wherein indications of forces asserted against the first slave robotic mechanism are also provided to the second master input device so that the second user may sense such forces.

23. The medical robotic system according to claim 21, wherein indications of forces at the first master input device are provided to the second master input device so that the second user may sense such forces.

24. The medical robotic system according to claim 21, wherein the at least one processor further causes the first master input device to track the manipulation of the second master input device by the second user so that the first user may sense such manipulation.

25. The medical robotic system according to claim 24, wherein indications of forces asserted against the first slave robotic mechanism are also provided to the second master input device so that the second user may sense such forces.

26. The medical robotic system according to claim 24, wherein indications of forces at the first master input device are provided to the second master input device so that the second user may sense such forces.

27. The medical robotic system according to claim 19, wherein the at least one processor is configured to generate the first slave command using a relative weighting of the first command and the second command as indicated by a relative weight command.

28. The medical robotic system according to claim 27, wherein the relative weight command is provided by the first user through the first master input device.

29. The medical robotic system according to claim 19, wherein the at least one processor is configured to generate the first slave command only from the first command in response to an over-ride command provided by the first user through the first master input device.

30. The medical robotic system according to claim 1, further comprising:

second slave robotic mechanism configured to manipulate a second surgery-related device according to a second slave command, wherein the at least one processor is further configured to generate the second slave command using one or both of the first command or the second command.

31. The medical robotic system according to claim 30, wherein the first surgery-related device is a first surgical tool suitable for use in a minimally invasive surgical procedure.

32. The medical robotic system according to claim 31, wherein the second surgery-related device is a second surgical tool suitable for use in the minimally invasive surgical procedure.

33. The medical robotic system according to claim 31, wherein the second surgery-related device is a camera system suitable for use in the minimally invasive surgical procedure.

34. The medical robotic system according to claim 33, wherein the camera system includes an endoscope.

35. The medical robotic system according to claim 34, wherein images captured by the endoscope are concurrently provided to a first display viewable by the first user and a second display viewable by the second user.

36. The medical robotic system according to claim 34, wherein the endoscope is a stereoscopic endoscope and the images captured by the stereoscopic endoscope are processed by the at least one processor to provide three-dimensional views of the images on the first display and the second display.

37. The medical robotic system according to claim 31, wherein the first surgery-related device is a first camera system suitable for use in the minimally invasive surgical procedure, and the second surgery-related device is a second camera system suitable for use in the minimally invasive surgical procedure.

38. The medical robotic system according to claim 37, wherein the at least one processor is configured to generate a first slave command responsive to the first command so that the first user may control a first view generated by the first camera system, and a second slave command responsive to the second command so that the second user may control a second view generated by the second camera system.

39. The medical robotic system according to claim 30, wherein the at least one processor is configured to generate the first slave command using one or both of the first command and the second command according to a first selection command, and generate the second slave command using one or both of the first command and the second command according to a second selection command.

40. The medical robotic system according to claim 39, wherein the first selection command and the second selection command are provided by the first user through the first master input device.

41. The medical robotic system according to claim 40, wherein the at least one processor comprises a first processor associated with the first master input device, and a second processor associated with the second master input device.

42. The medical robotic system according to claim 1, wherein the at least one processor generates the first slave command using the second command, but not the first command.

43. The medical robotic system according to claim 42, wherein the at least one processor is further configured with a second slave robotic mechanism for simulating manipulation of the first surgery-related device according to a second slave command.

44. The medical robotic system according to claim 43, wherein the second slave robotic mechanism comprises a computer model of the first slave robotic mechanism so as to simulate the first slave robotic mechanism.

45. The medical robotic system according to claim 44, wherein the computer model includes information of kinematics of the first slave robotic mechanism.

46. The medical robotic system according to claim 44, wherein the computer model includes information of dynamics of the first slave robotic mechanism.

47. The medical robotic system according to claim 44, wherein the computer model includes information of a control system controlling the first slave robotic mechanism.

48. The medical robotic system according to claim 44, wherein the computer model includes physical constraints imposed on the first slave robotic mechanism.

49. The medical robotic system according to claim 48, wherein the physical constraints include those derived from pre-operative data for a minimally invasive surgical procedure.

50. The medical robotic system according to claim 49, wherein the pre-operative data includes anatomical features of a patient upon whom the minimally invasive surgical procedure is to be performed.

51. The medical robotic system according to claim 49, wherein the pre-operative data includes diagnostic imaging information of a patient upon whom the minimally invasive surgical procedure is to be performed.

52. The medical robotic system according to claim 49, wherein the pre-operative data includes pre-operative planning information of a patient upon whom the minimally invasive surgical procedure is to be performed.

53. The medical robotic system according to claim 43, wherein the second master input device is positioned so as to be out of the physical sight of the first user of the first master input device.

54. The medical robotic system according to claim 53, wherein the at least one processor includes a first processor associated with the first master input device and a second processor associated with the second master input device, wherein the first processor and the second processor are configured to communicate with each other.

55. The medical robotic system according to claim 54, wherein the second slave robotic mechanism is modeled in the second processor as computer model of the first slave robotic mechanism so as to simulate the first slave robotic mechanism.

56. The medical robotic system according to claim 43, wherein the at least one processor is programmed to generate the second slave command using the first command, but not the second command.

57. The medical robotic system according to claim 56, wherein indications of forces asserted against the first slave robotic mechanism are provided to the first master input device so that the first user may sense such forces.

58. The medical robotic system according to claim 57, wherein the second master input device is positioned so as to be out of the physical sight of the first user of the first master input device.

59. The medical robotic system according to claim 58, wherein the at least one processor includes a first processor associated with the first master input device and a second processor associated with the second master input device, wherein the first processor and the second processor are configured to communicate with each other.

60. The medical robotic system according to claim 59, wherein the second slave robotic mechanism is modeled in the second processor as a computer model of the first slave robotic mechanism so as to simulate the first slave robotic mechanism.

61. The medical robotic system according to claim 60, wherein the first slave robotic mechanism is configured to communicate with the first processor and the second processor.

62. The medical robotic system according to claim 60, where indications of simulated forces being asserted against the second slave robotic mechanism are provided to the second master input device so that the second user may sense such simulated forces without experiencing any transmission delay that would be incurred if the indications of the forces being asserted against the first slave robotic mechanism were provided instead to the second master input device over the communication network.

63. A multi-user medical robotic system for collaboration in minimally invasive surgical procedures, comprising:

first and second master input devices;

first and second slave robotic mechanisms;

a switch mechanism operable by a first operator for selectively associating the first and the second slave robotic mechanisms with the first and the second master input devices so that the first operator manipulating the first master input device and a second operator manipulating the second master input device may perform a minimally invasive surgical procedure at a surgical site in collaboration with each other; and

first and second headsets respectively worn by the first and the second operators so that they may communicate with each while performing the minimally invasive surgical procedure in collaboration with each other.

64. The multi-user medical robotic system according to claim 63, further comprising:

first and second displays respectively viewable by the first and the second operators so that they may each view the surgical site while performing the minimally invasive surgical procedure in collaboration with each other.

65. The multi-user medical robotic system according to claim 64, wherein the first slave robotic mechanism includes a first tool positioned at a distal end of the first slave robotic mechanism, and the first tool is manipulated by the first user when the first master input device is selectively associated with the first slave robotic mechanism and manipulated by the second user when the second master input device is selectively associated with the first slave robotic mechanism.

66. The multi-user medical robotic system according to claim 65, wherein the second slave robotic mechanism includes a second tool positioned at a distal end of the second slave robotic mechanism, and the second tool is manipulated by the first user when the first master input device is selectively associated with the second slave robotic mechanism and manipulated by the second user when the second master input device is selectively associated with the second slave robotic mechanism.

67. The multi-user medical robotic system according to claim 65, wherein the second slave robotic mechanism includes an endoscope positioned at a distal end of the second slave robotic mechanism, and the endoscope is manipulated by the first user when the first master input device is selectively associated with the second slave robotic mechanism and manipulated by the second user when the second master input device is selectively associated with the second slave robotic mechanism.

68. The multi-user medical robotic system according to claim 63, wherein the switch mechanism is a button pressable by a thumb of the first operator.

69. The multi-user medical robotic system according to claim 63, wherein the switch mechanism is a foot pedal pressable by a foot of the first operator.

70. The multi-user medical robotic system according to claim 63, wherein the switch mechanism is voice activated by the first operator.

71. A multi-user medical robotic system for training in minimally invasive surgical procedures, comprising:

mentor and trainee master input devices respectively manipulatable by a mentor and a trainee;

a first slave robotic mechanism;

a switch mechanism operable by the mentor for selectively associating the first slave robotic mechanism with the mentor master input device and the trainee master input device so that either or both the mentor or the trainee may control operation of the first slave robotic mechanism to perform a minimally invasive surgical procedure; and

a mentor microphone proximate to the mentor and a trainee hearing device proximate to the trainee so that mentor may speak to the trainee while the mentor is performing the minimally invasive surgical procedure.

72. The multi-user medical robotic system according to claim 71, wherein forces associated with the first slave robotic mechanism are reflected back to the trainee master input device when the first slave robotic mechanism is selectively associated with the mentor master input device so that the trainee master input device tracks movement of the mentor master input device.

73. The multi-user medical robotic system according to claim 72, further comprising:

first and second headsets respectively worn by the mentor and the trainee so that they may communicate with each while the mentor is performing the minimally invasive surgical procedure.

74. The multi-user medical robotic system according to claim 72, further comprising:

first and second displays respectively viewable by the mentor and the trainee so that they may each view the surgical site while the mentor is performing the minimally invasive surgical procedure.

75. The multi-user medical robotic system according to claim 71, wherein forces associated with the first slave robotic mechanism are reflected back to the mentor master input device when the first slave robotic mechanism is selectively associated with the trainee master input device so that the mentor master input device tracks movement of the trainee master input device.

76. The multi-user medical robotic system according to claim 75, further comprising:

a mentor microphone proximate to the mentor and a trainee hearing device proximate to the trainee so that mentor may speak to the trainee while the trainee is performing the minimally invasive surgical procedure.

77. The multi-user medical robotic system according to claim 75, further comprising:

first and second headsets respectively worn by the mentor and the trainee so that they may communicate with each while the trainee is performing the minimally invasive surgical procedure.

78. The multi-user medical robotic system according to claim 75, further comprising:

first and second displays respectively viewable by the mentor and the trainee so that they may each view the surgical site while the trainee is performing the minimally invasive surgical procedure.

79. The multi-user medical robotic system according to claim 71, further comprising:

a second slave robotic mechanism, wherein the second slave robotic mechanism is selectively associated with the mentor master input device when the first slave robotic mechanism is selectively associated with the trainee master input device, and the second slave robotic mechanism is selectively associated with the trainee master input device when the first slave robotic mechanism is selectively associated with the mentor master input device.

80. The multi-user medical robotic system according to claim 79, wherein the first slave robotic mechanism is selectively associated with the mentor master input device, and first forces associated with the first slave robotic mechanism are reflected back to the trainee master input device so that the first forces may be sensed by the trainee.

81. The multi-user medical robotic system according to claim 80, wherein second forces associated with the second slave robotic mechanism are reflected back to the mentor master input device so that they may be sensed by the mentor.

82. The multi-user medical robotic system according to claim 81, wherein the second slave robotic mechanism includes at least one motor responding to movement of the mentor master input to cause movement of the second slave robotic mechanism in a corresponding fashion.

83. The multi-user medical robotic system according to claim 81, wherein the second slave robotic mechanism includes a computer model for simulating at least one motor responding to movement of the mentor master input to simulate movement of the second slave robotic mechanism in a corresponding fashion.

84. The multi-user medical robotic system according to claim 71, wherein the switch mechanism is a button pressable by a thumb of the first operator.

85. The multi-user medical robotic system according to claim 71, wherein the switch mechanism is a foot pedal pressable by a foot of the first operator.

86. The multi-user medical robotic system according to claim 71, wherein the switch mechanism is voice activated by the first operator.

87. A surgeon training medical robotic system comprising:

a first slave robotic mechanism configured to manipulate a first surgery-related device according to a first slave command;

a first trainee master control station configured to generate a first trainee command indicative of a first desired position for the first surgery-related device according to a first trainee input; and

a mentor master control station configured to generate the first slave command using the first trainee command and a first sensed position of the first surgery-related device, and provide the first slave command to the first slave robotic mechanism so as to cause the first slave robotic mechanism to move the first surgery-related device to the first desired position.

88. The surgeon training robotic medical system according to claim 87, wherein the mentor master control station is further configured to over-ride the first trainee command and generate the first slave command using a mentor command indicative of a mentor desired position for the first surgery-related device and the first sensed position of the first surgery-related device, and provide the first slave command to the first slave robotic mechanism so as to cause the first slave robotic mechanism to move the first surgery-related device to the mentor desired position.

89. The surgeon training robotic medical system according to claim 87, further comprising:

a slave cart having a plurality of slave robotic mechanisms including the first slave robotic mechanism and another slave robotic mechanism configured to manipulate an endoscope according to an endoscope positioning command, wherein images captured by the endoscope are concurrently provided to said first trainee master control station and said mentor master control station for viewing by their respective operators.

90. The surgeon training robotic medical system according to claim 87, further comprising:

an audio system configured for audio communication between a trainee operator of said first trainee master control station and a mentor operator of said mentor master control station.

91. The surgeon training robotic medical system according to claim 87, further comprising:

a second trainee master control station configured to generate a second trainee command indicative of a second desired position for the first surgery-related device according to a second trainee input;

wherein the mentor master control station is further configured to generate the first slave command using one of the first trainee command or the second trainee command as selected by a mentor operator of the mentor master control station, and the first sensed position of the first surgery related device.

92. The surgeon training robotic medical system according to claim 91, wherein the mentor master control station is provided with routing information, and the mentor master control station is configured to associate at least one of the first and the second trainee master control stations with the first robotic slave robotic mechanism according to the routing information.

93. The surgeon training robotic medical system according to claim 92, wherein the mentor master control station is configured with a routing table, and the routing information is provided in the routing table.

94. The surgeon training robotic medical system according to claim 92, wherein the mentor master control station is configured so that the mentor operator may define the routing information.

95. The surgeon training robotic medical system according to claim 94, wherein the mentor master control station includes a display configured with a user interface to facilitate defining of the routing information by the mentor operator.

96. The surgeon training robotic medical system according to claim 95, wherein the display includes a display screen displaying a plurality of icons indicating the routing information to the mentor operator.

97. The surgeon training robotic medical system according to claim 95, wherein the display includes a plurality of light-emitting-diodes indicating the routing information to the mentor operator.

98. The surgeon training robotic medical system according to claim 94, wherein the mentor master control station includes a manipulator and the mentor master control station is configured such that the routing information may be defined by the mentor operator using the manipulator.

99. The surgeon training robotic medical system according to claim 94, wherein the mentor master control station includes a foot pedal and the mentor master control station is configured such that the routing information may be defined by the mentor operator using the foot pedal.

100. The surgeon training robotic medical system according to claim 91, wherein the mentor master control station is provided with shared command information, and the mentor master control station is configured to generate the first slave command using the first trainee command or the second trainee command and the mentor command according to the shared command information.

101. The surgeon training robotic medical system according to claim 100, wherein the mentor master control station is configured with a shared command filter, and the shared command information is provided as parameters for the shared command filter.