Method for graphically providing continuous change of state directions to a user of a medical robotic system

• US 8,315,720 B2
• Filed: 09/26/2008
• Issued: 11/20/2012
• Est. Priority Date: 09/26/2008
• Status: Active Grant

• Current Assignee: Intuitive Surgical Operations, Inc. (Intuitive Surgical Holdings, LLC)
• Original Assignee: Intuitive Surgical Operations, Inc. (Intuitive Surgical Holdings, LLC)
• Inventors: Mohr, Paul W., Robinson, David W.
• Primary Examiner(s): Black, Thomas
• Assistant Examiner(s): Lewandroski, Sara
• Application Number:
  US12/238,631
• Publication Number:
  US 20100082039A1
• Time in Patent Office: 1,516 Days
• Field of Search: 700/257, 700/264, 700/249, 700/245, 700/3, 700/83, 700/180, 700/184, 715/3, 715/83, 715/180, 715/184, 715/705, 901/2, 901/6, 901/9, 901/50
• US Class Current: 700/83
• CPC Class Codes: A61B 2090/0811 for the position of a particular part of an instrument with respect to the rest of the instrument, e.g. position of the anvil of a stapling instrument
• CPC Class Codes: A61B 34/25 User interfaces for surgical systems
• CPC Class Codes: A61B 34/30 Surgical robots
• CPC Class Codes: A61B 34/37 Master-slave robots A61B34/35 takes precedence
• CPC Class Codes: A61B 90/37 Surgical systems with images on a monitor during operation
• CPC Class Codes: B25J 9/1689 Teleoperation
• CPC Class Codes: B25J 9/1692 Calibration of manipulator
• CPC Class Codes: G05B 2219/39024 Calibration of manipulator
• CPC Class Codes: G05B 2219/45123 Electrogoniometer, neuronavigator, medical robot used by surgeon to operate
• CPC Class Codes: Y10S 901/02 Arm motion controller
• CPC Class Codes: Y10S 901/06 Communication with another machine
• CPC Class Codes: Y10S 901/09 Closed loop, sensor feedback controls arm movement

Abstract

Continuous change of state directions are graphically provided on a display screen to assist a user in performing necessary action(s) for transitioning between operating modes in a medical robotic system or performing corrective action. A graphical representation of a target state of an element of the medical robotic system is displayed on a display screen viewable by the user. Current states of the element and indications directing the user to manipulate the element towards the target state are continuously determined and graphical representations of the continuously determined current states and indications are displayed on the display screen along with that of the target state.

Claims

1. A method for graphically providing continuous change of state directions to a user of a medical robotic system, the method comprising:

displaying a graphical representation indicative of a target state of the user manipulatable input device on a display screen viewable by the user, wherein a medical device is being held at a held state until the user manipulatable input device is associated to the medical device, and wherein the target state of the user manipulatable input device is a state of the user manipulatable input device at which the user manipulatable input device is aligned with the held state of the medical device;

continuously determining current states of the user manipulatable input device and indications directing the user to change the state of the user manipulatable input device towards the target state while the user manipulatable input device is being manipulated by the user; and

displaying graphical representations indicative of the continuously determined current states of the user manipulatable input device and graphical representations of the continuously determined indications along with the graphical representation indicative of the target state on the display screen to assist the user in aligning states of the user manipulatable input device to the medical device prior to associating the user manipulatable input device to the medical device.

2. The method according to claim 1, further comprising:

associating the user manipulatable input device to the medical device after the user manipulatable input device has reached the target state.

3. The method according to claim 1, further comprising:

determining the held state of the medical device by using sensed joint positions of a slave manipulator coupled to the medical device for manipulating the medical device.

4. The method according to claim 3, further comprising:

determining the target state as a state of the input device that commands the slave manipulator to manipulate the medical device to the held state of the medical device through robotic control of the medical device.

5. The method according to claim 1, wherein the user manipulatable input device is manipulatable in a plurality of degrees of freedom and the indications directing the user to manipulate the user manipulatable input device towards the target state comprise one or more graphical cues indicating which of, and how, the plurality of degrees of freedom of the user manipulatable input device should be manipulated for directing the user manipulatable input device towards the target state.

6. The method according to claim 1, wherein the medical device has an orientable end effector and the user manipulatable input device has an orientable hand grip, and the target state of the user manipulatable input device comprises an orientation of the hand grip that corresponds to an orientation of the end effector in the held state of the medical device.

7. The method according to claim 1, wherein the medical device has end effector jaws and the user manipulatable input device has hand manipulatable pinchers, and the target state of the user manipulatable input device comprises an opening angle of the hand manipulatable pinchers that corresponds to an opening angle of the end effector jaws in the held state of the medical device.

8. The method according to claim 1, wherein the graphical representation indicative of the target state comprises a depiction of the target state of the user manipulatable input device and the graphical representations of the current states comprise depictions of the current states of the user manipulatable input device.

9. The method according to claim 1, wherein the graphical representation indicative of the target state comprises a depiction indicating the held state of the medical device and the graphical representations of the current states comprise depictions of the commanded states of the medical device if the user manipulatable input device was associated at the time with the medical device.

10. A medical robotic system comprising:

an input device;

a medical device switchably associable to the input device, wherein the medical is held in an held state prior to being associated to the input device, and wherein the medical device changes state in response to a change of state of the input device after the medical device is associated to the input device;

a display screen disposed so as to be viewable by a user of the input device; and

a processor programmed to graphically provide, prior to associating the medical device to the input device, continuous change of state directions to a user of the medical robotic system by;

displaying a graphical representation indicative of a target state of the input device on the display screen wherein the target state of the input device is a state of the input device at which the input device is aligned with the held state of the medical device, continuously determining current states of the input device along with indications directing the user to manipulate the input device towards the target state, and displaying graphical representations indicative of the continuously determined current states of the input device and graphical representations of the continuously determined indications along with the graphical representation indicative of the target state on the display screen.

11. The medical robotic system according to claim 10, further comprising:

a slave manipulator adapted to manipulate the medical device;

wherein the processor is programmed to control the slave manipulator to manipulate the medical device in response to user manipulation of the input device when the medical device is associated to the input device, and wherein the processor is programmed to determine the held state of the medical device by using sensed joint positions of the slave manipulator.

12. The medical robotic system according to claim 11, wherein the processor is programmed to determine the target state as a state of the input device that would command the slave manipulator to manipulate the medical device to the held state of the medical device if the medical device was associated at the time to the input device.

13. The medical robotic system according to claim 10, wherein the input device is manipulatable in a plurality of degrees of freedom and the indications directing the user to manipulate the input device towards the target state comprise one or more graphical cues indicating which of, and how, the plurality of degrees of freedom of the input device should be manipulated for directing the input device towards the target state.

14. The medical robotic system according to claim 10, wherein the medical device has an orientable end effector and the input device has an orientable hand grip, and the target state of the input device comprises an orientation of the hand grip that corresponds to an orientation of the end effector in the held state of the medical device.

15. The medical robotic system according to claim 10, wherein the medical device has end effector jaws and the input device has hand manipulatable pinchers, and the target state of the element comprises an opening angle of the hand manipulatable pinchers that corresponds to an opening angle of the end effector jaws in the held state of the medical device.

16. The medical robotic system according to claim 10, wherein the graphical representation indicative of the target state comprises a depiction of the target state of the input device and the graphical representations of the current states comprise depictions of the current states of the input device.

17. The medical robotic system according to claim 10, wherein the processor is programmed to associate the medical device to the input device after the input device has reached the target state.

18. The medical robotic system according to claim 10, wherein the graphical representation indicative of the target state comprises a depiction indicating the held state of the medical device and the graphical representations of the current states comprise depictions of the commanded states of the medical device if the user manipulatable input device was associated at the time with the medical device.