Method for graphically providing continuous change of state directions to a user of a medical robotic system

• US 8,892,224 B2
• Filed: 10/14/2013
• Issued: 11/18/2014
• Est. Priority Date: 09/26/2008
• Status: Active Grant

• Current Assignee: Intuitive Surgical Operations, Inc. (Intuitive Surgical Holdings, LLC)
• Original Assignee: Intuitive Surgical Operations, Inc. (Intuitive Surgical Holdings, LLC)
• Inventors: Mohr, Paul W., Robinson, David W.
• Primary Examiner(s): Black, Thomas G
• Assistant Examiner(s): Nelson, Sara
• Application Number:
  US14/052,835
• Publication Number:
  US 20140039521A1
• Time in Patent Office: 400 Days
• Field of Search: 700/257, 700/264, 700/249, 700/245, 700/3, 700/83, 700/180, 700/184, 700/28, 700/32, 700/33, 715/3, 715/83, 715/180, 715/184, 715/705, 901/2, 901/6, 901/9, 901/50
• US Class Current: 700/83
• CPC Class Codes: A61B 2090/0811 for the position of a particular part of an instrument with respect to the rest of the instrument, e.g. position of the anvil of a stapling instrument
• CPC Class Codes: A61B 34/25 User interfaces for surgical systems
• CPC Class Codes: A61B 34/30 Surgical robots
• CPC Class Codes: A61B 34/37 Master-slave robots A61B34/35 takes precedence
• CPC Class Codes: A61B 90/37 Surgical systems with images on a monitor during operation
• CPC Class Codes: B25J 9/1689 Teleoperation
• CPC Class Codes: B25J 9/1692 Calibration of manipulator
• CPC Class Codes: G05B 2219/39024 Calibration of manipulator
• CPC Class Codes: G05B 2219/45123 Electrogoniometer, neuronavigator, medical robot used by surgeon to operate
• CPC Class Codes: Y10S 901/02 Arm motion controller
• CPC Class Codes: Y10S 901/06 Communication with another machine
• CPC Class Codes: Y10S 901/09 Closed loop, sensor feedback controls arm movement

Abstract

Continuous change of state directions are graphically provided on a display screen to assist a user in performing necessary action(s) for transitioning between operating modes in a medical robotic system or performing corrective action. A graphical representation of a target state of an element of the medical robotic system is displayed on a display screen viewable by the user. Current states of the element and indications directing the user to manipulate the element towards the target state are continuously determined and graphical representations of the continuously determined current states and indications are displayed on the display screen along with that of the target state.

Claims

1. A method for graphically providing continuous change of state directions to a user of a medical system that includes an input device which is operated by the user, an actuatable joint which is operatively coupled to the input device, and a display screen which is viewable by the user, the method comprising:

determining whether the actuatable joint would move within a threshold distance from a range of motion limitation of the actuatable joint in response to a current state of the input device; and

if it is determined that the actuatable joint would move within the threshold distance from the range of motion limitation of the actuatable joint in response to the current state of the input device, then

(i) displaying a graphical representation of a target state of the input device on the display screen, wherein the target state corresponds to a most recent state of the input device which did not result in the actuatable joint moving within the threshold distance from the range of motion limitation of the actuatable joint,

(ii) continuously determining current states of the input device and current indications directing the user to manipulate the input device towards the target state, and

(iii) displaying graphical representations of the current states of the input device and graphical representations of the current indications along with the graphical representation of the target state of the input device on the display screen.

2. The method according to claim 1, wherein the input device has a pincher formation, and wherein the displaying graphical representations of the current states of the input device on the display screen comprises:

displaying a graphical representation of the pincher formation that moves on the display screen so that a current distance between the displayed graphical representation of the pincher formation and the displayed graphical representation of the target state of the input device indicates a difference between the current state of the input device at that time and the target state of the input device.

3. The method according to claim 1, wherein the displaying graphical representations of the current indications on the display screen comprises:

displaying each graphical representation of the current indication as an arrow pointing from the displayed graphical representation of the current state of the input device to the displayed graphical representation of the target state of the input device so as to direct the user to manipulate the current state of the input device towards the target state of the input device.

4. The method according to claim 1, further comprising:

if it is determined that the actuatable joint would move within the threshold distance from the range of motion limitation of the actuatable joint in response to the current state of the input device, then providing force feedback on the input device until the current state of the input device reaches the target state of the input device.

5. The method according to claim 1, further comprising:

determining whether the actuatable joint would reach the range of motion limitation of the actuatable joint in response to the current state of the input device; and

if it is determined that the actuatable joint would reach the range of motion limitation of the actuatable joint in response to the current state of the input device, then suspending operative coupling between the input device and the actuatable joint, so that the actuatable joint does not move in response to the current state of the input device.

6. A medical system comprising:

a slave manipulator having a plurality of actuatable joints;

an input device operatively coupled to the plurality of actuatable joints;

a display screen viewable by a user of the medical system; and

a processor programmed to;

determine whether an actuatable joint of the plurality of actuatable joints would move within a threshold distance from a range of motion limitation of the actuatable joint in response to a current state of the input device; and

if it is determined that the actuatable joint would move within the threshold distance from the range of motion limitation of the actuatable joint in response to the current state of the input device, then

(i) cause a graphical representation of a target state of the input device to be displayed on the display screen, wherein the target state corresponds to a most recent state of the input device which did not result in the actuatable joint moving within the threshold distance from the range of motion limitation of the actuatable joint,

(ii) continuously determine current states of the input device and current indications directing the user to manipulate the input device towards the target state; and

(iii) cause graphical representations of the current states of the input device and graphical representations of the current indications to be displayed along with the graphical representation of the target state on the display screen.

7. The medical system according to claim 6, wherein the input device has a pincher formation, and wherein the processor is programmed to cause the graphical representations of the current states of the input device to be displayed on the display screen by causing a graphical representation of the pincher formation to move on the display screen so that a current distance between the displayed graphical representation of the pincher formation and the displayed graphical representation of the target state of the input device distance indicates a difference between the current state of the input device at that time and the target state of the input device.

8. The medical system according to claim 6, wherein the processor is programmed to cause graphical representations of the current indications to be displayed on the display screen by causing each graphical representation of the current indication to be displayed as an arrow pointing from the displayed graphical representation of the current state of the input device to the displayed graphical representation of the target state of the input device so as to direct the user to manipulate the current state of the input device towards the target state of the input device.

9. The medical system according to claim 6, wherein the processor is programmed to:

if it is determined that the actuatable joint would move within the threshold distance of the range of motion limitation of the actuatable joint in response to the current state of the input device, then command force feedback on the input device until the current state of the input device reaches the target state of the input device.

10. The medical system according to claim 6, wherein the processor is programmed to:

determine whether the actuatable joint of the plurality of actuatable joints would reach the range of motion limitation of the actuatable joint in response to the current state of the input device; and

if it is determined that the actuatable joint would reach the range of motion limitation of the actuatable joint in response to the current state of the input device, then suspend operative coupling between the input device and the plurality of actuatable joints, so that the plurality of actuatable joints do not move in response to the current state of the input device.