Synthetic representation of a surgical instrument

• US 9,788,909 B2
• Filed: 11/11/2013
• Issued: 10/17/2017
• Est. Priority Date: 06/29/2006
• Status: Active Grant

• Current Assignee: Intuitive Surgical Operations, Inc. (Intuitive Surgical Holdings, LLC)
• Original Assignee: Intuitive Surgical Operations, Inc. (Intuitive Surgical Holdings, LLC)
• Inventors: Larkin, David Q., Hoffman, Brian D., Mohr, Paul W.
• Primary Examiner(s): Downey, John R
• Application Number:
  US14/076,833
• Publication Number:
  US 20140135792A1
• Time in Patent Office: 1,436 Days
• Field of Search: 606 1
• CPC Class Codes: A61B 1/00154 using guiding arrangements for insertion
• CPC Class Codes: A61B 1/00165 with light-conductive means, e.g. fibre optics A61B1/07 takes precedence
• CPC Class Codes: A61B 1/00193 adapted for stereoscopic vision
• CPC Class Codes: A61B 1/04 combined with photographic or television appliances
• CPC Class Codes: A61B 17/00234 for minimally invasive surgery A61B17/0218, A61B17/0469, A61B17/12013, A61B17/1285, A61B17/29, A61B17/320016 take precedence
• CPC Class Codes: A61B 2017/00314 Separate linked members
• CPC Class Codes: A61B 2017/00323 Cables or rods
• CPC Class Codes: A61B 2034/2059 Mechanical position encoders
• CPC Class Codes: A61B 2034/2061 using shape-sensors, e.g. fiber shape sensors with Bragg gratings
• CPC Class Codes: A61B 2034/2065 Tracking using image or pattern recognition
• CPC Class Codes: A61B 2034/301 for introducing or steering flexible instruments inserted into the body, e.g. catheters or endoscopes
• CPC Class Codes: A61B 2034/305 Details of wrist mechanisms at distal ends of robotic arms
• CPC Class Codes: A61B 2034/306 Wrists with multiple vertebrae
• CPC Class Codes: A61B 2090/371 with simultaneous use of two cameras
• CPC Class Codes: A61B 34/10 Computer-aided planning, simulation or modelling of surgical operations
• CPC Class Codes: A61B 34/20 Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
• CPC Class Codes: A61B 34/30 Surgical robots
• CPC Class Codes: A61B 34/37 Master-slave robots A61B34/35 takes precedence
• CPC Class Codes: A61B 90/36 Image-producing devices or illumination devices not otherwise provided for
• CPC Class Codes: B25J 9/1671 characterised by simulation, either to verify existing program or to create and verify new program, CAD/CAM oriented, graphic oriented programming systems
• CPC Class Codes: B25J 9/1689 : Teleoperation
• CPC Class Codes: B25J 9/1692 : Calibration of manipulator
• CPC Class Codes: G05B 2219/39449 : Pendant, pda displaying camera images overlayed with graphics, augmented reality
• CPC Class Codes: G05B 2219/40607 : Fixed camera to observe workspace, object, workpiece, global
• CPC Class Codes: G05B 2219/45123 : Electrogoniometer, neuronavigator, medical robot used by surgeon to operate
• CPC Class Codes: G05B 2219/45169 : Medical, rontgen, x ray
• CPC Class Codes: G06T 7/0012 : Biomedical image inspection

Abstract

A synthetic representation of a tool for display on a user interface of a robotic system. The synthetic representation may be used to show force on the tool, an actual position of the tool, or to show the location of the tool when out of a field of view. A three-dimensional pointer is also provided for a viewer in the surgeon console of a telesurgical system.

Claims

1. A method for operatively coupling an end effector to an input device, the method comprising:

displaying an image of an object in a first window on a display screen;

displaying a synthetic representation of the end effector at a location in a second window on the display screen, wherein the location indicates a position of the end effector relative to a position of the object in a common reference frame;

operatively coupling an input device to a synthetic rendering of a pointing device, the synthetic rendering of the pointing device being displayed in the first window on the display screen along with one or more user interface selection devices, the synthetic rendering of the pointing device being an icon appearing as a smaller version of the synthetic representation of the end effector on the display screen;

receiving an indication that a user has interactively selected the synthetic representation of the end effector being displayed on the display screen by the user interacting with the input device so that the synthetic rendering of the pointing device selects the synthetic representation of the end effector from the one or more user interface selection devices; and

operatively decoupling the input device from the synthetic rendering of the pointing device and operatively coupling the input device to the end effector in response to the received indication.

2. The method of claim 1, further comprising:

receiving a captured image of the object from an image capture device, wherein the end effector is outside a field-of-view of the image capture device;

wherein the displaying an image of an object in a first window on a display screen comprises displaying the captured image on the display screen.

3. The method of claim 1, further comprising:

determining the position of the object relative to the common reference frame;

determining the position of the end effector relative to the common reference frame by using information of joint positions of a robot arm coupled to the end effector; and

determining the location to display the synthetic representation of the end effector in the second window by using the determined positions of the object and the end effector relative to the common reference frame.

4. The method of claim 1, wherein the object is an original end effector which is initially operatively coupled to the input device, and further comprising:

operatively decoupling the input device from the original end effector before operatively coupling the input device to the synthetic rendering of the pointing device.

5. The method of claim 4, further comprising:

aligning an orientation of the input device to an orientation of the end effector after decoupling the input device from the synthetic rendering of the pointing device and before operatively coupling the input device to the end effector.