Synthetic representation of a surgical instrument

• US 9,801,690 B2
• Filed: 05/17/2017
• Issued: 10/31/2017
• Est. Priority Date: 06/29/2006
• Status: Active Grant

• Current Assignee: Intuitive Surgical Operations, Inc. (Intuitive Surgical Holdings, LLC)
• Original Assignee: Intuitive Surgical Operations, Inc. (Intuitive Surgical Holdings, LLC)
• Inventors: Larkin, David Q., Hoffman, Brian D., Mohr, Paul W.
• Primary Examiner(s): Downey, John R
• Application Number:
  US15/597,558
• Publication Number:
  US 20170245948A1
• Time in Patent Office: 167 Days
• CPC Class Codes: A61B 1/00154 using guiding arrangements for insertion
• CPC Class Codes: A61B 1/00165 with light-conductive means, e.g. fibre optics A61B1/07 takes precedence
• CPC Class Codes: A61B 1/00193 adapted for stereoscopic vision
• CPC Class Codes: A61B 1/04 combined with photographic or television appliances
• CPC Class Codes: A61B 17/00234 for minimally invasive surgery A61B17/0218, A61B17/0469, A61B17/12013, A61B17/1285, A61B17/29, A61B17/320016 take precedence
• CPC Class Codes: A61B 2017/00314 Separate linked members
• CPC Class Codes: A61B 2017/00323 Cables or rods
• CPC Class Codes: A61B 2034/2059 Mechanical position encoders
• CPC Class Codes: A61B 2034/2061 using shape-sensors, e.g. fiber shape sensors with Bragg gratings
• CPC Class Codes: A61B 2034/2065 Tracking using image or pattern recognition
• CPC Class Codes: A61B 2034/301 for introducing or steering flexible instruments inserted into the body, e.g. catheters or endoscopes
• CPC Class Codes: A61B 2034/305 Details of wrist mechanisms at distal ends of robotic arms
• CPC Class Codes: A61B 2034/306 Wrists with multiple vertebrae
• CPC Class Codes: A61B 2090/371 with simultaneous use of two cameras
• CPC Class Codes: A61B 34/10 Computer-aided planning, simulation or modelling of surgical operations
• CPC Class Codes: A61B 34/20 Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
• CPC Class Codes: A61B 34/30 Surgical robots
• CPC Class Codes: A61B 34/37 Master-slave robots A61B34/35 takes precedence
• CPC Class Codes: A61B 90/36 Image-producing devices or illumination devices not otherwise provided for
• CPC Class Codes: B25J 9/1671 characterised by simulation, either to verify existing program or to create and verify new program, CAD/CAM oriented, graphic oriented programming systems
• CPC Class Codes: B25J 9/1689 : Teleoperation
• CPC Class Codes: B25J 9/1692 : Calibration of manipulator
• CPC Class Codes: G05B 2219/39449 : Pendant, pda displaying camera images overlayed with graphics, augmented reality
• CPC Class Codes: G05B 2219/40607 : Fixed camera to observe workspace, object, workpiece, global
• CPC Class Codes: G05B 2219/45123 : Electrogoniometer, neuronavigator, medical robot used by surgeon to operate
• CPC Class Codes: G05B 2219/45169 : Medical, rontgen, x ray
• CPC Class Codes: G06T 7/0012 : Biomedical image inspection

Abstract

A synthetic representation of a tool for display on a user interface of a robotic system. The synthetic representation may be used to show force on the tool, an actual position of the tool, or to show the location of the tool when out of a field of view. A three-dimensional pointer is also provided for a viewer in the surgeon console of a telesurgical system.

Claims

1. A method for using an input device as a computer mouse, wherein the input device is operatively coupled to an end effector whose captured image is being displayed on a viewer, the method comprising:

operatively decoupling the input device from the end effector;

displaying a pointer appearing as synthetic representation of the end effector on the viewer, so that the synthetic representation of the end effector appears as a smaller version of the captured image of the end effector being displayed on the viewer;

operatively coupling the input device to the pointer so that manipulation of the input device results in movement of the pointer on the viewer;

displaying a selectable item on the viewer; and

receiving an indication that the pointer has moved on the viewer in response to manipulation of the input device, so as to select the selectable item.

2. The method of claim 1,

wherein the selectable item is included in a drop-down menu being displayed on the viewer.

3. The method of claim 2, further comprising:

operatively decoupling the input device from the pointer; and

conditioned upon the input device being operatively decoupled from the pointer,

no longer displaying the pointer on the viewer; and

operatively recoupling the input device to the end effector.

4. The method of claim 3, further comprising:

conditioned upon the input device being operatively decoupled from the pointer,

aligning an orientation of the input device to an orientation of the end effector before operatively recoupling the input device to the end effector.

5. The method of claim 1,

wherein the viewer is a stereovision viewer;

and

wherein conditioned upon the input device being operatively coupled to the pointer,

the pointer moves in a three-dimensional image displayed on the stereovision viewer in response to manipulation of the input device.

6. The method of claim 1,

wherein the viewer is a stereovision viewer;

and

wherein conditioned upon the input device being operatively coupled to the pointer,

the pointer appears as a three-dimensional synthetic representation of the end effector on the viewer.

7. A system comprising:

an end effector;

a viewer displaying a captured image of the end effector;

an input device operatively coupled to the end effector; and

a processor programmed to;

operatively decouple the input device from the end effector;

cause a pointer to be displayed on the viewer, the pointer appearing as synthetic representation of the end effector, the synthetic representation of the end effector appearing as a smaller version of the captured image of the end effector being displayed on the viewer;

operatively couple the input device to the pointer so that manipulation of the input device results in movement of the pointer on the viewer;

cause a selectable item to be displayed on the viewer; and

receive an indication that the pointer has moved on the display in response to manipulation of the input device, so as to select the selectable item.

8. The system of claim 7,

wherein the selectable item is included in a drop-down menu being displayed on the viewer.

9. The system of claim 8,

wherein the processor is further programmed to:

operatively decouple the input device from the pointer;

conditioned upon the input device being operatively decoupled from the pointer, not cause the pointer to be displayed on the viewer; and

operatively recouple the input device to the end effector.

10. The system of claim 9,

wherein the processor is further programmed to:

conditioned upon the input device being operatively decoupled from the pointer,

cause an orientation of the input device to be aligned with an orientation of the end effector before operatively recoupling the input device to the end effector.

11. The system of claim 7,

wherein the viewer is a stereovision viewer;

and

wherein processor is further programmed to;

conditioned upon the input device being operatively coupled to the pointer,

cause the pointer to move in a three-dimensional image, which is being displayed on the stereovision viewer, in response to manipulation of the input device.

12. The system of claim 7,

wherein the viewer is a stereovision viewer;

and

wherein the processor is further programmed to;

conditioned upon the input device being operatively coupled to the pointer,

cause the pointer to appear as a three-dimensional synthetic representation of the end effector on the viewer.