Medical robotic system providing computer generated auxiliary views of a camera instrument for controlling the position and orienting of its tip

• US 9,516,996 B2
• Filed: 09/17/2014
• Issued: 12/13/2016
• Est. Priority Date: 06/27/2008
• Status: Active Grant

• Current Assignee: Intuitive Surgical Operations, Inc. (Intuitive Surgical Holdings, LLC)
• Original Assignee: Intuitive Surgical Operations, Inc. (Intuitive Surgical Holdings, LLC)
• Inventors: Diolaiti, Nicola, Larkin, David Q., Gomez, Daniel H., Mustufa, Tabish, Mohr, Paul W., Lilagan, Paul E.
• Primary Examiner(s): Sweet, Thomas J
• Assistant Examiner(s): CHOU, WILLIAM B
• Application Number:
  US14/488,408
• Publication Number:
  US 20150065793A1
• Time in Patent Office: 818 Days
• Field of Search: 600/103, 600/104, 600/117, 600/145, 600/173, 600/106, 600/109, 600/160
• US Class Current: 1/1
• CPC Class Codes: A61B 1/00006 of control signals
• CPC Class Codes: A61B 1/00009 of image signals during a use of endoscope
• CPC Class Codes: A61B 1/00042 for mechanical operation
• CPC Class Codes: A61B 1/00045 Display arrangement
• CPC Class Codes: A61B 1/0005 combining images e.g. side-by-side, superimposed or tiled
• CPC Class Codes: A61B 1/00087 Tools A61B17/00234 takes precedence
• CPC Class Codes: A61B 1/00149 using articulated arms
• CPC Class Codes: A61B 1/00154 using guiding arrangements for insertion
• CPC Class Codes: A61B 1/05 characterised by the image sensor, e.g. camera, being in the distal end portion
• CPC Class Codes: A61B 1/313 for introducing through surgical openings, e.g. laparoscopes
• CPC Class Codes: A61B 2017/00119 alarm; indicating an abnormal situation
• CPC Class Codes: A61B 2034/102 Modelling of surgical devices, implants or prosthesis
• CPC Class Codes: A61B 2034/2074 Interface software
• CPC Class Codes: A61B 2034/301 for introducing or steering flexible instruments inserted into the body, e.g. catheters or endoscopes
• CPC Class Codes: A61B 2034/305 Details of wrist mechanisms at distal ends of robotic arms
• CPC Class Codes: A61B 2090/364 Correlation of different images or relation of image positions in respect to the body
• CPC Class Codes: A61B 2090/365 augmented reality, i.e. correlating a live optical image with another image
• CPC Class Codes: A61B 34/20 Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
• CPC Class Codes: A61B 34/30 Surgical robots
• CPC Class Codes: A61B 34/37 Master-slave robots A61B34/35 takes precedence
• CPC Class Codes: A61B 34/74 : Manipulators with manual electric input means
• CPC Class Codes: A61B 34/77 : Manipulators with motion or force scaling
• CPC Class Codes: A61B 5/065 : Determining position of the probe employing exclusively positioning means located on or in the probe, e.g. using position sensors arranged on the probe
• CPC Class Codes: A61B 90/03 : Automatic limiting or abutting means, e.g. for safety
• CPC Class Codes: A61B 90/36 : Image-producing devices or illumination devices not otherwise provided for
• CPC Class Codes: A61B 90/361 : Image-producing devices, e.g. surgical cameras
• CPC Class Codes: G06T 7/70 : Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G09B 23/285 : for injections, endoscopy, bronchoscopy, sigmoidscopy, insertion of contraceptive devices or enemas

Abstract

A medical robotic system includes an entry guide with surgical tools and a camera extending out of its distal end. To supplement the view provided by an image captured by the camera, an auxiliary view including articulatable arms of the surgical tools and/or camera is generated from sensed or otherwise determined information about their positions and orientations and displayed on a display screen from the perspective of a specified viewing point. Intuitive control is provided to an operator with respect to the auxiliary view while the operator controls the positioning and orienting of the camera.

Claims

1. A method implemented by a processor for positioning and orienting a distal tip of an articulated image capturing instrument while the distal tip of the articulated image capturing instrument is extending out of an entry guide, the method comprising:

determining, by using the processor, a position and orientation of the distal tip of the articulated image capturing instrument;

generating, by using the processor, an auxiliary view of a portion of the articulated image capturing instrument that is extending out of the entry guide by the processor using the determined position and orientation of the distal tip of the articulated image capturing instrument, wherein the auxiliary view is from a three-dimensional viewing point from which the portion of the articulated image capturing instrument is viewable throughout an available workspace of the portion of the articulated image capturing instrument;

displaying, by using the processor, the auxiliary view on a display; and

controlling, by using the processor, positioning and orienting the distal tip of the articulated image capturing instrument in response to movement of an input device by;

accounting for misalignment between an orientation of the input device with respect to the auxiliary view being displayed on the display and the determined orientation of the distal tip of the articulated image capturing instrument with respect to a control reference frame, and mapping movement of the input device with respect to the auxiliary view being displayed on the display screen to a commanded movement of the distal tip of the articulated image capturing instrument with respect to the control reference frame.

2. The method of claim 1, wherein the control reference frame has an origin located at a point in space where the articulated image capturing instrument extends out of the entry guide.

3. The method of claim 1, wherein the control reference frame has an origin and orientation aligned with a perspective of the auxiliary view being displayed on the display.

4. The method of claim 1,

wherein the method further comprises determining, by using the processor, a position and orientation of a distal tip of an articulated tool instrument when the distal tip of the articulated tool instrument is extending out of the entry guide;

wherein the generated auxiliary view further includes a portion of the articulated tool instrument that is extending out of the entry guide as determined by the processor using the determined position and orientation of the distal tip of the articulated tool instrument; and

wherein the portion of the articulated tool instrument is viewable throughout an available workspace of the portion of the articulated tool instrument from the three-dimensional viewing point.

5. The method of claim 1, wherein generating, by using the processor, an auxiliary view of a portion of the articulated image capturing instrument that is extending out of the entry guide comprises:

generating, by using the processor, a view of a computer model of the portion of the articulated image capturing instrument from the perspective of the three-dimensional viewing point.

6. The method of claim 1, wherein mapping movement of the input device with respect to the auxiliary view being displayed on the display screen to a commanded movement of the distal tip of the articulated image capturing instrument with respect to the control reference frame comprises at least one of cancelling translational offsets and scaling the commanded movement of the distal tip of the articulated image capturing instrument with respect to the control reference frame.

7. The method of claim 6, wherein scaling the commanded movement of the distal tip of the articulated image capturing instrument with respect to the control reference frame comprises scaling the commanded movement of the distal tip of the articulated image capturing instrument with respect to the control reference frame by using a user-selectable scaling factor.