Methods of determining tissue distances using both kinematic robotic tool position information and image-derived position information

• US 8,792,963 B2
• Filed: 09/30/2011
• Issued: 07/29/2014
• Est. Priority Date: 09/30/2007
• Status: Active Grant

• Current Assignee: Intuitive Surgical Operations, Inc. (Intuitive Surgical Holdings, LLC)
• Original Assignee: Intuitive Surgical Operations, Inc. (Intuitive Surgical Holdings, LLC)
• Inventors: Zhao, Wenyi, Hasser, Christopher J., Nowlin, William C., Hoffman, Brian D.
• Primary Examiner(s): Jung, Unsu
• Assistant Examiner(s): LAURITZEN, AMANDA L
• Application Number:
  US13/249,821
• Publication Number:
  US 20120020547A1
• Time in Patent Office: 1,033 Days
• Field of Search: 600/424, 600/427, 600/437, 600/407, 382/153, 700/245, 606/130
• US Class Current: 600/424
• CPC Class Codes: A61B 1/00193 adapted for stereoscopic vision
• CPC Class Codes: A61B 1/00194 adapted for three-dimensional imaging
• CPC Class Codes: A61B 1/018 for receiving instruments
• CPC Class Codes: A61B 1/04 combined with photographic or television appliances
• CPC Class Codes: A61B 1/3132 for laparoscopy
• CPC Class Codes: A61B 2034/2065 Tracking using image or pattern recognition
• CPC Class Codes: A61B 2090/061 for measuring dimensions, e.g. length
• CPC Class Codes: A61B 2090/364 Correlation of different images or relation of image positions in respect to the body
• CPC Class Codes: A61B 2090/3983 Reference marker arrangements for use with image guided surgery
• CPC Class Codes: A61B 34/20 Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
• CPC Class Codes: A61B 34/30 Surgical robots
• CPC Class Codes: A61B 34/37 Master-slave robots A61B34/35 takes precedence
• CPC Class Codes: A61B 5/061 Determining position of a probe within the body employing means separate from the probe, e.g. sensing internal probe position employing impedance electrodes on the surface of the body A61B1/00158 takes precedence
• CPC Class Codes: A61B 5/1076 for measuring dimensions inside body cavities, e.g. using catheters A61B3/1005 takes precedence
• CPC Class Codes: A61B 90/361 Image-producing devices, e.g. surgical cameras
• CPC Class Codes: B25J 13/089 Determining the position of the robot with reference to its environment
• CPC Class Codes: B25J 15/0019 End effectors other than grippers
• CPC Class Codes: B25J 9/1656 characterised by programming, planning systems for manipulators
• CPC Class Codes: B25J 9/1689 Teleoperation
• CPC Class Codes: G06T 2207/10021 Stereoscopic video; Stereoscopic image sequence
• CPC Class Codes: G06T 2207/10068 : Endoscopic image
• CPC Class Codes: G06T 2207/30004 : Biomedical image processing
• CPC Class Codes: G06T 2207/30241 : Trajectory
• CPC Class Codes: G06T 7/246 : using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06V 10/255 : Detecting or recognising potential candidate objects based on visual cues, e.g. shapes
• CPC Class Codes: G06V 2201/034 : of medical instruments
• CPC Class Codes: G09B 23/28 : for medicine
• CPC Class Codes: G16H 20/40 : relating to mechanical, radiation or invasive therapies, e.g. surgery, laser therapy, dialysis or acupuncture
• CPC Class Codes: G16H 30/40 : for processing medical images, e.g. editing
• CPC Class Codes: G16H 40/67 : for remote operation

Abstract

In one embodiment of the invention, a method is disclosed to locate a robotic instrument in the field of view of a camera. The method includes capturing sequential images in a field of view of a camera. The sequential images are correlated between successive views. The method further includes receiving a kinematic datum to provide an approximate location of the robotic instrument and then analyzing the sequential images in response to the approximate location of the robotic instrument. An additional method for robotic systems is disclosed. Further disclosed is a method for indicating tool entrance into the field of view of a camera.

Claims

1. A method of determining a distance between first and second points on tissue, the method comprising:

touching tissue at a first point with a first tool tip of a first robotic tool;

touching the tissue at a second point with a second tool tip of a second robotic tool;

fusing kinematics derived position information and image derived position information of the first robotic tool in a state-space model by using a tool tracking system to estimate a first tool tip location;

fusing kinematics derived position information and image derived position information of the second robotic tool in a state-space model by using the tool tracking system to estimate a second tool tip location; and

comparing the first tool tip location and the second tool tip location by using the tool tracking system to determine a distance between the first and second points on the tissue.

2. The method of claim 1, wherein

the tissue comprises a tumor;

the first point is on a first external surface of the tumor;

the second point is on a second external surface of the tumor; and

the comparing determines a diameter of the tumor.

3. The method of claim 1, wherein

the tissue comprises an organ;

the first point is on a first external surface of the organ;

the second point is on a second external surface of the organ; and

the comparing determines a diameter of the organ.

4. The method of claim 1, further comprising:

processing sensor data of the first robotic tool by using the tool tracking system to generate the kinematics derived position information of the first robotic tool; and

processing sensor data of the second robotic tool by using the tool tracking system to generate the kinematics derived position information of the second robotic tool.

5. The method of claim 1, wherein

the fusing of the kinematics derived position information and the image derived position information of the first robotic tool in the state-space model is performed by the tool tracking system using a sequential Bayesian approach;

and

the fusing of the kinematics derived position information and the image derived position information of the second robotic tool in the state-space model is performed by the tool tracking system using the sequential Bayesian approach.

6. The method of claim 1, further comprising:

capturing a sequence of images of the first robotic tool and the second robotic tool at a surgical site;

processing the sequence of images of the first robotic tool and the second robotic tool at the surgical site by using the tool tracking system to generate the image derived position information of the first and second robotic tools.

7. The method of claim 6, wherein

the processing of the sequence of images of the first robotic tool and the second robotic tool at the surgical site comprises an image-by-synthesis approach wherein synthesized images of computer models of the first and second robotic tools are compared against images of the first and second robotic tools in the sequence of images.

8. The method of claim 6, wherein

the processing of the sequence of images of the first robotic tool and the second robotic tool at the surgical site comprises a feature matching approach wherein features of the first and second robotic tools are identified in the sequence of images.

9. The method of claim 6, wherein

the processing of the sequence of images of the first robotic tool and the second robotic tool at the surgical site comprises a sequence matching approach wherein objects or features in a sequence of images captured from a view of the image capturing device are matched against objects or features in a sequence of images captured from a different view of the image capturing device.

10. The method of claim 6, wherein

the processing of the sequence of images of the first robotic tool and the second robotic tool at the surgical site comprises a sequence matching approach wherein objects or features in a sequence of images captured from a view of the image capturing device are matched against objects or features in a sequence of synthesized images.