Virtual measurement tool for minimally invasive surgery

• US 9,155,592 B2
• Filed: 06/16/2009
• Issued: 10/13/2015
• Est. Priority Date: 06/16/2009
• Status: Active Grant

• Current Assignee: Intuitive Surgical Operations, Inc. (Intuitive Surgical Holdings, LLC)
• Original Assignee: Intuitive Surgical Operations, Inc. (Intuitive Surgical Holdings, LLC)
• Inventors: Itkowitz, Brandon D., Zhao, Tao, Dimaio, Simon, Zhao, Wenyi, Hasser, Christopher J., Curet, Myriam J., Mohr, Catherine J., Stein, Hubert
• Primary Examiner(s): Towa, Rene
• Assistant Examiner(s): ABOUELELA, MAY A
• Application Number:
  US12/485,545
• Publication Number:
  US 20100318099A1
• Time in Patent Office: 2,310 Days
• Field of Search: 600/587, 606/130, 700/245, 700/250, 700/264
• US Class Current: 1/1
• CPC Class Codes: A61B 17/00234 for minimally invasive surgery A61B17/0218, A61B17/0469, A61B17/12013, A61B17/1285, A61B17/29, A61B17/320016 take precedence
• CPC Class Codes: A61B 17/282 Jaws
• CPC Class Codes: A61B 18/1482 having a long rigid shaft for accessing the inner body transcutaneously in minimal invasive surgery, e.g. laparoscopy
• CPC Class Codes: A61B 2017/00477 Coupling A61B2017/0046 takes precedence
• CPC Class Codes: A61B 2034/105 Modelling of the patient, e.g. for ligaments or bones
• CPC Class Codes: A61B 2090/061 for measuring dimensions, e.g. length
• CPC Class Codes: A61B 34/25 User interfaces for surgical systems
• CPC Class Codes: A61B 34/30 Surgical robots
• CPC Class Codes: A61B 5/1072 measuring distances on the body, e.g. measuring length, height or thickness A61B5/1076 takes precedence
• CPC Class Codes: A61B 8/4209 by using holders, e.g. positioning frames
• CPC Class Codes: A61B 8/483 involving the acquisition of a 3D volume of data

Abstract

Robotic and/or measurement devices, systems, and methods for telesurgical and other applications employ input devices operatively coupled to tools so as to allow a system user to manipulate tissues and other structures being measured. The system may make use of three dimensional position information from stereoscopic images. Two or more discrete points can be designated in three dimensions so as to provide a cumulative length along a straight or curving structure, an area measurement, a volume measurement, or the like. The discrete points may be identified by a single surgical tool or by distances separating two or more surgical tools, with the user optionally measuring a structure longer than a field of view of the stereoscopic image capture device by walking a pair of tools “hand-over-hand” along the structure. By allowing the system user to interact with the tissues while designating the tissue locations, and by employing imaging data to determine the measurements, the measurement accuracy and ease of measurement may be enhanced.

Claims

1. A method for measuring a length along a tissue structure, the method comprising:

grasping a first three-dimensional (3-D) location on the tissue structure with a first grasping tool in response to a first command received at a first input command device operatively coupled using one or more processors to the first grasping tool so as to allow a system user to manipulate the first grasping tool control console;

grasping a second 3-D location on the tissue structure with a second grasping tool in response to a second command received at a second input command device operatively coupled using the one or more processors to the second grasping tool so as to allow a system user to manipulate the second grasping tool;

configuring the one or more processors to measure, using the one or more processors, three dimensional distances between grasped locations of the tissue structure during hand-over-hand sampling;

with the one or more processors, measuring a first three dimensional distance between the first 3-D location and the second 3-D location so as to provide a first three dimensional distance measurement;

releasing the first grasping tool from grasping the first 3-D location and then grasping a third 3-D location on the tissue structure with the first grasping tool in response to a third command received at the first input command device, the first, second, and third 3-D locations being in sequence along the tissue structure;

with the one or more processors, measuring a second three dimensional distance between the second 3-D location and the third 3-D location so as to provide a second three dimensional distance measurement; and

with the one or more processors, outputting a sum of the measured first and second three dimensional distances;

wherein the tissue structure comprises a tissue structure within a minimally invasive surgical site; and

further comprising;

grasping a sequence of additional tissue structure locations by alternately moving the first and second grasping tools and grasping the tissue structure alternatingly with the first and second grasping tools in response to commands received at the first and second input command devices;

configuring the one or more processors to process stereoscopic image data; and

with the one or more processors, receiving stereoscopic image data comprising left and right stereoscopic images of the tissue structure and of the first and second grasping tools from an image capture device and outputting to a display an image of the tissue structure and of the first and second grasping tools;

wherein the image is outputted in real time in the course of movement of the first and second grasping tools; and

wherein the image includes an image of a stretch line that is superimposed on the image of the tissue structure from a prior grasped tissue location to a currently grasping grasping tool.

2. The method of claim 1, wherein the first 3-D location, the second 3-D location, and the third 3-D location are 3-D tissue locations along the tissue structure, and wherein the 3-D tissue locations are designated by actuation of the grasping tools in response to commands received at the first and second input command devices.

3. The method of claim 2, wherein releasing of the first grasping tool from the first 3-D tissue location while the second grasping tool grasps the second 3-D tissue location designates the second 3-D tissue location, and wherein the designation of the 3-D second tissue location initiates the first three dimensional distance measurement between the first 3-D location and the second 3-D) location.

4. The method of claim 3, wherein releasing of the second grasping tool from the second 3-D tissue location in response to commands received at the second input command device while the first grasping tool grasps the third 3-D tissue location designates the third 3-D tissue location, and wherein the designation of the third 3-D tissue location initiates the second three dimensional distance measurement between the second location and the third location.

5. The method of claim 1, further comprising:

with the one or more processors, outputting to a display, an image capture device field of view including the tissue structure, wherein the third location is not visible in the field of view while measuring the first three dimensional distance between the first location and the second location, and manipulating the tissue structure with the grasping tools or moving the field of view in response to commands received at the first and second input command devices so as to output an image of the third location while measuring the second three dimensional distance between the second location and the third location.

6. The method of claim 1, further comprising with the one or more processors, receiving image data comprising left and right stereoscopic images of the tissue structure from an image capture device, wherein the three dimensional distances are measured by determining three dimensional distances between the first and second locations and the second and third locations using the image data.

7. The method of claim 1, wherein the image includes an image of a stretch line that is superimposed on the image of the tissue structure between the currently grasping tool and a currently moving measurement grasping tool.

8. A method for measuring a length along a tissue structure, the method comprising:

grasping a first three-dimensional (3-D) location on the tissue structure with a first tool in response to a first command received at a first input command device operatively coupled using one or more processors to the first grasping tool so as to allow a system user to manipulate the first grasping tool control console;

grasping a second 3-D location on the tissue structure with a second tool in response to a second command received at a second input command device operatively coupled using the one or more processors to the second grasping tool so as to allow a system user to manipulate the second grasping tool;

configuring the one or more processors to measure, using the one or more processors, three dimensional distances between grasped locations of the tissue structure during hand-over-hand sampling;

with the one or more processors, measuring a first three dimensional distance between the first 3-D location and the second 3-D location so as to provide a first three dimensional distance measurement;

releasing the first grasping tool from grasping the first 3-D location and then grasping a third 3-D location on the tissue structure with the first grasping tool in response to a third command received at the first input command device, the first, second, and third 3-D locations being in sequence along the tissue structure;

with the one or more processors, measuring a second three dimensional distance between the second 3-D location and the third 3-D location so as to provide a second three dimensional distance measurement; and

with the one or more processors, outputting a sum of the measured first and second three dimensional distances;

wherein the tissue structure comprises a tissue structure within a minimally invasive surgical site, and further comprising;

grasping a sequence a plurality of additional tissue structure locations by alternately moving the first and second grasping tools and grasping the tissue structure alternatingly with the first and second grasping tools in response to commands received at the first and second input command devices;

with the one or more processors, measuring a sequence of additional three dimensional distances between a sequence of additional tissue structure 3-D locations so as to provide a sequence of additional three dimensional distance measurements;

wherein the three dimensional distances are each defined by a pair of the locations; and

further comprising;

configuring the one or more processors to process stereoscopic image data, with the one or more processors, during the acts of grasping and releasing with the first and second grasping tools and measuring three-dimensional distances, outputting to a display in real time in the course of movement of the first and second grasping tools;

a) a graphical indicator of a measurement location along each grasping tool, b) a stretch line between a measurement location indicated by the graphical indicator along the first grasping tool and a measurement location indicated by the graphical indicator along the second grasping tool, c) a distance measurement between the first and second grasping tools, and d) a current cumulative measurement including the sum of distance measurements for one or more of the additional three dimensional distances, wherein each of a)-d) is superimposed on an image of the minimally invasive surgical site.

9. A method for measuring a length along a tissue structure, the method comprising:

grasping a first three-dimensional (3-D) location on the tissue structure with a first tool in response to a first command received at a first input command device operatively coupled using one or more processors to the first grasping tool so as to allow a system user to manipulate the first grasping tool control console;

grasping a second 3-D location on the tissue structure with a second tool in response to a second command received at a second input command device operatively coupled using the one or more processors to the second grasping tool so as to allow a system user to manipulate the second grasping tool;

configuring the one or more processors to measure, using the one or more processors, three dimensional distances between grasped locations of the tissue structure during hand-over-hand sampling;

with the one or more processors, measuring a first three dimensional distance between the first 3-D location and the second 3-D location so as to provide a first three dimensional distance measurement;

releasing the first grasping tool from grasping the first 3-D location and then grasping a third 3-D location on the tissue structure with the first grasping tool in response to a third command received at the first input command device, the first, second, and third 3-D locations being in sequence along the tissue structure;

with the one or more processors, measuring a second three dimensional distance between the second 3-D location and the third 3-D location so as to provide a second three dimensional distance measurement; and

with the one or more processors, outputting a sum of the measured first and second three dimensional distances;

with the one or more processors, receiving stereoscopic image data comprising left and right stereoscopic images of the tissue structure and of the first and second grasping tools from an image capture device and outputting to a display an image of the tissue structure and of the first and second grasping tools and of a representation of the measured first and second three dimensional distances;

wherein the image is outputted in real time in the course of movement of the first and second grasping tools;

wherein the tissue structure comprises a tissue structure within a minimally invasive surgical site, and further comprising;

grasping a sequence a plurality of additional tissue structure locations by grasping the tissue structure alternatingly with the first and second grasping tools in response to commands received at the first and second input command devices;

wherein the first 3-D location, the second 3-D location, and the third 3-D location are 3-D tissue locations along the tissue structure;

wherein the 3-D tissue locations are designated by actuation of the grasping tools in response to commands received at the first and second input command devices; and

wherein while the first tool remains grasped, the second tool is allowed to grasp and release multiple times in response to commands received at the first input command device until a desired tissue location is achieved;

wherein a three dimensional distance measurement between the plurality of additional locations is measured by releasing the first tool in response to commands received at the first input command device while the second grasping tool grasps the desired tissue location; and

wherein location designation responsibility between the first grasping tool and the second grasping tool switches when the first grasping tool is released.

10. A method for measuring a length along a tissue structure, the method comprising:

grasping a first three-dimensional (3-D) location on the tissue structure with a first tool in response to a first command received at a first input command device operatively coupled using one or more processors to the first grasping tool so as to allow a system user to manipulate the first grasping tool control console;

grasping a second 3-D location on the tissue structure with a second tool in response to a second command received at a second input command device operatively coupled using the one or more processors to the second grasping tool so as to allow a system user to manipulate the second grasping tool;

configuring the one or more processors to measure, using the one or more processors, three dimensional distances between grasped locations of the tissue structure during hand-over-hand sampling;

with the one or more processors, measuring a first three dimensional distance between the first 3-D location and the second 3-D location so as to provide a first three dimensional distance measurement;

releasing the first grasping tool from grasping the first 3-D location and then grasping a third 3-D location on the tissue structure with the first grasping tool in response to a third command received at the first input command device, the first, second, and third 3-D locations being in sequence along the tissue structure;

with the one or more processors, measuring a second three dimensional distance between the second 3-D location and the third 3-D location so as to provide a second three dimensional distance measurement; and

with the one or more processors, outputting a sum of the measured first and second three dimensional distances;

with the one or more processors, receiving stereoscopic image data comprising left and right stereoscopic images of the tissue structure and of the first and second grasping tools from an image capture device and outputting to a display an image of the tissue structure and of the first and second grasping tools and of a representation of the measured first and second three dimensional distances;

wherein the image is outputted in real time in the course of movement of the first and second grasping tools;

wherein the tissue structure comprises a tissue structure within a minimally invasive surgical site, and further comprising;

grasping a sequence a plurality of additional tissue structure locations by grasping the tissue structure alternatingly with the first and second grasping tools in response to commands received at first and second input command devices;

further comprising;

manipulating the tissue structure with the grasping tools in response to commands received at the first and second input command devices so as to straighten or stretch a length of the tissue structure when an associated three dimensional distance between the first and second grasping tools is measured.

11. A system to measure a length along a tissue structure, the system comprising:

a manipulator coupled using one or more processors to manipulate a first grasping tool in response to movement commands and coupled using the one or more processors to manipulate a second grasping tool in response to movement commands;

the one or more processors are configured to;

receive a first command to grasp a first 3-D location on the tissue structure with the first grasping tool and in response to receiving the first command, to determine and transmit an associated first instruction to a manipulator system coupled with the first grasping tool;

receive a second command to grasp a second 3-D location on the tissue structure with the second grasping tool and in response to receiving the second command, to determine and transmit an associated second instruction to the manipulator system coupled with the second grasping tool;

measure, using the one or more processors, a first three dimensional distance between the first 3-D location and the second 3-D location so as to provide a first three dimensional distance measurement; and

couple the grasping tools to an output such that an input command from a user induces the one or more processors to sum three-dimensional distances between the 3-D locations so as to measure a hand-over-hand length along the tissue structure;

wherein the one or more processors designates the second 3-D tissue location, currently grasped by the second grasping tool as a grasping location, in response to commands to release jaws of the first grasping tool from the first 3-D location; and

wherein the one or more processors adds the first three dimensional distance measurement between the first 3-D location and the second 3-D location to the hand-over-hand length after designating the second tissue 3-D location as the grasping location so that the first grasping tool may be commanded to grasp and release a plurality of candidate third tissue 3-D locations without altering the hand-over-hand length; and

the one or more processors configured to receive stereoscopic image data comprising left and right stereoscopic images of the tissue structure and of the first and second grasping tools from an image capture device and outputting to a display an image of the tissue structure and of the first and second grasping tools and of a representation of the measured first and second three dimensional distances;

wherein the image is outputted in real time in the course of movement of the first and second grasping tools.

12. A system to measure a length along a structure, the system comprising:

a manipulator coupled using one or more processors to manipulate a first grasping tool in response to movement commands and coupled using the one or more processors to manipulate a second grasping tool in response to movement commands;

the one or more processors configured to;

receive a first command to grasp a first 3-D location on the tissue structure with the first grasping tool and in response to receiving the first command, to determine and transmit an associated first instruction to a manipulator system coupled with the first grasping tool;

receive a second command to grasp a second 3-D location on the tissue structure with the second grasping tool and in response to receiving the second command, to determine and transmit an associated second instruction to the manipulator system coupled with the second grasping tool;

measure, using the one or more processors, a first three dimensional distance between the first 3-D location and the second 3-D location so as to provide a first three dimensional distance measurement; and

couple the grasping tools to an output such that an input command from a user induces the one or more processors to sum three dimensional distances between the 3-D locations so as to measure a hand-over-hand length along the tissue structure;

wherein the one or more processors is configured to defer a summation of three dimensional distances until a 3-D location is indicated per a command from the user so as to manipulate the tissue structure with the grasping tools during measurement of the hand-over-hand length and allow the grasping tools to straighten or stretch a measured length of the tissue structure when an associated distance between the first and second grasping tools is measured; and

the one or more processors configured to receive stereoscopic image data comprising left and right stereoscopic images of the tissue structure and of the first and second grasping tools from an image capture device and outputting to a display an image of the tissue structure and of the first and second grasping tools and of a representation of the measured first and second three dimensional distances;

wherein the image is outputted in real time in the course of movement of the first and second grasping tools.

13. A system to measure a length along a structure, the system comprising:

a manipulator coupled using one or more processors to manipulate a first grasping tool in response to movement commands and coupled using the one or more processors to manipulate a second grasping tool in response to movement commands;

the one or more processors configured to;

receive a first command to grasp a first 3-D location on the tissue structure with the first grasping tool and in response to receiving the first command, to determine and transmit an associated first instruction to a manipulator system coupled with the first grasping tool;

receive a second command to grasp a second 3-D location on the tissue structure with the second grasping tool and in response to receiving the second command, to determine and transmit an associated second instruction to the manipulator system coupled with the second grasping tool;

measure, using the one or more processors, a first three dimensional distance between the first 3-D location and the second 3-D location so as to provide a first three dimensional distance measurement; and

couple the grasping tools to an output such that an input command from a user induces the processor to sum three dimensional distances between the 3-D locations so as to measure a hand-over-hand length along the tissue structure;

wherein the one or more processors is further configured to output an image capture device field of view to a display;

wherein the one or more processors is configured to receive and execute commands for articulation of the tools to grasp and manipulate the structure so as to bring a third 3-D location on the tissue structure into the field of view, and after the third 3-D location on the structure is in the field of view, to determine a distance between the second 3-D location and the third 3-D location; and

the one or more processors configured to receive stereoscopic image data comprising left and right stereoscopic images of the tissue structure and of the first and second grasping tools from an image capture device and outputting to a display an image of the tissue structure and of the first and second grasping tools and of a representation of the measured first and second three dimensional distances;

wherein the image is outputted in real time in the course of movement of the first and second grasping tools.

14. A system to measure a length along a structure, the system comprising:

a manipulator coupled using one or more processors to manipulate a first grasping tool in response to movement commands and coupled using the one or more processors to manipulate a second grasping tool in response to movement commands;

the one or more processors configured to;

receive a first command to grasp a first 3-D location on the tissue structure with the first grasping tool and in response to receiving the first command, to determine and transmit an associated first instruction to a manipulator system coupled with the first grasping tool;

receive a second command to grasp a second 3-D location on the tissue structure with the second grasping tool and in response to receiving the second command, to determine and transmit an associated second instruction to the manipulator system coupled with the second grasping tool;

measure, using the one or more processors, a first three dimensional distance between the first 3-D location and the second 3-D location so as to provide a first three dimensional distance measurement; and

couple the grasping tools to an output such that an input command from a user induces the one or more processors to sum three dimensional distances between the 3-D locations so as to measure a hand-over-hand length along the tissue structure;

wherein the one or more processors is configured to determine and transmit instructions to the manipulator system to grasp and release multiple times using the second grasping tool, in response to received commands, until a desired tissue 3-D location is achieved;

wherein a distance measurement between the additional 3-D locations is measured in response to release of the first grasping tool while the second tool grasps the desired 3-D tissue location; and

wherein 3-D location designation responsibility between the first grasping tool and the second grasping tool switches in response to release of the first grasping tool, and the one or more processors configured to receive stereoscopic image data comprising left and right stereoscopic images of the tissue structure and of the first and second grasping tools from an image capture device and outputting to a display an image of the tissue structure and of the first and second grasping tools and of a representation of the measured first and second three dimensional distances;

wherein the image is outputted in real time in the course of movement of the first and second grasping tools.