Virtual measurement tool for minimally invasive surgery

• US 9,492,240 B2
• Filed: 06/16/2009
• Issued: 11/15/2016
• Est. Priority Date: 06/16/2009
• Status: Active Grant

• Current Assignee: Intuitive Surgical Operations, Inc. (Intuitive Surgical Holdings, LLC)
• Original Assignee: Intuitive Surgical Operations, Inc. (Intuitive Surgical Holdings, LLC)
• Inventors: Itkowitz, Brandon D., Zhao, Tao, Di Maio, Simon, Zhao, Wenyi, Hasser, Christopher J., Curet, Myrian J., Mohr, Catherine J., Stein, Hubert
• Primary Examiner(s): Fernandez, Katherine
• Assistant Examiner(s): Negarestan, Farshad
• Application Number:
  US12/485,503
• Publication Number:
  US 20100317965A1
• Time in Patent Office: 2,709 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: A61B 1/00193 adapted for stereoscopic vision
• CPC Class Codes: A61B 2017/00477 Coupling A61B2017/0046 takes precedence
• CPC Class Codes: A61B 2034/2065 Tracking using image or pattern recognition
• CPC Class Codes: A61B 2090/061 for measuring dimensions, e.g. length
• CPC Class Codes: A61B 34/25 User interfaces for surgical systems
• CPC Class Codes: A61B 34/30 Surgical robots
• CPC Class Codes: A61B 34/35 for telesurgery
• CPC Class Codes: A61B 34/37 Master-slave robots A61B34/35 takes precedence
• CPC Class Codes: A61B 90/06 Measuring instruments not otherwise provided for
• CPC Class Codes: A61B 90/361 Image-producing devices, e.g. surgical cameras

Abstract

Robotic and/or measurement devices, systems, and methods for telesurgical and other applications employ input devices operatively coupled to tools so as to allow a system user to manipulate tissues and other structures being measured. The system may make use of three dimensional position information from stereoscopic images. Two or more discrete points can be designated in three dimensions so as to provide a cumulative length along a straight or curving structure, an area measurement, a volume measurement, or the like. The discrete points may be identified by a single surgical tool or by distances separating two or more surgical tools, with the user optionally measuring a structure longer than a field of view of the stereoscopic image capture device by walking a pair of tools “hand-over-hand” along the structure. By allowing the system user to interact with the tissues while designating the tissue locations, and by employing imaging data to determine the measurements, the measurement accuracy and ease of measurement may be enhanced.

Claims

1. A surgical system for measuring tissue, the system comprising:

a first robotic tool having a tool tip;

a stereoscopic image capture device orientable toward a surgical site for capturing left and right stereoscopic images of the tissue at the surgical site and capturing left and right stereoscopic images of the first robotic tool, the image capture device in use generating stereoscopic image data corresponding to the left and right stereoscopic images of the tissue at the surgical site and the first robotic tool;

a stereoscopic display coupled to the image capture device so as to show the captured left and right stereoscopic images of the tissue at the surgical site and the captured left and right stereoscopic images of the first robotic tool to a system user;

an input for designating tissue locations indicated by the first robotic tool, wherein the input is coupled to a master controller coupled with the first robotic tool for receiving movement commands from the system user to guide the first robotic tool to designate a first tissue location, with the first robotic tool tip disposed over the first tissue location as seen from a dominant eye of the system user, without touching the first tissue location, the first tissue location having one or more first points of interest within both of the left and right stereoscopic images of the tissue at the surgical site shown on the stereoscopic display; and

a processor configured to track Cartesian position of the first robotic tool tip and to match an image of the one or more points of interest within one of the left and right stereoscopic images of the tissue at the surgical site to an image of the one or more points of interest within the other of the left and right stereoscopic images of the tissue at the surgical site, based upon local characteristics of the one or more points of interest, so as to determine a three-dimensional location of the tissue that corresponds to the matched one or more points of interest, wherein the three-dimensional location corresponds to the first designated tissue location indicated by the first robotic tool, and to superimpose a first graphical indicator on the stereoscopic images of the tissue at the determined three-dimensional location of the tissue that corresponds to the matched points of interest, without touching the first designated tissue location, the processor being further configured for generating a measurement between at least the first graphical indicator and a second designated tissue location;

wherein the first graphical indicator is superimposed on the stereoscopic images of the tissue at the determined three-dimensional location that corresponds to the matched one or more points of interest, after the first robotic tool is moved to designate the first designated tissue location.

2. The system of claim 1, wherein the processor superimposes a graphical tool measurement marker on the stereoscopic images of the tissue at the surgical site coincident with the first robotic tool while the user guides the first robotic tool to the first designated tissue location.

3. The system of claim 1, wherein the processor is configured to receive tissue designating commands sequentially from the system user so as to facilitate indication of the second designated tissue location with the first robotic tool.

4. The system of claim 1, wherein the processor is configured to receive a tissue designating command from the system user so as to simultaneously indicate the first designated tissue location with the first robotic tool and the second designated tissue location with a second robotic tool.

5. The system of claim 1, wherein the processor couples the input to the stereoscopic display so as to superimpose a moving line extending between the first designated tissue location and the first robotic tool while the system user guides the first robotic tool to the second designated tissue location.

6. The system of claim 5, wherein the processor is configured to determine an offset display location such that the measurement is displayed superimposed on the stereoscopic images of the tissue and separated from the moving line.

7. The system of claim 1, wherein the processor couples the input to the stereoscopic display so as to receive additional input designation commands so as to define an enclosed area, wherein the measurement comprises an area measurement.

8. The system of claim 7, wherein the first designated tissue location and the second designated tissue location comprise discrete locations bordering the enclosed area, with each of the first designated tissue location and the second designated tissue location having an associated location designation.

9. The system of claim 1, wherein the processor determines a three dimensional location of a tissue surface based on the stereoscopic image data, and wherein the processor designates the first designated tissue location by snapping the graphical indicator from the first robotic tool tip to the tissue surface.

10. The system of claim 1, wherein the tissue comprises a physiologically moving tissue, wherein the left and right stereoscopic images of the tissue comprises still stereoscopic images of the moving tissue, the system further comprising a three dimensional input device coupled to the display by the processor so that a three dimensional graphical indicator superimposed on the still stereoscopic images of the moving tissue is positioned using the input device so as to indicate the second designated tissue location.

11. The system of claim 1, wherein the processor derives the measurement at least in part from kinematic data of the first robotic tool.

12. The system of claim 1, wherein the first graphical indicator is configured to appear at a same depth as the tissue at the surgical site.

13. The system of claim 1, wherein the processor is configured to perform region matching to match the image of the one or more points of interest within the one of the left and right stereoscopic images of the tissue at the surgical site to the image of the one or more points of interest within the other of the left and right stereoscopic images of the tissue at the surgical site, based upon local characteristics of the images of the one or more points of interest.

14. The system of claim 1, wherein the processor configured to perform feature matching to match the image of the one or more points of interest within the one of the left and right stereoscopic images of the tissue at the surgical site to the image of the one or more points of interest within the other of the left and right stereoscopic images of the tissue at the surgical site, based upon local characteristics of the images of the one or more points of interest.

15. The system of claim 1, wherein the processor configured to perform feature interpolation to match the image of the one or more first of interest within the one of the left and right stereoscopic images of the tissue at the surgical site to the image of the one or more points of interest within the other of the left and right stereoscopic images of the tissue at the surgical site, based upon local characteristics of the images one or more points of interest.

16. A robotic system for measuring tissue, the system comprising:

a first robotic tool having a tool tip;

a stereoscopic image capture device orientable toward a surgical site for capturing left and right stereoscopic images of the tissue at the surgical site and capturing left and right stereoscopic images of the first robotic tool, the image capture device in use generating stereographic image data corresponding to the left and right stereoscopic images of the tissue at the surgical site and the first robotic tool;

a stereoscopic display coupled to the image capture device so as to show the captured left and right stereoscopic images of the tissue and the captured left and right stereoscopic images of the first robotic tool to a system user;

an input for designating tissue locations indicated by the first robotic tool, wherein the input is coupled to a master controller coupled with the first robotic tool for receiving movement commands from the system user to guide the first robotic tool to designate a first tissue location, with the first robotic tool disposed over the first tissue location as seen from a dominant eye of the system user, without touching the tissue location, the first tissue location having one or more points of interest within both of the left and right stereoscopic images of the tissue at the surgical site shown on the stereoscopic display; and

a processor configured to track Cartesian position of the first robotic tool tip, and to match an image of the one or more points of interest within one of the left and right stereoscopic images of the tissue at the surgical site to an image of the one or more points of interest within the other of the left and right stereoscopic images of the tissue at the surgical site, based upon local characteristics of the image of the one or more points of interest, so as to determine a three-dimensional location of tissue that corresponds to the matched one or more points of interest, wherein the three-dimensional location corresponds to the first designated tissue location indicated by the first robotic tool and to superimpose a first graphical indicator on the stereoscopic images of the tissue at the determined three-dimensional location that corresponds to the matched one or more points of interest, without touching the first designated tissue location, the processor further configured for generating a measurement between at least the first graphical indicator and a second tissue location,

wherein the first graphical indicator is superimposed on the stereoscopic images of the tissue at the determined three-dimensional location after the first robotic tool is moved to designate the first designated tissue location; and

wherein the processor couples the input to the stereoscopic display so as to receive additional input designation commands indicating additional tissue locations, so as to determine a plurality of three dimensional offset distances between sequential pairs of tissue locations, and so as to sum the three dimensional offset distances as a cumulative measurement value and superimpose the cumulative measurement value on the stereoscopic images of the tissue at the surgical site.

17. The system of claim 16, further comprising a second robotic tool, wherein the first robotic tool and the second robotic tool each comprises at least one jaw, wherein the movement commands comprises a jaw articulation input for the at least one jaw jaws so that the processor is configured to determine the plurality of three dimensional offset distances by summing offset distances in a hand-over-hand length measurement along the tissue.