Surgical system with haptic feedback based upon quantitative three-dimensional imaging

• US 10,555,788 B2
• Filed: 09/28/2016
• Issued: 02/11/2020
• Est. Priority Date: 03/28/2014
• Status: Active Grant

• Current Assignee: Intuitive Surgical Operations, Inc. (Intuitive Surgical Holdings, LLC)
• Original Assignee: Intuitive Surgical Operations, Inc. (Intuitive Surgical Holdings, LLC)
• Inventors: Panescu, Dorin, Jones, Daniel H., Allenby, Christopher
• Primary Examiner(s): Leubecker, John P
• Assistant Examiner(s): Ghimire, Shankar Raj
• Application Number:
  US15/300,263
• Publication Number:
  US 20170181808A1
• Time in Patent Office: 1,231 Days
• CPC Class Codes: A61B 1/00055 for alerting the user
• CPC Class Codes: A61B 1/044 for absorption imaging
• CPC Class Codes: A61B 1/06 with illuminating arrangements
• CPC Class Codes: A61B 2017/00119 alarm; indicating an abnormal situation
• CPC Class Codes: A61B 2034/2059 Mechanical position encoders
• CPC Class Codes: A61B 34/10 Computer-aided planning, simulation or modelling of surgical operations
• CPC Class Codes: A61B 34/20 Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
• CPC Class Codes: A61B 34/30 Surgical robots
• CPC Class Codes: A61B 34/32 operating autonomously
• CPC Class Codes: A61B 34/37 Master-slave robots A61B34/35 takes precedence
• CPC Class Codes: A61B 34/76 Manipulators having means for providing feel, e.g. force or tactile feedback

Abstract

A system is provided to provide haptic feedback during a medical procedure comprising: a quantitative three-dimensional (Q3D) endoscope; a surgical instrument disposed to deform a tissue structure; a haptic user interface device configured to provide an indication of tissue structure deformation in response to information indicative of the measure of tissue structure deformation; and a processor configured to produce a Q3D model that includes information indicative of a measure of tissue structure deformation and to provide the information indicative of the measure of tissue structure deformation to the haptic user interface device.

Claims

1. A system to provide haptic feedback during a medical procedure comprising:

a surgical instrument;

a quantitative three-dimensional (Q3D) endoscope having an imaging sensor array comprising at least three coplanar imaging sensors having coplanar overlapping fields of view, wherein each imagining sensor includes a pixel array that is separate from the pixel arrays of other imaging sensors, wherein the Q3D endoscope is operable to capture Q3D information indicative of deformation distance of different locations of a tissue structure placed in contact with the surgical instrument within the fields of view;

wherein the surgical instrument is operable to move across a tissue surface structure to contact and impart a force to deform different tissue surface locations of the tissue structure placed within the overlapping fields of view;

at least one processor configured to produce a Q31) model that provides a map of measures of tissue structure deformation for the contact between the instrument and the tissue surface at the different locations of the tissue structure surface, based upon the captured Q3D information; and

a haptic user interface device configured to produce haptic feedback indicative of the map of the measures of tissue structure deformation at the contact locations of the tissue structure surface.

2. The system of claim 1,

wherein the haptic user interface device includes a shape display.

3. The system of claim 2,

wherein the haptic user interface shape display is configured to:

provide a three dimensional contour that models at least a portion of the tissue structure, and

provide an indication of tissue structure deformation at a location on a tissue structure surface in response to information indicative of a measure of tissue structure deformation at the tissue structure location.

4. The system of claim 1,

wherein the haptic user interface device abuts against hand or finger grips.

5. The system of claim 1,

wherein the haptic user interface device includes a surgical instrument manipulation control device.

6. The system of claim 1 further including:

an alarm source configured to produce an alarm in response to the processor providing information indicative of a measure of tissue structure deformation that is outside of a clinically safe range.

7. The system of claim 6,

wherein the alarm source includes an audible alarm source and the alarm includes an audible alarm.

8. The system of claim 6,

wherein the alarm source includes a visual display and the alarm includes a visual alarm.

9. The system of claim 1 further including:

a force sensor associated with the surgical instrument to measure a force imparted to the tissue structure during deformation of the tissue structure; and

a visual display configured to provide a visual indication of force measured by the force sensor.

10. The system of claim 1,

wherein the haptic user interface shape display is configured to,

provide a three dimensional contour that models at least a portion of the tissue structure, and

provide an indication of tissue structure deformation at a location on a tissue structure surface in response to information indicative of a measure of tissue structure deformation at the tissue structure location.

11. A system for determining force exerted onto tissue during medical procedures comprising:

a quantitative three-dimensional (Q3D) endoscope having an imaging sensor array comprising at least three coplanar imaging sensors having coplanar overlapping fields of view, wherein each imagining sensor including a pixel array that is separate from the pixel arrays of other imaging sensors, wherein the Q3D endoscope is operable to capture Q3D information indicative of deformation distance of different locations of a tissue structure placed in contact with a surgical instrument within the fields of view;

wherein the surgical instrument is operable to move across a tissue surface structure to contact and impart a force to deform different tissue surface locations of a tissue structure placed within the overlapping fields of view; and

at least one processor configured to produce a Q3D model that includes information indicative of deformation distance of the different locations of the tissue structure deformation, based upon the captured Q3D information and to determine a force exerted by the instrument onto said tissue structure at at least one measured location of tissue structure deformation based upon, at least in part, a measure of tissue deformation at the at least one measured location of tissue structure deformation.

12. The system of claim 11,

wherein the processor produces an output indicative of said determined force.

13. The system of claim 12,

wherein the output drives a display or a haptic feedback interface device.

14. A system for providing virtual tissue palpation during medical procedures comprising:

a surgical instrument;

a quantitative three-dimensional (Q3D) endoscope having an imaging sensor array comprising at least three coplanar imaging sensors having coplanar overlapping fields of view, wherein each imagining sensor including a pixel array that is separate from the pixel arrays of other imaging sensors, wherein the Q3D endoscope is operable to capture Q3D information indicative of deformation distance of different locations of a tissue structure placed in contact with the surgical instrument within the fields of view;

wherein the surgical instrument is operable to move across a tissue surface structure to contact and impart a force to deform different tissue surface locations of a tissue structure placed within the overlapping fields of view; and

at least one processor configured to produce a Q31) model that includes information indicative of a measure of tissue structure deformation, based upon the captured Q3D information, to determine the tissue stiffness at said locations, and to provide a virtual palpation output based at least in part upon said determined tissue stiffness.

15. The system of claim 14,

wherein the virtual palpation output drives a display or a haptic feedback interface device.

16. The system of claim 15 further including:

a processor configured to automatically drive the surgical instrument to a series of locations within the overlapping fields of view.

17. The system of claim 15 further including:

means configured to provide an alert in response to the virtual palpation output exceeding a safe range, or increases or decreases with respect to a safe threshold.