Efficient vision and kinematic data fusion for robotic surgical instruments and other applications
First Claim
1. A robotic-method for performing surgery on a patient through a minimally invasive aperture, the method comprising:
- positioning, by a robotic surgical system, an elongate shaft of a tool through the minimally invasive aperture so that the shaft pivots at an actual pivotal center adjacent the minimally invasive aperture;
acquiring, by an image capture device, an image of a surgical site within the patient, the surgical site image including an image of a distal portion of the tool within the field of view of the image capture device, the image of the distal portion including an image of markers, the image capture device being inserted into the patient;
determining, by the robotic surgical system using the image of the markers, an image-based location of the actual pivotal center;
determining image-based pivotal center correction information using the image-based location of the actual pivotal center; and
determining, by the robotic surgical system, a position and/or orientation of the tool by imposing the image-based pivotal center correction information onto joint-based data, the joint-based data being used by a command module to calculate movements of a tool manipulator supporting the tool.
2 Assignments
0 Petitions
Accused Products
Abstract
Robotic devices, systems, and methods for use in telesurgical therapies through minimally invasive apertures make use of joint-based data throughout much of the robotic kinematic chain, but selectively rely on information from an image capture device to determine location and orientation along the linkage adjacent a pivotal center at which a shaft of the robotic surgical tool enters the patient. A bias offset may be applied to a pose (including both an orientation and a location) at the pivotal center to enhance accuracy. The bias offset may be applied as a simple rigid transformation from the image-based pivotal center pose to a joint-based pivotal center pose.
-
Citations
32 Claims
-
1. A robotic-method for performing surgery on a patient through a minimally invasive aperture, the method comprising:
-
positioning, by a robotic surgical system, an elongate shaft of a tool through the minimally invasive aperture so that the shaft pivots at an actual pivotal center adjacent the minimally invasive aperture; acquiring, by an image capture device, an image of a surgical site within the patient, the surgical site image including an image of a distal portion of the tool within the field of view of the image capture device, the image of the distal portion including an image of markers, the image capture device being inserted into the patient; determining, by the robotic surgical system using the image of the markers, an image-based location of the actual pivotal center; determining image-based pivotal center correction information using the image-based location of the actual pivotal center; and determining, by the robotic surgical system, a position and/or orientation of the tool by imposing the image-based pivotal center correction information onto joint-based data, the joint-based data being used by a command module to calculate movements of a tool manipulator supporting the tool. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14)
-
-
15. A robotic method for performing surgery on a patient through a minimally invasive aperture with a robotic system, the robotic system including a tool having an elongate shaft and a command module that determines movements of a tool manipulator supporting the tool by determining a joint-based pivotal center pose of the tool, the method comprising:
-
positioning, by the robotic system, the elongate shaft of the tool through the minimally invasive aperture so that the shaft pivots at an actual pivotal center adjacent the minimally invasive aperture, the actual pivotal center and the shaft defining an actual pivotal center pose; acquiring, by an image capture device of the robotic system, an image of a surgical site within the patient, the surgical site image including an image of a distal portion of the tool within the field of view of the image capture device, the image of the distal portion of the tool including an image of markers, the image capture device being inserted into the patient; determining, by the robotic system using the image of the markers, an image-based pose of the actual pivotal center pose; calculating, by the robotic system, a rigid transformation between the image-based pose of the actual pivotal center pose and the joint-based pivotal center pose using a recursive filter; and determining, by the robotic system, a location and/or orientation of the tool by using the rigid transformation to mitigate an error between the joint-based pivotal center pose and the actual pivotal center pose.
-
-
16. A robotic method for use with a robotic system including a manipulator movably supporting a tool and a command module generating movement commands of the manipulator so as to generate desired movements of the tool based on joint data from the manipulator, wherein environmental constraints induce an error in a joint-based pivotal center pose of the tool, the joint-based pivotal center pose being calculated by the command module along a linkage of the manipulator or tool, the joint-based pivotal center pose being defined by a joint-based location of an actual pivotal center and an orientation of a shaft of the tool per joint data, the shaft of the tool entering the patient through an incision in a patient, the shaft pivoting at the actual pivotal center, the actual pivotal center being adjacent the incision, the method comprising:
-
acquiring, by the robotic system, an image of a surgical site within the patient, the surgical site image including an image of a distal portion of the tool within the field of view of the image capture device, the image of the distal portion of the tool including an image of markers, the image capture device being inserted into the patient selectively determining, by the robotic system using the image of the markers, an image-based pivotal center pose of the tool reflecting the environmental constraints, the image-based pivotal center pose of the tool being defined by an image-based location of the actual pivotal center and an orientation of the shaft per the image of the distal portion of the tool; calculating, by the robotic system, a rigid transformation between the image-based pivotal center pose and the joint-based pivotal center pose; and determining, by the robotic system, a location of the actual pivotal center and orientation of the shaft of the tool by using the rigid transformation so as to mitigate the error in the joint-based pivotal center pose of the tool induced by the environmental constraints. - View Dependent Claims (17, 18)
-
-
19. A robotic system for performing surgery on a patient through a minimally invasive aperture, the system comprising:
-
a tool having a proximal end, a distal end, and an elongate shaft, the distal end of the tool being configured to be insertable through the minimally invasive aperture, the distal end of the tool including markers, and the shaft being configured to pivot at a pivotal center adjacent the minimally invasive aperture; an image capture device, the image capture device being configured to be insertable in the patent, the image capture device being configured to acquire an image of a surgical site within the patient, the surgical site image including an image of a distal end of the tool within the field of view of the image capture device, the image of the distal end of the tool including an image of markers; a tool manipulator supporting the proximal end of the tool; and a processor system coupling the image capture device to the tool manipulator, the processor system including a tool correction module and a command module, the command module being coupled to the tool manipulator, the command module being configured to transmit tool movement commands to the tool manipulator, the command module being configured to calculate a joint-based pivotal center in response to joint signals from the tool manipulator, the tool correction module being configured to generate a corrected tool position and/or orientation by determining an image-based location of the pivotal center using the image of the markers, by determining image-based pivotal center correction information using the image-based location of the pivotal center, and by correcting the joint-based pivotal center with the image-based pivotal center correction information.
-
-
20. A correction system comprising:
a tool correction module coupling an image capture device of a robotic surgical system to a command module of the robotic surgical system, the robotic surgical system comprising a tool having a proximal end, a distal end, and an elongate shaft, the distal end of the tool being configured to be insertable through a minimally invasive aperture in a patient, the distal end of the tool including markers, and the shaft being configured to pivot at a pivotal center adjacent the minimally invasive aperture;
the image capture device being configured to be insertable in the patent and the image capture device being configured to acquire an image of a surgical site within the patient, the surgical site image including an image of a distal end of the tool within the field of view of the image capture device, the image of the distal end of the tool including an image of markers;
a tool manipulator supporting the proximal end of the tool; and
a processor system including the command module coupled to the tool manipulator, the command module being configured to transmit tool movement commands to the tool manipulator, and the command module being configured to calculate a joint-based location of the pivotal center in response to joint signals; andthe tool correction module being configured to determine a position and/or orientation of the tool by determining an image-based location of the pivotal center using the image of the markers, by determining image-based pivotal center correction information using the image-based location of the pivotal center, and by correcting the joint-based location of the pivotal center with the image-based pivotal center correction information such that an error between the joint-based pivotal center and the actual pivotal center is mitigated. - View Dependent Claims (21, 22, 23, 24, 25, 26, 27, 28)
-
29. A robotic system comprising:
-
a manipulator movably supporting a tool having an elongate shaft; a command module generating movement commands of the manipulator so as to generate desired movements of the tool based on joint data from the manipulator, wherein environmental constraints induce an error in a joint-based pivotal center pose calculated by the command module along a linkage of the manipulator or tool, the joint-based pivotal center pose being defined by a joint-based location of an actual pivotal center and an orientation of a shaft of the tool per joint data, the shaft of the tool entering the patient through an incision in a patient, the shaft pivoting at the actual pivotal center, the actual pivotal center being adjacent the incision; an image capture device for acquiring an image of a surgical site within the patient, the surgical site image including an image of a distal portion of the tool within the field of view of the image capture device, the image of the distal portion of the tool including an image of markers, if the image capture device is inserted into the patient; means for image-based correction coupling the image capture device to the command module, the correction means comprising means for determining an image-based pivotal center pose of the tool from the image of the markers, the image-based pivotal center pose reflecting the environmental constraints, the image-based pivotal center pose of the tool being defined by the image-based pivotal center location and an orientation of the shaft per the image of the distal portion of the tool, means for calculating a rigid transformation between the image based pivotal center pose of the tool and the joint-based pivotal center pose, and means for using the rigid transformation to mitigate the error in the joint-based pivotal center pose of the tool induced by the environmental constraints. - View Dependent Claims (30, 31, 32)
-
Specification