Robotic visualization and collision avoidance

• US 10,413,373 B2
• Filed: 08/16/2016
• Issued: 09/17/2019
• Est. Priority Date: 08/16/2016
• Status: Active Grant

• Current Assignee: Cilag Gmbh International (Johnson & Johnson)
• Original Assignee: Ethicon, LLC (Johnson & Johnson)
• Inventors: Yates, David C., Harris, Jason L., Shelton, IV, Frederick E., Eckert, Chad E.
• Primary Examiner(s): Sample, Jonathan L
• Application Number:
  US15/237,902
• Publication Number:
  US 20180049829A1
• Time in Patent Office: 1,127 Days
• Field of Search: 700245-264
• CPC Class Codes: A61B 17/07207 the staples being applied sequentially
• CPC Class Codes: A61B 17/32 Surgical cutting instruments A61B18/042 takes precedence; suture cutters A61B17/0467; instruments for ligaturing or cutting A61B17/128; instruments for rupturing the amniotic membrane A61B17/4208; specially adapted knives for eye surgery A61F9/0133
• CPC Class Codes: A61B 2017/00017 Electrical control of surgical instruments
• CPC Class Codes: A61B 2017/00119 alarm; indicating an abnormal situation
• CPC Class Codes: A61B 2017/07271 characterised by its cartridge
• CPC Class Codes: A61B 2017/07278 characterised by its sled or its staple holder
• CPC Class Codes: A61B 2017/07285 characterised by its cutter
• CPC Class Codes: A61B 2090/061 for measuring dimensions, e.g. length
• CPC Class Codes: A61B 2090/0811 for the position of a particular part of an instrument with respect to the rest of the instrument, e.g. position of the anvil of a stapling instrument
• CPC Class Codes: A61B 2090/3937 Visible markers
• CPC Class Codes: A61B 34/30 Surgical robots
• CPC Class Codes: A61B 34/37 Master-slave robots A61B34/35 takes precedence
• CPC Class Codes: A61B 90/06 Measuring instruments not otherwise provided for

Abstract

Methods and devices are provided for robotic surgery, and in particular for controlling various motions of a tool based on visual indicators. In general, a surgical tool can include an elongate shaft and an end effector coupled to a distal end of the elongate shaft and including first and second jaws. The tool can have at least one visual indicator disposed thereon and configured to indicate a size, position, or speed of movement of the tool or components of the tool.

Claims

1. A surgical robotic system, comprising:

an electromechanical arm configured for movement in multiple axes;

an electromechanical tool configured to couple to the electromechanical arm and having an elongate shaft with an end effector coupled to a distal end thereof, the end effector having a cutting element configured to be advanceable therethrough;

a controller configured to control movement of the electromechanical arm and to control actuation of the end effector on the electromechanical tool;

wherein the electromechanical tool includes a first visual indicator thereon indicative of a length scale, the cutting element includes a second visual indicator thereon, the first and second visual indicators are effective to allow an action of the cutting element to be visually measured relative to the electromechanical tool, and the controller is configured to modify the action of the cutting element based on the visually measured action.

2. The system of claim 1, wherein the second visual indicator is configured to indicate a travel speed of the cutting element.

3. The system of claim 1, wherein the first visual indicator comprises a plurality of markings spaced apart along at least one of the shaft and the end effector.

4. The system of claim 1, wherein the first visual indicator comprises a grid pattern formed on the end effector.

5. The system of claim 1, wherein the end effector includes a plurality of cut-outs formed in the end effector that enable viewing of the second visual indicator on the cutting element therethrough as the cutting element is advanced through the end effector.

6. The system of claim 5, wherein second visual indicator includes a grid pattern formed therein that is visible through the plurality of cut-outs.

7. The system of claim 1, wherein the electromechanical tool includes a housing on a proximal end of the shaft that couples to a tool driver on a distal end of the electromechanical arm, the tool driver including a plurality of motors for actuating the end effector.

8. The system of claim 1, further comprising at least one camera operatively coupled to the controller and configured to visually measure the action of the cutting element.

9. The system of claim 2, wherein the controller is configured to compare the travel speed of the cutting element to an expected speed, and is configured to modify the action of the cutting element based on a difference between the travel and expected speeds.

10. The system of claim 2, wherein the controller is configured to compare the travel speed of the cutting element to an expected speed, and is configured to estimate a thickness of tissue grasped by the end effector and through which the cutting element transects based on a difference between the travel and expected speeds.

11. The system of claim 1, wherein the second visual indicator is configured to indicate an actual position of the cutting element relative to the electromechanical tool;

and

wherein the controller is configured to compare the actual position of the cutting element to an expected position, and is configured to modify the action of the cutting element based on a difference between the actual and expected positions.

12. The system of claim 1, further comprising a proximity zone at a distal end of the end effector, wherein the controller is configured to reduce a speed of advancement of the cutting element through the end effector when the cutting element reaches the proximity zone.

13. A surgical robotic system, comprising:

an electromechanical arm configured for movement in multiple axes;

an electromechanical tool removably coupled to the electromechanical arm, the electromechanical tool having an elongate shaft and an end effector, a first visual indicator being positioned on the electromechanical tool, and a second visual indicator being positioned on the end effector;

at least one camera configured to visually monitor the first and second visual indicators; and

a controller configured to control movement of the electromechanical arm and to control actuation of the end effector, the controller configured to determine a location of the end effector relative to the electromechanical tool based on the first and second visual indicators and to modify actuation of the end effector based on the location of the end effector relative to the electromechanical tool.

14. The system of claim 13, wherein the end effector includes a cutting element configured to be advanceable therethrough, the second visual indicator is positioned on the cutting element;

and

wherein the controller is configured to compare an actual location of the cutting element moving through the end effector to an expected location, and to alter a speed of the cutting element when the actual location does not correspond to the expected location.

15. The system of claim 13, wherein the end effector includes a cutting element that is advanced therethrough, the second visual indicator is positioned on the cutting element;

and

wherein the controller is configured to compare an actual speed of travel of the cutting element moving through the end effector to an expected speed of travel, and to alter a speed of the cutting element when the actual speed does not correspond to the expected speed.

16. The system of claim 15, wherein the controller is configured to calculate the actual speed of travel of the cutting element based on locations of the first and second visual indicators over time.

17. The system of claim 13, wherein at least one of the first and second visual indicators comprises a plurality of markings spaced apart from one another.

18. The system of claim 13, wherein at least one of the first and second visual indicators comprises a grid pattern.

19. The system of claim 13, wherein the end effector includes a cutting element that is advanced therethrough, and a plurality of cut-outs formed in the end effector that enable viewing of the cutting element therethrough as the cutting element is advanced through the end effector.

20. The system of claim 13, wherein the electromechanical tool includes a housing on a proximal end of the shaft that couples to a tool driver on a distal end of the electromechanical arm, the tool driver including a plurality of motors for actuating the end effector, and the controller is configured to modify actuation of the end effector by changing an amount of power supplied to at least one of the plurality of motors.