Robotic surgical system with virtual control panel for tool actuation

• US 10,568,701 B2
• Filed: 12/19/2016
• Issued: 02/25/2020
• Est. Priority Date: 12/19/2016
• Status: Active Grant

• Current Assignee: Cilag Gmbh International (Johnson & Johnson)
• Original Assignee: Ethicon, LLC (Johnson & Johnson)
• Inventors: Swayze, Jeffrey S., Young, Joshua, Beckman, Andrew
• Primary Examiner(s): Holloway, Jason
• Application Number:
  US15/383,026
• Publication Number:
  US 20180168733A1
• Time in Patent Office: 1,163 Days
• CPC Class Codes: A61B 1/04 combined with photographic or television appliances
• CPC Class Codes: A61B 1/06 with illuminating arrangements
• CPC Class Codes: A61B 1/313 for introducing through surgical openings, e.g. laparoscopes
• CPC Class Codes: A61B 17/29 Forceps for use in minimally invasive surgery
• CPC Class Codes: A61B 2034/2065 Tracking using image or pattern recognition
• CPC Class Codes: A61B 2034/2068 using pointers, e.g. pointers having reference marks for determining coordinates of body points
• CPC Class Codes: A61B 2034/256 having a database of accessory information, e.g. including context sensitive help or scientific articles
• CPC Class Codes: A61B 2090/373 using light, e.g. by using optical scanners
• CPC Class Codes: A61B 2090/3937 Visible markers
• CPC Class Codes: A61B 34/20 Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
• CPC Class Codes: A61B 34/25 User interfaces for surgical systems
• CPC Class Codes: A61B 34/30 Surgical robots
• CPC Class Codes: A61B 34/37 Master-slave robots A61B34/35 takes precedence
• CPC Class Codes: A61B 5/0036 including treatment, e.g., using an implantable medical device, ablating, ventilating
• CPC Class Codes: A61B 5/0084 for introduction into the body, e.g. by catheters A61B5/1459 takes precedence
• CPC Class Codes: A61B 5/064 using markers A61B5/062 takes precedence
• CPC Class Codes: A61B 5/743 Displaying an image simultaneously with additional graphical information, e.g. symbols, charts, function plots
• CPC Class Codes: A61B 90/361 Image-producing devices, e.g. surgical cameras
• CPC Class Codes: G06V 10/147 Details of sensors, e.g. sensor lenses fingerprint or palmprint sensors G06V40/13; vascular sensors G06V40/145; eye sensors G06V40/19
• CPC Class Codes: G06V 2201/034 of medical instruments

Abstract

A surgical system includes a detector, comprising an array of pixels configured to detect light reflected by a surgical instrument and generate a first signal comprising a first dataset representative of a visible image of the surgical instrument. The surgical system also includes a processor configured to receive the first signal, generate a modified image of the surgical instrument that includes a control panel. The control panel includes one or more control elements representative of one or more operating parameters of the surgical instrument. The processor is further configured to receive an input to the control panel from a user, the input being effective to change one of the operating parameters. The processor is also configured to generate a command signal based on the input to change the one of the operating parameters.

Claims

1. A surgical system comprising:

a detector, comprising an imaging device configured to detect light reflected by a surgical instrument and a target tissue and generate a first signal comprising a first dataset representative of an image of the surgical instrument and the target tissue;

a processor configured to;

receive the first signal,

receive a second signal indicative of coupling between the surgical instrument and the target tissue,

generate, after the receipt of the second signal, a modified image of the surgical instrument and the target tissue that includes a control panel, wherein the control panel comprises one or more control elements representative of one or more operating parameters of the surgical instrument,

receive an input to the control panel from a user via an input device configured to be held by the user, the input being effective to change one of the operating parameters,

generate a command signal based on the input to the control panel to change one of the operating parameters.

2. The system of claim 1, wherein the modified image is displayed on a display device.

3. The system of claim 1, wherein the processor identifies data from the first dataset data that is representative of the surgical instrument based on a position of the detector with respect to the surgical instrument.

4. The system of claim 1, wherein at least one of position and shape of the control panel in the modified image is adjusted based on the orientation of the surgical instrument.

5. The system of claim 1, wherein the position of the control panel with respect to the surgical instrument remains fixed.

6. The system of claim 1, further comprising an input device to receive the input from the user, wherein the input device is a user interface device.

7. The system of claim 1, wherein the processor generates a pointer that moves relative to the modified image based on the input by the user.

8. The system of claim 7, wherein the command signal is generated when the user actuates one of the one or more control elements using a user interface device.

9. The system of claim 1, wherein the surgical instrument is one with functionality including at least one of tissue cutting, tissue stapling, and delivery of energy to tissue.

10. The system of claim 1, wherein the processor receives a second signal representative of information related to the surgical instrument.

11. The system of claim 10, wherein the control panel is generated based on the second signal and data related to the surgical instrument that is stored in a database.

12. The system of claim 1, further comprising:

a robotic arm; and

a tool assembly removably coupled to the robotic arm and comprising a shaft extending distally from a housing and the surgical instrument is coupled to a distal end of the shaft.

13. The system of claim 2, wherein the display device is at least one of a wall mounted display, a table mounted display, and a display on an accessory worn by a surgeon.

14. The system of claim 6, wherein the processor generates a pointer that moves relative to the modified image, based on the input by the user.