Devices and methods for presenting and regulating auxiliary information on an image display of a telesurgical system to assist an operator in performing a surgical procedure

• US 7,107,090 B2
• Filed: 12/05/2002
• Issued: 09/12/2006
• Est. Priority Date: 12/08/1998
• Status: Expired due to Term

• Current Assignee: Intuitive Surgical Operations, Inc. (Intuitive Surgical Holdings, LLC)
• Original Assignee: Intuitive Surgical Limited (Intuitive Surgical Holdings, LLC)
• Inventors: Salisbury, J. Kenneth Jr., Mintz, David S., Guthart, Gary S., Niemeyer, Gunter D., Cooper, Thomas G., Younge, Robert G.
• Primary Examiner(s): Smith, Ruth S.
• Application Number:
  US10/314,001
• Publication Number:
  US 20030220541A1
• Time in Patent Office: 1,377 Days
• Field of Search: 600/407, 600/103, 600/109, 600/102, 600/427, 901/2, 901/4, 901/36, 901/14, 700/65, 700/83, 700/85, 700/264, 700/245
• US Class Current: 600/427
• CPC Class Codes: A61B 1/00 Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes examination of body cavities or body tracts using ultrasonic, sonic or infrasonic waves A61B8/12; endoscopic instruments for taking cell samples or for biopsy A61B10/04; for surgical purposes A61B17/00; endoscopic surgical cutting instruments A61B17/32; surgical instruments using a laser beam being directed along or through a flexible conduit A61B18/22; Illuminating arrangements therefor for the eyes A61B3/00
• CPC Class Codes: A61B 1/0005 combining images e.g. side-by-side, superimposed or tiled
• CPC Class Codes: A61B 1/00149 using articulated arms
• CPC Class Codes: A61B 1/313 for introducing through surgical openings, e.g. laparoscopes
• CPC Class Codes: A61B 2017/00477 Coupling A61B2017/0046 takes precedence
• CPC Class Codes: A61B 2034/305 Details of wrist mechanisms at distal ends of robotic arms
• CPC Class Codes: A61B 2090/506 using a parallelogram linkage, e.g. panthograph
• CPC Class Codes: A61B 34/30 Surgical robots
• CPC Class Codes: A61B 34/35 for telesurgery
• CPC Class Codes: A61B 34/37 Master-slave robots A61B34/35 takes precedence
• CPC Class Codes: A61B 34/70 Manipulators specially adapted for use in surgery
• CPC Class Codes: A61B 8/12 in body cavities or body tracts, e.g. by using catheters catheters per se A61M25/00
• CPC Class Codes: A61B 8/483 involving the acquisition of a 3D volume of data
• CPC Class Codes: A61B 8/5238 for combining image data of patient, e.g. merging several images from different acquisition modes into one image for image registration algorithms G06T7/30
• CPC Class Codes: A61B 90/36 Image-producing devices or illumination devices not otherwise provided for
• CPC Class Codes: A61B 90/361 Image-producing devices, e.g. surgical cameras
• CPC Class Codes: G16H 20/40 relating to mechanical, radiation or invasive therapies, e.g. surgery, laser therapy, dialysis or acupuncture
• CPC Class Codes: G16H 40/63 for local operation
• CPC Class Codes: G16H 40/67 for remote operation

Abstract

Systems and methods for performing robotically-assisted surgical procedures on a patient enable an image display device to provide an operator with auxiliary information related to the surgical procedure, in addition to providing an image of the surgical site itself. The systems and methods allow an operator to selectively access and reference auxiliary information on the image display device during the performance of a surgical procedure.

Claims

1. A robotically controlled surgical system comprising:

a surgical instrument;

an image capture device positionable so as to capture an image of a surgical site;

at least one image display coupled to the image capture device so as to display the image of the surgical sire;

a master control input device;

a source of auxiliary information; and

at least one processor coupled to the surgical instrument, the image capture device, the at least one image display, the master control input device, and the source of auxiliary information, and coupling the master control input device to the surgical instrument and the source of auxiliary information so as to allow manipulation of the surgical instrument in response to manipulation of the master control input device while the master control input device is operatively associated with the surgical instrument, and allow manipulation of the auxiliary information on the image display in response to manipulation of the master control input device while the operative association between the master control input device and the surgical instrument is interrupted.

2. The system of claim 1, wherein the source of the auxiliary information is a data capturing/gathering device.

3. The system of claim 2, wherein the auxiliary information is captured in real-time.

4. The system of claim 1, wherein the source of the auxiliary information is a data storage device.

5. The system of claim 4, wherein the auxiliary information comprises preoperative information specific to a surgical procedure to be performed.

6. The system of claim 4, wherein the auxiliary information comprises preoperative information specific to the patient on which a surgical procedure is to be performed.

7. The system of claim 1, wherein the processor is configured to adjust an image of the auxiliary information in a discrete window on the image display.

8. The system of claim 7, wherein the adjusting of the auxiliary information in a discrete window on the image display further comprises varying the position of the discrete window across the image display.

9. The system of claim 7, wherein the adjusting of the auxiliary information in a discrete window on the image display further comprises varying the dimensions of the discrete window across the image display.

10. The system of claim 7, wherein the adjusting of the auxiliary information in a discrete window on the image display further comprises varying the orientation of the discrete window across the image display.

11. The system of claim 1, wherein the manipulation of the auxiliary information on the image display comprises displaying the auxiliary information on the image display such that the auxiliary information is overlaid on the image of the surgical site.

12. The system of claim 1, wherein the auxiliary information comprises an image corresponding to the surgical site, and wherein the manipulation of the auxiliary information on the image display involves alternately displaying the auxiliary information image and the image of the surgical site on the image display.

13. The system of claim 1, wherein the auxiliary information comprises an image corresponding to the surgical site, and wherein the manipulation of the auxiliary information on the image display comprises at least generally merging the auxiliary information image with the image of the surgical site so that the images are superimposed and are both visible on the image display.

14. The system of claim 1, wherein the auxiliary information comprises a member selected from the group consisting of a CAT image, an MRI image, an X-ray image, and an ultrasound image.

15. The system of claim 1, wherein the auxiliary information comprises a member selected from the group consisting of an ECG signal, an EEG signal, a blood pressure signal, and a heartbeat signal.

16. The system of claim 1, wherein the auxiliary information comprises a member selected from the group consisting of a warning message, a clock indicating time, an e-mail service, a temperature reading, oxygen levels in a patient'"'"'s blood, and information from a remote computer.

17. The system of claim 1, wherein the auxiliary information comprises a member selected from the group consisting of a streaming video corresponding to a surgical procedure to be performed, a generic anatomical image, and a patient specific anatomical image.

18. The system of claim 1, wherein the auxiliary information comprises a member selected from the group consisting of a three-dimensional model, a raw volumetric image, a computer generated model, and a computer generated image.

19. The system of claim 1, wherein the auxiliary information comprises a member selected from the group consisting of an image of the surgical instrument, an image of the surgical instrument location, and an image of art entry port location through which the surgical instrument extends.

20. The system of claim 1, wherein the auxiliary information is an indication of the orientation of the image of the surgical site relative to a frame of reference.

21. The system of claim 1, wherein the image capture device captures a real time image of the surgical site, and wherein the auxiliary information comprises an image of the surgical site which is more panoramic than the real time image of the surgical site such that the auxiliary information image as manipulated on the image display extends around a periphery of the real-time image of the surgical site.

22. The system as claimed in claim 1, wherein the auxiliary information comprises an image corresponding to a constraint, and wherein the auxiliary information when adjusted on The image display is superimposed with the image of the surgical site such that the image of the constraint is arranged to indicate that movement of the surgical instrument through the image of the constraint is either to be avoided or not possible.

23. The system as claimed in claim 22, the processor adjusting the image of the constraint relative to the image of the surgical site in response to the manipulation of the master control input device.

24. The system as claimed in claim 23, wherein the master control input device can be selectively operatively associated with the image of the constraint, said processor adjusting the image of the constraint relative to the image of the surgical site comprising placing the image of the constraint relative to the image of the surgical site in a location corresponding to a position beyond which the surgeon desires not to move any surgical tool.

25. The system as claimed in claim 1, wherein the image capture device captures a real time image of the site, the system further comprising a second image capture device that captures a second image related to the surgical procedure from a vantage point different to a vantage point from which the real time image of the site is captured, the processor adjusting the second image on the image display in response to the manipulation of the master control input device.

26. The system of claim 1, wherein the manipulation of the auxiliary information on the image display comprises a least one of positionally adjusting the auxiliary information, orientationally adjusting the auxiliary information, or scaling the auxiliary information.

27. The system of claim 1, wherein the processor is configured to effect master/slave movement of the surgical instrument in response to a first manipulation of the master control input device, and to adjust the image of the auxiliary information on the image display in response to a second manipulation of the master control input device.

28. A method of preparing for or performing a robotic surgical procedure on a patient, the method comprising:

a) displaying an image of auxiliary information relevant to the surgical procedure on an image display;

b) effecting a master/slave movement of a surgical instrument of the robotic surgical system using a master control input device while the master control input device is operatively associated with the surgical instrument; and

c) adjusting the image of the auxiliary information on the image display of the robotic surgical system using the master control input device while operative association between the master control input device and the surgical instrument is interrupted.

29. A method of preparing for or performing a robotic surgical procedure on a patient, the method comprising:

a) viewing an image of a surgical site on an image display;

b) effecting a master/slave movement of a surgical instrument of the robotic surgical system using a master control input device while the master control input device is operatively associated with the surgical instrument; and

c) selectively accessing a source of auxiliary information, displaying the selected auxiliary information on the image display, and manipulating the auxiliary information on the image display using the master control input device while operative association between the master control input device and the surgical instrument is interrupted.