INTERACTIVE USER INTERFACES FOR ROBOTIC MINIMALLY INVASIVE SURGICAL SYSTEMS
First Claim
1. A method for a minimally invasive surgical system, the method comprising:
- capturing one or more camera images of a surgical site;
displaying the one or more captured camera images of the surgical site on at least one display device at a surgeon console;
switching out of a following mode in which one or more input devices of the surgeon console are used to couple motion into minimally invasive surgical instruments and into a masters-as-mice mode in which the one or more input devices are used to interact with a graphical user interface;
overlaying the graphical user interface including an interactive graphical object onto the one or more captured camera images displayed on the at least one display device at the surgeon console, wherein the interactive graphical object is related to a physical object in the surgical site or a function thereof and is manipulated by the one or more input devices of the surgeon console in the masters-as-mice mode; and
rendering a pointer within the one or more captured camera images displayed on the at least one display device for user interactive control of the interactive graphical object, wherein in the masters-as-mice mode the pointer is manipulated in three dimensions within the one or more captured camera images by at least one of the one or more input devices of the surgeon console having at least three degrees of freedom.
4 Assignments
0 Petitions
Accused Products
Abstract
In one embodiment of the invention, a method for a minimally invasive surgical system is disclosed. The method includes capturing and displaying camera images of a surgical site on at least one display device at a surgeon console; switching out of a following mode and into a masters-as-mice (MaM) mode; overlaying a graphical user interface (GUI) including an interactive graphical object onto the camera images; and rendering a pointer within the camera images for user interactive control. In the following mode, the input devices of the surgeon console may couple motion into surgical instruments. In the MaM mode, the input devices interact with the GUI and interactive graphical objects. The pointer is manipulated in three dimensions by input devices having at least three degrees of freedom. Interactive graphical objects are related to physical objects in the surgical site or a function thereof and are manipulatable by the input devices.
-
Citations
41 Claims
-
1. A method for a minimally invasive surgical system, the method comprising:
-
capturing one or more camera images of a surgical site; displaying the one or more captured camera images of the surgical site on at least one display device at a surgeon console; switching out of a following mode in which one or more input devices of the surgeon console are used to couple motion into minimally invasive surgical instruments and into a masters-as-mice mode in which the one or more input devices are used to interact with a graphical user interface; overlaying the graphical user interface including an interactive graphical object onto the one or more captured camera images displayed on the at least one display device at the surgeon console, wherein the interactive graphical object is related to a physical object in the surgical site or a function thereof and is manipulated by the one or more input devices of the surgeon console in the masters-as-mice mode; and rendering a pointer within the one or more captured camera images displayed on the at least one display device for user interactive control of the interactive graphical object, wherein in the masters-as-mice mode the pointer is manipulated in three dimensions within the one or more captured camera images by at least one of the one or more input devices of the surgeon console having at least three degrees of freedom. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12)
-
-
13. A method for a minimally invasive surgical system having a surgeon console with one or more input devices, the method comprising:
-
activating a graphical user interface (GUI) mode at the surgeon console to overlay graphics of a menu system and a pointer onto one or more images of a surgical site; switching from a first mode for one or more master input devices used to couple motion into a minimally invasive surgical tool during a surgical procedure to a second mode for the one or more master input devices to control the pointer and interact with the menu system, wherein in the second mode the pointer is manipulated in three dimensions within the one or more images of the surgical site by at least one master input device having at least three degrees of freedom; with at least one master input device, moving the pointer within the one or more images of the surgical site to be over one of one or more menu buttons of the menu system; with the pointer over one menu button, selecting the one menu button with the at least one master input device to further control the surgical system; and minimizing the menu system to avoid obscuring the images of the one or more minimally invasive surgical instruments and the surgical site. - View Dependent Claims (14, 15, 16, 17, 18, 19, 20, 21, 22)
-
-
23. A method for a minimally invasive surgical system, the method comprising:
-
displaying one or more captured camera images of a surgical site on at least one display device at a first surgeon console, the one or more captured camera images including an image of a first tool; overlaying a first menu system, a pointer, and a first graphical tool icon onto the one or more captured camera images of the surgical site, the first graphical tool icon associated with the image of the first tool in the one or more video images of the surgical site; if the pointer is moved to be over a first menu button in the first menu system in response to movement of at least one input device and the first menu button is selected by the at least one input device, overlaying a first pull down menu list of a plurality of menu options each associated with a medical image data set for selection; if the pointer is moved to be over a second menu button in the first menu system in response to movement of at least one input device and the second menu button is selected by the at least one input device, overlaying a second menu system associated with defining and adjusting virtual fixtures; and if the pointer is moved to be over the first graphical tool icon in response to movement of at least one input device and the first graphical tool icon is selected by the at least one input device, overlaying a second pull down menu list of a plurality of menu options each associated with the first tool. - View Dependent Claims (24, 25, 26, 27, 28, 29)
-
-
30. A method for a minimally invasive surgical system, the method comprising:
-
displaying one or more captured camera images of a surgical site on at least one display device at a first surgeon console, the one or more captured camera images including an image of a first tool; overlaying a first menu system and a pointer onto the one or more captured camera images of the surgical site, receiving a signal moving the pointer over the first menu system and overlaying a plurality of first menu buttons onto the one or more captured camera images of the surgical site; receiving a signal selecting a first menu button in the first menu system; in response to the selection of the first menu button, overlaying a second menu system onto the one or more captured camera images of the surgical site, the second menu system including a plurality of second menu buttons associated with a context of the first menu button to further control the minimally invasive surgical system or provide further information regarding the surgical procedure. - View Dependent Claims (31, 32, 33)
-
-
34. A method for a minimally invasive surgical system, the method comprising:
-
displaying one or more captured camera images of a surgical site onto a stereoscopic display device at a first surgeon console, the one or more captured camera images appearing to be three dimensional (3D) to a user and including an image of a first tool; overlaying a first tool icon and a pointer onto the one or more captured camera images of the surgical site to appear to be 3D to a user, the first tool icon located near an image of the first tool in the one or more captured camera images, wherein the pointer is manipulated in three dimensions within the one or more captured camera images by at least one input device of a surgeon console having at least three degrees of freedom; receiving a signal moving the pointer over the first tool icon; receiving a signal selecting the first tool icon; in response to the selection of the first tool icon, overlaying a first menu system onto the one or more captured camera images of the surgical site, the first menu system including a plurality of first menu buttons associated with a context of the first tool to further control the first tool or provide information associated with the first tool. - View Dependent Claims (35, 36, 37, 38, 39)
-
-
40. A minimally invasive surgical system comprising:
-
a stereoscopic endoscope to capture camera images of a surgical site and a first tool; a stereoscopic video display device of a surgeon console to display stereo video images to a user to view images with a three dimensional (3D) appearance; a processor coupled to the stereoscopic endoscope and the stereoscopic video display device, in response to stored program instructions the processor configured to overlay a first tool icon and a pointer onto the one or more captured camera images of the surgical site, the first tool icon located near the first tool, receive a signal moving the pointer over the first tool icon; receive a signal selecting the first tool icon; and in response to the selection of the first tool icon, overlay a first menu system onto the one or more captured camera images of the surgical site, the first menu system including a plurality of first menu buttons associated with the first tool to further control the first tool or provide information associated with the first tool.
-
-
41-86. -86. (canceled)
Specification