AUGMENTED STEREOSCOPIC VISUALIZATION FOR A SURGICAL ROBOT USING A CAMERA UNIT WITH A MODIFIED PRISM

• US 20090268011A1
• Filed: 06/30/2008
• Published: 10/29/2009
• Est. Priority Date: 04/26/2008
• Status: Active Grant

• Current Assignee: Intuitive Surgical Operations, Inc. (Intuitive Surgical Holdings, LLC)
• Original Assignee: Intuitive Surgical Incorporated (Intuitive Surgical Holdings, LLC)
• Inventors: McDowell, Ian, Stern, John D., Scott, David, Mohr, Catherine J., Zhao, Wenyi, Hoffman, Brian D.
• Granted Patent: US 8,803,955 B2
• Time in Patent Office: Days
• US Class Current: 348/45
• CPC Class Codes: A61B 1/000096 using artificial intelligence
• CPC Class Codes: A61B 1/0005 combining images e.g. side-by-side, superimposed or tiled
• CPC Class Codes: A61B 1/00149 using articulated arms
• CPC Class Codes: A61B 1/00186 with imaging filters
• CPC Class Codes: A61B 1/00193 adapted for stereoscopic vision
• CPC Class Codes: A61B 1/00194 adapted for three-dimensional imaging
• CPC Class Codes: A61B 1/043 for fluorescence imaging
• CPC Class Codes: A61B 1/05 characterised by the image sensor, e.g. camera, being in the distal end portion
• CPC Class Codes: A61B 1/063 for monochromatic or narrow-band illumination
• CPC Class Codes: A61B 1/0638 providing two or more wavelengths
• CPC Class Codes: A61B 1/313 for introducing through surgical openings, e.g. laparoscopes
• CPC Class Codes: A61B 2034/301 for introducing or steering flexible instruments inserted into the body, e.g. catheters or endoscopes
• CPC Class Codes: A61B 2090/364 Correlation of different images or relation of image positions in respect to the body
• CPC Class Codes: A61B 34/30 Surgical robots
• CPC Class Codes: A61B 34/37 Master-slave robots A61B34/35 takes precedence
• CPC Class Codes: A61B 5/0071 by measuring fluorescence emission
• CPC Class Codes: A61B 5/0075 by spectroscopy, i.e. measuring spectra, e.g. Raman spectroscopy, infrared absorption spectroscopy A61B5/0071 takes precedence
• CPC Class Codes: A61B 5/0084 for introduction into the body, e.g. by catheters A61B5/1459 takes precedence
• CPC Class Codes: A61B 90/30 Devices for illuminating a surgical field, the devices having an interrelation with other surgical devices or with a surgical procedure
• CPC Class Codes: A61B 90/361 Image-producing devices, e.g. surgical cameras
• CPC Class Codes: G01N 21/6456 : Spatial resolved fluorescence measurements; Imaging
• CPC Class Codes: H04N 13/207 : using a single 2D image sensor
• CPC Class Codes: H04N 13/239 : using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence

Abstract

An endoscope with a stereoscopic optical channel is held and positioned by a robotic surgical system. A first capture unit captures: a visible first color component of a visible left image combined with a fluorescence left image from first light from one channel in the endoscope; a visible second color component of the visible left image from the first light; and a visible third color component of the visible left image from the first light. A second capture unit captures: a visible first color component of a visible right image combined with a fluorescence right image from second light from the other channel in the endoscope; a visible second color component of the visible right image from the second light; and a visible third color component of the visible right image from the second light. An augmented stereoscopic outputs a real-time stereoscopic image including a three-dimensional presentation including the visible left and right images and the fluorescence left and right images.

Claims

1. A robotic surgical system comprising:

an endoscope, held and positioned by the robotic surgical system, wherein the endoscope comprises a stereoscopic optical channel including a first channel for transporting first light from tissue and a second channel for transporting second light from the tissue;

an image capture system comprising;

a first capture unit coupled to the first channel, wherein the first capture unit captures a visible first color component of a visible left image combined with a fluorescence left image from the first light, a visible second color component of the visible left image from the first light, and a visible third color component of the visible left image from the first light; and

a second capture unit coupled to the second channel, wherein the second capture unit captures a visible first color component of a visible right image combined with a fluorescence right image from the second light, a visible second color component of the visible right image from the second light, and a visible third color component of the visible right image from the second light; and

an augmented stereoscopic display system coupled to the image capture system, wherein the augmented stereoscopic display system outputs a real-time stereoscopic image of at least a portion of the tissue comprising a three-dimensional presentation including the visible left and right images and the fluorescence left and right images.

2. The robotic surgical system of claim 1:

wherein the first capture unit includes a prism;

wherein the prism separates the first light into the visible first color component of the visible left image combined with a fluorescence wavelength range component of the fluorescence left image, the visible second color component of the visible left image, and the visible third color component of the visible left image; and

wherein the fluorescence wavelength range component of the fluorescence left image is separated and removed from wavelength ranges of the first, second, and third visible color components.

3. The robotic surgical system of claim 1:

wherein the second capture unit includes a prism;

wherein the prism separates the second light into the visible first color component of the visible right image combined with a fluorescence wavelength range component of the fluorescence right image, the visible second color component of the visible right image, and the visible third color component of the visible right image; and

wherein the fluorescence wavelength range component of the fluorescence right image is separated and removed from wavelength ranges of the first, second, and third visible color components.

4. The robotic surgical system of claim 1:

wherein the visible first color component is comprises a blue or a green color component.

5. The robotic surgical system of claim 1:

wherein the endoscope further comprises an illumination channel; and

wherein the robotic surgical system further comprises a combination light source, coupled to the illumination channel and

wherein the combination light source comprises;

a first light source for providing a first illumination light; and

a second light source for providing a second illumination light;

wherein the second illumination light is different from the first illumination light; and

wherein said second illumination light excites the fluorescence.

6. The robotic surgical system of claim 5 further comprising:

a fiber optic bundle connecting the illumination channel to the combination light source.

7. The robotic surgical system of claim 5 further comprising:

a third light source;

wherein the third light source excites a second fluorescence at a wavelength different from the fluorescence excited by the second light source; and

wherein the second and the third light sources can be interchanged while the fiber optic bundle remains connected to the illumination channel and to the combination light source.

8. The robotic surgical system of claim 5:

wherein the augmented stereoscopic display system further comprises a display mode select coupled to the second illumination light source;

wherein the display mode select comprises a plurality of selectable modes;

wherein upon selection of a first mode, the augmented stereoscopic display system outputs the real-time stereoscopic image; and

wherein upon selection of a second mode, the second illumination light is stopped and only the first illumination light is provided to the tissue; and

wherein said augmented stereoscopic display system outputs another real-time stereoscopic image comprising only visible left and right images.

16. The robotic surgical system of claim 5:

wherein the augmented stereoscopic display system further comprises a display mode select coupled to the first illumination light source;

wherein the display mode select comprises a plurality of selectable modes;

wherein upon selection of a first mode, the augmented stereoscopic display system outputs the real-time stereoscopic image;

wherein upon selection of a second mode, the first illumination light is stopped and only the second illumination light is provided to the tissue; and

wherein the augmented stereoscopic display system outputs another real-time stereoscopic image comprising only fluorescence left and right images.

9-15. -15. (canceled)

17. A robotic surgical system comprising:

a stereoscopic endoscope that captures a left stereoscopic tissue image and a right stereoscopic tissue image;

wherein the left and right stereoscopic tissue images each comprise light in a first visible color component wavelength range, light in a second visible color component wavelength range, light in a third visible color component wavelength range, and light in a fluorescence component wavelength range; and

wherein the fluorescence component wavelength range is substantially different from the first, second, and third visible color component wavelength ranges;

a left image capture unit comprising a first sensor, a second sensor, and a third sensor;

wherein the first sensor receives substantially only light in the first visible color component wavelength range and light in the fluorescence component wavelength range,

wherein the second sensor receives substantially only light in the second visible color component wavelength range, and

wherein the third sensor receives substantially only light in the third visible color component wavelength range; and

a right image capture unit comprising a fourth sensor a fifth sensor, and a sixth sensor;

wherein the fourth sensor receives substantially only light in the first visible color component wavelength range and light in the fluorescence component wavelength range,

wherein the fifth sensor receives substantially only light in the second visible color component wavelength range, and

wherein the sixth sensor receives substantially only light in the third visible color component wavelength range.

18. The system of claim 17:

wherein the first visible color component wavelength range comprises a blue wavelength peak; and

wherein the fluorescence component wavelength range comprises a wavelength peak outside the visible spectrum.

19. The system of claim 17:

wherein the first visible color component wavelength range comprises a green wavelength peak; and

wherein the fluorescence component wavelength range comprises a wavelength peak outside the visible spectrum.

20. The system of claim 17:

wherein either one or both of the left image capture unit and the right image capture unit comprises a prism assembly that separates the corresponding stereoscopic tissue image into the first visible color component wavelength range combined with the fluorescence component wavelength range, the second visible color component wavelength range, and the third visible color component wavelength range.

21. The system of claim 17:

wherein the endoscope is held and positioned by the robotic surgical system.

22. A method comprising:

capturing a left stereoscopic tissue image and a right stereoscopic tissue image,

wherein the left and right stereoscopic tissue images each comprise light in a first visible color component wavelength range, light in a second visible color component wavelength range, light in a third visible color component wavelength range, and light in a fluorescence component wavelength range, and

wherein the fluorescence component wavelength range is substantially different from the first, second, and third visible color component wavelength ranges;

sensing the captured left image by directing substantially only light in the first visible color component wavelength range and light in the fluorescence component wavelength range to a first sensor, directing substantially only light in the second visible color component wavelength range to a second sensor, and directing substantially only light in the third visible color component wavelength range to a third sensor; and

sensing the captured right image by directing substantially only light in the first visible color component wavelength range and light in the fluorescence component wavelength range to a fourth sensor, directing substantially only light in the second visible color component wavelength range to a fifth sensor, and directing substantially only light in the third visible color component wavelength range to a sixth sensor.

23. The method of claim 22 further comprising:

using one or more prisms to separate light from a tissue image into the first visible color, second visible color, third visible color, and fluorescence component wavelength ranges.

24. The method of claim 22:

wherein light in the first visible color component wavelength ranges comprises a blue wavelength peak.

25. The method of claim 22:

wherein light in the first visible color component wavelength ranges comprises a green wavelength peak.

26. The method of claim 22 further comprising:

illuminating tissue in a first instance with a first light source and a second light source; and

illuminating tissue in a second instance with only the first light source;

wherein the first light source illuminates the tissue for sensing the first, second, and third visible color component wavelength ranges; and

wherein the second light source illuminates the tissue for sensing the fluorescence component wavelength range.

27. The method of claim 22 further comprising:

illuminating tissue in a first instance with a first light source and a second light source; and

illuminating tissue in a second instance with only the second light source;

wherein the first light source illuminates the tissue for sensing the first, second, and third visible color component wavelength ranges; and

wherein the second light source illuminates the tissue for sensing the fluorescence component wavelength range.

28. The method of claim 22 further comprising:

illuminating tissue in a first instance with a first light source and a second light source; and

illuminating tissue in a second instance with the first light source and a third light source;

wherein the first light source illuminates the tissue for sensing the first, second, and third visible color component wavelength ranges;

wherein the second light source illuminates the tissue for sensing a first fluorescence component wavelength range; and

wherein the third light source illuminates the tissue for sensing a second fluorescence component wavelength range different from the first fluorescence wavelength range.