METHODS AND APPARATUS FOR DISPLAYING ENHANCED IMAGING DATA ON A CLINICAL IMAGE

• US 20110082369A1
• Filed: 10/07/2009
• Published: 04/07/2011
• Est. Priority Date: 10/07/2009
• Status: Active Grant

• Current Assignee: Intuitive Surgical Operations, Inc. (Intuitive Surgical Holdings, LLC)
• Original Assignee: Intuitive Surgical Incorporated (Intuitive Surgical Holdings, LLC)
• Inventors: Mohr, Paul, MOHR, CATHERINE, McDowall, Ian
• Granted Patent: US 8,706,184 B2
• Time in Patent Office: Days
• US Class Current: 600/431
• CPC Class Codes: A61B 1/000094 extracting biological structures
• CPC Class Codes: A61B 1/043 for fluorescence imaging
• CPC Class Codes: A61B 34/37 Master-slave robots A61B34/35 takes precedence
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/10048 Infrared image
• CPC Class Codes: G06T 2207/10064 Fluorescence image
• CPC Class Codes: G06T 2207/10068 Endoscopic image
• CPC Class Codes: G06T 2207/30004 Biomedical image processing
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction

Abstract

In one embodiment of the invention, an apparatus includes a display device. The display device displays a desaturated image of tissue captured in the visible electro-magnetic (EM) spectrum from a body cavity; and a first color enhanced image combined with the desaturated image. The first color enhanced image represents the first data captured from the body cavity outside the visible electromagnetic spectrum. The relative brightness between the desaturated image and the first color enhanced image is set to emphasize the first data over the tissue captured in the visible electromagnetic spectrum to provide improved information content.

Claims

14. An apparatus comprising:

a display device displaying

a desaturated image of tissue captured in the visible electro-magnetic (EM) spectrum from a body cavity;

a first color enhanced image combined with the desaturated image, the first color enhanced image representing first data captured from the body cavity outside the visible electromagnetic spectrum; and

wherein a relative brightness between the desaturated image and the first color enhanced image is set to emphasize the first data over the tissue captured in the visible electro-magnetic (EM) spectrum to provide improved information content.

1. A method for a minimally invasive surgical system, the method comprising:

desaturating a visible color image of tissue captured in the visible electro-magnetic (EM) spectrum from a surgical site within a patient into a desaturated image;

coloring an enhanced image representing data captured from the surgical site outside the visible electro-magnetic (EM) spectrum with a visible color to form a color enhanced image;

combining the color enhanced image and the desaturated image together to form a blended image of the surgical site for display on a display device.

2. The method of claim 1, further comprising:

introducing a biomarker into the patient that can be excited with electro-magnetic radiation outside the visible electro-magnetic (EM) spectrum;

illuminating the surgical site within the patient with visible light in the visible electro-magnetic (EM) spectrum;

capturing the visible color image of the surgical site within the patient in response to the visible light; and

capturing data from the surgical site in response to the introduction of the biomarker into the patient.

3. The method of claim 2, further comprising:

prior to capturing data, exposing the surgical site within the patient to electro-magnetic radiation outside the visible electro-magnetic (EM) spectrum to excite the biomarker; and

wherein the data is captured from the surgical site in further response to the excitation of the biomarker.

4. The method of claim 3, wherein

the biomarker introduced into the patient is indocyanine green;

and

the electro-magnetic radiation outside the visible electro-magnetic (EM) spectrum is near-infrared (NIR) to excite the indocyanine green to fluoresce and emit electro-magnetic radiation.

5. The method of claim 1, further comprising:

prior to combining, setting a first relative brightness between the color enhanced image and the desaturated image.

6. The method of claim 4, wherein

the relative brightness is a predetermined level, and

the desaturation of the visible color image is at a predetermined level of desaturation.

7. The method of claim 6, further comprising:

varying the setting of the relative brightness between the color enhanced image and the desaturated image to a second relative brightness to vary information content in the blended image.

8. The method of claim 6, further comprising:

varying the level of desaturation to desaturate the visible color image of the surgical site to vary information content in the desaturated image.

9. The method of claim 8, wherein

the visible color image of the surgical site is fully desaturated to remove all color, and

the desaturated image is a black and white image.

10. The method of claim 1, wherein

the enhanced image is one of a near-infra-red (NIR) fluorescence image, visible light fluorescence image, or a raster scan of spectral characteristics of the surgical site including clinical information with spatial variation.

11. The method of claim 1, wherein

the enhanced image is formed of point measurements of a measureable tissue parameter.

12. The method of claim 11, wherein

the measureable tissue parameter is tissue impedance, point detection of cancer cells, or point detection of a predetermined cell type.

13. The method of claim 1, wherein

the combining of the color enhanced image and the desaturated image includes temporal and spatial registration of the color enhanced image with the desaturated image.

15. The apparatus of claim 14, wherein

the display device further displays

a second color enhanced image combined with the first color enhanced image and the desaturated image, the second color enhanced image representing second data captured from the body cavity outside the visible electromagnetic spectrum, the second color enhanced image having a different color than the first color enhanced image;

and

wherein the relative brightness between the desaturated image and the second color enhanced image is set to emphasize the second data over the tissue captured in the visible electro-magnetic (EM) spectrum.

16. The apparatus of claim 14, further comprising:

a user interface including a user selectable device to adjust the relative brightness between the desaturated image and the first color enhanced image displayed on the display device.

17. The apparatus of claim 14, further comprising:

a user interface including a first user selectable device to adjust the brightness of the desaturated image displayed on the display device and a second user selectable device to adjust the brightness of the second false colored image displayed on the display device.

18. The apparatus of claim 14, wherein

the desaturated image of tissue is captured with a sensor sensitive to ranges of wavelengths in the visible electromagnetic spectrum;

and

the data represented by the first color enhanced image is captured with a sensor sensitive to ranges of wavelengths outside the visible electromagnetic spectrum.

19. The apparatus of claim 14, wherein

a portion of the desaturated image includes a desaturated image of surgical tools in the body cavity.

20. The apparatus of claim 14, wherein

the data represented by the first color enhanced image is captured in response to a biomarker introduced into tissue.

21. The apparatus of claim 20, wherein

the biomarker is indocyanine green (ICG).

22. The apparatus of claim 14, wherein

the data represented by the first color enhanced image is captured with an X-ray of bone tissue under surface tissue.

23. A machine readable media have stored thereon instructions that when executed by a machine causes the machine to perform operations comprising:

reducing color in color images captured of tissue in a surgical site in the visible electro-magnetic (EM) spectrum within a patient to form desaturated images;

making non-visible characteristic tissue features captured in the surgical site visible in color enhanced images; and

combining the color enhanced images and the desaturated images together to form combined images for display on at least one display device.

24. The machine readable media of claim 22, wherein

the non-visible characteristic tissue features of the surgical site are made visible over the tissue captured in desaturated images by a false color.

23-1. The machine readable media of claim 22, wherein

the non-visible characteristic tissue features of the surgical site are captured outside the visible electro-magnetic (EM) spectrum.

25. A method comprising:

obtaining a visible color image of a surgical site illuminated by visible light;

desaturating the visible color image of the surgical site into a visible gray image;

obtaining a fluorescent image of fluorescing tissue in the surgical site, wherein the fluorescing tissue appears in a visible color; and

generating a displayable image comprising a combination of the visible fluorescent image and the visible gray image.

26. The method of claim 25, wherein the desaturating comprises:

substantially removing all color from the visible color image to form the visible gray image.

27. The method of claim 25, wherein the obtaining of the fluorescent image comprises:

exposing the surgical site to electromagnetic radiation outside the visible electromagnetic spectrum.

28. The method of claim 25, further comprising:

coloring the fluorescing tissue in the fluorescent image with the visible color.

29. The method of claim 25, further comprising:

displaying the displayable image on a display device; and

adjusting a level of brightness of one or both of the visible gray image and the fluorescent image.

30. The method of claim 25, further comprising:

displaying the displayable image on a display device; and

adjusting a level of desaturation of the visible color image.