Compact fluorescent endoscopy video system

• US 20020035330A1
• Filed: 07/13/2001
• Published: 03/21/2002
• Est. Priority Date: 07/14/2000
• Status: Active Grant

• Current Assignee: Stryker Corporation
• Original Assignee: Xillix Technologies Corp.
• Inventors: Fengler, John J.P., Cline, Richard W., Boehm, Joachim W.
• Granted Patent: US 6,821,245 B2
• Time in Patent Office: Days
• US Class Current: 600/476
• CPC Class Codes: A61B 1/000095 for image enhancement
• CPC Class Codes: A61B 1/00057 provided with means for testing or calibration
• CPC Class Codes: A61B 1/00186 with imaging filters
• CPC Class Codes: A61B 1/041 Capsule endoscopes for imaging
• CPC Class Codes: A61B 1/043 for fluorescence imaging
• CPC Class Codes: A61B 1/045 Control thereof
• CPC Class Codes: A61B 1/051 Details of CCD assembly
• CPC Class Codes: A61B 1/063 for monochromatic or narrow-band illumination
• CPC Class Codes: A61B 1/0638 providing two or more wavelengths
• CPC Class Codes: A61B 1/0646 with illumination filters
• CPC Class Codes: A61B 1/0655 Control therefor
• CPC Class Codes: A61B 1/0669 at proximal end of an endoscope
• CPC Class Codes: A61B 1/2676 Bronchoscopes
• CPC Class Codes: A61B 5/0071 by measuring fluorescence emission
• CPC Class Codes: A61B 5/0084 for introduction into the body, e.g. by catheters A61B5/1459 takes precedence
• CPC Class Codes: G01N 21/6458 Fluorescence microscopy fluorescence microscopes per se G02B21/0076 and G02B21/16
• CPC Class Codes: G16Z 99/00 Subject matter not provided for in other main groups of this subclass

Abstract

A fluorescence endoscopy video system includes a multimode light source that produces light for color and fluorescence imaging modes. Light from the light source is transmitted through an endoscope to the tissue under observation. The system also includes a compact camera for color and fluorescence imaging. Images obtained through the endoscope are optically divided and projected onto one or more image sensors by a fixed beam splitter in the camera. The fixed beam splitter eliminates the need for inserting a movable mirror into the light path between the endoscope and the image sensors. Image signals from the camera are processed in the system processor/controller where a contrast enhancement function can be applied. The contrast enhancement function increases the color contrast between normal tissue and tissue suspicious for early cancer. Finally, the system also includes a calibration feature whereby the system performance can be maintained when used with different endoscopes.

Claims

1. A fluorescence endoscopy video system including:

a multimode light source for producing white light, fluorescence excitation light or fluorescence excitation light with a reference reflectance light;

an endoscope for directing the light from the light source into a patient to illuminate a tissue sample and to collect the reflected light or fluorescence light produced by the tissue;

a camera positioned to receive the light collected by the endoscope, the camera including;

a color image sensor;

a low light image sensor; and

a beam splitter for splitting the light received from the endoscope into at least two beams and directing those beams onto the low light and color image sensors;

one or more filters positioned in front of the low light and color image sensors for selectively transmitting light of desired wavelengths;

one or more optical imaging components that project images onto both the color image sensor and the low light image sensor;

an image processor/controller for digitizing, processing and encoding the image signals received from the sensor(s) as a video signal; and

a video monitor for displaying the video signals.

2. The system of claim 1, wherein the camera is attached to the portion of the endoscope that remains outside of the body.

3. The system of claim 1, wherein the camera is built into the insertion portion of the endoscope.

4. The system of claim 2 or 3, wherein the beam splitter directs a greater percentage of light collected by the endoscope to the low light sensor and a lesser percentage to the color image sensor.

5. The system of claim 2 or 3, further comprising a filter positioned in the light path of the light source that simultaneously transmits the fluorescence excitation light and an amount of reference reflectance light not in a fluorescence detection wavelength band, wherein the amount of reference reflectance light transmitted is a fraction of the fluorescence excitation light, such that the intensity of the reflected reference light projected onto the color image sensor matches the requirements of the sensor, at the same time as maintaining sufficient fluorescence excitation so as to match the intensity of fluorescence projected onto the low light sensor to the requirements of the sensor, and wherein the light source filter blocks light from the light source at wavelengths in the fluorescence detection wavelength band from reaching the low light sensor to the extent that the light received by the low light sensor is substantially composed of light resulting from tissue fluorescence and minimally composed of light originating from the light source.

6. The system of claim 5, wherein the filter in front of the color image sensor blocks reflected excitation light and transmits reference reflectance light to the extent that the light received by the color image sensor is substantially composed of reference reflectance light and minimally composed of excitation light, the amount of reference reflectance light transmitted by the filter being such that the intensity of the reference light projected onto the color image sensor matches the requirements of the sensor, and the filter in front of the low light image sensor blocks reflected excitation and reference light, and transmits substantially fluorescence light to the extent that the light received by the low light sensor is substantially composed of light resulting from tissue fluorescence and minimally composed of light originating from the light source.

7. The system of claim 6, wherein the fluorescence, transmitted by the filter in front of the low light image sensor, is green light.

8. The system of claim 6, wherein the fluorescence, transmitted by the filter in front of the low light image sensor, is red light.

9. The system of claim 7, wherein the reference reflectance light transmitted by the light source filter is red light.

10. The system of claim 8, wherein the reference reflectance light transmitted by the light source filter is green light.

11. The system of claim 7, wherein the reference reflectance light transmitted by the light source filter is near-infrared light.

12. The system of claim 8, wherein the reference reflectance light transmitted by the light source filter is near-infrared light.

13. The system of claim 7, wherein the reference reflectance light transmitted by the light source filter is blue light.

14. The system of claim 8, wherein the reference reflectance light transmitted by the light source filter is blue light.

15. The system of claim 2 or 3, further comprising a filter positioned in the light path of the light source that transmits the fluorescence excitation light and blocks light at wavelengths in the detected fluorescence bands from reaching both the low light sensor and the color image sensor to the extent that the light received by the sensors is substantially composed of light resulting from tissue fluorescence and minimally composed of light originating from the light source.

16. The system of claim 15, wherein the filter in front of the low light image sensor blocks reflected excitation light and transmits fluorescence light in a green wavelength band to the extent that the light received by the low light sensor is substantially composed of light resulting from tissue fluorescence and minimally composed of light originating from the light source, and the filter in front of the color image sensor blocks reflected excitation light and transmits fluorescence light in a red wavelength band to the extent that the light received by the color image sensor is substantially composed of light resulting from tissue fluorescence and minimally composed of light originating from the light source.

17. The system of claim 2 or 3, wherein the camera also includes a reference image sensor and a dichroic splitter and filter assembly that divides the light not projected to the color image sensor into two spectral components and projects the two spectral components onto the low light image sensor and reference image sensor.

18. The system of claim 17, wherein the dichroic splitter and filter assembly splits the light into green and red bands while blocking out of band light to the extent that the light received by the sensors is substantially composed of light of green or red wavelengths and minimally composed of light of out of band wavelengths.

19. The system of claim 17, wherein the dichroic splitter and filter assembly splits the light into blue and red bands while blocking out of band light to the extent that the light received by the sensors is substantially composed of light of blue or red wavelengths and minimally composed of light of out of band wavelengths.

20. The system of claim 17, wherein the dichroic splitter and filter assembly splits the light into red and near-infrared bands while blocking out of band light to the extent that the light received by the sensors is substantially composed of light of red or near-infrared wavelengths and minimally composed of light of out of band wavelengths.

21. The system of claim 17, wherein the dichroic splitter and filter assembly splits the light into green and near-infrared bands while blocking out of band light to the extent that the light received by the sensors is substantially composed of light of red or near-infrared wavelengths and minimally composed of light of out of band wavelengths.

22. The system of claim 2 or 3, further comprising a dichroic splitter and filter assembly that divides the light not projected to the color image sensor into two spectral components and projects the two spectral components in the same image plane onto separate imaging areas of the low light image sensor.

23. The system of claim 22, wherein the dichroic splitter and filter assembly splits the light into green and red bands while blocking out of band light to the extent that the light received by the sensors is substantially composed of light of green or red wavelengths and minimally composed of light of out of band wavelengths.

24. The system of claim 22, wherein the dichroic splitter and filter assembly splits the light into blue and red bands while blocking out of band light to the extent that the light received by the sensors is substantially composed of light of blue and red wavelengths and minimally composed of light of out of band wavelengths.

25. The system of claim 22, wherein the dichroic splitter and filter assembly splits the light into red and near-infrared bands while blocking out of band light to the extent that the light received by the sensors is substantially composed of light of red and near-infrared wavelengths and minimally composed of light of out of band wavelengths.

26. The system of claim 22, wherein the dichroic splitter and filter assembly splits the light into green and near-infrared bands while blocking out of band light to the extent that the light received by the sensors is substantially composed of light of red and near-infrared wavelengths and minimally composed of light of out of band wavelengths.

28. The system of claim 27, wherein the camera is attached to the portion of the endoscope that remains outside of the body.

29. The system of claim 27, wherein the camera is built into the insertion portion of the endoscope.

30. The system of claim 28 or 29, wherein the beam splitter comprises:

a number of prisms having at least one partially reflecting surface and a number of reflecting surfaces arranged to divide incoming light into at least two separate beams, each of which has a substantially identical optical path length.

31. The system of claim 28 or 29, further comprising a filter positioned in the light path of the light source that simultaneously transmits the fluorescence excitation light and an amount of reference reflectance light not in a fluorescence detection wavelength band, wherein the amount of reference reflectance light transmitted is a fraction of the fluorescence excitation light such that the intensity of the reflected reference light projected onto the high sensitivity color image sensor matches the requirements of the sensor, at the same time as maintaining sufficient fluorescence excitation. so as to match the intensity of fluorescence also projected onto the high sensitivity color image sensor to the requirements of the sensor, and wherein the light source filter blocks light from the light source at wavelengths in the fluorescence detection wavelength band from reaching the high sensitivity color image sensor to the extent that the light received by the high sensitivity color image sensor is substantially composed of light resulting from tissue fluorescence and minimally composed of light originating from the light source.

32. The system of claim 31, wherein one filter in front of one imaging area of the high sensitivity color image sensor blocks reflected excitation light and reference light, and transmits substantially fluorescence light to the extent that the light received by the high sensitivity color image sensor is substantially composed of light resulting from tissue fluorescence and minimally composed of light originating from the light source, the amount of reference reflectance light transmitted by the filter being such that the intensity of the reference light projected onto the color image sensor matches the requirements of the sensor, and a second filter in front of a second imaging area of the high sensitivity color image sensor blocks reflected excitation light and transmits reference reflectance light to the extent that the light received by the second imaging area of the high sensitivity color image sensor is substantially composed of reference reflectance light and minimally composed of excitation light, the amount of reference reflectance light transmitted by the second filter being such that the intensity of the reflected reference light passing through the second filter and projected onto the high sensitivity color image sensor matches the intensity of the fluorescence light projected onto the first area of the high sensitivity color image sensor and matches the requirements of the sensor.

33. The system of claim 32, wherein the fluorescence, transmitted by at least one filter in front of the high sensitivity color image sensor, is green light

34. The system of claim 32, wherein the fluorescence, transmitted by at least one filter in front of the high sensitivity color image sensor, is red light.

35. The system of claim 33, wherein the reference reflectance light, not in the detected fluorescence band, transmitted by the light source filter is red light.

36. The system of claim 34, wherein the reference reflectance light, not in the detected fluorescence band, transmitted by the light source filter is green light.

37. The system of claim 33, wherein the reference reflectance light, not in the detected fluorescence band, transmitted by the light source filter is near-infrared light.

38. The system of claim 34, wherein the reference reflectance light, not in the detected fluorescence band, transmitted by the light source filter is near-infrared light.

39. The system of claim 33, wherein the reference reflectance light, not in the detected fluorescence band, transmitted by the light source filter is blue light.

40. The system of claim 34, wherein the reference reflectance light, not in the detected fluorescence band, transmitted by the light source filter is blue light.

41. The system of claim 28 or 29, further comprising a filter positioned in the light path of the light source that transmits the fluorescence excitation light and blocks light, not at wavelengths in the detected fluorescence bands, from reaching the imaging areas of the high sensitivity color image sensor to the extent that the light received by the sensor is substantially composed of light resulting from tissue fluorescence and minimally composed of light originating from the light source.

42. The system of claim 41, wherein one filter in front of one imaging area of the high sensitivity color image sensor blocks reflected excitation light and transmits fluorescence light in a green wavelength band to the extent that the light received in the first area of the high sensitivity color image sensor is substantially composed of light resulting from tissue fluorescence in a green wavelength band and minimally composed of light originating from the light source, and a second filter in front of a second imaging area of the high sensitivity color image sensor blocks reflected excitation light and transmits fluorescence light in a red wavelength band to the extent that the light received in the second area of the high sensitivity color image sensor is substantially composed of light resulting from tissue fluorescence in a red wavelength band and minimally composed of light originating from the light source.

70. The system of claim 1, 27, 43, 55, 60, or 65, further comprising a fluorescence/reflectance reference that produces known levels of fluorescence and reflectance light upon illumination with light from the multimode light source, wherein the image processor is programmed to adjust the gain of the one or more image sensors in response to the levels of fluorescence and reflectance light produced.

71. The system of claim 70, wherein the fluorescence/reflectance reference comprises a dye and an amount of scatterers in a solid.

27. A fluorescence endoscopy video system including:

a multimode light source for producing white light, fluorescence excitation light or fluorescence excitation light and reference reflectance light;

an endoscope for directing the light from the light source into a patient to illuminate a tissue sample and to collect reflected light and fluorescence light produced by the tissue;

a camera positioned to receive the light collected by the endoscope, the camera including;

a high sensitivity color image sensor;

a beam splitter for splitting the light received from the endoscope into at least two beams and one or more optical imaging components that, in combination, project images in the same image plane onto separate imaging areas of the high sensitivity color image sensor; and

one or more filters positioned in front of the separate imaging areas of the high sensitivity color image sensor;

an image processor for digitizing, processing and encoding image signals from the separate imaging areas of the high sensitivity color image sensor; and

a video monitor for displaying the images.

43. A fluorescence endoscopy video system including:

a multimode light source for producing white light, fluorescence excitation light or fluorescence excitation light with a reference reflectance light;

an endoscope for directing light from the light source into a patient to illuminate a tissue sample and to collect the reflected light or fluorescence light produced by the tissue;

a camera positioned to receive the light collected by the endoscope including;

a high sensitivity color image sensor having integrated filters;

a filter selectively positioned in front of the high sensitivity color image sensor for passing light at green and longer wavelengths and for blocking light at blue and shorter wavelengths to the extent that light reaching the sensor is substantially composed of light at green and longer wavelengths and minimally composed of light at blue and shorter wavelengths, and which allows the light in red and green wavelength bands to be further filtered by the integrated filters on the high sensitivity color image sensor; and

one or more optical imaging components that projects images onto the high sensitivity color image sensor;

an image processor that receives image signals from a high sensitivity color image sensor and combines image signals from pixels having filters with the same filter characteristics to form separate images formed by light in each of the two wavelength bands; and

a video monitor for simultaneously superimposing the separate images.

44. The system of claim 43, wherein the camera is attached to the portion of the endoscope that remains outside of the body.

45. The system of claim 43, wherein the camera is built into the insertion portion of the endoscope.

46. The system of claim 44 or 45, further comprising a filter positioned in the light path of the light source that simultaneously transmits the fluorescence excitation light and an amount of reference reflectance light not in a fluorescence detection wavelength band, the amount of reference reflectance light transmitted by the filter being a fraction of the fluorescence excitation light, such that the intensity of the reflected reference light projected onto the high sensitivity color image sensor matches the requirements of the sensor, at the same time as maintaining sufficient fluorescence excitation so as to match the intensity of fluorescence also projected onto the high sensitivity color image sensor to the requirements of the sensor, and wherein the light source filter blocks light from the light source at wavelengths in the fluorescence detection wavelength band from reaching the high sensitivity color image sensor to the extent that the light received by the fluorescence sensor is substantially composed of light resulting from tissue fluorescence and minimally composed of light originating from the light source.

47. The system of claim 46, wherein the filter in front of the high sensitivity color image sensor blocks reflected excitation light, and transmits fluorescence and reference light to the extent that the light received by the high sensitivity color image sensor is substantially composed of light resulting from tissue fluorescence and reflected reference light and minimally composed of fluorescence excitation light.

48. The system of claim 47 wherein the fluorescence, transmitted by the filter in front of the high sensitivity color image sensor, is green light

49. The system of claim 47 wherein the fluorescence, transmitted by the filter in front of the high sensitivity color image sensor, is red light.

50. The system of claim 48, wherein the reference reflectance light, not in the detected fluorescence band, transmitted by the light source filter and the filter in front of the high sensitivity color sensor is red light.

51. The system of claim 49, wherein the reference reflectance light, not in the detected fluorescence band, transmitted by the light source filter and the filter in front of the high sensitivity color sensor is green light.

52. The system of claim 48, wherein the reference reflectance light, not in the detected fluorescence band, transmitted by the light source filter and the filter in front of the high sensitivity color sensor is near-infrared light.

53. The system of claim 49, wherein the reference reflectance light, not in the detected fluorescence band, transmitted by the light source filter and the filter in front of the high sensitivity color sensor is near-infrared light.

54. The system of claim 44 or 45, further comprising a filter positioned in the light path of the light source that transmits the fluorescence excitation light and blocks light at wavelengths in the fluorescence detection wavelength bands from reaching the imaging areas of the high sensitivity color image sensor to the extent that the light received by the sensor is substantially composed of light resulting from tissue fluorescence and minimally composed of light originating from the light source.

55. A fluorescence endoscopy video system including:

a multimode light source for producing white light, fluorescence excitation light or fluorescence excitation light with a reference reflectance light;

an endoscope for directing the light from the light source into a patient to illuminate a tissue sample and to collect the reflected light or fluorescence light produced by the tissue;

a camera positioned to receive the light collected by the endoscope, the camera including;

a color image sensor;

a low light image sensor;

a beam splitter for splitting the light received from the endoscope into at least two beams and directing those beams onto the low light and color image sensors;

one or more filters positioned in front of the low light and color image sensors for selectively transmitting light of desired wavelengths; and

one or more optical imaging components that project images onto both the color image sensor and the low light image sensor;

a processor/controller for digitizing fluorescence and reference image signals received from the low light and color image sensors, the processor/controller including a memory device that stores a sequence of instructions that cause the processor/controller to adjust the intensity of the fluorescence or reference image signal on a pixel by pixel basis as a function of an analysis of the signals received from the low light and color image sensors, and then to encode the adjusted image signals received from the sensors as a video signal; and

a video monitor for displaying the images.

56. The system of claim 55, wherein the camera is attached to the portion of the endoscope that remains outside of the body.

57. The system of claim 55, wherein the camera is built into the insertion portion of the endoscope.

58. The system of claim 56 or 57, wherein the analysis of the digitized image signals that the processor/controller carries out utilizes a ratio of an intensity of the reference reflectance light and the fluorescence light received from the tissue.

59. The system of claim 56 or 57, wherein the analysis of the digitized image signals that the processor/controller carries out utilizes an intensity of pixels that neighbor the one or more pixels and adjusts the processed image signals based on the neighboring pixel intensities.

60. A fluorescence endoscopy video system including:

a multimode light source for producing white light, fluorescence excitation light or fluorescence excitation light and reference reflectance light;

an endoscope for directing the light from the light source into a patient to illuminate a tissue sample and to collect reflected light and fluorescence light produced by the tissue;

a camera positioned to receive the light collected by the endoscope, the camera including;

a high sensitivity color image sensor;

a beam splitter for splitting the light received from the endoscope into at least two beams and one or more imaging optical components that, in combination, project images in the same image plane onto separate imaging areas of the high sensitivity color image sensor; and

one or more filters positioned in front of the separate imaging areas of the high sensitivity color image sensor;

a processor/controller for digitizing fluorescence and reference image signals received from the separate imaging areas of the high sensitivity color image sensor;

the processor/controller including a memory device that stores a sequence of instructions that cause the processor/controller to adjust the intensity of the fluorescence or reference image signal on a pixel by pixel basis as a function of an analysis of the signals received from the different image areas of the high sensitivity color image sensor; and

then encodes the adjusted image signals received from the sensors as a video signal; and

a video monitor for displaying the images.

61. The system of claim 60, wherein the camera is attached to the portion of the endoscope that remains outside of the body.

62. The system of claim 60, wherein the camera is built into the insertion portion of the endoscope.

63. The system of claim 61 or 62, wherein the analysis of the digitized image signals that the processor/controller carries out utilizes a ratio of an intensity of the reference reflectance light and the fluorescence light received from the tissue.

64. The system of claim 61 or 62, wherein the analysis of the digitized image signals that the processor/controller carries out utilizes an intensity of pixels that neighbor the one or more pixels and adjusts the processed image signals based on the neighboring pixel intensities.

65. A fluorescence endoscopy video system including:

a multimode light source for producing white light, fluorescence excitation light or fluorescence excitation light with a reference reflectance light;

an endoscope for directing light from the light source into a patient to illuminate a tissue sample and to collect the reflected light or fluorescence light produced by the tissue;

a camera positioned to receive the light collected by the endoscope including;

a high sensitivity color image sensor having integrated filters;

a filter selectively positioned in front of the whole of the high sensitivity color image sensor for passing light at green and longer wavelengths and for blocking light at blue and shorter wavelengths to the extent that light reaching the sensor is substantially composed of light at green and longer wavelengths and minimally composed of light at blue and shorter wavelengths, and which allows the light in red and green wavelength bands to be further filtered by the integrated filters on the high sensitivity color image sensor; and

one or more optical imaging components that projects images onto the high sensitivity color image sensor;

a processor/controller that receives image signals from a high sensitivity color image sensor and combines image signals from pixels having filters with the same filter characteristics to form separate images formed by light in each of the two wavelength bands;

the processor/controller including a memory device that stores a sequence of instructions that cause the processor/controller to adjust the intensity of the fluorescence or reference image signal on a pixel by pixel basis as a function of an analysis of the signals received from pixels with filters of different wavelength bands on the high sensitivity color image sensor; and

then encodes the adjusted image signals received from the sensors as a video signal; and

a video monitor for simultaneously superimposing the separate video images.

66. The system of claim 65, wherein the camera is attached to the portion of the endoscope that remains outside of the body.

67. The system of claim 65, wherein the camera is built into the insertion portion of the endoscope.

68. The system of claim 66 or 67, wherein the analysis of the digitized image signals that the processor/controller carries out utilizes a ratio of an intensity of the reference reflectance light and the fluorescence light received from the tissue.

69. The system of claim 66 or 67, wherein the analysis of the digitized image signals that the processor/controller carries out utilizes an intensity of pixels that neighbor the one or more pixels and adjusts the processed image signals based on the neighboring pixel intensities.

72. A reference for a fluorescence imaging system, comprising:

a target including a mixture of one or more dyes and a light scattering material, wherein the one or more dyes are selected to emit fluorescence in two or more wavelength bands in a manner similar to known diseased tissue when excited with an excitation light.

73. The reference of claim 72, wherein the one or more dyes and light scattering material are dissolved in a liquid which, following activation, hardens to become a solid.

74. The reference of claim 73, wherein the one or more dyes and light scattering material are dissolved in an epoxy.