Imaging systems and methods for displaying fluorescence and visible images

• US 11,025,867 B2
• Filed: 11/03/2016
• Issued: 06/01/2021
• Est. Priority Date: 12/22/2006
• Status: Active Grant

• Current Assignee: Stryker Corporation
• Original Assignee: Stryker European Operations Limited (Stryker Corporation)
• Inventors: Westwick, Paul R., Potkins, David, Fengler, John
• Primary Examiner(s): Rahaman, Mohammed S
• Application Number:
  US15/343,038
• Publication Number:
  US 20170064258A1
• Time in Patent Office: 1,671 Days
• CPC Class Codes: A61B 1/041 Capsule endoscopes for imaging
• CPC Class Codes: A61B 1/043 for fluorescence imaging
• CPC Class Codes: A61B 1/045 Control thereof
• CPC Class Codes: A61B 1/051 Details of CCD assembly
• CPC Class Codes: A61B 1/0638 providing two or more wavelengths
• CPC Class Codes: A61B 1/0646 with illumination filters
• CPC Class Codes: A61B 1/0669 at proximal end of an endoscope
• CPC Class Codes: A61B 1/0676 at distal tip of an endoscope
• CPC Class Codes: A61B 1/0684 using light emitting diodes [LED]
• CPC Class Codes: A61B 1/128 provided with means for regulating temperature
• CPC Class Codes: A61B 3/18 Arrangement of plural eye-testing or -examining apparatus
• CPC Class Codes: H04N 23/10 for generating image signals from different wavelengths
• CPC Class Codes: H04N 23/555 for picking-up images in sites, inaccessible due to their dimensions or hazardous conditions, e.g. endoscopes or borescopes
• CPC Class Codes: H04N 23/56 provided with illuminating means
• CPC Class Codes: H04N 23/84 for processing colour signals
• CPC Class Codes: H04N 23/843 Demosaicing, e.g. interpolating colour pixel values
• CPC Class Codes: H04N 25/136 using complementary colours
• CPC Class Codes: H04N 7/183 for receiving images from a single remote source

Abstract

An endoscopic video system and method using a camera with a single color image sensor, for example a CCD color image sensor, for fluorescence and color imaging and for simultaneously displaying the images acquired in these imaging modes at video rates in real time is disclosed. The tissue under investigation is illuminated continuously with fluorescence excitation light and is further illuminated periodically using visible light outside of the fluorescence excitation wavelength range. The illumination sources may be conventional lamps using filters and shutters, or may include light-emitting diodes mounted at the distal tip of the endoscope.

Claims

1. A method of displaying a colorized luma image comprising:

illuminating tissue under observation with white light from a white light illumination source and excitation light;

simultaneously detecting at an image sensor a color image from reflectance of the white light from the white light illumination source and a fluorescence image produced by illuminating the tissue under observation with the excitation light to simultaneously produce image data comprising both color image data and fluorescence image data;

computing a luma image from the detected color image data;

colorizing the luma image based on the detected fluorescence image data; and

displaying the colorized luma image.

2. The method of claim 1, wherein the excitation light is near infra-red light.

3. The method of claim 1, wherein the fluorescence image is produced by exciting a fluorescent dye with the excitation light.

4. The method of claim 3, wherein the fluorescent dye comprises indocyanine green (ICG).

5. The method of claim 1, wherein the color image and the fluorescence image are detected using a CCD or CMOS image sensor.

6. The method of claim 5, wherein the color image and the fluorescence image are detected using an image sensor and a light blocking filter to block the excitation light from reaching the image sensor.

7. The method of claim 1, wherein color image and fluorescence image are detected using a single CCD image sensor.

8. The method of claim 1, wherein color of the displayed colorized luma image is indicative of health of the tissue under observation.

9. The method of claim 1, comprising normalizing the detected fluorescence image and colorizing the luma image based on the normalized detected fluorescence image.

10. The method of claim 9, wherein the fluorescence image is normalized using a red image.

31. The method of claim 3, wherein the dye is indocyanine green (ICG).

11. A system for visualizing tissue under observation comprising:

an illumination device having a white light source that emits white light, and an excitation light source that emits excitation light having a predetermined wavelength for exciting a fluorescent dye;

one or more image sensors for simultaneously detecting a color image and a fluorescence image to simultaneously produce image data comprising both color image data and fluorescence image data, wherein at least one of the one or more image sensors that is used for detecting the color image is also used to detect the fluorescence image; and

a processing system configured to compute a luma image from the detected color image, colorize the luma image based on the detected fluorescence image, and output a colorized luma image to a display, wherein the system is configured to simultaneously detect produce image data comprising both color image data from reflectance of the white light from the white light source and fluorescence image data.

12. The system of claim 11, wherein the system is an endoscope system.

13. The system of claim 11, wherein the illumination device comprises a near infra-red excitation light source.

14. The system of claim 11, wherein the excitation light source emits excitation light having a predetermined wavelength for exciting indocyanine green (ICG) fluorescent dye.

15. The system of claim 11, wherein the one or more image sensors comprise one or more CCD or CMOS image sensors.

16. The system of claim 11, further comprising a light blocking filter to block the excitation light from reaching at least one of the one or more image sensors.

17. The system of claim 11, comprising a single sensor for detecting the color image and the fluorescence image.

18. The system of claim 11, wherein the processing system is configured to colorize the luma image to indicate health of the tissue under observation.

19. The system of claim 11, wherein the processing system is configured to normalize the detected fluorescence image and colorize the luma image based on the normalized detected fluorescence image.

20. The system of claim 19, wherein the fluorescence image is normalized using a red image.

21. A method of displaying a colorized luma image of tissue of a subject, the method comprising:

administering a fluorescent dye to the subject;

illuminating the tissue with white light from a white light illumination source and excitation light;

detecting a color image from reflectance of the white light from the white light source, and simultaneously detecting a fluorescence image produced by illuminating the tissue with the excitation light to simultaneously produce image data comprising both color image data and fluorescence image data, wherein at least one of image sensor that is used for detecting the color image is also used to detect the fluorescence image;

computing a luma image from the detected color image data;

colorizing the luma image based on the detected fluorescence image data; and

displaying the colorized luma image.

22. The method of claim 21, wherein the excitation light is near infra-red light.

23. The method of claim 21, wherein the fluorescent dye comprises indocyanine green (ICG).

24. The method of claim 21, wherein the color image and the fluorescence image are detected using a CCD or CMOS image sensor.

25. The method of claim 24, wherein the color image and the fluorescence image are detected using an image sensor and a light blocking filter to block the excitation light from reaching the image sensor.

26. The method of claim 21, wherein color image and fluorescence image are detected using a single CCD image sensor.

27. The method of claim 21, wherein color of the displayed colorized luma image is indicative of health of the tissue under observation.

28. The method of claim 21, comprising normalizing the detected fluorescence image and colorizing the luma image based on the normalized detected fluorescence image.

29. The method of claim 28, wherein the fluorescence image is normalized using a red image.

30. The method of claim 21, wherein the dye is indocyanine green (ICG).