Image processing device for fluorescence observation

• US 7,324,674 B2
• Filed: 06/23/2003
• Issued: 01/29/2008
• Est. Priority Date: 06/26/2002
• Status: Expired due to Fees

• Current Assignee: Olympus Corporation
• Original Assignee: Olympus Corporation
• Inventors: Gono, Kazuhiro, Akimoto, Shunya, Ozawa, Takeshi, Imaizumi, Katsuichi
• Primary Examiner(s): Bella; Matthew C.
• Assistant Examiner(s): LIEW, ALEX KOK SOON
• Application Number:
  US10/601,496
• Publication Number:
  US 20040037454A1
• Time in Patent Office: 1,681 Days
• Field of Search: None
• US Class Current: 382/128
• CPC Class Codes: A61B 1/000094 extracting biological structures
• CPC Class Codes: A61B 1/000095 for image enhancement
• CPC Class Codes: A61B 1/0005 combining images e.g. side-by-side, superimposed or tiled
• CPC Class Codes: A61B 1/00186 with imaging filters
• CPC Class Codes: A61B 1/043 for fluorescence imaging
• CPC Class Codes: A61B 1/0638 providing two or more wavelengths
• CPC Class Codes: A61B 1/0646 with illumination filters
• CPC Class Codes: A61B 1/0655 Control therefor
• CPC Class Codes: A61B 1/0669 at proximal end of an endoscope

Abstract

An image processing device having a very straightforward construction, for obtaining an image in which normal tissue and diseased tissue are easy to identify, comprises: an image synthesizing section that generates a synthesized image by synthesizing an image signal of reflected light image produced by illumination light obtained by illuminating body tissue with illumination light and an image signal of a fluorescent image obtained by illuminating the body tissue with excitation light; and a gain adjustment section that adjusts the gain of the image signal of the reflected light image and/or the image signal of the fluorescent image such that the boundary of the hues of the normal tissue and diseased tissue found from the optical characteristics of the respective tissues is contained in a predetermined range with respect to a prescribed standard chromaticity diagram, depending on whether the body tissue that is displayed on the synthesized image generated by the image synthesizing section is normal tissue or diseased tissue.

Claims

1. An image processing device for fluorescence observation comprising:

a changeover filter located in an optical path of a light beam generated by a light source device and impinging on body tissue, said light beam including a spectra from an infrared wavelength band to a visible light band, said changeover filter alters said light beam into illumination light or excitation light;

an image synthesizing section that generates a synthesized image by synthesizing an image signal of a reflected light image sensed by an image pickup element and obtained by illuminating body tissue with said illumination light and an image signal of a fluorescent image sensed by said image pickup element and obtained by illuminating said body tissue with said excitation light, wherein said body tissue is sequentially and non-concurrently illuminated by said illumination light and said excitation light; and

a gain adjustment section that adjusts a gain of said image signal of said reflected light image and/or said image signal of said fluorescent image such that a boundary of hues of normal tissue and diseased tissue found using optical characteristics of said normal tissue and said diseased tissue is included in a predetermined range with respect to a prescribed standard chromaticity diagram, depending on whether said body tissue that is represented in said synthesized image generated by said image synthesizing section is normal tissue or diseased tissue.

2. The image processing device for fluorescence observation according to claim 1, wherein the image signal of the reflected light image comprises an image signal of a first reflected light image obtained by illuminating body tissue with illumination light of a wavelength band including the optical absorption band of hemoglobin and an image signal of a second reflected light image obtained by illuminating the body tissue with illumination light of a wavelength band including the optical non-absorption band of hemoglobin;

the image synthesizing section synthesizes the image signal of the first reflected light image, the image signal of the second reflected light image and the image signal of the fluorescent image; and

the gain adjustment section adjusts at least one gain of the image signal of the first reflected light image, the image signal of the second reflected light image and the image signal of the fluorescent image.

3. The image processing device for fluorescence observation according to claim 2, wherein the image synthesizing section synthesizes the image signal of the first reflected light image, the image signal of the second reflected light image and the image signal of the fluorescent image as respectively different hues.

4. The image processing device for fluorescence observation according to claim 1, wherein

the image signal of the reflected light image comprises an image signal of a first reflected light image obtained by illuminating body tissue with illumination light of a wavelength band including the optical absorption band of hemoglobin and an image signal of a second reflected light image obtained by illuminating body tissue with illumination light of a wavelength band including the optical non-absorption band of hemoglobin;

the image synthesizing section performs synthesis with the first reflected light image allocated to a red color, the second reflected light image allocated to a blue color and the fluorescent image allocated to a green color; and

the gain adjustment section adjusts the gain of the three image signals such that the boundary of the hues of the normal tissue and diseased tissue represented in the synthesized image is included in a range defined by the four points (0.21, 0.53), (0.18, 0.50), (0.23, 0.44) and (0.25, 0.49) with respect to the CIE 1976 UCS chromaticity diagram.

5. The image processing device for fluorescence observation according to claim 1, wherein

the image signal of the reflected light image comprises an image signal of a first reflected light image obtained by illuminating body tissue with illumination light of a wavelength band including the optical absorption band of hemoglobin and an image signal of a second reflected light image obtained by illuminating body tissue with illumination light of a wavelength band including the optical non-absorption band of hemoglobin;

the image synthesizing section performs synthesis with the first reflected light image allocated to a blue color, the second reflected light image allocated to a red color and the fluorescent image allocated to a green color; and

the gain adjustment section adjusts the gain of the thee image signals such that the boundary of the hues of the normal tissue and diseased tissue represented in the synthesized image is included in a range defined by the four points (0.21, 0.53), (0.18, 0.50), (0.23, 0.44) and (0.25, 0.49) with respect to the CIE 1976 UCS chromaticity diagram.

6. The image processing device for fluorescence observation according to claim 1, wherein the boundary of the hues is the point of intersection in the respective probability distribution functions obtained based on the average color tone of pixels in regions of interest that are set in regard to body tissue and diseased tissue, respectively.

7. The image processing device for fluorescence observation according to claim 4, having a normal image mode for generating an image signal of a normal image obtained by successively illuminating body tissue with light of red, green and blue color produced from white light.

8. The image processing device for fluorescence observation according to claim 5, having a normal image mode for generating an image signal of a normal image obtained by successively illuminating body tissue with light of red, green and blue color produced from white light.

9. The image processing device for fluorescence observation according to claim 2, wherein the illumination light of the wavelength band including the optical absorption band of hemoglobin includes 550 nm wavelength.

10. The image processing device for fluorescence observation according to claim 2, wherein the illumination light of the wavelength band including the optical non-absorption band of hemoglobin includes 610 nm wavelength.

24. The image processing device of claim 1, wherein said image pickup element sequentially produces said reflected light image and said fluorescent image.

25. The image processing device of claim 1, wherein said illumination light has a wavelength band containing at least a portion of 470 nm to 700 nm.

26. The image processing device of claim 1, wherein said changeover filter sequentially provides said illumination light and said excitation light.

27. The image processing device of claim 1, wherein said image pickup element is selected from the group consisting of one charge coupled element, one charge modulation device, one C-MOS image pickup element, one amplified MOS imager, and one back-illuminated charge coupled device.

32. The image processing device of claim 1, wherein said changeover filter comprises a nominal illumination filter for transmitting said illumination light and a fluorescent illumination filter for transmitting said excitation light that are concentric and have different radial distances from a center of said changeover filter, and wherein said image processing device further comprises a moving mechanism for moving said changeover filter to select a portion of said changeover filter to be placed in said optical path of said light beam between said nominal illumination filter and said fluorescent illumination filter.

11. An image processing device for fluorescence observation comprising:

a changeover filter located in an optical path of a light beam generated by a light source device and impinging on body tissue, said light beam including a spectra from an infrared wavelength band to a visible light band, said changeover filter alters said light beam into illumination light or excitation light;

image synthesizing means that generates a synthesized image by synthesizing an image signal of a reflected light image sensed by an image pickup element and obtained by illuminating body tissue with said illumination light and an image signal of a fluorescent image sensed by said image pickup element and obtained by illuminating said body tissue with said excitation light, wherein said body tissue is sequentially and non-concurrently illuminated by said illumination light and said excitation light; and

gain adjustment means that adjusts a gain of said image signal of said reflected light image and/or said image signal of said fluorescent image such that a boundary of hues of normal tissue and diseased tissue found using optical characteristics of said normal tissue and said diseased tissue is included in a predetermined range with respect to a prescribed standard chromaticity diagram, depending on whether said body tissue that is represented in said synthesized image generated by said image synthesizing section is normal tissue or diseased tissue.

12. The image processing device for fluorescence observation according to claim 11, wherein

the image signal of the reflected light image comprises an image signal of a first reflected light image obtained by illuminating body tissue with illumination light of a wavelength band including the optical absorption band of hemoglobin and an image signal of a second reflected light image obtained by illuminating the body tissue with illumination light of a wavelength band including the optical non-absorption band of hemoglobin;

the image synthesizing means synthesizes the image signal of the first reflected light image, the image signal of the second reflected light image and the image signal of the fluorescent image; and

the gain adjustment means adjusts at least one gain of the image signal of the first reflected light image, the image signal of the second reflected light image and the image signal of the fluorescent image.

13. The image processing device for fluorescence observation according to claim 11, wherein

the image signal of the reflected light image comprises an image signal of a first reflected light image obtained by illuminating body tissue with illumination light of a wavelength band including the optical absorption band of hemoglobin and an image signal of a second reflected light image obtained by illuminating body tissue with illumination light of a wavelength band including the optical non-absorption band of hemoglobin;

the image synthesizing means performs synthesis with the first reflected light image allocated to a red color, the second reflected light image allocated to a blue color and the fluorescent image allocated to a green color; and

the gain adjustment means adjusts the gain of the three image signals such that the boundary of the hues of the normal tissue and diseased tissue represented in the synthesized image is included in a range defined by the four points (0.21, 0.53), (0.18, 0.50), (0.23, 0.44) and (0.25, 0.49) with respect to the CIE 1976 UCS chromaticity diagram.

14. The image processing device for fluorescence observation according to claim 11, wherein

the image signal of the reflected light image comprises an image signal of a first reflected light image obtained by illuminating body tissue with illumination light of a wavelength band including the optical absorption band of hemoglobin and an image signal of a second reflected light image obtained by illuminating body tissue with illumination light of a wavelength band including the optical non-absorption band of hemoglobin,

the image synthesizing means performs synthesis with the first reflected light image allocated to a blue color, the second reflected light image allocated to a red color and the fluorescent image allocated to a green color, and

the gain adjustment means adjusts the gain of the three image signals such that the boundary of the hues of the normal tissue and diseased tissue represented in the synthesized image is included in a range defined by the four points (0.21, 0,53), (0.18, 0.50), (0.23, 0.44) and (0.25, 0.49) with respect to the CIE 1976 UCS chromaticity diagram.

33. The image processing device of claim 11, wherein said changeover filter comprises a nominal illumination filter for transmitting said illumination light and a fluorescent illumination filter for transmitting said excitation light that are concentric and have different radial distances from a center of said changeover filter, and wherein said image processing device further comprises a moving mechanism for moving said changeover filter to select a portion of said changeover filter to be placed in said optical path of said light beam between said nominal illumination filter and said fluorescent illumination filter.

15. An image processing device for fluorescence observation comprising:

a light source device that emits a light beam comprising illumination light including two different wavelength bands, a wavelength band including an optical absorption band of hemoglobin and another wavelength band including an optical non-absorption band of hemoglobin, and excitation light in yet another wavelength band for exciting fluorescence;

a changeover filter located in an optical oath of said light beam, wherein said changeover filter alters said light beam into illumination light or excitation light;

an image pickup section that comprises an image pickup element that picks up respectively two reflected light images produced by reflected light obtained by reflection after illuminating body tissue with said illumination light of said two different wavelength bands from said light source device and a fluorescent image produced by fluorescence excited by illuminating body tissue with said excitation light from said light source device wherein said body tissue is sequentially and non-concurrently illuminated by said illumination light and said excitation light; and

an image processing section that generates a processed image by signal processing of image signals of said two reflected light images obtained by said image pickup section and an image signal of said fluorescent image;

wherein said image processing section comprises;

a signal input section that inputs thee image signals consisting of said image signals of said two reflected light images picked up by said image pickup section and said image signal of said fluorescent image;

an image synthesizing section that generates a synthesized image by image synthesis of said image signals of said two reflected light images and said fluorescent image signal, wherein one of said image signals of said two reflected light images is an image signal of said wavelength band including said optical absorption band of hemoglobin, and wherein the other of said image signals of said two reflected light images is an image signal of said wavelength band including said optical non-absorption band of hemoglobin; and

a gain adjustment section that adjusts a gain of at least one of said image signals of said two reflected light images and said fluorescent image signal that are input by said signal input section such that a boundary of hues of normal tissue and diseased tissue represented in said synthesized image synthesized by said image synthesis section is included in a range defined by the four points (0.21, 0.53), (0.18, 0.50), (0.23, 0.44) and (0.25, 0,49) with respect to the CUB 1976 UCS chromatic city diagram.

16. The image processing device for fluorescence observation according to claim 15, wherein the image synthesizing section effects synthesis by allocating one of the two reflected light images to a red color, allocating the other to a blue color and allocating the fluorescent image to a green color.

17. The image processing device for fluorescence observation according to claim 15, wherein the image synthesizing section effects synthesis by allocating one of the two reflected light images to a blue color, allocating the other to a red color and allocating the fluorescent, image to a green color.

18. The image processing device for fluorescence observation according to claim 15, wherein the boundary of the hues is the point of intersection in the respective probability distribution functions obtained based on the average color tone of pixels in regions of interest that are set in regard to body tissue and diseased tissue, respectively.

34. The image processing device of claim 15, wherein said changeover filter comprises a nominal illumination filter for transmitting said illumination light and a fluorescent illumination filter for transmitting said excitation light that are concentric and have different radial distances from a center of said changeover filter, and wherein said image processing device further comprises a moving mechanism for moving said changeover filter to select a portion of said changeover filter to be placed in said optical path of said light beam between said nominal illumination filter and said fluorescent illumination filter.

19. An image processing device for fluorescence observation comprising:

a light source device that emits a light beam comprising illumination light including two different wavelength bands, a wavelength band including an optical absorption band of hemoglobin and another wavelength band including an optical non-absorption band of hemoglobin, and excitation light in yet another wavelength band for exciting fluorescence;

a changeover filter located in an optical path of said light beam, wherein said changeover filter alters said light beam into illumination light or excitation light;

an image pickup section comprises an image pickup that element that picks up respectively two reflected light images produced by reflected light obtained by reflection after illuminating body tissue with illumination light of said two different wavelength bands from said light source device and a fluorescent image produced by fluorescence excited by illuminating said body tissue with said excitation light from said light source device, wherein said body tissue is sequentially and non-concurrently illuminated by said illumination light and said excitation light; and

an image processing section that generates a processed image by signal processing of image signals of said two reflected light images obtained by said image pickup section and an image signal of said fluorescent image;

wherein said image processing section comprises;

signal input means that inputs three image signals consisting of said image signals of said two reflected light images picked up by said image pickup section and said image signal of said fluorescent image;

image synthesizing means that generates a synthesized image by image synthesis of said image signals of said two reflected light images and said fluorescent image signal, wherein one of said image signals of said two reflected light images is an image signal of said wavelength band including said optical absorption band of hemoglobin, and wherein the other of said image signals of said two reflected light images is an image signal of said wavelength band including said optical non-absorption band of hemoglobin; and

gain adjustment means that adjusts a gain of said three image signals that are input by the signal input means such that a boundary of hues of normal tissue and diseased tissue represented in said synthesized image synthesized by said image synthesis section is included in a range defined by the four points (0.21, 0.53), (0.18, 0.50), (0.23, 0.44) and (0.25, 0.49) with respect to the CIE 1976 UCS chromaticity diagram.

20. The image processing device for fluorescence observation according to claim 19, wherein the image synthesizing section effects synthesis by allocating one of the two reflected light images to a red color, allocating the other to a blue color and allocating the fluorescent image to a green color.

21. The image processing device for fluorescence observation according to claim 19, wherein the image synthesizing section effects synthesis by allocating one of the two reflected light images to a blue color, allocating the other to a red color and allocating the fluorescent image to a green color.

22. The image processing device for fluorescence observation according to claim 19, wherein the fluorescence wavelength band is a wavelength band including 520 nm wavelength and the two reflected light bands are wavelength bands comprising respectively 550 nm wavelength and 600 nm wavelength.

23. The image processing device for fluorescence observation according to claim 22, wherein the wavelength width of the two reflected light bands is no more than 20 nm.

28. The image processing device of claim 19, wherein said image pickup element sequentially produces said two reflected light images and said fluorescent image.

29. The image processing device of claim 19, wherein said illumination light including said two different wavelength bands has a wavelength band containing at least a portion of 470 nm to 700 nm.

30. The image processing device of claim 19, wherein said changeover filter sequentially provides said illumination light of the two different wavelength bands and said excitation light.

31. The image processing device of claim 19, wherein said image pickup element is selected from the group consisting of one charge coupled element, one charge modulation device, one C-MOS image pickup element, one amplified MOS imager, and one back-illuminated charge coupled device.

35. The image processing device of claim 19, wherein said changeover filter comprises a nominal illumination fitter for transmitting said illumination light and a fluorescent illumination filter for transmitting said excitation light that are concentric and have different radial distances from a center of said changeover filter, and wherein said image processing device further comprises a moving mechanism for moving said changeover filter to select a portion of said changeover filter to be placed in said optical path of said light beam between said nominal illumination filter and said fluorescent illumination filter.